Power Moves: SQL Server on Linux, Rails, and Docker

Tue, 06 Jun 2017 00:00:00 -0400

Microsoft is releasing new versions of SQL Server on Linux more and more frequently. Last month brought us SQL Server 2017 Community Technology Preview 2.0 and like the others before, it's a breeze to use with Docker.

Around the same time, Rails 5.1 was released. Writing modern web applications using every technology from the backend database to frontend JavaScript has never been more enjoyable. Rails 5.1 also raises intergration with SQL Server to an all-time record high.

Now... today... this very moment... it is time to get merge-obsessed and combine our developer super powers to form some sweet new Power Moves. GO!

Your Environment

Before moving on, we should pick a secure password for our local SQL Server administrator. The SA_PASSWORD variable is used both by Rails and SQL Server on Docker. I recommend setting this in your Bash, ZSH or whatever config your shell uses.

$ export SA_PASSWORD=mysecretpassword

Install Docker

If you do not have it already, please download and install Docker. They offer a free Community Edition for most platforms. Once you have downloaded and verified your install, please update your Docker resource preferences to allow up to 4GB of memory. SQL Server will need a bit more than the 2GB default.

Install SQL Server

This command will download the latest SQL Server image from Microsoft's official hub on Docker, currently SQL Server 2017 CTP v2.0.

$ docker pull microsoft/mssql-server-linux

Now, start SQL Server using the previous environment settings. The first command will create a persistent data volume while the second starts SQL Server via Docker and attaches that volume to the container. All new databases will persist to the mssqldata volume.

$ docker create \
  -v /var/opt/mssql \
  --name mssqldata microsoft/mssql-server-linux

$ docker run -p 1433:1433 \
  -e "ACCEPT_EULA=Y" \
  -e "SA_PASSWORD=$SA_PASSWORD" \
  -d --volumes-from mssqldata \
  microsoft/mssql-server-linux

Create a Rails Application

This guide does not cover Ruby installation. Rails requires Ruby v2.2.2 or higher and if you are new to Ruby or do need to install the latest version, I recommend rbenv for POSIX systems or RubyInstaller if you are on Windows.

Let's install the latest version of Rails and Bundler using RubyGems.

$ gem install rails bundler
Successfully installed rails-5.1.1
Successfully installed bundler-1.15.1

To create your new Rails application with SQL Server as the database, we pass the -d flag with a value of sqlserver. This will install the SQL Server Adapter and TinyTDS dependencies – the Ruby bindings to DBLIB. Finally, we test that we can boot our new Rails application and load it in a browser. If all goes well, you should see the following image.

$ rails new my_app -d sqlserver
$ cd my_app
$ ./bin/rails server

=> Booting Puma
=> Rails 5.1.1 application starting in development on http://localhost:3000

Ultimate Power Move

A static welcome page is fine and all, but let's see if we can make sure our SQL Server database works with ActiveRecord.

$ ./bin/rails db:setup

Created database 'my_app_development'
Created database 'my_app_test'

Success! The db:setup command has shown us that it has created two databases for us and that everything is bootstraped to SQL Server correctly. We can create a model/table now via a migration.

$ ./bin/rails generate model User name:string email:string
$ ./bin/rails db:migrate

== CreateUsers: migrating ======================================
-- create_table(:users)
   -> 0.0024s
   -> -1 rows
== CreateUsers: migrated (0.0025s) =============================

Success again! Here we crated a User model/table with a few attributes. Running the migration creates the table and columns as needed and we can even fire up a console to test it all out.

$ ./bin/rails console

> User.count
  SQL (0.9ms)  USE [my_app_development]
   (2.1ms)  SELECT COUNT(*) FROM [users]
=> 0

Thanks!

There are a lot of good people working hard every day to make your experience with Ruby & SQL Server amazing. From engineers at Microsoft all the way to individual contributors on the Rails SQL Server repos.

In the coming weeks, we hope to do so much more. What might be next? Maybe better integration to JRuby within the adapter. Maybe native cross-platform tools to script large databases when the Rails DSL can not represent that schema. All of this is in the works.

I am very proud to be a part of this community and hope you find time to stop by and help, or drop a friendly note/issue about your experiences with Ruby & SQL Server. Lastly, thanks to CustomInk for supporting our passion projects and individual causes.

Types of Pain

Thu, 14 Jul 2016 00:00:00 -0400

Just sharing some pains for the first commit(s) to the SQL Server Rails v5 support effort. These are very raw findings I captured while working. The idea is to share the pains/gains in making a 3rd party adapter for Rails 5. Goals include:

Have fun with ActiveRecord v5.
Learn where things are hard for us because we may be doing it wrong!
Learn where things are hard for 3rd party adapters in general.
Make contributions to Rails when it makes sense.
Share how the SQL Server adapter is different.

The title of this post is for humor only. The internals of Rails' ActiveRecord have moved forward with leaps and bounds. The ones most interesting for me & the adapter are the new type system. I wrote a little about Sean Griffin's excellent work in a post called ActiveRecord 4.2's Type Casting last year. This year, I was totally expecting Sean to deliver on his promises to decouple types and quoting to help make ActiveRecord better.

Forward: Our Prepared Statement Methodology

Is completely different than most other adapters whereby we use the sp_executesql stored procedure to cache frequently used query plans at the database itself. See this 2011 Engine Yard Blog which talks about this methodology.

Schema Reflection

The SchemaStatements#columns is the first thing that usually breaks when we upgrade for major Rails releases. Like many other low level adapter parts, when it breaks... NOTHING works till you get it fixed. I had a particularly fun time with the changes needed this time around.

Our lowest level method to look up all things schema related is our column_definitions method. The SQL here is responsible for joining many info schema tables to give us everything we need to know for the current database as well as other schemas. Even though there are caches in ActiveRecord just for schema reflection, we found this SQL has to be fast. Hence we create our own binds to leverage sp_executesql's statement cache abilities.

Previously we created binds by making a new column object using the INFORMATION_SCHEMA.COLUMNS.TABLE_NAME of type nvarchar(128) with whatever table name value requested by ActiveRecord. Optionally we did the same with the TABLE_SCHEMA if a table name identifier was fully qualified. This kept our DB reflection fast. However, binds no longer contain column information. This is what I found in the ActiveRecord::Relation class where binds and substitutions are made:

QueryAttribute.new(arel_attr.name, value, klass.type_for_attribute(arel_attr.name))

In some ActiveRecord tests, we see the following:

ActiveRecord::Relation::QueryAttribute.new "id", 20, Post.type_for_attribute("id")

It was now apparent to me how ActiveRecord broke columns out of the type/cast/quote layer by using columns further up the stack. The Relation::QueryAttribute is actually a subclass of ActiveRecord::Attribute. My solution to fix schema reflection was to manually create some query attributes for our binds.

nv128 = SQLServer::Type::UnicodeVarchar.new limit: 128
binds << Relation::QueryAttribute.new('TABLE_NAME', identifier.object, nv128)
binds << Relation::QueryAttribute.new('TABLE_SCHEMA', identifier.schema, nv128) unless identifier.schema.blank?
results = sp_executesql(sql, 'SCHEMA', binds)

Now we were able to reflect on the database again. However, because we need to know the SQL Server data type for our binds to work with sp_executesql, we have extended our types to respond to a sqlserver_type similar to the simplified type. This way, we can ask our binds for type.sqlserver_type and get the tidy type arguments needed for sp_executesql. That was a lot of type'in 😀

module ActiveRecord
  module ConnectionAdapters
    module SQLServer
      module Type
        module Sql

          def sqlserver_type
            type.to_s
          end

        end
        ::ActiveModel::Type::Value.include SQLServer::Type::Sql
      end
    end
  end
end

Updating Our Types

This is where things get hard for us. Unlike other databases, quoting a Ruby type for the database is often dependent on the SQL Server data type of a specific column. For example, a string might be easy for MySQL. But for SQL Server, only non-national (single byte) strings/types can be quoted with single quotes. For multi-byte strings of database type nchar, nvarchar, ntext, and nvarchar(max) – we have to quote using the N'string' "national" prefix. Fun! More interesting are supporting our date and time types. Many of which have different fractional precision and quoting is important to each.

ActiveRecord's types have been moved to ActiveModel and each needs their own serialize(value) implementation vs the old type_cast_for_database(value) which now resides at the connection and a type caster map object. We easily updated our type objects to use the serialize method. But I have yet to dig down to see how many types I will have to couple to quoting as the project moves forward.

At first glance, our column objects are getting smaller and the type objects are getting fatter.

During Inserts

Our exec_insert relied on knowing two critical things. First, if a identity column is being inserted. Second, the target table name of that insert so that we enable identity inserts (table scope) for the duration of that insert statement. Since binds no longer have column information, we are going to have to start trusting the pk parameter signaling that an ID has been set. We were somewhat doing this anyway in our sql_for_insert method. Also, once again, we need to pick up the pattern of using regular expressions on the sql value to pull the table name out. It appears the PostgreSQL adapter does this as well. It will be interesting to see if this slows us down or not.

Microsoft's First Open Source DataCamp

Mon, 19 Oct 2015 00:00:00 -0400

Rarely do the words "open source" come to mind when the Microsoft Corporation is mentioned. As the maintainer of the SQL Server stack for both Ruby & Rails, and a long time Apple/UNIX fan, I have been engineering solutions to make working with SQL Server as easy as possible. Often times the work has required outside in hacks to a closed & proprietary set of technologies. That changed last week.

Both Sean Griffin and I were invited to represent the Ruby/Rails community at Microsoft's first #OSSDataCamp. The idea is simple; to open up and engage our community at an intimate level. The results so far are incredibly positive and these new relationships feel lasting and honest.

This post contains interesting details if you want to learn more about Microsoft's Azure platform. We were lucky enough to be exposed to some candid inner workings of their latest service offerings which look to be a strong competitor to Amazon's Web Services.

We Want To Learn From You

That sentiment was echoed by everyone I met. From engineers to leadership and various product managers in-between, Microsoft is hungry to learn and this time it does not feel like the old consume & conquer strategies from the IronRuby days. Instead their new culture closely reflects that of CustomInk's own Inkers. Our hosts were the most upbeat and polite folks I have seen.

Windows Support - A Diversity Issue

As Sean Griffin put it, we cannot require new engineers to buy a MacBook Pro to learn Ruby. Be it a Rails Girls workshop or your local meetup group, a poor Windows experience can literally lose good, diverse talent in our community. Thanks to projects like RailsInstaller, and the modern replacement RubyInstaller, we are stronger than we have historically been. But we can do better.

To that end, Microsoft has kindly agreed to provide key projects like Rails, TinyTDS, and the ActiveRecord SQLServerAdapter with funded continuous integration using AppVeyor services. Such a small cost can greatly impact our feedback loop and identify pain points quickly.

A Changing Culture

Change is hard on all sides. For me, getting into the mindset that Microsoft can help was a big step. When Sean found issues with execjs that led back to Windows Script Host, they introduced us to that team. Likewise, the SQL Server adapter has always needed a bullet-proof cross platform binary to dump the schema. "We can help and here are the engineers..." was the reply. Wow!

Some issues sit well in Microsoft's sphere to solve. What most impressed me was how eager they were to address them. Being a large company, often times the immediate answer was to schedule a series of 1-on-1s with the team(s) responsible. Connections and conversations like this quickly led to even pondering if Ruby could be installed on Windows by default. Or maybe even build cross platform SQL Server development binaries! 😱

Quid Pro Quo

So what did Microsoft learn most from us? Other than Rubyists keep it weird? Likely their biggest takeaway was that Sean Griffin loves to play the Keep Talking and Nobody Explodes game. I mean he really loves to play this game.

It's simple, use 2-way, fast-paced, verbal communication to quickly identify and defuse a virtual ticking time bomb. As it turned out, this is a great team building exercise. Becoming a bomb defusing expert is high on my list and I can not wait to play with a group of my fellow Inkers. Thanks Sean!

Coming Soon To Ruby/Rails

We got a lot of work done along with establishing some big milestones for those using Windows and/or SQL Server with Ruby. Here are some things you can look forward to:

Upgrade TinyTDS to the latest FreeTDS v0.95 which supports the TDS v7.3 protocol.
Making the adapter pass all tests for Rails 5 under active development.
Allow TinyTDS to be statically compiled on every platform w/OpenSSL for easier Azure support.
ActiveRecord JSON support via a v4.2 update. Leverages SQL 16 Box/Azure built-in JSON.
Binstub wrappers for all statically compiled FreeTDS binaries.
First class Azure support for Rails which includes proper DB setup and clone tasks.
Allow the rails-sqlserver adapter to use an alternate jTDS connection mode.

These items are tracked in the TinyTDS and/or the SQL Server Adapter milestones on GitHub. Follow along or contribute if anything interests you.

Microsoft Azure In Brief

Azure != SQL Server, it is much more. Think AWS (Analytics, IoT, CDN, Etc) with a better frontend portal. Microsoft's Azure platform and PaaS offerings are amazing and accessible for anyone looking to rapidly build out their new web application suite of microservices. Seriously, you really should consider it next time you are looking for a Heroku-style marketplace. Here is a quick brief:

1 out of 4 VMs are Linux.
7 million requests a second.
1.6 million SQL databases.

Command Line & Application Interfaces

Portal smortal, who wants GUIs? The real power is in the command line and automation. Just like Heroku's toolbelt, Azure has cross platform binaries that are open source on GitHub. This client has rich 2FA and is packed with interfaces. Services can even be stood up with a simple JSON template hosted on GitHub. Need an application interface? They have a Ruby SDK too.

Services Marketplace

Their marketplace and open source service offerings are seriously impressive. One that stood out is their Elasticsearch service and how well it can integrate with any data source. They literally built an entire frontend GUI that can help you build out super impressive search indexes.

The message, "We give you 99.99% uptime, you worry about how search works, and we take care of the rest." really stood out to me as a market differentiator. Maintaining shard strategies and the complexities of Elasticsearch is not what I want to do.

SQL Server 2016

I can't even begin to go into all the amazing features of SQL Server 2016 now available on Azure. A quick list includes:

Built-In JSON Support.
Always encrypted data with transparent app/client usage.
Data Lake infinite storage with built-in Revolution-R analytics.
Diverse sharding and multi-tenant structures.

One feature that stood out was SQL Server's ability to stretch or archive historical data transparently and securely. This can be done entirely via their hosted service or coupled with your on-premises SQL Server databases. All without the application caring about managing that cold <=> hot data.

Many Thanks!

Special thanks to Microsoft as a whole. Especially to Andrea Lam and Meet Bhagdev for their personal & professional support during our trip. You made us feel right at home. Also in attendance for the first #OSSDataCamp were Michael Manfre, Michiya Takahashi, and Tim Graham. All of whom represented the Python/Django community and helped our open source cause.

This is a process and one that I hope continues with Ruby and other languages. It will succeed if Microsoft actively contributes back to open source and in some cases, champions certain projects with an earnest level of excellence.

To the right is Tobias Ternstrom and myself. The reason I am so happy in that photo is that Tobias literally helped make FETCH happen in SQL Server. Supporting a sane limit/offset model allowed projects like the ActiveRecord adapter to delete hundreds of lines of code while becoming more maintainable. Thanks so much Tobias! All ORMs everywhere love you.

And lastly, I want to give a shout out to Lars Kanis who has been a huge contributor over the past many months by making TinyTDS work well on Windows. Let him know it if you see his pull requests.

Customizing Rake Tasks In Rails 4.1 And Higher

Sun, 08 Feb 2015 00:00:00 -0500

I have been overriding, invoking, and executing custom Rake tasks since I was an early Ruby developer. Tweaking your project's automated tasks are likely the closest thing Rails developers come to building their own light saber. Most popular are adding or changing how the Rails test suite behaves. For example, adding Capybara to your project.

Recently I have been upgrading projects from 3.2 to 4.2 and one thing that really stood out to me was how Rails testing tasks are created and run. Most obvious is that the default test task now runs all model, controller, mailer, helper, job, and integration tests in a single process now. Upon investigation, I found that these major changes were introduced in Rails 4.1. Go check out their new testing.rake file if you are interested. Much cleaner than before!

So what does this mean for you? Let me describe a real scenario I had to address in one of my projects. This project uses two custom test additions. The first is called Konacha, a gem that leverages the asset pipeline to easily test my projects JavaScript with Mocha & Chai under Capybara with PhantomJS. The second addition uses the Capybara::DSL within the standard Rails test/integration namespace and directory structure.

When moving to Rails 4.2 I found that my integration tests were mixed together with everything else. This was a problem since I really wanted my slower integration tests to run after all my other tests. No matter what I tried, I could not get the setup I wanted. So like most problems, I solved this by diving into the code and learning.

Deconstructing Rails::TestTask

So we already had a look at the new testing.rake file above. But looking closer, we can see how the default test task is defined right at the top. Also of interest to us is that test:run task.

require 'rake/testtask'
require 'rails/test_unit/sub_test_task'

task default: :test

desc "Runs all tests in test folder"
task :test do
  Rails::TestTask.test_creator(Rake.application.top_level_tasks).invoke_rake_task
end

namespace :test do
  # ...
  Rails::TestTask.new(:run) do |t|
    t.pattern = "test/**/*_test.rb"
  end
  # ...

That description for test is pretty helpful and quite accurate. Indeed even though we have distinct test tasks like test:models and test:mailers, this default task is somehow aggregating each into a single run. To learn more about this implementation, we need to open up the sub_test_task.rb file required at the top.

In here we can see two new utility classes. One called TestCreator and another called TestInfo. The TestCreator#invoke_rake_task instance method is what we came here for.

def invoke_rake_task
  if @info.files.any?
    create_and_run_single_test
    reset_application_tasks
  else
    Rake::Task[ENV['TEST'] ? 'test:single' : 'test:run'].invoke
  end
end

Without knowing too much about the @info object initialized by TestCreator, we can make a pretty good guess that test:run is what we came here to find. By looking back at the test:run tasks pattern of "test/**/*_test.rb" it seems clear that this is the test responsible for running all of our tests at once. Knowing that, let's start adding code to the project's Rakefile and make our new tests tasks do our bidding.

Customizing Your Rakefile

Here is the first cut at customizing our Rails project's Rakefile. These come right after the default Rails.application.load_tasks line.

Rake::Task['test:run'].clear

namespace :test do

  task 'js' => 'konacha:run'

  Rails::TestTask.new(:_run) do |t|
    t.test_files = FileList['test/**/*_test.rb'].exclude(
      'test/integration/**/*_test.rb'
    )
  end

  task :run => ['test:js', 'test:_run', 'test:integration']

end

The first thing that is happening is we want to remove the existing test:run task using the clear method. We then open up the test task namespace and make a few additions. In order they are:

Create a simple test:js alias for the Konacha provided konacha:run.
Create our test:run replacement but now named test:_run. Note how we use test_files vs a pattern so we can use the exclude method on FileList to remove all integration tests.
Make a new test:run task that calls each other task in the order we want.

If you were to run rake test now, we would see that our integrations are still inter-mixed with all the other Rails test cases. But why!?!

Final Rakefile Separating Integrations

So why did our integration tests still run with all the other Rails test cases? The answer lies in one simple override that Rails::TestTask implements. Normally the Rake::TestTask calls an instance method named define which basically builds a string and executes a Ruby subprocess. The Rails::TestTask class implements the define method like so:

def define
  task @name do
    if ENV['TESTOPTS']
      ARGV.replace Shellwords.split ENV['TESTOPTS']
    end
    libs = @libs - $LOAD_PATH
    $LOAD_PATH.unshift(*libs)
    file_list.each { |fl|
      FileList[fl].to_a.each { |f| require File.expand_path f }
    }
  end
end

See what is happening there? At the end of the day, all Rails::TestTask tasks do is require the test files. So now it makes perfect sense as to why it appeared that each task was being merged. With this knowledge, we can now customize our setup.

Rather than using Rails test task, we will just bail out and use the default Rake::TestTask instead. When doing so, we will have to push the test directory to the libs too. This is one of the few things done for us by Rails::TestTask.

Rake::Task['test:run'].clear
Rake::Task['test:integration'].clear

namespace :test do

  task 'js' => ['konacha:run']

  Rake::TestTask.new(:_run) do |t|
    t.libs << "test"
    t.test_files = FileList['test/**/*_test.rb'].exclude(
      'test/integration/**/*_test.rb'
    )
  end

  Rake::TestTask.new('integration' => 'test:prepare') do |t|
    t.libs << 'test'
    t.pattern = 'test/integration/**/*_test.rb'
  end

  task :run => ['test:js', 'test:_run', 'test:integration']

end

Now we are cooking. All of our JavaScript tests will run first, then all normal Rails test cases, and finally our integration tests. By customizing the test:run task we get the full benefit of the default test task implementation which allows us to pass file and/or TESTOPTS arguments. This happens to be one of my favorite features of Rails 4.1 and up too.

$ rake test test/models/user_test.rb
$ rake test test/integration/app_stories_test.rb

And it all just works! Thanks for reading!

ActiveRecord SQLServer v4.2.0 - Code Name Kantishna

Sun, 25 Jan 2015 00:00:00 -0500

After 4 weeks of work, the ActiveRecord SQL Server adapter is now ready for Rails v4.2.0 and in pre-release status. The list of features is impressive. But first, some bad news. Code named Kantishna, after a small community in Denali area of Alaska, the adapter is only for SQL Server 2012 and higher. This is a good thing because these versions of SQL Server have something we have needed for a very long time, simple offset and limit windowing functions.

FETCH Can Happen

Sure there are hacks like ROW_NUMBER and in theory these work. In reality it has caused our code to be incredibly complex and infected all parts of the adapter. Problems were often solved by using regular expressions and affected performance. Even worse, I think it kept people from contributing to the project. Worry not, if you are on an older version of SQL Server, please feel free to use an older version of Rails and the adapter.

For the rest, FETCH can happen and that is indeed what we are using, OFFSET and FETCH. Here is a great article that describes its basic usage. To see the benefits, take a quick look at our latest Arel visitor and note the difference.

Whats New

For full details check out the latest CHANGELOG on our project repo. But here is a great summary. The v4.2.0 adapter is the strongest version to date!

TYPE CASTING: A few weeks ago I wrote a full article on the new Type objects in ActiveRecord 4.2. These objects help us do a much better job of ensuring data going in and out of the DB are proper. They will play a huge role in Rails 5.0 when the API becomes public. In Rails 4.2 they help us quote values and output proper schema.rb files. We have aliased our type namespace to ActiveRecord::Type::SQLServer and these objects will help us win at every datatype FreeTDS supports.

OBJECT IDENTIFICATION: Inspired by the PostgreSQL adapter, we now have a plain ruby object which is responsible for identifying servers, databases, objects, and schemas. No more ad-hoc quoting which caused us to blow up when needing to use quoted/unquoted identifier parts at different times. Our new identifier rules should allow legacy systems to easily use dbo. or other schema prefixes in table names and everything from DDL statements to our schema cache & column reflection will just work.

NATIONAL DEFAULTS: Not new, but all Rails simplified types now default to matching SQL Server "national" or Unicode types. If you need to a non-national type or other special type, we have full column extensions in place. For example, need a char(24) vs a full length national varying character, just use the t.char method and proper Rails options in your schema migrations.

Whats Coming

The v4.2.0 adapter is now in pre-release and we only have a few issues to track down and fix. Check out the issues in our Github milestone if you are interested in helping. The biggest feature will be turning on foreign key support!

But why stop there? There is so much more to help with. From benchmark and performance tuning, to Database task support. We sure could use your help. For now, give it a try. Thanks!

gem 'activerecord-sqlserver-adapter', '~> 4.2.0.pre'

ActiveRecord 4.2's Type Casting

Tue, 06 Jan 2015 00:00:00 -0500

Last month Rails 4.2 was released and if you have been keeping up with my posts, I even covered how you can upgrade from 3.2 to 4.2 in one step! This speaks volumes to how easy it is to adopt to outward facing API changes within our beloved framework. But often times, version changes bring implementation changes that we never see. For example, all of Aaron Patterson's work for AdequateRecord Pro™ are performance optimizations that affect no outward API interface at all. Unless you read the source, many of these awesome changes go unnoticed and that's a damn shame because some of them can make our lives easier.

Today I want to share some of the new hotness I found while working on the latest ActiveRecord SQL Server Adapter. Specifically, how ActiveRecord type casts values. Up until Rails 4.2, all type casting was done in class methods like value_to_date implemented on the ActiveRecord::ConnectionAdapters::Column object. Sean Griffin does a great job explaining this convoluted process. Warning, it's kind of boring and a chore to read.

This process has been around for as long as I can remember. It made it really hard to write good abstract OO code that casts values going into and out of the database. All that has changed with the new ActiveRecord::Type namespace. All objects within this namespace are simple POROs with very obvious and well documented interfaces. The base class is ActiveRecord::Type::Value and below is a slightly trimmed down version of that object, sans comments. Take a quick read.

module ActiveRecord
  module Type
    class Value

      def type_cast_from_database(value)
        type_cast(value)
      end

      def type_cast_from_user(value)
        type_cast(value)
      end

      def type_cast_for_database(value)
        value
      end

      def type_cast_for_schema(value)
        value.inspect
      end

      def changed?(old_value, new_value, _new_value_before_type_cast)
        old_value != new_value
      end

      def changed_in_place?(*)
        false
      end


      private

      def type_cast(value)
        cast_value(value) unless value.nil?
      end

      def cast_value(value)
        value
      end

    end
  end
end

Do you see what I see? This is amazing. I see an object that finally handles all of the following.

Casting raw DB values.
Casting user input to prepare for DB quoting.
Casting for default values in schema dumpers.
Avoid ActiveRecord::ConnectionAdapters::Column code bloat.
So much more!

Case in point, a lot of database connection gems still return raw strings for every value. Sub classes of Value can define their own type_cast_from_database implementation to deal with this. For example, here is the Integer object's default behavior. Super easy!

def type_cast_from_database(value)
  return if value.nil?
  value.to_i
end

One thing that Rails core team did to make this even better allows us to type check our Ruby values ahead of time during attribute assignment vs. when we save to the database. This is now done in the Integer class using the limit attribute parsed from the SQL type. Here are the salient points of that class.

module ActiveRecord
  module Type
    class Integer < Value

      def initialize(*)
        super
        @range = min_value...max_value
      end

      private

      def cast_value(value)
        case value
        when true then 1
        when false then 0
        else
          result = value.to_i rescue nil
          ensure_in_range(result) if result
          result
        end
      end

      def ensure_in_range(value)
        unless range.cover?(value)
          raise RangeError, "#{value} is out of range for #{self.class} with limit #{limit || 4}"
        end
      end

      def max_value
        limit = self.limit || 4
        1 << (limit * 8 - 1) # 8 bits per byte with one bit for sign
      end

      def min_value
        -max_value
      end
    end
  end
end

Any type aliased to use the Integer value object will now type check that the value is within the accepted database range. As far as I can tell, only Integer objects in Rails core do this, but I plan on implementing these checks for Decimal and other values too. Here is how SQL Server's smallint(2) SQL type attribute behaves.

@obj.small_int_value = -32_768
@obj.small_int_value = -32_769  # => RangeError!

@obj.small_int_value = 32_767
@obj.small_int_value = 32_768   # => RangeError!

There is so much more that we can do with these objects. The PostgreSQL adapter already casts the JSON data type. I can even see SQL Server returning a Nokogiri object for an XML data type. The sky is the limit. The core Value object allows the SQL Server Adapter to implement guards for different connection modes. Our TinyTDS connection returns all DB values mapped to their proper Ruby primitive. To avoid wasting precious time, we bypass all Rails type casting in one single place now.

These objects are a great step forward and they should open up all sorts of possibilities for gems to extend our DB objects. Thanks so much to Sean Griffin and anyone else working on ActiveRecord to make it better, faster, and easier to use!

Resources

How Rails' Type Casting Works

Custom Oak Display Stands

Tue, 16 Dec 2014 00:00:00 -0500

A few months ago I decided to address my ongoing neck pain by raising the height for all 3 of my displays. I knew that good ergonomics dictated the top of a display needed to be around eye level. However, my eyes still spent most of their time looking downward. Frankly, I blame bottom oriented chat interfaces as well as my tendency to scroll and focus on code/content at the bottom of files.

My solution back then was to grab any random box or book and move my displays' center to around eye level. Four months later, my MVP display change worked, no more neck pain! But I did have a new eye sore, those janky risers. I take great pride in my home office which includes solid pine paneling and oak mission furniture. In short, it was time for a wood working project. Here is a peak at the results, read on for the details.

The Rockwell BladeRunner

At CutomInk & Booster, we know all too well that the Blade Runner font makes a winning custom t-shirt. For that reason alone, I have been on the hunt for a Rockwell BladeRunner table saw and this project seemed like a perfect excuse. The BladeRunner would allow me to easily cut oak planks to perfectly match the shape of each of my displays' base.

I first made a template on paper, pictured below. This allowed me to get some crude dimensions on the size and shape of the lumber I would need. I highly recommend that you trace your final templates on cardboard. The studier template will make it easier to transfer an outline to your wood pieces. Also, note how every good woodworking project needs beer, ham, and cheese!

From here I marked and cut each plank on my new BladeRunner. A few things to watch out for. First, oak is obviously hard! Make sure to get a few spare blades. Second, it really helps if you can secure the BladeRunner to your work table. I did this using the same clamps that I would use later to hold my laminated pieces together. Securing the BladeRunner will allow you to use a bit of turning force when cutting rounded edges.

Putting Things On Top Of Other Things

In the fine tradition of placing things on top of other things, my custom stands will rely on laminating a stack of cut planks. My HP display needed to be 6" tall which required 8 total 3/4" pieces. To make the final stand appear more sculpted, each plank after the 4th downward was cut 1/8" wider than the previous. My Thunderbolt display stand followed the same technique but only required two pieces.

Depending on your final shape and dimensions, you will need some good clamps to hold everything together. Make sure to apply your glue with a craft brush so that the entire glue area is covered.

Ad Hoc Sanding Station

Given that I had to taper sand a self made oak "stump", I knew my work was cut out for me. Ideally this type of sanding required some sort of fixed vertical belt sander. My solution was to mount my Porter Cable upside down in my work bench's vice grip. I even built a dust hood out of an Amazon Prime box (we got tons of those) hooked up to the Shop-Vac.

This setup worked really well. I placed a shop light in the box too so that I could watch the piece while I sanded it. This helped me keep from over sanding which would have been easy to do given that I was using a 30-grit ceramic belt.

Finishing Touches

Here are all 3 pieces after all the sanding was done. I went from the 30-grit, 60, 120, and then 300. I did everything past the 30-grit using a simple hand block. Using a belt sander for finner grits easily leads to burn marks. Lastly, I made sure to round off the hard corners on the edges of the wood too.

For the finish, I chose a simple traditional cherry stain since this both matches my office's pine paneling and mission desk. The picture below shows one coat of satin polyurethane.

After allowing the stain and polyurethane to dry for a day, I smoothed off any rough edges with a soft steel wool. The last step was to add a handful of small felt guards to the bottom of each stand. This would allow me to easily adjust all my displays without scratching my desk.

The Results

Back to a clean and clutter free work environment with no clunky boxes or books in the way :)

From Rails 3.2 to 4.2

Tue, 16 Sep 2014 00:00:00 -0400

Last week I set out to upgrade HomeMarks, a personal bookmarking project of mine. This application sat on a very recent upgrade to Rails 3.2. It is written as an API to both an iOS and HTML JavaScript interface. It is by no means huge and should represent a nominal service oriented application. Here are some stats:

8 Models (450 LOC)
11 Controllers (550 LOC)
2 Mailers (50 LOC)
8 Libraries (500 LOC)
Using Ruby 2.1.2.

The application is heavily tested with an emphasis on controller and integrations. It uses a standard Rails test setup with simple minitest-spec-rails usage. Integrations are done using the Capybara DSL with the Poltergeist driver. JavaScript tests use Konacha which leverage MochaJS and the Chai assertion library.

With the introductions out of the way let's do some fun upgrade work! Here is a step by step process of how I tackled the task and I hope you find it useful.

Grease The Wheels

The first step I always recommend for any Rails upgrade is to update your entire bundle with your current Rails semver as the only gem constraint. The goal here is to have no gem versions other than ~> 3.2 for Rails itself in your Gemfile - hence allowing Bundler to do all the work during your big 4.2 update. For this reason you should delete any explicit gem version specs and run bundle update.

During this time you will want to remove any non-essential gems as well. Great examples are test related gems. Spending time debugging any non-essential gem during an upgrade process is the worst time spent of all. Likewise, identify gems that are obsolete in your target upgrade. In my application, good examples were the mail_view and the quiet_assets gems. Hopefully your application has a tagging convention for notes when upgrading. At CustomInk, we use PENDING: [Rails4]... style comment tags.

Once done, update your master branch with this work and get that deployed. This is your new base for the push to Rails 4.2.

Make A Template

We want to reference a fresh 4.2 application as a guide. This will come in handy in many ways later on. During the time of this article Rails 4.2 was in beta1, so all examples will use that version. Please adjust your commands/examples as newer versions are released.

$ gem install rails 4.2.0.beta1
$ rbenv rehash
$ rails new myapp

The Big Bundle Update

We need to focus on the Gemfile first. Start by changing that ~> 3.2 twiddle-wakka to our target version of ~> 4.2.0.beta1 as our Rails gem version. Go through your notes and delete those obsolete gems too and make any adjustments to any pessimistic version if you noted any.

In Rails 4.2, there is no such thing as an assets group for gems. So I removed my group :assets and flattened my asset gems to the root of the Gemfile. I do suggest maintaining an comment and clustering your asset gems in your Gemfile as a way to keep them organized. Due to sass-rails being in beta, I ended up with something like this.

gem 'bcrypt-ruby'
gem 'pg'
gem 'rails', '~> 4.2.0.beta1' # PENDING: [Rails 4.2] Remove wakka.

# Assets
gem 'bourbon'
gem 'coffee-rails'
gem 'jquery-rails'
gem 'sass-rails', '~> 5.0.0.beta1' # PENDING: [Rails 4.2] Remove wakka.
gem 'uglifier'

Lastly, I find it extremely useful to delete the Gemfile.lock now. In my experience Bundler will have a much better time coping by allowing the entire dep graph to be rebuilt. Now go for the big update with a simple bundle install.

$ bundle install

If you run into any problems here, take them on one by one. In some cases you could be using a gem that does not optimistically include 4.x versions. Before proceeding, find each project's issue tracker and do a little research. If you encounter resistance, look broadly first and avoid hacking around it. This could be your time to shine and help with an open source pull request.

Once you are bundled, you are ready for the next step. Avoid the temptation to launch your application or running tests! You are no where near ready for that, so slow your roll. We still have some good work to do.

The Mimic Process

This is a great time to open the template application we created above. It is a very straightforward process to go through each of the application's files & folders and mimic the template within your own application.

In my experience I have seen a lot of pain in converting over config/environments files to the newer format. If you have never done so, I highly recommend following a simple practice that makes upgrading of these files easier in the future. Always keep the environment file largely untouched and add your configs to the bottom of each below a comment like this.

Rails.application.configure do

  # ... Rails generated ...

  # My Configs
  # ----------
  config.action_mailer.show_previews

end

This makes updating these files in the future go much quicker. Here are some general notes I had when doing my entire mimesis process.

Environments & Initializers

Removed config.filter_parameters from application.rb to config/initializers/filter_parameter_logging.rb
The config.assets.precompile and config.assets.version have moved to config/initializers/assets.rb
Many files now use the Rails.application singleton resource vs MyApp::Application constant.

Concerns Directories

I fully embraced the new concern directories. I created both app/controllers/concerns/.keep and app/models/concerns/.keep. I found there were a few files in my lib directory that were actually concerns. For example, I moved both my AuthenticatedSystem and RenderInvalidRecord modules to the controller concerns. This allowed me to remove any hacks I had for setting the auto load path on the lib directory too.

Bin Directory & Spring

I have been using both bundler bin stubs and Spring in all of my 3.2 applications. Now that Rails supports both the local bin directory and Spring in an integrated way, I wanted to follow the golden path of least resistance.

The first step was to blow away my entire bin directory and just copy over the one from the application template. These new bin files for rake and rails leverage the Spring preloader. Make sure to delete your local or global bundle config for installing bins too. Do this with bundle config --delete bin. Now, instead of having Bundler install bin stubs for every gem which could conflict with the preloader bins, we should be explicit on a per gem basis. For example, this would install the bins for guard.

$ bundle binstubs guard

Lastly, if you have never done so, change your shell's PATH to look for the local ./bin directory before any others. I do this after my rbenv initialization.

eval "$(rbenv init -)"
export PATH="./bin:$PATH"

Routes File

Removed my match methods in favor of get verb method.

New Test Directories

Renamed test/functional folder to test/controllers
Renamed test/unit folder to test/models
Created a test/mailers folder and moved all my mailer tests from units to it.
Renamed ActionController::IntegrationTest to ActionDispatch::IntegrationTest.

Framework Changes

Now is the time where you can start to run your tests and identify what needs to change. I recommend starting with the model tests and moving on from there. Though I am a big fan of Guard for automatic test runs, Rails now has a new option when using rake. Just pass the filename after the test argument. For example, this would run a single model test and since the new bin/rake file uses Spring, you can run this command over and over again very quickly.

$ rake test test/models/user_test.rb

Below are things I found while moving through my tests. I have organized them by framework. They are by no means comprehensive and your application may expose more differences between Rails 3.2 and 4.2.

ActiveRecord

You are going to see a lot of ArgumentError: Unknown key: ... errors. The reason is that Rails 4.0 now requires that scopes use a callable object such as a Proc or lambda. I saw these errors mostly on :order and the :readonly option arguments. Here are a few before/after examples.

# Old
has_many :foos, order: 'position'
has_many :bars, through: :foos, readonly: false, order: 'foos.position, bars.position'

# New
has_many :foos, -> { order('position') }
has_many :bars, -> { order('foos.position, bars.position').readonly(false) }, through: :foos

Seems the callable object has to be the second argument and you can easily chain the scopes within. I took a wild guess that the readyonly scope took an optional false argument and was handsomely rewarded.

I also got a few good ArgumentError: The provided regular expression is using multiline anchors (^ or $) ... errors. This was easy to fix by using the suggested \A and \z instead. Really happy to see the framework warning on this common errors when using regular expressions for validations.

I deleted all the attr_accesible declarations from my models and switched to strong parameters in the controllers. If you are new to strong parameters, check out this article which goes into great depth on the topic. Alternatively, you can start using strong parameters before you upgrade to Rails 4.x by using the backward compatible strong parameters gem. A great strategy to ease large application transitions. Thanks to Chris Mar for pointing out this approach to our team.

The class update_all no longer takes a second conditions argument. I always disliked methods that took two option hashes and this is a great change. For example:

# Old
Book.update_all({author: 'David'}, {title: 'Rails'})

# New
Book.where(title: 'Rails').update_all(author: 'David')

Lastly, the all method no longer takes finder options.

# Old
User.all(conditions:{email:'ken@metaskills.net'}).map(&:id)

# New
User.where(email:'ken@metaskills.net').all.map(&:id)

ActionMailer

Using _path url methods in mailers will now result in a DEPRECATION WARNING... cannot be used here as a full URL is required message. Unless you were manually augmenting these paths to have a host, this is a good thing and will keep developers from including partial URLs in mailers.

MailView is now fully integrated into ActionMailer. Read this article for full details. If you have never used MailView in a Rails 3 application, it allows you to develop your mails in the browser as if they were controller view. When moving from old MailView support to Rails 4.x usage, follow these steps:

Add config.action_mailer.show_previews to config/environments/development.rb file.
Moved previous mailer previews to new test/mailers/previews directory.

ActionPack

Partials can no longer have - hyphens in the filename. I had to change a few.

There is a new and better request forgery protection in Rails 4.x. Read the source link for full error messages and documentation. In my case, I had already redefined a protected protect_against_forgery? for a special controller and needed to only add skip_before_action :verify_authenticity_token to my filters to fully work around the security warning.

Asset Pipeline

The biggest issue that I had is that asset compilation no longer generates non-digest filenames for each asset. This github issue explains the rationale, but I do believe there are some corner cases where you do want to reference the non-digest asset filename.

If after careful examination you find yourself in need of this corner case, do not disable digests for all files. Instead install the non-stupid-digest-assets gem which allows you to whitelist specific asset files and thereby including a non-digest filename along with the fingerprinted filenames for said asset(s). I recommend putting the NonStupidDigestAssets.whitelist settings at the bottom of the new config/initializers/assets.rb file.

Testing

The Rails testing task strategy has changed a lot. By default now, when you run rake test all models, mailers, controllers and integrations are run in one collective suite run. If you are interested in learning how, read the testing.rake source. I also talked about how to add different directories to this process within a github issues under the minitest-spec-rails project where Mike Moore contributed a few helpful hints too.

The reason I mention this is that it is somewhat common to expect your Capybara enhanced integrations to run in a distinct process. Because of this, it is also common to see monkey patches to the ActiveRecord connection pool to support integration tests that leverage DB transactions. Depending on your setup, you may be required to make a few tweaks.

In Closing

Upgrading Rails applications used to be a pain! My recent upgrade only took two evenings of my spare time. In my opinion Rails 3.1 and up have become much more stable for both application and gem authors to leverage the framework. Thus making upgrades approachable. Keep your applications small and focused as a way to win the upgrade wars!

In closing, thanks for reading and I hope you found this information helpful. If you have any questions, feel free to ask in the comments. Cheers!

Dynamic SVGs Using <defs> Elements & JavaScript

Fri, 29 Aug 2014 00:00:00 -0400

The scalable vector graphics format has a really nifty way to define and reuse objects. It does this by allowing objects or paths to be defined in the <defs> element and then used one or many times with the <use> element. It is a great way to keep your SVG's file size low. Even better, it makes for a great programmatic interface to dynamically compose an image.

While designing the third iteration of my personal MetaSkills.net blog, I decided to carry the retro space theme into the background of the fixed left navigation area. The design called for something like this image below. Besides the planet, it has three distinct types of stars that are scattered all over the place. Each has a different scale and opacity with plans to animate a few.

So where to start? We could export a final pre-designed SVG image from any type of vector program – but that would bloat the file size since it would copy and paste each star's path. It would also make animating distinct stars tricky and require post production edits. Lastly, who has time to randomly scatter stars? Not me! So let's program up a solution that solves all of this for us. Here is the base SVG structure that we are going to work with.

<svg
  version="1.1"
  width="300px" height="700px"
  viewBox="0 0 300 700"
  preserveAspectRatio="xMidYMax slice"
  xmlns="http://www.w3.org/2000/svg"
  xmlns:xlink="http://www.w3.org/1999/xlink">
  <defs>
    <!-- Define star objects here. -->
  </defs>
  <style type="text/css">
    .star    { fill: #95a8b1; }
    .burst   { fill: #e4522a; }
    .steeler { fill: #fffbe1; }
  </style>
  <rect x="0" y="0" width="100%" height="100%" fill="#052838" />
  <g id="starfield">
    <!-- Use stars objects here. -->
  </g>
  <script type="text/ecmascript">
    <!-- Dynamically create stars here. -->
  </script>
</svg>

Our image will have a slender vertical portrait area to match the general proportions of the sidebar. The SVG is setup to preserve the aspect ratio in such a way that allows the bottom planet to always be in view when scaled. We have a <defs> area where we can add reusable objects, an in-line style sheet for our awesome CSS, and a <g> starfield element to hold each star. Lastly, a place to write some JavaScript.

Base Star Objects

Adding these to our SVG is really easy. I started by creating a canvas in my favorite vector program, Sketch. We want to make sure our exported paths start at 0,0 position and are are generally the size you expect to use them. This will help you later when translating these paths to different positions.

I exported each star type as an SVG and then copied the path into the <defs> area of our template above. When you do this make sure that each path has its own unique id. Also, you may want to optimize your paths. Many programs export 6 or more decimal points for a path's data. I recommend running them through some sort of SVG optimiser or tweaking your export settings if your art program has them. Now we have our reusable path objects, see below. Examples have truncated d attributes for brevity.

<defs>
  <path id="star" d="M2.395,0.409 C2.026,0.973 ... Z"></path>
  <path id="burst" d="M6.092,4.816 L1.661,3.4 ... Z"></path>
  <path id="steeler" d="M3.231,6.211 L0,5.830 ... Z"></path>
</defs>

Using Defined Objects

If we were to view our SVG now, it would be pretty boring and contain only the colored background rect. So lets add some stars by using the defined objects above. We do this with the <use> element by setting each xlink:href to the id of that object we want to target. To move and size each star, we use the SVG transform attribute to both translate and scale each star. Adding these elements right before the JavaScript tag will create the following image.

<use xlink:href="#star" transform="translate(10,10) scale(1.0)" class="star" />
<use xlink:href="#burst" transform="translate(30,20) scale(1.0)" class="burst" />
<use xlink:href="#steeler" transform="translate(60,10) scale(1.5)" class="steeler" />

Still kind of boring... and who has time to stochastically place several dozen stars around the canvas? Time for some JavaScript automation.

SVGs & JavaScript

Yes! SVGs are distinct documents that can have their own CSS and JavaScript. But in order for this JavaScript to execute, we must embed it into the parent HTML document. Linking to an SVG via an <img> tag will not allow this due to security concerns. We can get around that issue by using an <object> tag instead.

<object data="our-awesome.svg" type="image/svg+xml"></object>

Now that our SVG's JavaScript will execute, how do we go about adding <use> elements to our SVG's document? Fortunately the techniques are very similar to how you would do this with vanilla JavaScript and the HTML DOM. The minor difference is that we have to use two namespaces. One for creating elements and the second for adding the xlink attribute. I recommend we start our JavaScript off with these namespaces as vars. We also want to create variables for our canvas width and height. Finally, we want a handle to our starfield group element.

var xmlns = "http://www.w3.org/2000/svg",
    xlinkns = "http://www.w3.org/1999/xlink";
var width  = 300,
    height = 700;
var starfield = document.getElementById("starfield");

Random JavaScript Helpers

Before we just start adding some stars, we need some random functions to help us position and scale our stars. I like to start low level and build up. So here is our random function that takes a min and max value.

function randomBase(min, max) {
  return Math.random() * (max - min) + min;
}

> randomBase(0,300)   // 229.5510318595916
> randomBase(-50,300) // -22.134714818093926

Building on that, we want to made a random position function to help place each star in our field. This helper only needs the max argument which will be either our width or height. Notice too how it blankets our canvas area by setting the minimal and maximum to -50 and +50. This will ensure each star could be placed slightly off canvas and keeping them from appearing clustered toward the middle. We also floor the return value, no decimals are needed for position.

function randomPos(max) {
  return Math.floor(randomBase(-50, max+50));
}

Here is our random scale function which takes a min and max threshold. This will allow us to adjust the weight of each star type in the cluster and possibly adjust the exported size differences from our image program into the <defs> area. This function returns a two decimal float value.

function randomScale(min, max) {
  return parseFloat(randomBase(min, max).toFixed(2));
}

> randomScale(0.5, 0.9) // 0.6
> randomScale(0.5, 0.9) // 0.85

Finally, here is a helper function that builds on top of randomScale. It takes a star name and returns the proper value.

function randomScaleNamed(name) {
  var scale;
  switch(name) {
  case 'star': scale = randomScale(0.3, 0.8); break;
  case 'burst': scale = randomScale(0.4, 0.9); break;
  case 'steeler': scale = randomScale(1.1, 2.0); break;
  default: scale = 1.0; }
  return scale;
}

Adding Use Elements

Now that we have the boring random functions out of the way, time to create our star-agnostic addElement function. This will create a new <g> element for each star's <use> element. The group is for placement and size, the use is for twinkle and shine animations.

function addElement(name) {
  var g   = document.createElementNS(xmlns, "g"),
      use = document.createElementNS(xmlns, "use"),
      t   = 'translate(' + randomPos(width) + ',' + randomPos(height) + ') ' +
            'scale(' + randomScaleNamed(name) + ')';
  use.setAttributeNS(null, "class", name);
  use.setAttributeNS(xlinkns, "xlink:href", "#" + name);
  g.setAttributeNS(null, "transform", t);
  g.appendChild(use);
  starfield.appendChild(g);
}

This function ends by placing the newly created element within the starfield. For example, calling addElement('burst') once would create this.

<g id="starfield">
 <g transform="translate(58,151) scale(0.9)">
   <use xlink:href="#burst" class="burst" />
 </g>
</g>

To add a bunch of stars!

function addElements(name, count) {
  while (count -= 1) { addElement(name); }
}

addElements('steeler', 15);
addElements('burst', 15);
addElements('star', 50);

Going Further

Thanks for reading this far! If you are interested in how I handled star animations using Bourbon & Sass, check out this gist on GitHub for my source code. It also shows how I animated the planet to spin at the bottom of the image and cover the background stars.

I hope you have as much fun as I have with animating SVGs using JavaScript and Sass!

Resources

Jekyll-Style Blogging On Rails

Sat, 19 Apr 2014 00:00:00 -0400

Do you want to use your existing Rails' layouts & business logic for your new Blog? Do you know and love Jekyll, but cant seem to get the two to play nice together?

In the spirit of doing the simplest thing that could possibly work, I set out on a solution to this problem. Besides mimicing Jekyll as closely as possible, my requirements were loosly defined as:

Use a directory of markdown files with a YYYY-MM-DD-blog-post-title.md naming convention.
Have fast syntax highlighting via fenced code blocks.
Expose model class finders for a single resource & collections.
Some model object to ask posts for their title, date, html, etc.
Make local development follow Rails conventions. Like page refreshes.
Provide a reasonable amount of caching in production.

Additionally if I had time or the gumption:

Use YAML front matter for different post titles & other options.
Preprocess markdown with ERB and allow Rails routes & helpers to be used.
Ask each post for an excerpt so we can render teasers elsewhere.
Create a rake task that tees up a new post using the current date.

Getting Started

Most times I like to start coding to high-level interfaces. It helps me think about the smaller details I will need to build later. A Rails controller is a good option but be careful and keep it simple! If you are the type of person that likes to jump to the end, check out the final gist of our completed work.

First, we are going to make our blog routes and controller. We want two actions, an index to list all blog posts and a show to render each post.

# In config/routes.rb
resources :blog, only: [:index, :show]

# In app/controllers/blog_controller.rb
class BlogController < ApplicationController

  layout 'site'
  before_filter :find_post, only: [:show]

  def index
    @posts = BlogPost.all
  end

  def show
    render text: @post.html, layout: 'site' if stale? @post, public: true
  end

  private

  def find_post
    @post = BlogPost.find params[:id]
    redirect_to blog_index_path unless @post
  end

end

Pretty simple looking, but this code exposes some neat details that we will have to build while accomplishing a lot too. Our index action shows we will need a class finder for all posts and assumes they will be ordered somehow. The show action which is behind our find_post filter tells us we need some type of key for our posts – likewise we have determined that we need an html instance method. Lastly, we are going to check when a post is stale somehow while setting the public Cache-Control header for proxies.

Our BlogPost Model

Our :id key will use the file name as our slug. Our markdown files will be located where our controller would normally find them, in the app/views/blog directory. Feel free to create your first post there, name it 2014-04-19-my-first-post.md using the current date. Now here is our first itteration of our new BlogPost model that allows us to find our posts.

class BlogPost

  attr_reader :slug

  class << self

    def all
      all_slugs.map{ |slug| new(slug) }.sort
    end

    def find(slug)
      all.detect { |post| post.slug == slug }
    end

    def directory
      Rails.root.join 'app', 'views', 'blog'
    end

    private

    def all_slugs
      @all_slugs ||= Dir.glob("#{directory}/*.md").map { |f| File.basename(f).sub(/\.md$/,'') }
    end

  end

  def initialize(slug)
    @slug = slug
  end

end

Notice how we memoize the private all_slugs class method to avoid hiting the filesystem again and again in production? This idea is definitly not complete and if needed, we can update it later. So, with the class methods out of the way, we can flesh out the model more. But first, we need to think about Jekyll.

Time For Jekyll

I've used Jekyll for a handful of static sites and blogs. Most of which have been running the next major version which is not quite out yet. Jekyll v2 is amazing and we totally need it for this project. Below are the gem deps you will need to bundle up. Remember, when Jekyll v2 is released, you can change your Gemfile as needed.

# In Gemfile
gem 'jekyll', '~> 2.0.0.alpha'
gem 'redcarpet'
gem 'rouge'

So, most of us are familiar with the Redcarpet gem. That will be the markdown converter we use, but what about rouge? The Rouge gem is a pure-ruby syntax highlighter. It can highlight over 60 languages, and output HTML or ANSI 256-color text. Its HTML output is compatible with existing stylesheets designed for pygments.

class BlogPost

  # ...

  private

  def to_html
    Jekyll::Converters::Markdown::RedcarpetParser.new({
      'highlighter' => 'rouge',
      'redcarpet' => {
        'extensions' => [
          "no_intra_emphasis", "fenced_code_blocks", "autolink",
          "strikethrough", "lax_spacing",  "superscript", "with_toc_data"
        ]
      }
    }).convert(markdown)
  end

end

It is not just sexy, Rouge is fast too! And by using it, we avoid bundling to pygments which has Python dependencies. We are going to let Jekyll do the heavy lifting and implement our private to_html method like so. This will give us Github flavored markdown using Rouge for our fenced code blocks.

Other BlogPost Details

In our BlogPost.all class method, we called sort on the collection with the expectation that the newest post comes first. To fulfill this we are going to use Ruby's <=> spaceship operator by comparing each post's date. This completes our controller's index requirement.

class BlogPost
  # ...
  def <=> other
    other.date <=> date
  end
  # ...
end

We want each post to have a title and date method. We can easily parse these form the slug. I even made a date_formatted method using ActiveSupport's Inflector that will be useful for our views. It returns date strings like April 19th, 2014. We also have a path method that we can use when linking to our posts from elsewhere in our site. It will use the slug as the :id param. This completes our controller's find_post requirement and gives us pretty URLs just like Jekyll.

class BlogPost
  # ...
  def title
    slug.sub(/\d{4}-\d{2}-\d{2}-/, '').titleize
  end

  def date
    Date.parse(slug)
  end

  def date_formatted
    day_format = ActiveSupport::Inflector.ordinalize(date.day)
    date.strftime "%B #{day_format}, %G"
  end

  def path
    "/blog/#{slug}"
  end
  # ...
end

Before implementing our markdown to HTML conversion, we should think about caching. Following Rails' conventions we implment a cache_key method that combines the blog namespace with the posts unique slug and timestamp. The updated_at timestamp is simply the last time the file was modified on disk. Convention is to convert this to an integer which is seconds since Unix Epoch. The cache_key will be used by the controller's show method when determining the ETag for the response.

class BlogPost
  # ...
  def updated_at
    File.mtime(file_path)
  end

  def cache_key
    ActiveSupport::Cache.expand_cache_key ['blog', slug, updated_at.to_i]
  end
  # ...
end

Now the fun part, converting our markdown to HTML. We are going to use the Rails.cache which in local development is a simple memory store and Memcached in production. The fetch method takes a key and a block. The block is only executed if there is a cache miss.

class BlogPost
  # ...
  def html
    Rails.cache.fetch("#{cache_key}/html") { to_html }
  end


  private

  def file_path
    self.class.directory.join "#{slug}.md"
  end

  def markdown
    Rails.cache.fetch("#{cache_key}/markdown") { File.read(file_path) }
  end
  # ...
end

Code Highlighting

The only thing left to do is style the highlighted output from the Rouge gem. Thankfully, there are many existing pygment theme options spread accross the internet. Here are a few to get you started.

pygments-css - CSS files created from pygment's built-in styles.
GitHub-Dark - GitHub Dark theme for Stylish. Browse the themes directory.
pygments-github-style - GitHub style for Pygments.

For the HomeMarks blog, I used the github jekyll-github.css from the last resource above. Have fun choosing your own and @import it into your existing CSS bundle in the manner that matches your own project.

Taking It Further?

So that's it, we can ship it! But what about those other great requirements like YAML front matter, ERB templates, Rails' routes/helpers, and more? The good news is that I found the time to do them all. You can find each implemented in the final Jekyll-Style Blogging On Rails gist.

def scope
  ApplicationController.helpers.clone.tap do |h|
    h.singleton_class.send :include, Rails.application.routes.url_helpers
  end
end

One aspect that I was particularly proud of was ERB pre-processing with a evaluation scope to Rails' routes and helpers. The solution turned out to be simple after digging into the railties IRB console helpers and ActionPack. My solution above used this scope with the Tilt's erubis template.

Where To Go From Here?

With this setup, you have a great foundation for Jekyll on Rails. Feel free to customize it as needed. Some ideas include:

Better caching in production. Currently hits the filesystem for cache_key.
Maybe make a jekyll-rails engine that does all of this for you?
Add an Emoji HTML filter to your posts.

Resources

Thanks for reading! Would love to hear your feedback too.

Thanks To My Mother For The Career I Have Today!

Sat, 15 Mar 2014 00:00:00 -0400

As a white male in the computer industry, I have never felt like I could add value to the conversations that call for diversity and respect to all those that share the same passions I do. Instead, since teaching myself to program several years ago, I have worked very hard to openly advocate the love I have for software development through our local @757rb user group and whomever would give me a platform to do so. Never murmuring a comment that came close to gender related.

However, today, I really felt compelled to share something that only recently occurred to me. I owe my entire career path and love of computers to my Mom. In fact, the more I think about it – she is my primary "you can do anything" influence that nurtured both my creative thinking and technical know how.

Rosie the Riveter

So let me introduce you to my Mom and share a bit of her story which seems to strike a chord even today. These photos are her working at the Norfolk Naval Shipyard here in Portsmouth, Virginia. Most everyone knows that during WWII there were many women working in places like the Shipyard doing jobs only men would do. After the war, there were little or no women I am told working in these positions. My Mom was the first female apprentice to join their ranks again on October 3rd 1965 and she was gunning for a position only the guys had.

I talked to my Mom today about women in software and she was kind enough to share a few of her personal stories with me. For example, unlike most girls her age, she took 4 years of mechanical drawing in High School. And being the only girl in the class, the teacher used to send her to the office on errands so he could chum up and tell dirty jokes to the boys. She also told me that even after successfully completing the apprenticeship program at the Shipyard and being promoted to the design division, that guys would ask her out just to test if she was a lesbian or not.

To be honest, it never occurred to me that these and many other things debased her experiences. She told me the hardest thing about being a woman in a male dominated career was that most interactions were polarized. In one group, her male counterparts would be overly helpful. In the other, overly negative or outright trying to sabotage her work. Nothing ever felt "normal". And I guess that is my biggest take away and source of empathy.

My Nerd Mentor

So despite her troubles, my Mom has and always will be successful. She exposed me and my twin Brother to as many different things as possible. She taught us how to draw, how to build things, and how to believe in ourselves. From skiing to firewalking and everything in between, she wanted us to learn and do. But most of all, her love of computers affected me the most.

To this day I joke that she loved her computer way too much. I have fond memories of the 1986 Mac Plus getting a front seat in the Dodge Caravan (with a seat belt) while I and my Brother had to ride in the back. Before then, she used to turn the house into a social software copy party and we spent family time notching disks before the guests would arrive.

Thanks Mom!

In closing, many thanks to my Mom for allowing me to share her story with you all. I am forever grateful to her teaching me that I can do anything I want and exposing me to computers and many other things. Maybe this is not uncommon? Maybe many more male software developers owe many things to a strong matriarchal influence? That would be really neat and I'd love to hear your story about it. Thanks for reading mine.

Instrumenting Your Code With ActiveSupport Notifications

Sun, 15 Dec 2013 00:00:00 -0500

Have you ever wondered how tools like New Relic are able to gain valuable metrics to your Rails application's internals? Or maybe you are interested in learning how to write your own libraries and gems so they can be instrumented using those same techniques? Once again the answer is to look deep into the Rails source code – and the answer is ActiveSupport::Notifications. A simple and powerful instrumentation API for Ruby available in Rails v3.0 and upward.

Today I want to share a working example of how you might use ActiveSupport::Notifications. Instead of presenting some contrived code, I thought it would be fun to freedom-patch a popular gem so that we can garner supernatural metrics that would have otherwise been lost or hidden deep in our log. For this exercise I have chosen the Subexec gem. Subexec is a simple library that spawns an external command with an optional timeout parameter. It is used by other gems like MiniMagick – a popular minimal replacement for RMagick.

Building Subexec::Notifications

Our goal will be to build a new gem called subexec-notifications that instruments all commands run by the Subexec library. Once completed, third-party subscribers would be able collect these metrics thereby opening up developer/operational insights into how long certain commands were taking and on which server(s).

Lucky for us, the Subexec gem has one interface, the Subexec#run! method. So our work is going to be straight forward. All we have to do is alias method chain that instance method and wrap it with some instrumentation. Assuming you are up to speed on this practice, here is our new implementation.

def run_with_notifications!
  payload = {sub: self, hostname: Socket.gethostname}
  ActiveSupport::Notifications.instrument "subexec.run", payload do
    run_without_notifications!
  end
end

This small snippet of code exemplifies how simple it is to instrument our code. The Notifications instrument class method takes two arguments, a string for the name and an optional payload hash. The name will be used by subscribers and the payload hash can contain anything you want.

Since the Subexec instance has tons of valuable information like the commands output, process id, exit status, and the command string itself - I decided to include it with our payload. The host name is provided with the payload to help us aggregate or subdivide our metrics for each server.

Believe it or not, that pretty much wraps up all that is needed for our new gem's code. Everything else like tests and gem structure are orthogonal to our learning today. But please, browse the entire subexec-notifications gem if you are interested in how it is put together.

Choosing A Metrics Service

So now we have a way to instrument all of our system commands, but how do we collect and view that data? To be honest, your options are incredibly numerous at this point. While learning ActiveSupport::Notifications myself, these two services kept appearing.

My examples below will use Librato since I found their service extremely simple to use. I was able to quickly get metrics submitted to them and viewable via their dashboard gauges. Librato also has a very nice presence on Github and some impressive tools for Ruby. Datadog is no slouch in any of these areas either. So please use what best fits your own needs.

IMPORTANT: The example Rails application code that follows makes direct use of the librato-metrics gem. This means that submissions will happen synchronously while your application is running. You would never do this in your Rails application! If you choose to use Librato, please use the librato-rails gem instead. Metrics are then delivered asynchronously behind the scenes so they won't affect the performance of your requests. Other possibilities would be to use background jobs or some other worker message queue.

Subscribing To Events

Assuming we have a Rails application that makes use of MiniMagick, Subexec or both, all we have to do now is bundle up our new notification gem along with librato-metrics.

# In Gemfile

gem 'subexec-notifications'
gem 'librato-metrics'

Now we need to subscribe to the subexec.run events that we instrumented in the subexec-notifications gem. For a Rails application, this is best done in an initializer named after the gem.

# In config/initializers/subexec_notifications.rb

ActiveSupport::Notifications.subscribe 'subexec.run' do |*args|
  Subscribers::SubexecLibrato.new(*args)
end

Publishing Metrics

As you can see in the code above, subscribing to an event will yield an array of arguments. Technically, these will be the name of the event, a few timestamps, a unique id, and the payload. Because dealing with individual arguments is not very object-oriented, I always recommend creating an event object using the ActiveSupport::Notifications::Event class. It consumes these arguments and gives you a clean interface to the duration of the event, payload, and more.

To accomplish this in one place for our publishing code, I created a simple base class for all our subscribers to inherit from. This base class creates our event object as well as a process method that subclasses must implement.

# In app/models/subscribers/base.rb

module Subscribers
  class Base

    attr_reader :event

    def initialize(*args)
      @event = ActiveSupport::Notifications::Event.new(*args)
      process
    end

    def process
      raise NotImplementedError
    end

  end
end

Now to the fun part, sending some metrics to Librato. Below is the full implementation of our SubexecLibrato event consumer. This creates two different types of metrics. One for each command/binary that was run and the other for the host the commands are run on. Each of these metrics will allow us to build some interesting gauges. The Librato site has a great developer section titled What Are Metrics that can guide you on what type of data you may want to submit.

# In app/models/subscribers/subexec_librato.rb

module Subscribers
  class SubexecLibrato < Base

    def process
      sub   = event.payload[:sub]
      dur   = event.duration
      type  = sub.command.split.first
      host  = event.payload[:hostname]
      time  = Time.current
      Librato::Metrics.submit 'subexec.hosts' => {measure_time: time, value: dur, source: host}
      Librato::Metrics.submit 'subexec.types' => {measure_time: time, value: dur, source: type}
    end

  end
end

Viewing Metrics

Here are what each of these metrics look like in Librato. To generate some commands, I wrote a small tests case that did some random MiniMagick commands along with a few echo and uptime commands. All of these ran on my local machine.

In Closing

Hopefully these simple examples we built will help get you excited both about instrumenting your application as well as collecting and viewing those metrics. If you are hungry for more, check out the links in the resources below. You can even dig deep into the Rails source to see where and how it uses ActiveSupport::Notifications. Thanks!

Other Resources

Using Dotenv In Rails

Thu, 03 Oct 2013 00:00:00 -0400

Environment variables as a configuration means are everywhere in Ruby. For instance, ActiveRecord will use the single DATABASE_URL environment variable for every part of it's configuration, no database.yml needed! If you are not on board with environment variables, check out The Twelve-Factor App for configuration. This is exactly how good software platforms like Heroku work, all through environment variables.

But using environment variables in a Rails application can be tricky. During local development, you may not want to set everything in your Bash or ZSH profile. Perhaps you want per project settings. And then there is the trouble of using different environment variables when running your tests. Enter the Dotenv project.

In my examples below of how to integrate Dotenv, I will be using @ecbypi's fork until this pull request for overloading environment variables is accepted. This pull request is key to overriding development or local settings for our Rails test environment. If you like how these examples look below, weigh in on that pull request and lobby for a fresh hot Dotenv release.

Let's get started. First, let's add the gem to your Rails project's Gemfile.

# In Gemfile
gem 'dotenv', github: 'ecbypi/dotenv', branch: 'overload-environment-variables'

Next up, open your config/application.rb and add this line right after the require 'rails/all' line. This will require the Dotenv gem, then load your own per-developer local settings (if present), then a .env file that matches the current Rails.env.

# In config/application.rb
require 'rails/all'
require 'dotenv' ; Dotenv.load ".env.local", ".env.#{Rails.env}"

Next, create individual .env.RAILS_ENV files with your configurations. YMMV on this one, but the idea is to check all of these in. The .env.development will hold all the default configurations for a new developer to get up and running quickly. The .env.test will hold all the configurations for your test runs. Past that, how you handle the production file or not, is left up to you. In my examples below, we are setting the environment variable for the Awesome gem.

$ echo "AWESOME_GEM_URL=https://awesome.dev"          > .env.development
$ echo "AWESOME_GEM_URL=https://sandbox.awesome.com"  > .env.test
$ echo "AWESOME_GEM_URL=https://awesome.com"          > .env.production

This covers our bases, but what if a developer wants to override a configuration. They can use good ol' environment variables and get the behavior they expect. Or they create their own .env.local file and set their configurations there. The benefit of the local file is that your Rails application via the console or the development server (for example Pow) will automatically get the same thing when using the .env.local file. Oh yea, it is a good idea to add the this file to your .gitignore.

$ echo "AWESOME_GEM_URL=https://my-awesome.dev"       > .env.local

Now, one last thing. We need to make sure that our tests do not use any real, development or local environment settings. This is where @ecbypi overload feature comes in handy. I have added this to the top of the Rails test/test_helper.rb file.

# In test/test_helper.rb
ENV["RAILS_ENV"] = "test"
+require 'dotenv' ; Dotenv.overload ".env.test"

Happy environment usage! Have you been solving environment and application configuration in a different way (don't say YAML), if so, I would love to hear about it!

Jekyll Tips And Tricks

Mon, 02 Sep 2013 00:00:00 -0400

Last week, our localy Ruby community in Norfolk, VA re-launched the 757rb.org website. This was the third static website I have build using Jekyll and the second since the official v1.0.0 release. It was also completely different than any creative approach I have taken on. Please, go ahead and check it out real quick, I will wait.

Many things have changed with Jekyll over the years and it is always fun to stay current with the latest technical aspects of the tools we use. So I thought I would put a few of my own techniques out there to see what others think. If you find any of these useful or know of better ways to solve them, let me know.

Use Task Scripts

I have no idea where I picked this up, but the general idea is to create a tasks directory that holds a set of useful shell scripts. My Jekyll projects usually contain the following executables.

jekyll
jekyll-livereload
deploy
optipng
post

The jekyll wrapper script is only useful to those projects that need to perform additional steps before or after building the site. Notice the usage of $* after the jekyll build command. This is akin to Ruby's reverse splat args. It allows you to send any arguments to your own script that the jekyll command would normally take.

I will cover jekyll-livereload and deploy later in this post. The optipng script is a useful command line utility that compresses assets. While the post script allows you to easily create a new post. For example:

$ ./tasks/post "Jekyll Tips And Tricks"

Development & Production Environments

Sometimes you do not want a production feature turned on when developing your site locally. Blogs backed by DISQUS comments would be a great example. Likewise, maybe you want to develop a new feature that you can see during local development only to be enabled at a later time. Thanks to v1.0 of Jekyll and up, this is easy to do with different configuration files.

# In _config.yml
production: false
timezone: "America/New_York"

# In _config_production.yml
production: true
timezone: "America/New_York"

Here we have the default _config.yml acting as our default development configuration. It sets the production site variable to false while the _config_production.yml sets it to true. Remember the deploy task I mentioned above? This is a great place to pass down the --config argument to your own jekyll task. Here is a partial example of a deploy task.

#!/usr/bin/env bash
set -e

./tasks/jekyll --config _config_production.yml

Now you can write code like the following examples in your layouts or includes. Jekyll will automatically take any additional top level configurations and turn them into properties on the site object.


<!-- Only show DISQUS comments in production. -->
{% if site.production %}
<script type="text/javascript">
  var disqus_shortname = '...';
  ...
</script>
{% endif %}

<!-- Developing a local only feature not ready for production. -->
{% unless site.production %}
<section class="promotion">
  <header>T-SHIRT FUNDRAISER!</header>
  ...
</section>
{% endunless %}

Use The Asset Pipeline

If you have written Jekyll sites prior to the v1.0 release you have likely concocted up your own scripts to use CoffeeScript, Sass or both. Love it or hate it, as Ruby developers we got it good with Sprockets.

But now Jekyll has the power of the asset pipeline too. Just install the jekyll-assets gem. Here are a few tips to get you up and running faster.

First, remember that Sprockets is built on top of a gem named Tilt which is a generic interface to multiple template engines. Installing tilt does not install template engines, so make sure to install the ones you want to work with by explicitly declaring them in your Gemfile.

gem 'jekyll-assets'
gem 'coffee-script'
gem 'compass'
gem 'sass'
gem 'uglifier'

I found the default configuration for jekyll-assets very confusing. It mimicked a production environment Rails setting. All the files used the digest as part of the file name and compression was on by default. This made it really hard to debug. So to make jekyll-assets more like Rails defaults. Use these configurations for each jekyll environment.

# In _config.yml
assets:
  js_compressor:
  css_compressor:
  cache:
  cachebust: none

# In _config_production.yml
assets:
  js_compressor: uglifier
  css_compressor: sass
  cache: _cache/assets
  cachebust: hard

Pow It Up

Sure the new jekyll release has a serve option to boot up a web server. But like good software developers, we should be too lazy for such things. If you develop on a Mac and use Pow already for your Rails applications, why not just hook it up to serve your Jekyll site(s) too. Here are the steps.

First, add both rack and rack-rewrite to your Gemfile.

gem 'rack'
gem 'rack-rewrite'

Now create a config.ru file at the root of your project with the following contents.

gem 'rack-rewrite'
require 'rack/rewrite'

use Rack::Rewrite do
  r301 %r{^([^\.]*[^\/])$}, '$1/'
  r301 %r{^(.*\/)$}, '$1index.html'
end

Lastly, sym link the _site directory of your Jekyll project to the public. Then symlink your Jekyll project directory to Pow as you normally would do any Rails or Rack app.

$ ln -s _site public
$ cd ~/.pow
$ ln -s /path/to/myjekyllapp

You can now access your generated site at http://myjekyllapp.dev via Pow.

Live Reloads

I love LiveReload and have purchased the native OS X client for $9 from the Mac App Store. Personally, I could never get the guard-livereload gem working and just opted for the GUI client. So this tip will only focus on using the native Mac app.

It solves a few problems. First, it overcomes LiveReload's lack of rbenv support. It also assumes that your jekyll site is rather large and takes several seconds to build. Finally, assuming that you are working on your latest post and all you need to do is build a single page over and over again.

Setup LiveReload like you normally would by dragging your project's folder to their setup window. From here we want to configure your jekyll project to "Run a custom command...". Click on "Options..." and add the full path to your jekyll-livereload task script.

Also, make sure to exclude any directories that you do not want LiveReload to monitor. One critical folder is the jekyll _site directory. Watching this directly usually results in a indefinite LiveReload loop. Not good.

Your mileage may vary, but here is what I have found works for me in my jekyll-livereload script. This forces the LiveReload sub shell script to load up rbenv again. It then uses the --limit_posts argument to build just the last post. I have found this quickly refreshes the browser page automatically for me while working on my latest Jekyll post in markdown.

#!/usr/bin/env bash
set -e

export LANG="en_US.UTF-8"
export PATH="$HOME/.rbenv/shims:$HOME/.rbenv/bin:$HOME/.rbenv/plugins/ruby-build/bin:$PATH"
eval "$(rbenv init -)"

jekyll build --limit_posts 1

Resources

In Loving Memory of Kenneth Earle Collins

Sat, 05 Jan 2013 00:00:00 -0500

Kenneth Earle Collins, 66, of Chesapeake, Virginia, passed away January 5th, 2013.

My father, known to his friends and family as Kenny, was born in Logan, West Virginia and was the son of the late Edgar and Frances Collins. He was a retired Master Deputy with the Chesapeake Sheriff's Department. Kenny is survived by his beloved wife of 7 years, Lois V. Collins; a sister, Lanna DeVites and husband John of Zuni, VA; 4 sons, Benjamin M. Collins and fiancée Samantha of Williamsburg, VA, Kenneth Collins and wife Kelly of Portsmouth, VA, Brian Seabold and wife Kelly of Chesapeake, VA, and George Seabold and wife Terri of Elizabeth City, North Carolina; a daughter, Lisa Berger and husband Keith Berger of VA Beach, VA; and 6 grandchildren, Marcus, Zachery, Grayson, Bryson, Jacob, and Trey.

Service

I am not sure when my father's parents moved from West Virginia, but my Dad did grow up here locally in the Hodges Manor section of Portsmouth, Virginia - right next to Elizabeth Manor Golf & Country Club. Dad graduated from Portsmouth High School with the class of 1965. He was drafted into the US Army and trained at Fort Jackson, South Carolina.

Dad, served two years in Vietnam from 1966 to 1968. Like most veterans, he never talked about his service. After he passed, I decided to use DD 214 to order replacement decorations. Thankfully, the Medals Of America company put in a rush order for me so we could decorate his uniform for the memorial service. He also saved his short-timers stick and jacket with hand sew patches and lettering. The text on the back of the jacket says "I am certain to go to Heaven because I spent my time in Hell."

Below is a list of his decorations:

National Defense Service Medal
Vietnam Service Medal
Republic of Vietnam Campaign Medal w/60 Device
Good Conduct Medal
Expert Badge (M-14 Rifle)

Shortly after serving in the Army, my Dad joined the Portsmouth Police Department where he served till mid 1974, a few years after I was born. During his time there he became a Detective, State Arson investigator, and served on the bomb squad. His detective work focused on the narcotics division and he served much of that time undercover. I have numerous family photos with my Dad sporting pot leafs on his shirt or a nice Budweiser cap. After he passed, I finally found a copy of his resignation letter with the force where he quoted salary and the need to be more of a "family man".

After resigning from the force, Dad started a private investigation company with a long time friend, Ken Riddleberger. It was aptly named Double K Incorporated and they primarily handled arson and insurance fraud cases. Dad did this till the late 80's. I really believe he thought of himself as Magnum PI. Often he would wake up in the morning and drive around calling out on the CB Radio till his partner Ken Riddleberger woke up and did the same. They would promptly head towards a local Texaco station where they would play Ms. PacMan while they discussed their case load.

Because investigation work often required a lot of travel, Dad eventually got a desk job as head of security at the Portsmouth Naval Base and eventually came back to law enforcement with the Chesapeake Sheriff's office. I am very proud of my father's service and work. Below are some shadow boxes I made to commemorate his service.

Lifestyle

Dad was an avid golfer! I mean really, he played every spare moment he could and passed the love of the game to both my brother and myself. We owned our first set of clubs at the age of 9 after Dad won a tournament at Bide-A-Wee golf course. Dad loved the game so much, the family opened up a small golf shop in Chesapeake called Links Golf Shop. Ben and I even attended the GolfWorks school for club manufacture and repair in Newark, OH.

Golf was not our only sport. For some reason Dad made me and my brother try our luck with many others. We played basketball, football, and baseball. Most years we played baseball and even though we both sucked and never had more than a 0 batting average, Dad was there for us and helped coach all of our teams.

After getting divorced in 1991, my Dad eventually remarried in July 2006 to Lois V. Collins. Dad and Lois met at a Parents Without Partners (PWP) group and dated for some time. Lois is a real class act and took care of my Dad during his last few years while battling lung cancer. I can not count the number of times she would drive them both to his favorite destinations or other out of state trips. Lois will always be a Collins and is family to me. I love her dearly.

Stories

My Mom, Linda Campbell - now remarried, recently told this story to my Brother and myself and I hope she does not mind sharing it here.

Your Dad and I must have been in junior high school, you know the age where you are scared and everything embarrasses you and you know if you do anything, it will be wrong? I remember feeling especially confident though because I had on an ironed white shirt and a brand new pair of Bermuda shorts that my mom had sewn for me. I was Sandra Dee and – well, we didn't really know each other or hang out at the time but we were with a bunch of neighborhood kids all hanging around on a hot summer day with nothing to do.

As it was back then, kids didn't hang out inside so we were all standing and sitting on a friends chain link fence and being shy about the opposite sex. I was one of the kids on the fence, don't know why, because the wire went above the top pipe bar and was not very comfortable at all to sit on. The kids generally agree to go beg something cold to drink from one of the mothers – mothers were always home back then, just like June Cleaver. So, I jumped down from the fence.

Uh-oh, my feet hit the ground, but most of my brand new Bermuda shorts were still attached to the wicked fence. Standing there in my white cotton undies and very little of my shorts, I wanted to crawl under the fence. Everyone was laughing and making fun – everyone but your Dad. He calmly went to the clothes line and pulled down an ironing board cover that had dried in the sunlight and approached me with his head turned sideways, and his arms spread wide, holding the ironing board cover for me like a mother holding a towel for a wet toddler emerging from the bathtub.

We didn't cross paths again until after graduation, when he worked at James' Texaco. I had just gotten my new spiffy, metallic blue Corvair with 4 in the floor and was gassing up at his station, trying to look prim and proper. He never mentioned the shorts eating fence until later but when he asked me on a date, I remembered his gallantry from many years earlier.

In Closing

The Collins family has always been close, despite long durations without seeing one another. Often times I went weeks without checking in on my immediate family. The great part about family is that you can always pick up right where you left off. Time has no meaning. But with my Dad's passing, I will really miss that next time when I want to call and talk to him.

HolyGrailHarness

Wed, 05 Dec 2012 00:00:00 -0500

A curated Rails application prototype that focuses on simple test patterns for Ruby & JavaScript!

Unlike normal Rails Application Templates or more modern Rails application generators like Rails Composer, the HolyGrailHarness is a basic Rails application that can be considered a prototype and customized via a simple setup script. It is also somewhat opinionated in that it promotes simple and powerful testing choices and focuses on using Ruby 1.9 and up, MiniTest::Spec, Capybara, Poltergeist/PhantomJS, and Konacha. More details on each component and what HolyGrailHarness provides are below.

The HolyGrailHarness is perfect for any of the following:

Bootstrapping your next Rails application.
Learning and promoting MiniTest::Spec
Modern JavaScript testing setups.
Teaching Rails and/or JavaScript at your next meetup.

Usage

Download the project.
Now from the root of "holy_grail_harness" directory.

puts "This code will be highlighted"

Make sure to replace my_app_name above with the name of your new Rails application. The setup script has a few options, but the end result will be a new Rails application all ready to go. So why not a normal Rails application template? Although, Rails application templates provide a really nice feature set. It was much easier to bootstrap a new Rails application using this prototype method. The end result is a cleaner Gemfile and application setup that can be vetted and tested from within HolyGrailHarness itself.

The script will rename your directory and prompt you to cd to that directory. Once you do that, run rake test:all to see that everything is working.

$ cd ../my_app_name
$ bundle exec rake test:all

Rails 3

This application prototype will focus on the latest Rails version. At this time, the bundle is locked down to v3.2.9. As Rails updates and is compatible with each component, so will this prototype application be updated. The bundle includes:

QuietAssets gem for silent pipeline logging.
Thin webserver. Primarily to be automatically used by Konacha but also good for development if you are not using something like Pow.

Testing

MiniTest::Spec All The Way Across The Sky!

Don't wait for Rails 4 to use MiniTest::Spec! This application is using the minitest-spec-rails gem which forces ActiveSupport::TestCase to subclass MiniTest::Spec. This means that you can start using the MiniTest's Spec or Unit structure and assertions directly within the familiar Rails unit, functional, or integration directories. For full details, check out the minitest-spec-rails documentation or some of the test shims within HolyGrailHarness. For example, a test/unit/user_test.rb might look like this.

require 'test_helper'

class UserTest < ActiveSupport::TestCase

  let(:bob)   { users(:bob) }
  let(:admin) { users(:admin) }

  it 'must respond true to #admin? for administrators only' do
    admin.must_be :admin?
    bob.wont_be   :admin?
  end

end

Capybara Integration Tests With Poltergeist Using PhantomJS

You don't need Cucumber to write good integration tests. Instead use the basic Capybara DSL directly within a Rails integration tests with the most bad ass driver available, Poltergeist, which is built on top of PhantomJS. Never again worry about installing Qt so you can compile capybara-webkit, just go download a pre-compiled PhantomJS binary for your specific platform and enjoy 20% faster integration test runs vs capybara-webkit.

Integration tests are still within the ActionDispatch::IntegrationTest class and as promised, MiniTest::Spec is available here too. Each test file needs to require the test_helper_integration which provides the following base features.

Sets page size to that of a 13" MacBook Air.
Resets Capybara sessions after each test.
Provides a #save_and_open_page, or #page! for short, screen shot method.
Ensures a single ActiveRecord DB connection for transactional test runs.
An #execjs helper for bridging Ruby and the JavaScript under test.

HolyGrailHarness comes with a integration test example in the test/integration/application_test.rb file. An integration test might look something like this.

require 'test_helper_integration'

class ApplicationTest < ActionDispatch::IntegrationTest

  before { visit root_path }

  let(:h1) { find 'h1' }

  it 'renders' do
    h1.must_be :present?
  end

end

Konacha JavaScript Tests Using PhantomJS

Move over Jasmine(rice), Konacha is the way to test your JavaScript now. Konacha is a Rails engine that allows you to test your JavaScript with the Mocha test framework and Chai assertion library. Konacha's killer feature is a sandboxed <iframe> for each test spec to run within as well as full Rails asset pipeline integration. The HolyGrailHarness does all the work to get your Konacha spec/javascripts directory all setup and ready to go. Highlights include:

An initializer that sets up Poltergeist as the Capybara driver.
A directory structure for model, view, and controller specs.
A spec_helper.js.coffee for your specs to require. Provides global setup, configurations and vendor requires.

HolyGrailHarness also has a spec/javascripts/spec_helper directory meant for helpers and extensions that should be available to all specs. We have included a fixtures.js.coffee file that demonstrates how to setup JSON data fixtures for use from anything to stubbing requests to instantiating new model objects. We also have a helpers.js.coffee file that exposes a few top level functions that make debugging your JavaScript easy. Below are the vendored JavaScript libraries that are required by the spec_helper.

Sinon.JS - For spies, stubs, faking time, etc.
jQuery Mockjax - Best way to mock jQuery's AJAX functions.
Chai jQuery - Chai assertions for jQuery.
jsDump - Used by the myLog() helper.

Because your CI system should run all your tests, the HolyGrailHarness has added a Rake task to the test namespace that runs the default rails test task (units, functional, integrations) then your Konacha tests.

$ rake test:all     # Runs all Rails tests, then Konacha tests.

Guard

TDD in style and run your tests when you hit save! Both guard-minitest and guard-konacha are bundled and ready to go. A basic Guardfile is already setup too. Unlike most, this one is split into two groups :ruby or :js. This lets you focus on either everything or a specific language for your tests.

$ guard             # Monitor both Ruby and JavaScript tests.
$ guard -g ruby     # Monitor Ruby tests.
$ guard -g js       # Monitor JavaScript tests.

The Guardfile assumes you are running OS X and wish to use the Ruby GNTP (Growl Notification Transport Protocol). If this is not the case, consult the Guard documentation on different system notification alternatives.

Factories And Fixtures

ActiveRecord YAML fixtures suck, but so do slow tests that rely on an empty database with excessive setups based on factories. The answer? Take advantage of the best each has to offer. Use factories to populate fixtures into the test database while leveraging database transactions during your test runs. The end result is a known factory story with the ability to create more test data as needed using the same factories. Allowing factories to properly hook into model logic means no more decomposing business logic into YAML text files. How?

The HolyGrailHarness bundles the named_seeds gem along with the factory_girl gem. The NamedSeeds library checks for the existence of a db/test/seeds.rb file and if present, loads that file. Just like Rails' own db/seeds.rb anything in this file goes. The only difference is that this seed file is populated right before you tests are run so they persist between transactions. You also get the benefit of using this same seed data in development as part of the normal Rails db:setup process. Read the full documentationn on their site on how to use it. Below is a brief example.

Create factories in the test/factories directory. Note, factories are best when they make valid garbage™, so the HolyGrailHarness also requires the forgery gem to help with that.

# In test/factories/user_factory.rb

FactoryGirl.define do

  factory :user do
    email      { Forgery::Email.address }
    first_name { Forgery::Name.first_name }
    last_name  { Forgery::Name.first_name }
    password   'test'
  end

end

When making seed data, be explicit with your attributes that may be forged in the factory, database seeds should be consistent and have meaningful attributes. In this example we are creating an admin user. Note too how we are using NamedSeeds.identify which mimics AcctiveRecord's fixture identity. This gives us a handle to the fixture within our tests. We also create the @admin instance variable because we might want to use that user later on in the fixture story.

# In db/test/seeds.rb

require 'factory_girl'
FactoryGirl.find_definitions rescue true
include FactoryGirl::Syntax::Methods

@admin = create :user, id: NamedSeeds.identify(:admin),
                       first_name: 'Admin', last_name: 'User', email: 'admin@test.com'

Lastly, in your test/test_helper.rb file, declare that you have a named seed to the users model. This will allow your tests to act just like those with ActiveRecord fixtures and use the users(:admin) helper to get to that seeded fixture.

# In test/test_helper.rb

class ActiveSupport::TestCase

  named_seeds :users

end

MVC JavaScript

The HolyGrailHarness wants you to use some type MV* structure for your JavaScript. The setup script supports Spine.js as an option, however you can decline and all traces of Spine.js will be removed. If so, the following features will still remain.

A single JavaScript namespace on the window object. This namespace creates a model, view, controller object structure that direly matches to the app/assets/javascripts/#{my_app_name}/(model|view|controller) directory structure within the Rails asset pipeline. This JavaScript namespace and matching directories will be changed to your new application name as part of the setup task. Here is an example of a User model whose corresponding file would be found in the app/assets/javascripts/my_app_name/models/user.js.coffee file.

class @MyAppName.App.Models.User extends View
  @configure 'User', 'id', 'email'

The main application.js file requires all vendor frameworks, then the index.js.coffee within your application name directory. Use this file to boot your JavaScript application and/or setup your root view controller.

Also included is the SpacePen view framework. SpacePen is a powerful and minimalist client-side view framework authored in CoffeeScript. It is actually a jQuery subclass which makes your views really easy to traverse and respond to controller events. Read my View Controller Patterns With Spine.js & SpacePen article to learn why views should not be dumb and how you can take advantage of SpacePen no matter what JavaScript MV* framework you use.

With Spine.js

If you choose to use Spine.js as your JavaScript MVC structure, the setup script will create a git submodule to the Spine repository to the vendor/assets/javascripts/spine directory. This allows your project to use the the source CoffeeScript files, which makes for a wonderful learning experience to both Spine.js and idomatic CoffeeScript.

By default the index.js.coffee will require all Spine components. This includes manager (stacks), ajax, route, and relation. Remove anything that you do not need. This file also defines the root view controller along with a MyAppName.App.Index.init() class level initialization function. This is called in the main application.html.erb layout file for you too. Likewise, the application init is done in the Mocha before filters mentioned above in both the spec_helper.js.coffee and fixtures.js.coffee files. If you examine these files closely, you will see how they make use of Mocha's done() callback so that you can cleanly abstract AJAX mocks and anything else related to your JavaScript application's boot process. Here is an example of how you might setup your initApplication().

@initApplication = (callback) =>
  bob = MyAppName.Test.Seeds.users.bob
  $.mockjax url: "/users/#{bob.id}", responseText: MyAppName.Test.Response.bobInitial.responseText
  MyAppName.App.Models.User.fetch id: bob.id
  MyAppName.App.Models.User.one 'refresh', callback

No JavaScript project should be without a local notification system to help keep disparate components up to date. Thankfully, Spine's event module makes a local PubSub system a breeze. The HolyGrailHarness has a notifications.js.coffee that exposes a class level bind() and trigger() to any event string/namespace you want. To make more simple, we recommend creating class level functions that expose the event name as the function name and pass the args to the handle() function. We have done this for the MyAppName.Notifications.appReady() to demonstrate. Calling this function will trigger the app.ready event and passing a function to this function will bind that function to the same event name.

Sass & Compass

Sass is the only way to write CSS for today's modern web applications. Compass is the CSS framework that no Sass user should go without. Together they provide a foundation for writing beautiful CSS using pre-built time saving functions. The HolyGrailHarness includes both the sass-rails and compass-rails gems.

To get you started on the right path, we have also created a basic structure within the app/assets/stylesheets asset pipeline directory to help you organize your Sass files. Here is the directory structure below.

├── application.css
├── application
│   ├── _layout.scss
│   ├── index.scss
│   ├── components
│   │   └── _foo.scss
└── shared
    ├── _animations.scss
    ├── _fonts.scss
    ├── _mixins.scss
    ├── _placeholders.scss
    ├── _variables.scss
    └── base.scss

The application.css file.

Never write CSS in application.css. Say what? I know right, but trust me. Just consider this file a top level bundle dependency that only requires other top level bundle assets. Here is the contents of that file. Notice how it requires a bundle called twitter and an index. One is for twitter bootstrap, see section below, and the other is the index to your own Sass framework.

/*
 *= require application/twitter
 *= require application/index
*/

The shared directory.

Think of this as your own Compass framework. The base.scss is your single file to @import to get everything loaded and ready to go. Nothing in any of the shared files should generate CSS! Importing shared/base should act just like importing compass. Use these files for setting your own variables and creating misc helper functions & mixins. There is a variables file for... variables! Another for animations, fonts and mixins too.

Pay special attention to the _placeholders.scss file. If you do not know about Sass 3.2's placeholder selectors (silent classes) and how they make presentational classes efficiently extended by semantic ones, then I highly suggest you read Dale Sande's presentation titled Sass 3.2 Silent Classes on Speaker Deck.

Below is the contents of the base.scss file, take note of the order. See too how we import the entire Compass framework. This means that all of your Sass code in any of the shared files can take full advantage of both Bootstrap and Compass' variables and mixins. Epic win!

// Think of this file as your own compass. Importing the base, never generates CSS.

@import "shared/variables";
@import "bootstrap/mixins";
@import "compass";
@import "compass/layout";
@import "compass/css3/user-interface";
@import "shared/fonts";
@import "shared/mixins";
@import "shared/animations";
@import "shared/placeholders";

The application directory.

Organize this as you see fit. We have started you off by creating a _layout.scss file for your general layout/structure styles. There is also a components directory which all sub files are imported via a glob. The idea is that components are not dependent upon another. Files that might go in here are things like datepicker, navigation, and general files named after components or widgets. Below is what the application/index.scss looks like.

@import "shared/base";
@import "./layout";
@import "components/*";

If you are more advanced with your CSS and like the idea of style guides, take a looks a the Toadstool style guide framework.

Twitter Bootstrap

Twitter Bootstrap is awesome, but LESS is not. That is why the HolyGrailHarness uses the bootstrap-sass gem that converts all the Bootstrap LESS files to Sass. Making them ready to import via the Rails asset pipeline.

As shown above in the Sass section, we require the application/twitter.scss bundle asset from the top level application.css bundle file. This twitter bundle file, contents below, take advantage of your shared variables before importing bootstrap from the gem. In this way you can define variables that tweak bootstrap. A good example would be button colors, column widths, etc. Later on in the file you can extend bootstrap styles to your liking. For instance, add more padding to buttons.

@import "shared/variables";
@import "bootstrap";
@import "font-awesome";

// Tweak or redefine Twitter classes below.

Font Awesome

The glyph icons included in Twitter Bootstrap are horrible for hi-resolution "retina" displays typically found on mobile devices. Thankfully the Font Awesome project provides a drop in replacement that instead uses icon fonts vs raster images.

The HolyGrailHarness vendors these font files and the needed font-awesome.scss file and requires them as part of the Twitter Bootstrap bundle shown above. More advanced users may prefer to only include the icon fonts needed in their application or a few custom icons. If that is the case, check out Font Custom, webfonts from the comfort of the command line.

Synchronized Times In Ruby & JavaScript Acceptance Tests Using Capybara, Timecop & Sinon.JS

Tue, 07 Aug 2012 00:00:00 -0400

Any good Ruby developer that tests time-dependent code has used the Timecop gem. Timecop provides dead simple time travel and freezing capabilities to Ruby's standard library. But what if you are working on a rich JavaScript application that is backed and tested by something like a Rails application and you want to alter the test browser's clock as well? The answer is pretty simple, but let's first examine all the parts at play here.

Rails & Capybara

My examples leverage a very simple Rails integration test setup using Capybara's DSL within a basic Rails integration test case. Much like Wyatt has described here and I have shown in a demo project on github. So if your integration test setup is different, transpose my code to fit your needs. Also, there are a few drivers for Capybara that support full JavaScript integration, most notably capybara-webkit. But the latest on the scene is Poltergeist which uses the badass PhantomJS project. I highly recommend you switch to this driver! That said, any Capybara driver that fully supports Capybara's #execute_script should work just fine.

Ruby & JavaScript Times

JavaScript date values represent time in milliseconds since Unix Epoch. Many 3rd-party JavaScript date libraries use millisecond integers for both instantiating and altering these objects. Ruby on the other hand has a much higher precision and thanks to ActiveSupport's core extensions to Ruby's date and time classes we can easily represent these values for JavaScript. Specifically, ActiveSupport adds a #to_i method that returns an integer which represents that date or time in seconds since Unix Epoch. In my example code, you will see that I multiple this by 1000 to get the millisecond representation.

ActiveSupport also provides an #advance core extension to all date and time classes. This method is Valuable As Fuck™ since it returns a new time instance that has moved backwards or forwards given a hash of options. For example, returning a time instance moved forward by 20 minutes would look like this @time.advance(minutes:20). See the documentation for all the options and remember, you can provide negative values to move backward. In short, the advance method is awesome!

Faking Time With Sinon.JS

Sinon.JS is a small stand-alone library that provides spies, stubs and mocks for your JavaScript. To be honest, I use Jasmine and supporting extensions for most of these features. However, Sinon.JS has one killer feature, faking time! Yup, it allows you to freeze JavaScript's clock to a specific time and tick it forward as needed. I want you to ponder the benefits of that for awhile. Imagine you have time sensitive JavaScript code that uses setInterval() or the like. Sinon.JS will actually allow you to tick time forward and still maintain compatibility with that code's behavior! Basically Sinon.JS is equal to our Ruby Timecop gem and then some! Check out their clock API or read the code if you want to learn more. Remember, it is safe to include Sinon.JS in any existing JavaScript project since it will not do anything unless you ask it too. So no fear in it clashing with your other JavaScript test setup.

Putting It All Together

So now the fun part, some code examples. First, you need to get Sinon.JS in your Rails JavaScript manifest. If you are smart, you have already setup a system where you can specify top level asset manifests for your JavaScript application per test environment. If not, you might want to take a look at two posts I previously published on setting up Jasminerice for testing Spine.JS applications.

These file snippets below assume you have an integration.js manifest which includes both a vendored Sinon.JS then a sub file which actually initializes Sinon.JS for our integration test run. In this case below, we are first processing a CoffeeScript file with ERB and then initializing Sinon.JS' fake timers to a default time. In my case, this is 8:30am central standard time. Now we can assure that Capybara's browser engine will be frozen at that time and ready to move forward for each test.

// File app/assets/javascripts/integration.js

//= require sinon-1.4.2
//= require integration/sinon
//= require application

# File app/assets/javascripts/integration/sinon.js.coffee.erb

<% central = ActiveSupport::TimeZone['Central Time (US & Canada)'] %>
<% today = central.parse('8:30am').to_i * 1000 %>

window.clock = sinon.useFakeTimers <%= today %>

Now, the Ruby code. Here is slimmed down version of my base integration test case which uses MiniTest::Spec to drive Capybara tests. The first thing I do before any integration test is use Timecop to travel to 8:30am. This means that both Ruby and JavaScript are synced to the exact millisecond in time. Any test that needs to move time forward must call the #advance_time test helper. This method takes a hash of options which is passed directly to the #advance method I previously talked about. It measures the milliseconds between now and the advancement and sends that directly to Sinon.JS' fake timers using Capybara's #execute_script method. So calling advance_time(seconds:20) in Ruby now moves time forward in both Ruby and JavaScript. Epic win!!!

# File test/test_helper_integration.rb

require "test_helper"
require "capybara/rails"

module ActionDispatch
  class IntegrationTest

  include Capybara::DSL

  before { travel '8:30am' }
  after  { Capybara.reset_sessions! }

  private

  def travel(parseable_time)
    Timecop.return
    Timecop.travel parse_time(parseable_time)
  end

  def parse_time(time)
    time.is_a?(String) ? Chronic.parse(time) : time
  end

  def advance_time(options)
    now_ms = Time.current.to_i * 1000
    Timecop.travel Time.current.advance(options)
    traveled_ms = (Time.current.to_i * 1000) - now_ms
    advance_sinon traveled_ms
  end

  def advance_sinon(ms)
    page.execute_script "if (window.clock) { window.clock.tick(#{ms}); }"
  end

end

I have found the following technique critical to fully testing a recent time based JavaScript application I have developed. I hope you find this technique useful as well and as always, please contribute your thoughts or questions below. Cheers!

Resources

Rails, Moment.js And Time Zones

Sun, 05 Aug 2012 00:00:00 -0400

Here are a few quick tips for the time zone aware Rails developer that finds themselves deep into JavaScript date objects. First, use the Moment.js JavaScrpt date library! Moment.js has a very rich API for parsing and working with times, very similiar to ActiveSupport's extensions. However, it does not have a solid way of moving times across zones. Especially if those zones may or may not observer daylight savings time (DST).

Many JavaScript time zone libraries require a huge set of geographic data to both identify zones and their observance of DST. These data files can add a significant overhead to JavaScript. But wouldn't it be great if there was a simple way of leveraging your Rails model's time zone settings? There is, but first we need to serialize an ActiveSupport::TimeZone object in JSON. Easy, just define an #as_json method like the one below. I suggest adding this to an initializer in your Rails config/initializers/active_support.rb directory. The key attribute here is the utc_total_offset. This will be a number in minutes that properly observes DST.

module ActiveSupport
  class TimeZone

    def as_json(options=nil)
      { :name                => name,
        :identifier          => tzinfo.identifier,
        :friendly_identifier => tzinfo.friendly_identifier,
        :utc_offset          => utc_offset,
        :utc_total_offset    => tzinfo.current_period.utc_total_offset }
    end

  end
end

Now assuming you have serialized a models time zone attribute using the full ActiveSupport::TimeZone object, we can easily use this information client side via a quick extension to Moment.js' prototype. Here is a moment.js.coffee file I have required in my Rails applications.

moment.fn.forTimeZone = (timeZone) ->
  currentOffset = (this.zone() * 60) * -1
  adjustedOfffset = if currentOffset > timeZone.utc_total_offset
                      timeZone.utc_total_offset - currentOffset
                    else
                      currentOffset - timeZone.utc_total_offset
  this.clone().add 'seconds', adjustedOfffset

So we let Ruby do all the hard work of telling us what time zones are observing DST without all the bloat to our JavaScript for parsing zone identifiers. Some sample output.

eastern = {
  friendly_identifier: "America - New York",
  identifier: "America/New_York",
  name: "Eastern Time (US & Canada)",
  utc_offset: -18000,
  utc_total_offset: -14400
}

noonPacific = 1344279600000 // "2012-08-06T12:00:00-07:00"
format = 'MMMM Do, YYYY \\at h:mma'

moment(noonPacific).format(format)
// "August 6th, 2012 at 12:00pm"

moment(noonPacific).forTimeZone(eastern).format(format)
// "August 6th, 2012 at 3:00pm"

Resources

Moment.js - A lightweight JavaScript date library.

View Controller Patterns With Spine.JS & SpacePen

Tue, 22 May 2012 00:00:00 -0400

The VC in 'MVC'

As Addy Osmani points out in his Scaling Your JavaScript Applications presentation, all JavaScript 'MVC' frameworks interpret MVC differently. These differences are an academic rabbit hole and if you are really interested about them, I recommend reading some of the resource links at the bottom of this post. One in particular by Jonas Nicklas really outlined how I think client side JavaScript applications should be developed. It is titled Why Serenade.js and in it, Jonas describes his reasons for yet another JavaScript MVC framework. In doing so, he outlines the responsibilities of the MVC triad as follows:

Views: Present data from the model and update it if it changes, notify controllers of user interaction events. Controllers: React to user interaction events by instructing the model to perform certain actions. Models: Handle business logic and persistence, notify the view of any changes to the data.

Seems straight forward right? In actuality it is easy to stray from and there is a lot of wiggle room for both the MV* framework authors and their developers to re-interpret one or all of these components. The two that I think are most open for debate are the View and the Controller stacks. For example, how many JavaScript frameworks do you know allow you to choose your own view framework? How many times have you heard too that views should be "logic-less" and how does each of these play into helping newcomers understand the proper way to code these precious views and controllers together? My answer, ask Apple!

Apple's View Controller Programming Guide

Some might be surprised that JavaScript applications for the browser can and should be coded much like modern desktop applications. Having developed iOS applications before, many of the same MVC principals and application structure apply. Apple has tons of documentation about general application design and one of my favorites is their View Controller Programming Guide. I highly recommend anyone developing MVC JavaScript applications to read this guide. I'll paraphrase a few points:

A view controller manages a discrete portion of your app’s user interface. Often that view is the root view for a more complex hierarchy of other views. View controllers typically fall in one of two categories, either content or container. Content view controllers are great for things like lists and images. Container view controllers coordinate with other view controllers like tab view controllers or navigation controllers.

It is also worth reading both the UIViewController and UIView class references too. So what should be your major take away from reading those? The first is that every controller in your JavaScript application manages a view. That view is going to be a DOM node that will most likely have a complex hierarchy of other views possibly managed by other controllers which themselves champion a node with a specific application concern. Second is that views are themselves valid objects built by classes with their own methods. It's best to think of UIViews as an instance of a jQuery object. Views should not be dumb either. They sometimes have handles back to model objects so they can establish bindings for updates. Views can also expose an interface that canonize how controllers should work with them and most often will have helper methods that break up their complexity. All of this is a good thing. If you disagree, stop reading now!

SpacePen - Markup On The Final Frontier

I found the SpacePen project on github a few months ago and I have never looked at another templating solution since. For the first time my JavaScript application's view and controller stacks feel natural. No longer do I find myself writing any view related code in my controllers. SpacePen allows me to write view objects that encapsulate their concerns, stay DRY and build Spine.JS web applications that hit the MVC mark. So what is SpacePen? Basically a CoffeeScript subclass of a jQuery object.

class SpacePort extends View

  @content: ->
    @div =>
      @h1 "Space Ships"
      @ol =>
        @li "Apollo"
        @li "Soyuz"
        @li "Space Shuttle"

view = new SpacePort
view.find('ol').append('<li>Star Destroyer</li>')

view.click 'li', ->
  alert "You clicked on #{$(this).text()}"

This simple example shows how your SpacePen view class defines its markup in a @content class function. This function yields a builder syntax where you can easily add attributes and values. My advice is to do basic structure only in your @content function and rarely take advantage of the fact that it also takes the same parameters argument passed to your constructor. I will explain more on that advice later. But first, I did say that views should expose interfaces to controllers and are first class code citizens that champion their own concerns right? Let's look at a common pattern, views building new nodes within itself.

class SpacePort extends View

  @content: ->
    @div =>
      @h1 "SpacePort"
      @ol outlet: 'shipList'

  addShip: (ship) ->
    @shipList.append "<li>#{ship.name}</li>"

view = new SpacePort
view.addShip "Enterprise"

The example has been changed slightly so that the view now exposes a public function called addShip() which takes an object to add to the view. It can easily implement this because I have declared the ordered list to be an outlet called shipList which will automatically be assigned as a property on each view instance. Calling append on that node is possible because like the view object itself, any outlet is a jQuery object as well.

Though contrived, this example illustrates a powerful technique. If the view was refactored to some other structure besides an ordered list with items, the controller interface would stay the same. Pulling the thread, it is easy to see how a complex view hierarchy could have views building and managing messages to their subviews too. This is the crux of my article, but first, lets talk about controllers.

Spine.JS Controllers

Spine.JS is my goto JavaScript MVC framework and if you are unfamiliar with it, check out their introduction or read why others have chosen to use it. The main reasons I love Spine.JS can be summed up like so. First, it is authored in CoffeeScript, which means I can read the code to learn it. Second, all model instances will reflect changes by any other instance. Lastly, the controllers are minimally implemented and nicely abstract managing a view and the way controllers respond to events that bubble up to it.

A Spine.JS controller always has an element associated to it which can be accessed via the el property. This element represents the view that controller manages. It will either be created for you or it can be assigned via the constructor. Many patterns in Spine.JS rely on appending elements to a controller and most of the functions for adding these subviews either take an element or an object that has an el property. This means building complex view/controller hierarchies by stacking Spine.JS controllers is both idiomatic and almost identical to how iOS view controllers work.

Using SpacePen Views With Spine.JS Controllers

With the overview of these components out of the way, let's jump right into how they can work together to punch the VC up a notch in your next JavaScript MVC application. These examples below are pared down version of a scheduler application I just completed. The model is similar to an iCal day interface for calendar events. In such, there would be a view controller that manages the presentation of your schedule "by day". This controller's view would have a header for the current day and a set of controls for going to the previous day, today and the next day. This controller would act as a navigation controller and correlate to Apple's notion of a "container" view controller. When needed, it would find or create a day controller and append it to one of its subviews. The code for that container controller and view is out of the scope of this article, but its one of many day "content" controllers and view are perfect examples. Here we go!

# In app/assets/javascripts/app/controller/day.js.coffee

class App.Controllers.Day extends Spine.Controller

  constructor: (params) ->
    @view = new App.Views.Day
    super el: @view

# In app/assets/javascripts/app/views/day.js.coffee

class App.Views.Day extends View

  @content: (params) ->
    @div class: 'day', =>
      # ...

  initialize: (params) ->
    @attr 'data-date', params.moment.format('YYYY-MM-DD')

So rule number one, always create an instance of a SpacePen view and assign that to the @view property of every Spine.JS controller. That @view will be assigned to the controller's @el property when it supers up.

Rule number two, keep the @content function for your views concerned with building the structure only. In my example, I set the date for the event using the initialize hook provided by SpacePen. This rule is important when you want to bind views to model events and update that view accordingly. All customization of the view to a model object should be done in a set/update helper of that view.

In Greater Detail

Here is the same controller and view example with a bit more detail. First up, the view. This day view CoffeeScript file has a public App.Views.Day view and a private EventView that represents a subview and would be bound to a model instance. This public day view exposes a few public functions for a controller to hook into. The addEvent function will actually build a new SpacePen views for the passed model and append it as a subview. Notice too the findEvent helper function which can find the DOM element for a model object and then use the view() function provided by SpacePen to get to the SpacePen instance for this node.

class App.Views.Day extends View

  @content: (params) ->
    @div class: 'day', =>
      # ...
      @div outlet: 'events', class: 'events'
      # ...

  initialize: (params) ->
    @date = params.moment.clone()
    @attr 'data-date', @date.format('YYYY-MM-DD')

  addEvent: (event) ->
    view = new EventView(event)
    @events.append

  updateEvent: (event) ->
    @findEvent(event)?.update(event)

  removeEvent: (event) ->
    @findEvent(event)?.remove()

  # Private

  findEvent: (event) ->
    selector = ".event[data-event-id='#{event.startsAtFormat()}']"
    @events.find(selector).view()

  afterAttach: ->
    @startSpinner()
    @loadEvents()

  # ...

class EventView extends View

  @content: (event) ->
    @div class: '.event', 'data-event-id': event.id =>
      @div outlet: 'description'
      @div outlet: 'duration'

  initialize: (event) ->
    @update event

  update: (event) ->
    @attr 'data-starts-at', event.startsAtFormat()
    @description.text event.description
    @duration.text event.duration

Next up, the expanded day controller example that manages the view above. In this example we bind to two of the the Spine.JS CalendarEvent model events. One for update and one for destroy. These events are then pushed down to the view and will do any work necessary. Just in case this controller is ever discarded for performance reasons, we hook into the release() function provided by Spine.JS controllers to remove the event listeners.

class App.Controllers.Day extends Spine.Controller

  @events:
    'click .event': 'selectEvent'

  constructor: (params) ->
    @date = params.moment
    @view = new App.Views.Day moment: @date
    super el: @view
    CalendarEvent.bind 'update', @updateEvent
    CalendarEvent.bind 'beforeDestroy', @destroyEvent

  release:
    CalendarEvent.unbind 'update', @updateEvent
    CalendarEvent.unbind 'beforeDestroy', @removeEvent
    super

  # Private

  selectEvent: ->
    # ...

  updateEvent: (event) ->
    @view.updateEvent event
    # ...

  removeEvent: (event) ->
    @view.removeEvent event
    # ...

Hope you found this mini introduction to SpacePen and Spine.JS controllers useful. There is literally tons of ways you can use these tools together to make your controllers and views fun to build and maintainable. Be excellent and let me know what works for you!

Resources

StoreConfigurable - A Lesson In Recursion In Ruby

Mon, 12 Mar 2012 00:00:00 -0400

When ActiveRecord 3.2 was released there was a small addition called ActiveRecord::Store which bills itself as a simple key/value store for your models. The code below is pulled right from their example usage.

class User < ActiveRecord::Base
  store :settings, accessors: [:color, :homepage]
end

@user = User.new(color: 'black', homepage: '37signals.com')
@user.color            # => 'black'
@user.settings[:color] # => 'black'
@user.settings[:remember_me] = true

Most people know that I love simple tools. But when I found myself considering ActiveRecord::Store, I found it seriously lacking for my particular use case. What I wanted was a config for a user class that could do the following:

Interchange property dot notation with indifferent string/symbol lookup.
Automatically grow as needed. For namespaces and nested organization.
Report state changes to the model from any node.

StoreConfigurable

The lack of these features is why I set out to build a little gem I call StoreConfigurable. A zero-configuration recursive Hash for storing a tree of options in a serialized ActiveRecord column which includes self-aware hooks that delegate dirty/changed state to your configs owner. Perfect right? Here is a simple example of its usage below. If you want to learn more, checkout the project on github with the full README.

class User < ActiveRecord::Base
  store_configurable
end

@user.config.remember_me = true
@user.config.sortable_tables.column    = 'created_at'
@user.config.sortable_tables.direction = 'asc'
@user.config.you.should.never.need.to.do.this.but.you.could.if.you.wanted.to = 'deep_value'
@user.config_changed? # => true

Rather than talking about how StoreConfigurable might be a good fit for you, I would instead like to discuss how my first practical usage of recursion in Ruby is implemented in StoreConfigurable. That along with how to leverage default Hash values is the topic for this post.

Default Hash Values

So first up, a little known feature of Ruby's Hash class is the ability for a default value to be returned when a key is missing. If you stop and think about this, the default is nil, which is a valid object in Ruby. But you can tell Hash objects to return other values when a key is missing. The first way to do this is to pass the default value as an argument to Hash.new(). The second is to pass a block to the new method which would return the default value. That block is passed in the current hash object and the key that is missing so you can do some fancy things if needed. Here are some quick examples of both techniques.

# Default value passed to new.

hash = Hash.new('default')  # => {}
hash[:foo]                  # => "default"

# Default value from new's block.

hash = Hash.new { |hash, key| "Missing #{key} for hash with id #{hash.object_id}." }
hash                        # => {}
hash[:foo]                  # => "Missing foo for hash with id 283829283."

Recursion

Recursion in Ruby can take many forms. My solution for StoreConfigurable was to always make the hash returned by the proxy object self replicate itself for key misses. Here is a fundamental example to make your own Hash object return new instances of itself when keys are missing.

class RecursiveHash < Hash
  Recursive = lambda { |h,k| h[k] = h.class.new }
  def initialize
    super(&Recursive)
  end
end

hash = RecursiveHash.new  # => {}
hash.class                # => RecursiveHash
hash[:foo]                # => {}
hash[:foo].class          # => RecursiveHash
hash[:a][:b][:c]          # => {}
hash                      # => {:foo=>{}, :a=>{:b=>{:c=>{}}}}

In this example, I create a Proc object via the lambda keyword and I use this as the block argument when you create an instance of RecursiveHash via the super method. Here I am using the & syntax to pass an existing Proc as a block. Hope I have not lost you so far :)

So default hashes in Ruby and simple recursion techniques are what make up some of the key points of StoreConfigurable. I also use recursion when loading the hash values from the YAML stored in the database. Here is what that technique looks like.

loader = lambda do |hash, key, value|
  if value.is_a?(Hash)
    value.each { |k,v| loader.call(hash.send(key), k, v) }
  else
    options.send "#{key}=", value
  end
end

data.each { |k,v| loader.call(config, k, v) }

Hopefully this dive into Ruby or maybe StoreConfigurable is helpful!

Resources

StoreConfigurable - A zero-configuration recursive Hash for storing a tree of options in a serialized ActiveRecord column with self-aware hooks that delegate dirty/changed state to your configs owner.
ActiveRecord::Store - Simple key/value store for your models.

Too LESS? Should You Be Using Sass?

Mon, 27 Feb 2012 00:00:00 -0500

First, a little bit of background. A while back ago I took a great deal of my personal time to try out Twitter's Bootstrap project for a Rails application of mine. This meant that I was willing to throw away some of the work I had done in Sass and rewrite it using LESS. No problem I thought, at first glance LESS and Sass look almost identical in functionality. So after some weeks of improving the toolchain for LESS with Ruby and then Rails, I set out to do just that – and what a miserable failure it was. My goal is to share my experiences to anyone considering using LESS and why it might be wrong for your Rails project.

Why Should My Opinion Matter?

For staters, I am the author of less-rails, the gem that allows you to effectively use LESS in the Rails asset pipeline. This gem made other gems of mine like less-rails-bootstrap possible. So now full LESS frameworks such as Twitter's Bootstrap could be leverage natively within the Rails ecosystem. Both of these gems greatly reduced the barrier to using LESS for the average Rails developer and I wrote them for one simple reason. I wanted to make sure LESS was as close to par with Sass in the Rails asset pipeline as possible. So before I rewrote one single line of Sass to LESS, it was going to have a fair shake. This was no small feat! Especially considering that LESS is the only JavaScript based CSS language to be extended with Rails' asset helpers like asset-data-url(). Meaning you can hook into assets from any other place in the pipeline directly from LESS, just as you would with Sass.

Other qualifications? Well I fancy myself a CSS nut and put my skills slightly above average. Most know me as an ActiveRecord or database nerd. But I am just as comfortable doing client side JavaScript and design implementation, especially using HTML5 and CSS3. Very few people know of my design background nor that I have been writing proper presentation layer CSS for over 14 years. Here in the past few years, I have learned to leverage both Sass and the Compass framework like a pro, even for this blog. Here lately, in the project I intended to move to LESS, my Sass usage has become quite impressive. I have extended Sass' color objects to allow HSV from RGB conversions for presentation code parity with similar iOS CoreGraphics drawing code in my native iPhone applications. I have even learned to write Sass mixins using custom functions and tricks I picked up while reading the Compass source. All this placed the bar pretty high for me on how LESS could take the place of Sass in my little experiment.

What Went Wrong?

So my bar was pretty high. I set out to not only rewrite my current Sass to LESS, but I fully expected to duplicate much of the things I learned to love about the Compass framework in LESS as well. And like a good open-source citizen, I even prepared a new project that I dubed protractor on github to share all my work. So what went wrong? Plenty! Every day, mixin by mixin, line by line, I felt MORE pain. LESS was fighting me and it was winning.

Some of the bugs I encountered were fixed, like parsing errors for CSS3 keyframes. Others had horrible workarounds that used more code while at the same time lowering legibility. Finally, I came across the one issue on the hundreds of those that exists on LESS' github page that stopped me cold. It was titled variable property and after learning about it, I decided to stop using LESS for my Rails projects. What are variable properties anyway. Let me show you a common pattern both I and Compass use in a basic contrived example. You may also want to read my comments in that github issue link too. Imagine I have some global colors that I want to mixin to other classes based on a dynamic property. So given this SCSS below:

$myWhite: rgb(240,240,240);
$myGray:  rgb(140,140,140);
$myBlack: rgb(30,30,30);

@mixin myColorClasses($property) {
  &.white   { #{$property}: $myWhite; }
  &.gray    { #{$property}: $myGray; }
  &.black   { #{$property}: $myBlack; }
}

.box {
  @include myColorClasses(background-color);
}

It would output something like the following. Notice how I use the variable string background-color and generate dynamic properties?

.box.white { background-color: #f0f0f0; }
.box.gray  { background-color: #8c8c8c; }
.box.black { background-color: #1e1e1e; }

Astute observers may point out that you can achieve the same with sub classes and be more efficient in the generated CSS too. You would be correct. I did say this was a contrived example. The point is that LESS is not a CSS preprocessor and without the ability to dynamically define property values you will be limited in the ways your LESS based CSS framework can compete with frameworks like Compass. Simply put, LESS will never stack up to Sass.

In All Fairness

I have mad respect for Alexis Sellier (@cloudhead). By far his skills and open source contributions well outrank my own. As software developers we need to recognize that there are no absolutes and as such, there is no one tool that is right. You have to weight a ton of other factors and choose the tools that are right for you. I firmly believe that as Ruby and or Rails developers that you should choose to learn Sass and Compass vs. LESS and I am basing my opinion heavily on three criteria.

First is that from a Ruby perspective, Sass will always be more easy to extend and hook into. Second, that LESS lacks a critical feature that I require to author CSS. Lastly, that LESS is not a CSS preprocessors by design and I believe they are the way forward to a better future via the CSSWG. This is what Sass does very very well. I want to treat CSS as a language and use features like loops, lists and custom functions.

Your mileage may vary and please make your own decisions. I would be more than happy to followup with any questions on this topic too, so please ask away in the comments below. Lastly, here are a few other articles that talk about LESS and Sass.

Pretty Console Logging With Guard::Jasmine & Black Coffee

Wed, 01 Feb 2012 00:00:00 -0500

OK I know I promised that we would start the dive into testing your Spine.JS application using Jasmine(rice) in my last article, but this is a good diversion. If you are new to my latest series on Spine.JS and Jasmine, scroll on down to the bottom to the related section and read back. However, for those that might be more familiar with Jasmine and specifically Guard::Jasmine and ever felt the pain that debugging from that terminal window was lacking, read on! Even if your new to Guard::Jasmine and Jasminerice I still suggest you setup these elegant hacks to make your testing go that much smoother.

So What Is The Problem?

Guard::Jasmine allows you to continuously test your JavaScript right from the terminal window just like your Ruby code. The only drawback is that the console debugging is less than helpful. Guard::Jasmine will not allow you do view a string version of every object nor see line numbers of calling files. Both of these are invaluable when your stuck in a testing hole and just need to inspect a few objects. So if you are tired of seeing [object Object] in your Guard::Jasmine output, let's fix it right away.

The first place we need to patch things up is Guard::Jasmine itself. In my last article I covered how to monkey patch Jasminerice in a config/initializers/jasminerice.rb file. My Guard::Jasmine freedom patch will be placed in that same file. Pasted below, this does two things vs the original. First it changes the report_specdoc_logs method to not pass true to the format_log_message method. Second, the format_log_message method itself now has the message regular expression changed to a multi-line scan. It will also look out for a custom prefix tag and allow it to pass through. This is for our pretty objects. Anything else now outputs the message with the file and line number, something previously stripped by Guard::Jasmine.

# coding: utf-8
if defined?(Jasminerice) && Jasminerice.environments.include?(Rails.env)

  # Other Jasminerice patches from:
  # http://metaskills.net/2012/01/17/rails-and-spine-js-jasmine-testing-part-2/
  # ...

  # Patch Guard::Jasmine to use a custom formatter for log messages that
  # allows multi-line objects to be printed with the line numbers.
  module Guard
    class Jasmine
      module Runner
        class << self

          def report_specdoc_logs(spec, options, level)
            if spec['logs'] && (options[:console] == :always || (options[:console] == :failure && !spec['passed']))
              spec['logs'].each do |log|
                Formatter.info(indent("    • #{ format_log_message(log) }", level))
              end
            end
          end

          def format_log_message(message)
            if message =~ /(.*?) in http.+?assets\/(.*)\?body=\d+\s\((line\s\d+)/m
              pp_prefix = '[myLog]'
              msg, file, line = $1, $2, $3
              if msg.starts_with? pp_prefix
                msg.sub pp_prefix, ''
              else
                "#{msg} in #{file} on #{line}"
              end
            else
              message
            end
          end

        end
      end
    end
  end

end

A JavaScript Pretty Print Library

So now that we have Guard::Jasmine not stripping multi-line console messages to one line and printing file and line numbers, we are ready to hook into it. But first we need a JavaScript library that pretty prints objects for us and wrap this all up behind our own interface to console.log. During my research I found a library called jsDump and decided to use a fork of the project on github. So go download that file and place it in vendor/assets/javascripts/jsDump.js of your Rails project. Next you will want to add it to your spec/javascripts/spec.js manifest like so. My below example is pulled right from my previous article and I have placed jsDump right after my jasmine files.

#= require jasminerice
#= require mock-ajax
#= require jsDump
#= require application
#= require jasmine-myapp
#= require_tree ./lib/
#= require_tree ./models/
#= require_tree ./controllers/
#= require_tree ./views/

Logging Helpers - Using BlackCoffee

In my last article I mentioned how I would talk more about the jasmine-myapp file in the manifest above. Now is the time. I recommend that this file is place to write all your top level helpers and functions for Jasmine or any other testing libraries in your JavaScript stack. Think of this file as your Ruby test_helper.rb or your own extensions to ActiveSupport::TestCase. My advice is that these functions all be written to pollute the time level namespace as other Jasmine helpers do, like the beforeEach and clearAjaxRequests. This makes things easier but be conscious of that decision and write functions keeping that in mind.

The only thing working against is is Sprockets/Tilt rendering CoffeeScript files in their own closure. Which is something you should really not fight! But in this case I think it is fine to have this particular file avoid that. Which allows us to (a) write our helper code in CoffeeScript and (b) use these functions as helpers in the global space like other Jasmine helpers. So enter my Sprockets BlackCoffee gem. This is a simple gem that exposes a CoffeeScript template that uses the --bare option to keep your file from being wrapped in a closure. All you have to do is give the file a .js.black_coffee extension and it will just work. So let's assume you have a spec/javascripts/jasmine-myapp.js.black_coffee created and the gem installed like so.

group :assets do
  # ...
  gem 'sprockets-blackcoffee'
end

Finally, here are examples of my logging helpers in the jasmine-myapp file. The myLogParser uses jsDump to get back a pretty formatted string of any object if that object is not already a string. The primary logging helper myLog will prefix your message with [myLog] so the Guard::Jasmine recognizes the message and outputs only the object. The last helper myLogLine will do just like the other, but will allow the file and line number tobe printed too.

myLog = (obj) ->
  console.log "[myLog]#{myLogParser(obj)}"

myLogLine = (obj) ->
  hmLogParser(obj)

myLogParser = (obj) ->
  if typeof obj is 'string' then obj else jsDump.parse obj

In Practice

Here is an example of a Jasmine spec where I was using console.log before the patches above and what it would output to the terminal.

#= include spec_helper

describe 'User', ->

  it 'has been configured with proper attributes', ->
    @user = new User id: 2, email: 'bob@test.com'
    console.log @user.attributes()
    expect(@user.email).toEqual 'bob@test.com'

User
  ✘ has been configured with proper attributes
    ➤ Expected 'foo@bar.com' to equal 'bob@test.com'.
    • [object Object]
ERROR: 1 specs, 1 failures
in 0.653 seconds

There is the totally helpful [object Object]. But if we now change our code to leverage out patches and change console.log to hmLog @user.attributes() to use our helper. We will get this. If you need a pretty object with line numbers. Just use hmLogLine.

User
  ✘ has been configured with proper attributes
    ➤ Expected 'foo@bar.com' to equal 'bob@test.com'.
    • {
       "email": "foo@bar.com",
       "id": 2
    }
ERROR: 1 specs, 1 failures
in 0.653 seconds

I hope this is helpful to anyone using Guard::Jasmine with a desire to see better output. If you continue to follow my series, my next post will be a deeper dive into Jasmine testing of Spine.JS applications.

Resources

Rails & Spine.JS - Jasmine Testing Part 2

Tue, 17 Jan 2012 00:00:00 -0500

So this is the third part to my mini series on Rails and Spine.JS. Part one covers an initial setup and how to include Spine.JS into your Rails project while part two is actually the first of a tome on how to test your Spine.JS application. Assuming you have covered the bases there, lets get right down to business and review some of the elegant hacks ™ yours truly came up with while testing my own Spine.jS application using Guard and Jasminerice.

That Engine Is Gonna Need An Overhaul

So did I mention that Jasminerice is a Rails engine and that you can run your tests by accessing the /jasmine URL in your development environment's browser? Good. As uncool as browser testing is, sometimes that it useful. For instance console.log statements will not show up in your Guard's test output. But hopefully that is an edge case and you are really using guard-jasmine and watching your specs run in the terminal window. The question now becomes how can I leverage Jasminerice and its associated engine to really test my Spine.JS application with a full-fledged DOM? The answer is simple, let's hack the Jasminerice engine to load up our application while running the Jasmine specs.

A Review Of Jasmine And HTML Fixtures

This might be a bit contraversial (or maybe just wrong) but I would like to examine why we might hack the Jasminerice engine for a few minutes. Jasmine bills itself as behavior-driven development for JavaScript, to which I disagree. I think Jasmine first and foremost is an all-purpose testing framework. Depending on how you use it determines what it becomes. In my case, I have been doing a lot of unit testing of my Spine.JS models and supporting libs as I learn the framework and build my system. So my usage now could be called TDD at both a unit and functional level. Later on, I plan on doing higher level integration testing with Jasmine, at this time it will be my BDD tool. Normally I do not get caught up in symantics but I think it is important to understand a few lexical terms before I start showing off how I use Jasmine to test my Spine.JS application.

Now that I have set my higher order bit for Jasmine as my unit, functional and itegration testing framework for Spine.JS – I would like to show how my solution below might differ from other practices. Experienced Jasmine users rave about extensions that allow you to load HTML fixtures to functionally test units of code. In fact, the Jasminerice gem includes a custom version of the jasmine-jquery library which among other things allows you to do just that. Let me be clear on this, there is nothing wrong with that! As with all things software, proper solutions depends on what you are doing and need. In my case, I believe that testing your Spine.JS application with Rails only needs to hook into your existing application without the need for fixtures and excessive mocking and stubbing.

My basis for this argument is founded on a few principas that I believe all rich client side JavaScript applications should follow. Most important, the application is a single page load and all other calls to the server happen from that one page via AJAX. Every other part of your Rails application then becomes an API client to the JavaScript application. All views are client side only, most likely in a JST namespace.

Hacking Jasminerice

Turns out this is really easy since Jasminerice is just a simple Rails engine with few moving parts. It has a SpecController that renders a basic layout which requires your application's JavaScript assets along with Jasmine. The goal is to tell it to render your single application page that loads up your Spine.JS app with a few additional hacks. When done, we are going to have Jasminerice in your full control. This means we get the same CSS and JavaScript as our real application along with a solid foundation to extend Jasminerice at our whim. This leaves us with a clean canvas suitable for Jasmine unit, functional and integraiton testing all wrapped up into one.

# In: config/initializers/jasminerice.rb

module Jasminerice

  module MyApplication
    extend ActiveSupport::Concern
    included do
      delegate :foo_path, to: ::Rails.application.routes_url_helpers
    end
    def jasminerice?
      true
    end
    def current_user
      @current_user ||= User.find(1)
    end
  end

  ApplicationHelper.send :include, MyApplication

  class SpecController < Jasminerice::ApplicationController
    def index
      render template: 'users/home', layout: 'application'
    end
  end

end if defined?(Jasminerice) && Jasminerice.environments.include?(Rails.env)

There we go, a simple example pulled from my current project. What we are doing here is freedom-patching that spec controller's index action to render both a template and layout from your application. I also have a MyApplication module which I include in the Jasminerice::ApplicationHelper for a few methods that my application view code would expect. In this case a current_user method and some delegation of URL helpers to the root Rails application. Do not get hung up on what user is returned there too. It is mostly moot as all AJAX calls will be stubbed. Lastly, I made a jasminerice? view helper and a matching one that I have in my own ApplicationHelper which returns false. I will show why and how this all fits together later on.

Now that we have full control and some reflection for Jasminerice, let's reviw our appliation's main layout. Here is an example of my app/views/layouts/application.html.haml file.

!!! 5
%html{:lang => 'en'}
  %head
    %meta{:charset => 'utf-8'}
    = csrf_meta_tags
    %title My Application
    - if jasminerice?
      = stylesheet_link_tag 'spec'
      = javascript_include_tag 'spec'
    - else
      = stylesheet_link_tag 'application'
      = javascript_include_tag 'application'
      %script jQuery(function(){ MyApp.App.Home.init(); });
  %body
    %section{id: 'app', data: {id: current_user.id}}

It is really that simple! It has a single condition that says if Jasminerice is loading this page, use the top level spec CSS and JavaScript asset manifest. If not, render my main application's CSS and JavaScript manifests. You will also notice that I decouple my Spine's application initialization from the main JS files and only initialize the app on page load when not running in Jasminerce. This allows us to stub AJAX calls in specs then initialization the Spine.JS app when we are fully ready. So now, let us take a look at what both the spec/javascripts/spec.css and spec/javascripts/spec.js manifests may look like.

/*
 *= require jasmine
 *= require application
 */
.jasmine_reporter { display: none; }

#= require jasminerice
#= require mock-ajax
#= require application
#= require jasmine-myapp
#= require_tree ./lib/
#= require_tree ./models/
#= require_tree ./controllers/
#= require_tree ./views/

These should be self explanatory. The spec.css manifest requires the jasmine styles, then our application's styles. It then hides the Jasmine reporter element. The spec.js manifest does something similiar, It first requires the jasminerice manifest, a Jasmine helper called jasmine-ajax but poorly named mock-ajax, then your application's JavaScript followed by a personal Jasmine helper that we will discuss later. The last lines include all the spec files in each of the lib, models, controller and views directories located in the parent spec/javascripts directory.

Congratulations! You are now in full control of both Spine.JS and Jasmine(rice) and how your specs are executed. You have a fully styled DOM that you can happily ignore as you test your unit code. And when the time comes, you can leverage that and you full Spine.JS application from a high level integration perspective. Tune in for my last part which will give some working examples of how to test your Spine.JS application with Jasmine.

Resources

Rails & Spine.JS - Jasmine Testing Part 1

Mon, 16 Jan 2012 00:00:00 -0500

In my previous article I talked a little bit about why I decided to use Spine.JS and how to include the CoffeeScript source into your Rails project using git submodules. Now I would like to talk about testing your brand new Spine.JS application. Afterward, be sure to read the second part to this article which covers more advanced aspects of your Spine.JS application specs.

Testing JavaScript

OK, so like any good programmer, you want to test your JavaScript web application, but how? Like most, I kept finding that Jasmine was the de facto testing framework that most Rails developers were using. For the newly aquatinted, Jasmine describes itself as behavior-driven and sports a clean spec style using describe and it blocks similar to RSpec or MiniTest::Spec. But maybe you, like me, quickly dismissed Jasmine since you sure as hell were not going to hit refresh or F5 in your browser every time you wanted to run your damn specs. After all, this is 2012 and Rails developers do not test with a browser! So why should I start now?

Luckily I am a big fan of Ruby's Guard gem, a simple library that responds to file system events. The guard project literally has TONS of other guard gems that automate everything from running test files to restarting your development server. Thankfully my search for JavaScript testing with Guard in mind brought me right back to Jasmine. Enter the guard-jasmine gem and the wonderful world of a headless JavaScript testing piped right down to your terminal window!

Guard, Jasmine & Jasminerice

So this is our holy trinity and to be honest, there are a lot of moving parts under the stack. Things will seem to get complicated quick, but don't worry. I will give you a brief overview of the moving parts and then get right down to the basics of how you can start using Guard and Jasmine to test your Sine.JS application.

First, let's cover Guard. It is a simple gem that uses a Guardfile at the root of your project to control how other guards are triggered. I'll give you an example Guardfile later. But for starters, read the documentation on what special libraries may be needed for file system events or notifications on your specific platform. In my case, I us Mac OS X and purchased the latest Growl 1.3. So my example Gemfile below will have the ruby_gntp gem included in the spec.

Next up is the guard-jasmine gem. My instructions assume you are running a Rails 3.1 or 3.2 app and that you are taking full advantage of the asset pipeline and CoffeeScript. Many of these details can be found on the guard-jamine's Rails 3.1 setup section of their readme page. The underlying components of guard-jasmine are two a libs named Jasminerice and PhantomJS. Jasminerice is a simple Rails engine that brings in the Jasmine source files to the asset pipeline while running a rack app mounted to /jasmine to run your specs from your current application. PhantomJS is yet another headless WebKit based on Qt that has a rich JavaScript API which guard-jasmine delegates to. Is your head spinning? Mine was too.

Put It All Together

Here is the bullet train to getting this stack up and running. First, you will need to get PhantomJS installed. If Homebrew is your thing, just do $ brew install phantomjs. Or you can download one of their precompiled binaries for your specific platform. This is what I opted to do and I just placed the phantomjs in my PATH somewhere.

Next, we need to get the gems in our Gemfile. Here is how mine are setup. I have them in both the :develpment and :test groups since Jasminerice runs in both of those Rails environments. I also have that ruby_gntp dependency since I am using Growl on Mac OS X, YMMV.

group :development, :test do
  gem 'jasminerice'
  gem 'guard-jasmine'
  gem 'ruby_gntp'
end

So that was easy, now on to our Guardfile. Here is mine below. Notice how I put my JavaScript related guards into a js group? This is a seldom used feature of Guard and it means I can monitor only my jasmine specs by starting guard off using $ guard -g js and my other guards in my ruby group, like minitest, will be ignored.

group 'js' do

  guard 'jasmine' do
    watch(%r{app/assets/javascripts/myapp/index\.js\.coffee$})  { "spec/javascripts" }
    watch(%r{app/assets/javascripts/myapp/(.+)\.js\.coffee$})   { |m| "spec/javascripts/#{m[1]}_spec.js.coffee" }
    watch(%r{spec/javascripts/(.+)_spec\.js\.coffee$})          { |m| "spec/javascripts/#{m[1]}_spec.js.coffee" }
    watch(%r{spec/javascripts/spec\.js$})                       { "spec/javascripts" }
    watch(%r{spec/javascripts/spec_helper\.js$})                { "spec/javascripts" }
    watch(%r{spec/javascripts/jasmine-myapp.*})                 { "spec/javascripts" }
  end

end

This setup assumes a few things. First that you are only testing your Spine.JS application and that those files are in the app/assets/javascripts/myapp directory. That myapp directory could just be app in your case if you used the spine-rails gem without the --app option. In my case, that folder is named homemarks. This Guardfile also assumes that your JavaScript app and specs are CoffeeScript files and that specs are in the spec/javascripts folder specified by Jasminerice. You are going to want to follow some file naming convention now too. For example if you have a Spine.JS app file in app/assets/javascripts/myapp/models/post.js.coffee, then you are going to want the matching spec in spec/javascripts/models/post_spec.js.coffee. So saving each of those files will trigger that specific spec to run. There are also so some watchers on important root files like your spec.js sprockets manifest and your Spine.JS app index. Changing any of those files will result in your entire spec suite running again.

To Be Continued...

I will go into more details on the spec_helper.js and jasmine-myapp files above in the second part of this article. For now you should be set to start writing specs like the one below and seeing them run by either visiting the /jasmine URL of your running Rails application or by using Guard in your terminal window.

describe 'App', ->

  it 'sets el', ->
    expect(MyApp.Application.el).toEqual $('#app')

  it 'sets the userId as a property on itself', ->
    expect(MyApp.Application.userId).toEqual @bob.id

Continue to part two...

Resources

Rails & Spine.JS - Using The CoffeeScript Source

Sun, 15 Jan 2012 00:00:00 -0500

Our options for JavaScript MVC frameworks are numerous these days. While working on the third major rewrite of my personal bookmarking application, HomeMarks, I decided to learn Backbone.js. Thankfully a local friend of mine recommended that I try Spine.JS. I was immediately hooked!

Why Spine.JS?

Spine.JS is is authored in CoffeeScript and that is a big deal for me. I will never write raw JavaScript again, which I consider, the deprecated syntax. So a JavaScript MVC framework that is written in CoffeeScript means I can read the source, learn from it and even debug it if necessary. Sure, I can read raw JavaScript or just rely on documentation. But nothing beats reading source code. A practice I think good developers follow. So here is an example of the Backbone.js model source compared to reading Spine.JS model source and I think you will see the difference.

Source code legibilty is not the only reason to use Spine.JS. It is also very lightweight and requires no other JavaScript dependencies. Take for example Backbone.js which requies the use of Underscore.js and think for a moment why. Underscore.js makes mundane tasks in JavaScript like itterators and event binding much more friendly. But this is all moot when you are using CoffeeScipt's loops and comprehensions. And take my advice, CoffeeScript has so much more to offer. One of my personal favorites is the existential operator!

Coffee Time!

So, are you with me and want to try out Spine? Great! But do not rush in and use the spine-rails gem. Sure it gives you a nice way to require the Spine JavaScript files in the asset pipeline and some fancy generators. But when you break it down, there are better ways to get Spine's source files and who the hell uses generators? I mean, the only useful one is the initial rails g spine:new generator. Past the initial project setup, the spine-rails gem really does nothing but lock you down to an explicit version of Spine tied to that gem release.

I highly advise that new-comers to Spine start off with the spine-rails gem and its new project generator. Then quickly switch to just including the spine source using a git submodule. This will give you the benefit of using the source CoffeeScript files and tracking Spine's git repo which is getting good active development. So let's live on the edge and read some source. First uninstall the spine-rails gem if you have it and add the Spine project as a git submodule to your git repo.

$ mkdir -p vendor/assets/javascripts
$ git submodule add git://github.com/maccman/spine.git vendor/assets/javascripts/spine

This adds the Spine project to your vendor/assets/javascripts/spine directory. Which means it can now be leveraged by Rails asset pipeline using Sprockets. So if you had used the spine-rails generator above and had your spine requires in app/assets/javascripts/app/index.js.coffee, you would now be able to change what should have looked like this:

#= require spine
#= require spine/manager
#= require spine/ajax
#= require spine/route

To something like the following. Since the spine/src directory is where the source CoffeeScript files from our submodule above and Sprockets will render these just fine, it all just works!

#= require spine/src/spine
#= require spine/src/manager
#= require spine/src/ajax
#= require spine/src/route

So now you can easily update your Spine dependency using a simple git workflow with the added benefit that you can open up any of the source CoffeeScript files and learn Spine from the inside out. You can even change the source or put in console debugging to see what is happening and your application files will recompile via Sprockets on the next request.

Resources

LESS Is More - Using Twitter's Bootstrap In The Rails 3.1 Asset Pipeline

Mon, 26 Sep 2011 00:00:00 -0400

This weekend I decided to experiment with LESS CSS by replacing the existing Sass and Compass code that had been built thus far a small project. Why? Three basic reasons. First, I wanted to see how LESS stacked up. Second, I was intrigued by some of LESS' features, in particular their namespace support. Lastly, I wanted to use Twitter's Bootstrap project as a baseline for my design. Since Rails 3.1 has been out for some time, I was expecting the move to LESS to go considerably smoother than my pre-release attempts with Compass and Sass. I was wrong.

So the tilt gem already supported LESS templates and that means that Sprockets and the asset pipeline rendering technically did too. But there was no official less-rails gem that setup a standard configuration for other gems to hook into. This meant that libraries that distributed the Twitter Bootstrap assets were often hard to use or had inconsistent results. For example, it was not possible to load the Bootstrap's assets in such as way that let you build on top of the LESS mixins they had build. Nor were those assets namespaced in such a way that would allow you to have a modal.less asset as it would have conflicted with Bootstrap's version. So since I apparently have a bunch of free time and I hate building on top of a bad foundation, I set out to learn both Rails::Railtie and Rails::Engine to build the proper tools for LESS and Bootstrap in the Rails 3.1 asset pipeline.

New LESS Gems

I introduce you to both the new less-rails and less-rails-bootstrap gems. Take a look at the source code if you ever wanted to learn how to implement a simple Rails::Railtie or Engine. But assuming you want to use Twitter's Bootstrap, here are a few examples to get you started. First just bundle up to less-rails-bootstrap, it will automatically pull in less-rails and less.rb via their dependencies. BTW, I am going to keep the semantic versioning of the less-rails-bootstrap gem in sync with the major and minor versions of the Bootstrap project.

gem 'less-rails-bootstrap', '~> 1.3.0'

From here, the easiest way to use Bootstrap is to require it in your application.css file. Doing so will compile the complete LESS libraries files for Bootstrap. Note how we namespace the files using a simple directory structure.

/*
 *= require twitter/bootstrap
*/

#foo {
  /* Your styles... */
}

Alternatively, in a file with the .css.less extension, you can import the entire Bootstrap LESS framework. This will allow you to use Bootstrap's variables and mixins in your CSS that follows. Remember, unlike other CSS frameworks, requiring or importing Bootstrap will include all the CSS for building a bootstrapped website. If you only want variables or mixins, you will have to import those discreet files, see next section.

@import "twitter/bootstrap";

#foo {
  .border-radius(4px);
}

Maybe all you want to use are the variables and mixins that come with Twitter Bootstrap. No problem, just import them individually from you own .css.less file. In this case only the #foo selector is output.

@import "twitter/bootstrap/variables";
@import "twitter/bootstrap/mixins";

.myButton(@radius: 5px) {
  .border-radius(@radius);
}

#foo {
  .myButton(10px);
}

Using the Bootstrap JavaScript files is just as easy. Again, you can include all them with a single directive from your application.js file. Optionally, you can require only the files you need like require twitter/bootstrap/modal.

//= require twitter/bootstrap

$(document).ready(function(){
  //...
});

The less-rails project has already started getting some good feedback. Soon we hope to implement all the features that you may have used in the sass-rails project, like asset pipeline helpers. One last thing, I wanted to say thanks to Nicolas Mérouze for opening up the old less-rails gem space on rubygems.org for the new gem!

Resources

Keep Trying

Sat, 17 Sep 2011 00:00:00 -0400

One part of Objective-C that I like is being able to send messages to nil objects safely and more so their KVC and KVO patterns. In Ruby I often use the #try method to safely send messages to objects that may be nil at runtime. But one thing I always wanted was a nice way to send a key path, basically a string of methods signatures, to an object in the same way. I give you my simple #try_keypath method :)

class Object
  def try_keypath(methods, *args, &block)
    methods.to_s.split('.').inject(self) { |result, method| result.try(method) }
  end
end

class NilClass
  def try_keypath(*args)
    nil
  end
end

Yup, it is that simple. Let's see how this would work. Here are a few basic classes that randomly return nested objects. So staring with the Foo object, we have the possibility to get to some deeply nested info.

class Foo
  def bar
    rand(2) == 1 ? Bar.new : nil
  end
end

class Bar
  def batz
    'wohoo'
  end
  def deep
    Deep.new
  end
end

class Deep
  def info
    {:winning => true}
  end
end

Finally, here is how it would look and return different results. Man I love Ruby!

Foo.new.try :bar # => #<Bar:0x108d7dfd0>
Foo.new.try :bar # => #<Bar:0x108d7ddf0>
Foo.new.try :bar # => nil
Foo.new.try :bar # => nil
Foo.new.try :bar # => #<Bar:0x108d7d4b8>
Foo.new.try :bar # => nil

Foo.new.try_keypath 'bar.batz' # => "wohoo"
Foo.new.try_keypath 'bar.batz' # => "wohoo"
Foo.new.try_keypath 'bar.batz' # => nil
Foo.new.try_keypath 'bar.batz' # => nil
Foo.new.try_keypath 'bar.batz' # => nil
Foo.new.try_keypath 'bar.batz' # => "wohoo"

Foo.new.try_keypath 'bar.deep.info' # => nil
Foo.new.try_keypath 'bar.deep.info' # => {:winning=>true}
Foo.new.try_keypath 'bar.deep.info' # => nil
Foo.new.try_keypath 'bar.deep.info' # => nil
Foo.new.try_keypath 'bar.deep.info' # => {:winning=>true}
Foo.new.try_keypath 'bar.deep.info' # => nil

Revisiting My Design Past

Sun, 11 Sep 2011 00:00:00 -0400

Not many people know that I used to be a designer back in the mid 90's. Some of you all may have ventured deep into the colophon section of my site and read my resume which has visual clues of that history. Either way, this page is dedicated to those that had no idea a programmer like myself could come from a design background. Besides, I think it would be cool to share some of my old portfolio pieces.

Beer Labels

Every good starving artist needs a watering hole. Mine used to be the Tap House Grill At Ghent. Doing menu and beer tap design was a great way to pay for the basic staples of life, food and beer! Most of my friends worked there as well. Even after all these years, The Taphouse Grill still seems to be going strong in the same location. So here are a few of those beer tap designs. The "Sex Ina Canoe" tap was a joke against Yuengling Lager and referenced the joke, What does American beer and sex in canoe have in common?

Logos

On top of doing tons of 4-color brochure and package design, I used to love doing logo work. The one for the Virginia Stage Company was never used and the middle was a logo for my old online moniker before MetaSkills. The last logo was for a friend's Mother's clothing company with heavy art direction.

Drawing Collaborations

One thing I really enjoyed doing was working with illustrations by my friend Jeff Stephenson. He sketched tons of different things while making his way thru the day. I believe some of his drawings were purely cathartic or helped him cope in one way or another. Most times I would rescue them from the trash bin and scan them in for artwork. The first below is a t-shirt design for the Taphouse Grill's Cindo De Micro event. I remember piecing different illustrations together with some heavily inspired artwork I gathered while shopping for frozen burritos to eat that night. The end result was awesome. Lastly, what can I about Superfly Bundt Cackes? I think the wings came out nicely.

Experimental

Even designers have to perform occupational maintenance. I was always reading design articles (before blogs were invented) and learning the different techniques and power of PhotoShop 4 and its numerous plug-ins. Below is a mixed bag of those results.

Design Is Important

I think it is critical that every designer or programmer have a basic skill in the other field. It adds tons of value and streamlines communication across the disciplines. For me however, I have made a career decision that I will never open a PSD file again or try to facilitate the design role. I can if I have to, but not doing it all the time means that the time I do spend designing is not as efficient as someone else in that role. My place is now spent pushing pixels via code!

How Do You Encapsulate Your JavaScript

Tue, 06 Sep 2011 00:00:00 -0400

I ask this question a lot! To Job candidates, friends, and almost any developer that says they work with JavaScript. I believe how you encapsulate your JavaScript is a good indicator on your level of expertise with the language. I find that most beginners have come to JavaScript via jQuery and often define their functions at the top level namespace in some application.js file. These functions are loosely organized and often have no way of sharing simple object state and behavior. A good object oriented programmer would never write their application code like that. JavaScript should be no different and I think jQuery has a part in the blame of those not knowing this.

I've been writing rich object oriented JavaScript for the better part of 4 years now using Prototype.js. That framework has always championed encapsulating your web application's behavior using a standard class and inheritance notation. Sam Stephenson, the author of Prototype.js, did a good job trying to educate what real JavaScript Object Notation (JSON) is about and how you can encapsulate behavior using Prototype's Class structure while paying homage to Dean Edward's Base.js to which it owes its origins.

So jQuery is the proper winner and choice for anyone wanting to use a JavaScript framework to make working with the DOM in web sites easier. It has awesome event handling and delegation. So many things that Prototype.js lacked. It even has great documentation. About the only bad thing about jQuery is the widely different interfaces to it's AJAX functions and the arguments passed to their callbacks. Oh, and they totally screwed new JavaScript developers by not sheperding them into learning some sort of way to encapsulate their object behavior. So this article is for those that are not using CoffeeScript's class system, Backbone.js's class system, or jQuery's deeply nested plugin class system that is hidden away. So lets get to fixing that. Here is 32 lines of dirt simple JavaScript inheritance.

/* Simple JavaScript Class
   By John Resig. MIT Licensed. <http://ejohn.org/blog/simple-javascript-inheritance/> */
(function(){
  var initializing = false, fnTest = /xyz/.test(function(){xyz;}) ? /\b_super\b/ : /.*/;
  this.JQClass = function(){};
  JQClass.extend = function(prop) {
    var _super = this.prototype;
    initializing = true;
    var prototype = new this();
    initializing = false;
    for (var name in prop) {
      prototype[name] = typeof prop[name] == "function" &&
        typeof _super[name] == "function" && fnTest.test(prop[name]) ?
        (function(name, fn){
          return function() {
            var tmp = this._super;
            this._super = _super[name];
            var ret = fn.apply(this, arguments);
            this._super = tmp;
            return ret;
          };
        })(name, prop[name]) :
        prop[name];
    }
    function JQClass() {
      if ( !initializing && this.init )
        this.init.apply(this, arguments);
    }
    JQClass.prototype = prototype;
    JQClass.prototype.constructor = JQClass;
    JQClass.extend = arguments.callee;
    return JQClass;
  };
})();

It is worth reading the comments John Resig's blog post for this piece of work as well as the full documentation. My code example above names this JQClass to avoid namespace collisions with other frameworks. Here is a simple example of its usage.

// Your namespace.

var MyApp = {
  models: {}
};

// Simple object example for page flash.

MyApp.models.flash = JQClass.extend({

  init: function(aPage) {
    this.page = $(aPage);
    this.alertDiv = this.page.find('div.flash.alert');
    this.noticeDiv = this.page.find('div.flash.notice');
  },

  alert:  function(content) { this._doFlashFor(this.alertDiv,content); },

  notice: function(content) { this._doFlashFor(this.noticeDiv,content); },

  clear: function() {
    # ...
  },

  _doFlashFor: function(flash, content) {
    $.mobile.silentScroll(0);
    this.clear();
    flash.html(content);
    flash.show();
  }

});

This code example is pulled from a jQuery mobile project I recently finished. The real usage of the flash objects would be numerous as there would be one per page and I have a top level delegate object that finds the active page and finds or create the flash DOM elements. But this should be a good enough example to show why you might choose a simple class and object inheritance system vs putting all your logic in $(document).ready({}) scopes. There are tons of ways to write good OO JavaScript and I hope some find this useful and explore some other ways.

jQuery Mobile & Rails

Wed, 24 Aug 2011 00:00:00 -0400

I just finished my first dive into using jQuery Mobile with a Rails application and wanted to share some techniques that came out along the way. Hopefully these will help you if your are using jQuery Mobile with Rails or want to test your mobile application's integration layer. This post assumes you are somewhat familiar with jQuery Mobile and its basic concepts. So let's jump right in with a series of helpful tips.

A Mobile Layout

In my application, I decided to use use a route namespace of "mobile" vs a sub-domain for all controllers and views to reside in. Do what works for you, but I find that using namespaces both helps keep my code organized and simple to maintain. Either way, you should have a layout specific for your mobile application. Mine is in app/views/layout/mobile.html.haml. As you can see here, I use HAML, so all view examples will be using it vs ERB. Here is a general layout.

!!! 5
%html{:lang => 'en'}
  %head
    %meta{:charset => 'utf-8'}
    %meta{:name => 'viewport', :content => 'width=device-width, initial-scale=1'}
    %meta{:name => 'format-detection', :content => 'telephone=no'}
    %title= 'My Mobile App'
    %link{:rel => 'stylesheet', :href => 'http://code.jquery.com/mobile/1.0b2/jquery.mobile-1.0b2.min.css'}
    %script{:src => 'http://code.jquery.com/jquery-1.6.2.min.js'}
    %script{:src => 'http://code.jquery.com/mobile/1.0b2/jquery.mobile-1.0b2.min.js'}
  %body
    = yield(:layout)

A few key points here. First, the meta tag for the viewport is recommended as a good base. The other meta tag for format-detection is to disable automatic detection and linking of phone numbers. Phone number detection is way too aggressive and often just links random numbers with periods and hyphens. This means if you want a phone number to call when touched, you will have to use the tel: link format with the phone number afterward. I recommend the aid of something like the phony gem for validating and parsing phone number formats. Lastly, the title attribute in the head is really moot. The jQuery Mobile framework will dynamically change the page title as new page DOM elements are loaded in.

Mobile Page IDs

A typical jQuery Mobile app will have one full page load. All other pages thereafter are loaded via AJAX and inserted into the DOM. Their docs suggest that every page (and form) have a unique id attribute. These ids can be used to link to the DOM page element when navigating around preloaded pages. Simple apps can get by with a few ids like "#home", "#contact_us", "#etc". But if you have a large application, you need a better system to keep track of things.

Thankfully if you are building your Rails applications in a RESTful manner, it is easy to leverage the route link helpers to generate your page ids. In the example below, I created a method called mobile_page_id that I placed into a shared concerns module directory. From here I mix this into my test helper, and application's view helper. It is that damn useful! You are going to want to use this everywhere!

# File: app/concerns/mobile_concerns.rb
module MobileConcerns
  module Helpers
    def mobile_page_id(path = request.path)
      path.sub(/\A\/mobile\//,'').gsub("/",'_')
    end
  end
end

# Mix into your applications helper.
module ApplicationHelper
  include MobileConcerns::Helpers
end

# Mix into your test helper of choice.
class MobileStoriesTest < ActionController::IntegrationTest
  include MobileConcerns::Helpers
end

The mobile_page_id method is primarily for views. I will cover its usage in testing further down. When called without an argument, it will take the current request path and translate it to a string suitable for a page id. So if you were rendering a /mobile/users/10/avatar page, the id would be users_10_avatar. If your application follows RESTful resources in your routes, this can pay dividends. When needed, you can pass a *_path helper method to get the same id. In this example the same id would come back for mobile_page_id(mobile_user_avatar_path(@user)). Remember, I am using a "mobile" namespace in my examples and hence my method above strips that out, flavor this helper to your needs. Here is an example of what you might find in app/views/users/avatar.html.haml using the mobile_page_id. You can see here that I am also setting a title local that is used in the %h1 tag and the data-title page element attribute. Doing this will show the same title in the header bar as well as the page title.

- title = "User Profile"

%div{:id => mobile_page_id, :data => {:role => 'page', :title => title}}

  %div{:data => {:role => 'header'}}
    %h1= title
  %div{:data => {:role => 'content'}}
    edit your profile...
  %div{:data => {:role => 'footer'}}
    /...

Easy Data Attributes

The jQuery Mobile framework will have you typing a lot of data-* attributes into your elements. It uses these from basic page behavior to UI themes. They are literally needed everywhere. If your typing out raw HTML these data attributes get old pretty quick. Thankfully both HAML and the latest Rails 3.1 tag helpers can keep things clean. Both allow hashes to be passed to the data attribute. Keys are dasherized and values are JSON-encoded except for string and symbols.

# HAML Shorthand For:
# <div id="foo" data-role="page" data-title="Title" data-rel="dialog"></div>

%div{:id => 'foo', :data => {:role => 'page', :title => 'Title', :rel => 'dialog'}}

# Rails 3.1 Tag Helpers
# <div data-name="Stephen" data-city-state="[&quot;Chicago&quot;,&quot;IL&quot;]" />

tag("div", :data => {:name => 'Stephen', :city_state => %w(Chicago IL)})

Dynamically Setting Layout

Remember how jQuery Mobile only loads the entire page once and then inserts all following pages using AJAX? This means that views after the initial page load do not need the layout when rendered. Depending on the setup of your mobile application, it could become a chore telling each action to conditionally render the layout or not. Imagine a page refresh or a user bookmarking a deep resource node of your mobile app. Thankfully with a little Rails-fu, we can conditionally set the layout to false if the request is an HTTP GET from an AJAX request. Below is an example of my base mobile controller that all mobile controllers inherit from. Now every action is able to load the entire mobile layout or not.

class Mobile::BaseController < ApplicationController

  LAYOUT = 'mobile'.freeze
  layout LAYOUT
  around_filter :dynamically_assign_layout

  private

  def dynamically_assign_layout
    self.class.layout false if request.get? && request.xhr?
    yield
  ensure
    self.class.layout LAYOUT
  end

end

Integration Testing & Capybara-Webkit

Those that know me are familiar that I do not use RSpec or Test::Unit but instead opt for a simple testing framework built into Ruby 1.9, MiniTest::Spec. I also use the Capybara-WebKit driver for my acceptance testing in both Rails 2.3 and Rails 3.1's standard integration test layer. Reference the integration_test_helper.rb file in each link above to learn how to use Capybara-WebKit with your rails app. Thanks to Wyatt Greene for his original article on the matter.

So why the fuss? Well Capybara-WebKit is a headless WebKit browser that you can direct right from your test suite. What makes it so awesome is that it renders web pages with full JavaScript support and is much faster than selenium based Capybara drivers. Since jQuery Mobile is entirely based on JavaScript with HTML & CSS3 - Capybara-WebKit is a perfect candidate to acceptance test your mobile application. The only gotcha is scoping your Capybara actions to a certain page that is dynamically loaded into the DOM. No problem! This is easily solvable using the page ids from the mobile_page_id helper method I mentioned above. So whatever your testing framework, here are a few helpers that are critical. Assume these are mixed into your test helper.

private

include MobileConcerns::Helpers # Mentioned above.

def current_page_id(path=nil)
  path = path || current_path
  "div##{mobile_page_id(path)}.ui-page-active"
end

def current_page(path=nil)
  find(current_page_id(path))
end

def within_current_page(path=nil)
  within(current_page_id(path)) { yield }
end

So let's go over these. First is the current_page_id method. Most of the time you are going to pass a path argument to this method, since the Capybara's current_path will only work correctly on the first page load, not each following AJAX page load which would change the location hash. See how this is using the mobile_page_id helper mixed in and described above? Next is the current_page helper. It finds the passed path/id with Capybara's find method. The within_current_page leverages Capybara's within helper to scope your action to that particular DOM element. Here is a classic example using these.

should 'be able to navigate to logged in user page and change email' do
  login_as @user
  visit mobile_homepage_path
  click_on 'My Account'
  user_path = mobile_user_path(@user)
  assert current_page(user_path)
  # Change email
  new_email = Forgery(:email).address
  within_current_page(user_path) do
    fill_in 'email', :with => new_email
    click_button 'Save Changes'
  end
  @user.reload.email.must_equal new_email
end

Hopefully you can see how this simple page id foundation can help you better test your jQuery Mobile app with Capybara-WebKit. What an awesome tool! I sometimes find it hard to believe we are now at the point where we can easily test this much JavaScript in a headless browser directed by Ruby.

Resources

Use Compass Sass Framework Files With The Rails 3.1.0.rc5 Asset Pipeline

Fri, 29 Jul 2011 00:00:00 -0400

This is a simple update to my original article for using compass's .scss files with the asset pipeline. This assumes you had this setup working in Rails 3.1.0.rc4, but should be helpful to anybody. The good news is that most everything is wired up.

First, you are going to need to update your Gemfile. Follow the latest conventions and make sure you have a :assets group. Use the example below. With the exception of the compass addition to their "rails31" branch, this is what a Rails 3.1.0.rc5 project would generate. It is a good idea to put all your asset pipeline related gems in this group. As the comment says, these would not be required for production as the assumption is would precompile your static assets on deploys. Note, make sure your bundle for sass-rails installs the matching rc5 too.

# Gems used only for assets and not required
# in production environments by default.
group :assets do
  gem 'sass-rails', "~> 3.1.0.rc"
  gem 'coffee-rails', "~> 3.1.0.rc"
  gem 'uglifier'
  gem 'compass', :git => 'git://github.com/chriseppstein/compass.git', :branch => 'rails31'
end

Now here is a crazy gotcha. Rails 3.1.0.rc5 has changed the application.rb bundler line a little to account for this new :assets group. Find the bundler line and make sure it looks like this.

Bundler.require *Rails.groups(:assets) if defined?(Bundler)

Lastly, we need to get the compass files in the the load path. The sass-rails gem has set us up for this. I choose to hook into it using my config/initializers/sass.rb like so. I imagine that compass will be doing this on its own at some point in the future too. After this you should be all set!

Rails.configuration.sass.tap do |config|
  config.load_paths << "#{Gem.loaded_specs['compass'].full_gem_path}/frameworks/compass/stylesheets"
end

Free The Enterprise With Ruby & Master Your Own Domain

Fri, 01 Jul 2011 00:00:00 -0400

For anyone that did not attend the first MADExpo (Mid-Atlantic Developer Expo) this week, here are my slides posted on slideshare. This talk centers around 3 basic sections.

The first is general Open Source and what it can do for you as a software developer to increase your skills and general knowledge. Second, it covers some key open source projects I have been working on. Specifically for SQL Server and and ActiveRecord which includes projects like TinyTDS and the latest accomplishments the SQL Server adapter had done for prepared statement support in Rails 3.1.

Lastly, I go over what tools have recently been developed specifically for the Windows community to make using Ruby & Rails an easy experience. One is the great Rails Installer project by Wayne E. Sequin and Luis Lavena. Basically a one click installer for everything from Git, Ruby, Rails, MySQL, TinyTDS & the SQL Server adapter. I even did a new Rails 3.1 project called AdventureWorks.Ruby which shows you how to get up and running with your legacy SQL Server databases. Specifically the hardest point, cloning your legacy schema to a fresh test database via the standard Rails' rake tasks.

Bind jQuery Event Handlers To This Object With CoffeeScript

Sun, 22 May 2011 00:00:00 -0400

Friends have told me that rich domain objects are seldom wielded when using jQuery to enhance behavior on web pages. I myself have always loved JavaScript as a rich dynamic language first and something for the DOM second. Hence most of my client-side JavaScript follows a robust object-oriented approach similar to Ruby. This is the main reason I have used Prototype.js for so long.

Since Rails announced both jQuery and CoffeeScript as the defaults in version 3.1, I decided it was high time I starting learning both. I had always known that jQuery bound the this keyword in event handlers to the DOM object. Something I found totally confusing and unacceptable to someone dealing with their own objects to encapsulate behavior. Today I found two ways of dealing with my problem, a jQuery way and a CoffeeScript way. First a code example.

class MyObject

  constructor: ->
    @myDomElement = $('#myDomElement')
    @._initBehavior()

  handler: (event) ->
    console.log(this)
    false

  _initBehavior: ->
    $(window).resize @handler

jQuery ->
  window.myObject = new MyObject();

Here is a simple class using CoffeeScript's methods. It initializes itself with a static property this.myDomElement to some DOM element on the page with an id of "myDomElement". It then attaches an event handler to the window's resize event and logs this along the way. Simple stuff, the only problem will be that the object logged will not be an instance of MyObject, but the raw DOM element, in this case the window object. One way of fixing this is to use jQuery's proxy function like so

_initBehavior: ->
  $(window).resize jQuery.proxy(@handler,this)

This works, but seems a little verbose to me and can clutter up your event initialization code. The other way is to use CoffeeScript's fat arrow operator. An excerpt from their project page explains it well.

The fat arrow => can be used to both define a function, and to bind it to the current value of this, right on the spot. This is helpful when using callback-based libraries like Prototype or jQuery, for creating iterator functions to pass to each, or event-handler functions to use with bind. Functions created with the fat arrow are able to access properties of the this where they're defined.

So all we have to do is change -> to => for any of our callbacks or event handlers and now this is our own object and not the DOM element. Hot damn!

class MyObject

  constructor: ->
    @myDomElement = $('#myDomElement')
    @._initBehavior()

  handler: (event) =>
    console.log(this)
    false

  _initBehavior: ->
    $(window).resize @handler

jQuery ->
  window.myObject = new MyObject();

The Browser Is Dead?

Sat, 21 May 2011 00:00:00 -0400

This twitter post by Dave Thomas sparked an interesting back and forth with DHH on how Rails 3.1 could be more opinionated towards web development for the browser. A short time before – it was announced that Rails would include CoffeeScript and Sass as defaults for JavaScript and CSS authoring. FWIW, both of these new defaults are in my opinion the best of the breed fore each task. If you have not done so, I suggest taking a quick read down this thread that I put together with screenshots from my Twitter.app.

The topic is one I have thought often about over the past 6 years. I started writing Rails applications before 1.0 was released and I have seen the changes and felt each one. Having written both web and native iOS apps for HomeMarks.com and retooling the whole application for each major Rails version, I think it gives me a good insight into the pain here. For instance when RJS in Rails first came out, HomeMarks took extreme advantage of it in very clever ways. Pushing rendered partials out and communicating to the browser's JavaScript environment was fun and rewarding at the time. It also felt really DRY at times.

As I started to learn more advanced JavaScript techniques for organizing my rich client code, I ended up rewriting HomeMarks again under the poorly named moniker "The AJAX Head Design Pattern". It even caught a little bit of attention by the Ajaxian and Chris Williams. Essentially what I ended up building was my own minimal backbone.js in that I had a rich MVC layer all written in JavaScript. This usage of Rails as an API foundation on top of a rich JavaScript application was in my opinion a few years ahead of the average curve. Though it was very rewarding from a JavaScript as a language first, DOM second development perspective & it yielded a very clean way to leverage the same RESTful controller actions used by JavaScript in the browser with a native iOS app that integrated with Core Data, it still felt like an over achievement or harder than it had to be.

So as I embark on my third complete rewrite of HomeMarks, I have to think about my tools and techniques again and what the end goal may look like. I have thrown away thousands of lines of OO-JavaScript written in Prototype.js and started rewriting them in CoffeeScript. Though there are still some tough decisions to be made on what my complete stack will look like and how HomeMarks v3 will hopefully again push the new web forward, I cant help but to agree with both Dave and DHH. I also share James A Rosen's opinion summed up in this tweet.

If I had known that RailsConf would be How-to-Use-Rails-as-a-Backend-for-Javascript-Conf, I would've gone!

Use Compass Sass Framework Files With The Rails 3.1 Asset Pipeline

Wed, 18 May 2011 00:00:00 -0400

The Sprockets 2 gem along with the Tilt gem make it really easy to write JavaScript or CSS using any templating language you desire. The rails defaults are CoffeeScript and Sass. About the best collection of Sass framework files for easy cross-browser CSS authoring are packaged in the compass framework. Compass even has great documentation for using their Sass framework. But what if you want to use those within the asset pipeline provided by Rails? Easy enough!

First, bundle up the compass, but do not require it. Add this to your Gemfile.

gem 'compass', :require => false

Next, add a Sass initializer in config/initializers/sass.rb and fill it in with the code below. This will add two more load paths to the Sass engine. The first is your default rails/sprockets asset path for stylesheets. It simply let's you build a deep folder structure in that directory and use relative paths from each file. The second will put the entire compass Sass framework files into the Sass load path.

Sass::Engine::DEFAULT_OPTIONS[:load_paths].tap do |load_paths|
  load_paths << "#{Rails.root}/app/assets/stylesheets"
  load_paths << "#{Gem.loaded_specs['compass'].full_gem_path}/frameworks/compass/stylesheets"
end

Now in your rails app/assets/stylesheets/foo.scss file you can use Sass' @import with paths to the compass framework.

@import "compass/css3/opacity";
#mylogo { @include opacity(0.5); }

That is an example loading up the opacity helpers. Your generated css file will look like this! CSS is never going to be the same again!

#mylogo {
  -ms-filter: "progid:DXImageTransform.Microsoft.Alpha(Opacity=50)";
  filter: progid:DXImageTransform.Microsoft.Alpha(Opacity=50);
  opacity: 0.5; }

Speaking At MADExpo

Fri, 06 May 2011 00:00:00 -0400

Over the past few months I have been actively working with both Luis Lavena and Wayne E Seguin to incorporate both TinyTDS and the SQL Server Adapter into the latest release of the Rails Installer for Windows. Like them, I firmly believe that Ruby's success is tightly bound to how well we can bring these tools to the Windows platform. So despite that I have no love for the Windows – I do want to see Ruby succeed and help spread the gospel. To this end, I have been working really hard at making both TinyTDS and the adapter a superb backend for ActiveRecord. In fact today, I just released a 3.1 beta that takes advantage of the new prepared statement support.

To go one step further in helping Ruby, I have decided to speak at a local conference called the Mid Atlantic Developer Expo or just MADExpo. The details for my talk are below and if you find any of the other talks interesting, I highly advise you to grab a ticket and help me bring the message of Ruby right to the belly of the beast!

Free The Enterprise With Ruby & Master Your Own Domain

On the heals of Luis Lavena's RailsConf talk "Infiltrating Ruby Onto The Enterprise Death Star Using Guerilla Tactics" comes a local and frank talk about the current state of Open Source Software (OSS) participation from Windows developers. Learn what OSS is, what motivates its contributors, and how OSS can make you a stronger developer. Be prepared to fall in love with writing software again!

We will start off with a 101 introduction to both the Ruby programming language and the Ruby on Rails web application framework. You will learn about ActiveRecord, a powerful ORM that maps rich objects to your databases, and the latest components to use it with SQL Server. As a Rails core contributor and author of the SQL Server stack, I will give you a modern insight into both that will allow you to leverage your legacy data with Ruby.

Lastly, I will review the bleeding edge tools being actively created for Windows developers to ease the transition to Ruby, Rails and OSS from a POSIX driven world. Many things have changed. It is time to learn and perform some occupational maintenance.

Using MiniTest::Spec With Rails

Sat, 26 Mar 2011 00:00:00 -0400

So after a few blog post on the subject of MiniTest::Spec, I finally have a simple solution for Rails 3 that leverages MiniTest's spec DSL. I introduce to you the minitest-spec-rails gem. MiniTestSpecRails defines a Test::Unit::TestCase class that subclasses MiniTest::Spec. It implements only what is needed to make Rails happy.

Once you bundle the gem in your Rails application, it will satisfy the require "test/unit/testcase" from ActiveSupport's test case. Tricking it to use MiniTest::Spec instead of MiniTest::Unit. Here is an example Gemfile that shows the usage of MiniTestSpecRails. That is all you have to do!

group :test do
  gem 'minitest-spec-rails'
end

Advantages?

Since MinitTest::Spec is built on top of MiniTest::Unit which is technically a replacement for Test::Unit, there is not a lot that can go wrong. With minitest-spec-rails, we finally have a working solution by replacing MiniTest::Spec as the superclass for ActiveSupport::TestCase. This solution is drop dead simple and does not require you to recreate a new test case in your test_helper.rb or to use generators supplied by gems like minitest-rails.

This means two things. First, older Rails applications can switch to MiniTest::Spec now by simply installing this gem. Second, as Rails itself evolves to eventually leverage MiniTest::Spec, your test case code will not have to change. About the only gotcha is a few missing assertions available in Test::Unit that are changed or no longer available in MiniTest. For example, assert_raise vs assert_raises and there is no such thing as assert_nothing_raised. Code you would have had to change eventually anyway.

New Assertion Styles

One thing that I try to do is to change my test assertions from the old Test::Unit style to the new MiniTest::Spec style. I commonly use this cheat sheet to remember them. For instance:

# This:
assert_not_nil @foo.bar

# Would Become This:
@foo.bar.wont_be_nil

Likewise, MiniTest::Spec is consistently getting new features added. One patch that I advocated for now allows the must_be and wont_be low-level assertions to work with a predicate method symbol. Meaning, if the symbol ends in a question mark, then it will implicitly call that mehod on the subject for truthy or falsy matching. For instance:

@user.must_be :valid?

Known Issues

None that I know of yet. Try it out and report any issues to our github project page. If you have been using minitest-spec-rails, I would love to hear about it too!

Resources

MiniTest::Spec For Rails - Drop in MiniTest::Spec support for Rails 3.
MiniBacktrace - Make the Rails backtrace cleaner work.
MiniTest Rails - Official minitest project for Rails that uses generators.

MiniBacktrace - For MiniTest & Rails

Tue, 22 Mar 2011 00:00:00 -0400

MiniBacktrace allows you to take advantage of the Rails.backtrace_cleaner when using MiniTest. This includes everyone using Rails 3 with Ruby 1.9.

Just add 'mini_backtrace' to your Gemfile's :test group and your should automatically see a huge difference. Any additions to the Rails.backtrace_cleaner should now work.

Rails.backtrace_cleaner.add_silencer { |line| line =~ /my_noisy_library/ }

PDFKit Overview & Advanced Usage

Sun, 20 Mar 2011 00:00:00 -0400

Last week I had the pleasure of rewriting 4 years of legacy PDF::Writer code to PDFKit. Why? Well drawing pdfs in ruby using libraries like PDF::Writer is like composing a webpage with an Etch A Sketch. In short, its a damn chore that involves a bunch of code that mixes both data and presentation. Sure there are gems like Prawn that make this much easier, but nothing beats drawing your pdf code in native HTML and CSS, and that is where wkhtmltopdf comes in.

Wkhtmltopdf is an open source project that uses the WebKit rendering engine to convert HTML to native PDF code. This is the muscle behind the PDFKit gem and other projects like WickedPdf. In this article I am only going to focus on PDFKit with Rails. But many topics will apply to both PDFKit and WickedPdf since they use wkhtmltopdf on the backside.

Installation

Installing the PDFKit gem is a no brainer. The hard part is getting the wkhtmltopdf binaries for you platform installed. Thankfully the google project page hosts a batch of static binaries that work on just about every platform. So go to their download page and pick the statically compiled binary that meets your needs. I highly suggest that you get the latest 0.10.0rc2 since some topics below take advantage of recent bug fixes. I have tested both the OSX and i386 on RHEL with success and the release candidate seems very production ready. I suggest placing wkhtmltopdf in /usr/local/bin/wkhtmltopdf.

$ which wkhtmltopdf
/usr/local/bin/wkhtmltopdf

The Basic Requirements

I knew that HTML to PDF generation has its drawbacks, specifically with common headers/footers and page breaks. I happily found out that wkhtmltopdf has a solution for all these problems and can layout PDF pages with pixel perfect precision. So let's skip over the basics and get right down to using PDFKit like a pro. We are going to build out the Rails HTML/CSS layouts and templates that will solve a series of common problems.

The major reason to use PDFKit and wkhtmltopdf is that we can use the same templating system in Rails that we use to generate other views. This means that we test our PDF view code just like any other Rails code using its built-in functional or integration test cases. Let me say that again, we can TEST our PDF code! A huge win if you have complex conditional view code. So let's get to it.

The Main PDF Layout

Sometimes it is useful to start at the end. So the first thing we need is a new layout for all of our pdf templates to use. Here is a HAML file that I recommend you name app/views/layouts/pdf.html.pdf.haml. Did you see that name? This is important! because Rails allows us to specify template names that can service more than one mime type format. So in this case, the layout will be found when rendering both HTML and .pdf formats.

!!! 5
%html{:lang => 'en'}
  %head
    %meta{:charset => 'utf-8'}
    %meta{:name => 'pdfkit-footer_html', :content => pdf_footer_url}
    = stylesheet_link_tag "#{request.protocol}#{request.host_with_port}/stylesheets/pdf.css"
  %body
    = yield(:layout)

I recommend that all PDF layouts, templates, and partials use the dual mime type naming convention. This will allow you to test your rendered HTML to PDF view code using common DOM techniques. Like assert_select in rails or maybe capybara's has_selector?. So given that you may have a print action for your orders controller, you would use print_orders_path(@order) for your functional tests with DOM assertions and print_orders_path(@order,:format=>:pdf) for real world usage and/or integration tests. Both formats will render the same partials, templates and layouts if you use that naming structure.

So that layout file is a real basic HTML5 doctype (which WebKit handles just fine) plus a few special elements. I'll cover that pdfkit-footer_html meta tag later on. For now, let's focus on that pdf.css stylesheet.

Your PDF CSS

You might be tempted to utilize your existing site's stylesheets for a base and then use media/print techniques to override and customize your printed versions. I am of the opinion that your PDF stylesheets should be very basic and easy to layout. To this end, I highly suggest that you start with an HTML reset CSS. In the example below, I have used Yahoo's CSS reset. This makes it so that every bit of layout is under your strict control with a common rendering of no margin or padding to throw you off.

/* Reset CSS. http://yui.yahooapis.com/3.2.0/build/cssreset/reset-min.css  */
html{color:#000;background:#FFF;}body,div,dl,dt,dd,ul,ol,li,h1,h2,h3,h4,h5,h6,
pre,code,form,fieldset,legend,input,textarea,p,blockquote,th,td{margin:0;padding:0;}
table{border-collapse:collapse;border-spacing:0;}fieldset,img{border:0;}address,caption,
cite,code,dfn,em,strong,th,var{font-style:normal;font-weight:normal;}li{list-style:none;}
caption,th{text-align:left;}h1,h2,h3,h4,h5,h6{font-size:100%;font-weight:normal;}
q:before,q:after{content:'';}abbr,acronym{border:0;font-variant:normal;}
sup{vertical-align:text-top;}sub{vertical-align:text-bottom;}input,textarea,
select{font-family:inherit;font-size:inherit;font-weight:inherit;}input,textarea,
select{*font-size:100%;}legend{color:#000;}

/* Your Base Foundation */
html,body { font-family: sans-serif; font-size:12px; }
h1 { font-size:18px; }
h2 { font-size:16px; }
h3 { font-size:14px; }
h4 { font-size:12px; }
strong { font-weight:900 !important; }
hr     { border:0; margin:0; padding:0; height:1px; color:#000; background-color:#000; }

/* Page Breaks */
.pb_before { page-break-before:always !important; }
.pb_after  { page-break-after:always !important; }
.pbi_avoid { page-break-inside:avoid !important; }

The second section of the CSS above is the place where you can put in a few custom styles that fit your needs. In my example I set a series of header font sizes, a base sans-serif font face and an hr tag that can be used as a simple rule. Feel free to add others here like basic styles for data tables, etc. The last section of the CSS file above are page break helpers. The latest version of wkhtmltopdf never breaks text in the middle of the line anymore. So most of the time the default page break behavior will work fine. But for those situations where you need more control, these 3 CSS declarations will serve most of your needs. Let's take a look at a few examples of their usage. Full details on CSS paged media can be found on the W3C's site.

Use the .pbi_avoid class on any block level element that you want to make sure is never broken across multiple pages. A great usage would be on each element of an orders line items. It can also be used on any large page element that will certainly fit on one page, but should never be broken up. This is perfect in places where you might have measured the remaining page space before drawing said element. The .pb_before class will always break to a new page. I found this very useful when printing composite PDF files that combined multiple other PDF actions. So here is another HAML template that renders 3 other PDF full page partials. Each partial can be 1 to many pages. By enclosing each in a

tag that forces a new page break makes sure that we always start a new page when rendering each document.

%div.pb_before
  = render :partial => 'pdf/orders/print'
%div.pb_before
  = render :partial => 'pdf/orders/invoice'
%div.pb_before
  = render :partial => 'pdf/orders/picklist'

Custom Headers/Footers

PDFKit and specifically wkhtmltopdf handles common page headers and footers just wonderfully, though it did take me some time to figure it out. I'll try to spare you the same pain by detailing the process for a custom footer on each page. In this example we will expect that our custom footer will be approximately .2 inches tall with a current page number next to a total page count.

Remember that pdfkit-footer_html meta tag in the pdf layout above? If not, here it is again.

%meta{:name => 'pdfkit-footer_html', :content => pdf_footer_url}

So what is going on here? Two things really. The first is a way for PDFKit to customize the command arguments passed down to wkhtmltopdf when the page is converted. PDFKit will take any meta tag with a name prefixed using "pdfkit-" and pass down the content attribute as the value to the suffix of the name attribute. In this case --foter-html http://myapp.com/pdf/footer will be used as a command argument to wkhtmltopdf when rendering templates using that layout file. Note, it is important to use fully qualified URLs for header and footer arguments.

When it comes to headers and footers, wkhtmltopdf takes the URL to an HTML page, renders it to native PDF code and embeds it automatically for you below or above your page margin. You can control the placement of these in one of two ways. The first is by adjusting the layout of the header/footer HTML page. The second is by adjusting the margin of the parent document. In my case, since I knew my footer was around .2 inches tall, I gave it's template an internal top margin of 10 pixels and told PDFKit to increase my global .5 inch page margins by .2 inches for the bottom margin using a rails initializer.

PDFKit.configure do |config|
  config.default_options = {
    :page_size     => 'Letter',
    :margin_top    => '0.5in',
    :margin_right  => '0.5in',
    :margin_bottom => '0.7in',
    :margin_left   => '0.5in'
  }
end

So now I know that whatever content I render in my http://myapp.com/pdf/footer page will fit just nicely on the bottom of each page. But how to generate that content and the custom page numbers? First, let's make a single pdf resource in our rails route file with a collection action for #footer. Now here is a controller for that resource with a single footer action.

class PdfController < ApplicationController

  def self.perform_caching ; true ; end
  caches_page :footer

  def footer
    render :layout => false
  end

  private

  def perform_caching
    true
  end

end

There is not much here past rendering a basic template file with no layout. All the rest is to achieve an important set of caching rules. Ideally the URL argument to --footer-html would be a static HTML file. However, if want to use Rails templating to render that file, it is important to cache the results. The parent document will request this URL for each page it renders, so you can see how one process could deadlock another if your were not careful. In my example above, I override ActionController's perform_caching class and instance methods so that all actions in this controller would cache. I recommend committing a cached footer html page to any source control you have for deployment.

With that out of the way, what about the content of the footer HTML page? Again, here is a HAML template I used. This is very much like my pdf layout with one important difference, some JavaScript that is used to parse the query parameters that wkhtmltopdf tacks onto each header/footer URL request. In the example below I am only using the current page page and total page count topage params and inserting those values into to elements. For a full list of all the query parameters, consult the wkhtmltopdf expanded help page.

!!! 5
%html{:lang => 'en'}
  %head
    %meta{:charset => 'utf-8'}
    = stylesheet_link_tag "#{request.protocol}#{request.host_with_port}/stylesheets/pdf.css"
    :javascript
      var pdfInfo = {};
      var x = document.location.search.substring(1).split('&');
      for (var i in x) { var z = x[i].split('=',2); pdfInfo[z[0]] = unescape(z[1]); }
      function getPdfInfo() {
        var page = pdfInfo.page || 1;
        var pageCount = pdfInfo.topage || 1;
        document.getElementById('pdfkit_page_current').textContent = page;
        document.getElementById('pdfkit_page_count').textContent = pageCount;
      }
  %body{:onload => 'getPdfInfo()'}
    %div#pdfkit_page_numbers
      %span Page:
      %span#pdfkit_page_current
      %span of
      %span#pdfkit_page_count

Hopefully this helps anyone looking to use PDFKit or any system that leverages the wkhtmltopdf project. If I missed anything or can help, please leave me a comment. Thanks!

GC Tune Ruby 1.9.2 With RVM

Wed, 09 Mar 2011 00:00:00 -0500

Here is a gist by Sokolov Yura (funny-falcon) that allows you to GC tune Ruby 1.9.2 just like Ruby Enterprise Edition (REE). So all of us using RVM have no reason not start using this right away. Here are the steps.

$ curl https://gist.github.com/raw/856296/patch-1.9.2-gc.patch > ~/192-gc.patch
$ rvm uninstall 1.9.2
$ rvm install 1.9.2 --patch ~/192-gc.patch

I have used this RVM hook below for awhile now. It automatically sets and unsets the proper ENV vars to GC tunes my REE. I have now updated it to apply the same GC settings to my newly patched 1.9.2 as well. I recommend this go into ~/.rvm/hooks/after_use.

case "$rvm_ruby_string" in
  *ree*|*ruby-1.9.2*)
    export RUBY_HEAP_MIN_SLOTS=1000000
    export RUBY_HEAP_SLOTS_INCREMENT=1000000
    export RUBY_HEAP_SLOTS_GROWTH_FACTOR=1
    export RUBY_GC_MALLOC_LIMIT=1000000000
    export RUBY_HEAP_FREE_MIN=500000
    export RUBY_FREE_MIN=$RUBY_HEAP_FREE_MIN
  ;;
  *)
    unset RUBY_HEAP_MIN_SLOTS RUBY_HEAP_SLOTS_INCREMENT RUBY_HEAP_SLOTS_GROWTH_FACTOR RUBY_GC_MALLOC_LIMIT RUBY_HEAP_FREE_MIN RUBY_FREE_MIN
  ;;
esac

Code Sharing With XCode 4 - An Alternative To Static Libraries

Thu, 03 Mar 2011 00:00:00 -0500

There are many great techniques for sharing code across XCode projects. If that code is tested and build-able by XCode as a static library, Clint Harris' Easy, Modular Code Sharing Across iPhone Apps: Static Libraries and Cross-Project References is really a solid source on the topic. However, this approach is a bit heavy handed for what I wanted to do. Basically if you just have some common header and implementation files that you want to use across multiple XCode projects, this article is for you. The goals for the common code is simple:

Common code needs to be under distinct version control.
Common code files need to be editable by XCode from the referencing project.

In this example I will use a contrived example folder called "MetaSupport" as simple directory of common code to use. It includes some global function, a MyApp namespace with class with accessors and a directory of Additions to core classes.

$ tree MetaSupport
├── Additions
│   ├── NSArray+MetaSupport.h
│   └── NSArray+MetaSupport.m
├── MyApp.h
├── MyApp.m
├── MyGlobal.h
└── MyGlobal.m

So let's get started. Below are the steps for adding shared code to our "KillerApp" iOS project. FIrst, right click on the XCode project in your project navigator. Select "New Group".

Select the newly created group in your project navigator. Now open up the file inspector. It can be found in View => Utilities => File Inspector. In the file inspector, set the "Group Name" to match your shared code folder. In this case I named it "MetaSupport". This is purely optional and simply helps you organize your code.

Drag all your shared files and folders from your "MetaSupport" directory to the newly created "MetaSupport" group folder. XCode will ask you to choose some options for adding these files. It is important to add folder references and not copy files. Check any targets that apply.

So not only can you see and edit the files belonging to your shared code. You can also commit changes to those files and they will be committed to the shared code's repository that they reside in, not the current KillerApp projects repo! Hence you can update your shared code for all projects from any project.

Modern SQL Server & Ruby/Rails Installer For Windows

Thu, 10 Feb 2011 00:00:00 -0500

If you have not done so, please go read my article Modern SQL Server & Rails on the Engine Yard blog. I want to thank Wayne E Seguin for inviting me to write the article. This article details many exciting things that have been happening for those using the SQL Server Adapter for Rails, most notably TinyTDS. The article outlines why everyone still using the ODBC connection mode should really switch over to TinyTDS.

Many exciting things are happening thanks to Wayne's new pet project, a ruby and rails installer for Windows. The adapter and TinyTDS will hopefully dove tail into that. The plan is to include components in the one click installer that will allow Windows users to access their legacy databases using TinyTDS and the SQL Server Adapter. We had even talked on the mailing list about including a demo rails app with the ruby windows installer that would access the northwind database much the same way .Net demos would.

Simple Memcached Reports In Rails

Thu, 27 Jan 2011 00:00:00 -0500

The Pragmatic Bookshelf's Using memcached: How to scale your website easily is way old. But then again, memcached is still very useful. One of the things I remember reading a long time ago was the stats for your process and how to pull out meaningful reports like a hit ratio, memory and a get to set ration.

I whipped these up today as a simple way of getting a report using Rails.cache.report, assuming your cache strategy is set to memcached. Hope others find it useful. BTW, I do not recommend the book, it is too low level for any useful meaning to the average Rails developer.

module ActiveSupport
  module Cache
    class MemCacheStore < Store

      include ActionView::Helpers::DateHelper
      include ActionView::Helpers::NumberHelper

      # Higher percentages, above 90%, are good.
      def hit_ratio
        stats.inject({}) do |memo, (server,stat)|
          value = (stat['get_hits'].to_f / stat['cmd_get'].to_f) * 100
          memo[server] = number_to_percentage(value,:precision=>2)
          memo
        end
      end

      # Another good statistic to look at is percentage of gets to sets. For a well-tuned application,
      # there should be more gets than sets. High percents are bad, negatives are great.
      def gets_to_sets
        stats.inject({}) do |memo, (server,stat)|
          sets = stat['cmd_set'].to_f
          gets = stat['cmd_get'].to_f
          more_sets = sets - gets
          value = (more_sets / gets) * 100
          memo[server] = number_to_percentage(value,:precision=>2)
          memo
        end
      end

      def uptime
        stats.inject({}) do |memo, (server,stat)|
          started_on = stat['uptime'].seconds.ago
          utime = distance_of_time_in_words started_on, Time.now
          memo[server] = {:started_on => started_on, :uptime => utime}
          memo
        end
      end

      def memory
        stats.inject({}) do |memo, (server,stat)|
          memory = number_to_human_size stat['limit_maxbytes']
          used = number_to_human_size stat['bytes']
          memo[server] = {:memory_total => memory, :memory_used => used}
          memo
        end
      end

      def report
        stats.keys.inject({}) do |memo, server|
          rpt = memory[server].merge(uptime[server])
          rpt[:hit_ratio] = hit_ratio[server]
          rpt[:gets_to_sets] = gets_to_sets[server]
          memo[server] = rpt
          memo
        end
      end

    end
  end
end

From Test::Unit & Shoulda To MiniTest::Spec & MiniShoulda

Tue, 25 Jan 2011 00:00:00 -0500

It seems the MiniTest replacement for Test::Unit in Ruby 1.9 has been presenting itself more and more in my daily readings. Recently was Jamis Buck's article titled The road to faster tests where he talks about optimizing basecamp's suite to run faster. And there again, some of the comments below mentioned MiniTest as a solution. That got me thinking on what would it take for me to switch from Test::Unit to MiniTest.

I love testing! But I am not nuts about large libraries. Looking at you RSpec! Like others, I believe that you do not need a bloated test framework to get good coverage with a lean and succinct style. A handful of assertions and a simple context structure provided by libraries like Shoulda is really all I need for both unit and a certain level of functional testing. It is my firm believe that large libraries and semi-magical test code based on things like RSpec will wane in the coming years. Those that base very large test suites on them will suffer the penalty for when an upgrade or test refactor is needed. I could be wrong, time will tell. For now let's cover how to move to MiniTest for those that like simple Test::Unit and or Shoulda.

MiniTest provides a drop in replacement for Test::Unit, so I wont cover the details of those basics. What most do not know is that MiniTest comes with a simple MiniTest::Spec DSL that uses describe and it blocks much like Shoulda's context and should methods. The similarities were so close that I set out to move all my projects to MiniTest::Spec. The goal would be that none of my exiting suites context/should/before/after code would require a change. The result...

MiniShoulda

MiniShoulda is a small ruby gem that puts a Shoulda DSL on top of MiniTest::Spec. All it really does it alias a few key methods. Here is the core of it now.

class MiniTest::Spec < MiniTest::Unit::TestCase

  class << self
    alias :setup    :before
    alias :teardown :after
    alias :should   :it
    alias :context  :describe
  end

  def self.should_eventually(desc)
    it("should eventually #{desc}") { skip("Should eventually #{desc}") }
  end

end

For now, MiniShoulda performs only one monkey patch to MiniTest. There is a critical bug in MiniTest that does not allow the describe/context blocks to use the existing scope of the parent class. I have a pull request in for the project. Please help out and add a thoughtful comment based on the merits of the patch and the description of the problem. No author likes to see simple +1 comments.

The MiniTest Switch

$ gem install mini_shoulda

Our gem spec will automatically pull in MiniTest version 2.0.2 or greater. Assuming you have a test helper file, the top requires will looks something like this. MiniTest needs that autorun file required or none of your tests will run.

require 'mini_shoulda'
require 'minitest/autorun'

What should I expect? For starters, expect a few of your tests to fail. Especially ones that were happenstantially running linearly and not exercised in isolation. I think there is a technical term for this? Anyways, MiniTest mixes up the test suite each run trough it using a random seed. So it is highly like to expose a few poorly written tests that depended on state leftover from a previous test. Also the MiniTest::Unit underneath MiniTest::Spec does not have a few assertions you may have used. One is assert_raise which is now the standard convention assert_raises with an "s". And finally assert_nothing_raised has been pulled completely since Test::Unit's documentation alluded to its uselessness for a long time.

What does MiniTest::Spec give you besides clean and powerful describe/it organization with before/after setups and teardowns? I highly recommend the MiniTest cheat sheet to find out. See the spec and mock methods at the bottom? As much as I have always like the basic unit testing assertions, I am looking forward to using the must/wont object assertions.

Resources

MiniShoulda Github Project
The MiniTest Describe Pull Resuest - HELP OUT! COMMENT!
MiniTest Cheat Sheet

Automating Heroku PG Backups

Mon, 03 Jan 2011 00:00:00 -0500

On December 1st, Heroku deprecated their bundles add-on in favor of their new PG Backups. Even though there are other solutions for automating backups using this new add-on, none of them met my needs. I like to have a daily DB backup history, just in case you find something bad that happened "n" days earlier. Below is a simple rake task suitable to place in your rails lib/tasks/heroku.task file. I'll explain some things I learned below when writing this.

require 'heroku'
require 'heroku/command'

HEROKU_BACKUP_BUCKET = "#{Heroku::Command::Base.selected_application}-backups"

namespace :heroku do

  desc "Use the `heroku pgbackups` with my S3 bucket."
  task :backup => :connect_to_s3 do
    info = capture_heroku_command 'pgbackups'
    if heroku_existing_backup?(info)
      last_backup_info = info.split("\n").last.split(" | ")
      last_backup_id = last_backup_info[0]
      last_backup_time = last_backup_info[1]
      puts "Deleting last backup - ID: #{last_backup_id} BackupTime: #{last_backup_time}"
      heroku_command 'pgbackups:destroy', last_backup_id
    end
    heroku_command 'pgbackups:capture'
    backup_url = capture_heroku_command 'pgbackups:url'
    backup_filename = "#{Heroku::Command::Base.selected_application}_#{Time.now.xmlschema}.dump"
    backup_data = Net::HTTP.get_response(URI.parse(backup_url)).body
    AWS::S3::S3Object.store backup_filename, backup_data, HEROKU_BACKUP_BUCKET
  end

  task :connect_to_s3 do
    AWS::S3::Base.establish_connection!(
      :access_key_id     => ENV['AMAZON_ACCESS_KEY_ID'],
      :secret_access_key => ENV['AMAZON_SECRET_ACCESS_KEY'])
    begin
      AWS::S3::Bucket.find(HEROKU_BACKUP_BUCKET)
    rescue AWS::S3::NoSuchBucket
      AWS::S3::Bucket.create(HEROKU_BACKUP_BUCKET)
    end
  end

end

private

def heroku_command(*cmds)
  Heroku::Command::Base.command(*cmds)
end

def capture_heroku_command(*cmds)
  stdout = STDOUT
  StringIO.new.tap do |out|
    def out.flush ; end
    $stdout = out
    heroku_command(*cmds)
  end.string.chomp
ensure
  $stdout = stdout
end

def heroku_existing_backup?(info)
  info !~ /no backups/i
end

Once installed you can run something like bundle exec rake heroku:backup or if you are not using bundler, rake heroku:backup. Assuming you have your S3 credentials setup in the ENV variables, it will find or create a private bucket on your S3 account and upload an app-named and time-stamped dump file to that new bucket. If necessary, it will delete your latest backup on Heroku to make room for this new one. The code should be easy to change, so flavor to taste.

I tried to use the Heroku commands built into their plugin without resorting to command interpolation. The only problem was that the Heroku gem always wants to print to standard out and flush the buffer. So I created a few private helper methods that temporarily shim in a $stdout replacement that does not flush. This let's me run the Heroku commands from code and capture what would have been printed to standard out.

Lastly, since I could not find a way to automate this on Heroku via their cron add-on, I simply added a launchd plist to my desktop Mac that hit a shell script in my project folder to run the rake task. It is way past my skills to try and get RVM to work in the launchd.plist system since it is not a true shell. This is why the shell script uses my system ruby (installed via MacPorts). Here is the shell script below and the launchd plist which I placed in ~/Library/LaunchAgents with a name like com.actionmoniker.backupMyApp.plist. Just run launchctl load ~/Library/LaunchAgents/com.actionmoniker.backupMyApp.plist and this will run at 4am every morning. If any one finds out how to automate the execution of this rake task on Heroku, please drop me a line!

#! /bin/zsh
source /Users/kencollins/.zshenv
cd /Users/kencollins/repos/myapp && /opt/local/bin/rake heroku:backup

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
  <key>Label</key>
  <string>com.actionmoniker.backupMyApp</string>
  <key>ProgramArguments</key>
  <array>
    <string>/Users/kencollins/repos/myapp/lib/bin/backup.sh</string>
  </array>
  <key>StartCalendarInterval</key>
  <dict>
    <key>Minute</key>
    <integer>0</integer>
    <key>Hour</key>
    <integer>4</integer>
  </dict>
</dict>
</plist>

Pyr Pressure

Wed, 29 Dec 2010 00:00:00 -0500

It's a shame I do not do lifestyle posts more often, but with the new blog I hope to change that. To that end, this article is dedicated to our Great Pyrenees Sasha who passed away from bone cancer last month. She was just 7 years old and we only had 3 weeks of good times from the point she was diagnosed with osteosarcoma, a form of bone cancer. I am sure everyone loves their pet(s), we even have another dog and 3 cats. But there is something about a large breed, perhaps more so a Great Pyrenees, that made our loss really painful. It was also the first time I heard about the rainbow bridge, total tear buster.

Luckily for us, we have a personal friend that breeds Great Pyrenees show dogs in Smithfield, VA and she just so happens to have had a new litter. Determined to turn a bad situation into something positive, we just up'ed our family by two more puppy dogs. You could say I RAID-1'ed my pyr puppies. So we now have a little boy and girl from the same litter. His name is Remy LeBeau and is on the left and her name is Sonya, pictured on the right.

They are getting tons of love, attention, and good food. Raising two puppy dogs is so much fun, but it takes a lot of time and patience, especially when training them to go out doors with all the snow we have gotten lately. In the future Kelly and I will enjoy walking these guys around Colonial Williamsburg. Since they look like sheep, our Border Collie will corral them indoors for us as he did for Sasha. We can already tell that Remy will be the protector of the household and probably weigh in around 150 pounds when he grows up.

Let It Go - Moving From Mephisto To Jekyll

Mon, 27 Dec 2010 00:00:00 -0500

Welcome to the new MetaSkills.net! I have moved from Mephisto to Jekyll and done a complete rewrite of the presentation layer to the latest web standards as a way of staying somewhat sharp with the latest HTML5/CSS3 hotness. In this article, I would like to cover all the gory details of what I learned about moving to Jekyll and the various techniques I have used for the site's CSS. But first a small history lesson.

If you are somewhat new to rails, Mephisto was one of the greatest open source blog engines a ruby developer could work with. But ruby, and especially rails, is a fast paced landscape, making it hard to keep any CMS project up to date. Options like file storage, comment moderation, templating languages, hosting locations and more are all very hard to embody in one project. I think that is why Mephisto's creator, Rick Olson (@technoweenie), announced that he was dropping the project. Good for him and many thanks for his hard work on all the open source projects he has contributed to! So let's talk about Jekyll et al.

Migrating Mephisto Data

So maybe you have an old Mephisto install that you would like to migrate to Jekyll? If so, I have posted a gist of some rails 3 models that I used to export my data from MySQL to static files following the format required by Jekyll. Simply populate a new rails 3 project with these, fire up a console and do Site.export_jekyll. This will create a folder in tmp/jekyll_posts and write a file for each Mephisto article. The posts will contain the YAML front matter for jekyll and hopefully a good starting point of the categories you had assigned to that post. As a bonus, the export will create a tmp/jekyll_posts/rewrites.txt file that will be very useful to include in a .htaccess file of your jekyll project. An example below.

RewriteRule ^2010/8/19/how-to-clean-a-campfire-room-of-uploads$ /2010/08/19/how-to-clean-a-campfire-room-of-uploads/ [L]

Also included in the gist is a Site.export_jskit method that will generate an XML file of your comments to import into Disqus. Though I was able to import my Mephisto comments into Disqus and see them linked from the admin interface, I have yet to get them to show up on the new site. I have a support request open with Disqus and will update this article and the gist with any new information.

Using Jekyll

The best way you can learn how to use Jekyll is to review code of the sites that use it. If you want to learn more about how I have implemented jekyll, you could browse my source on github.

Like many others, I have created a tasks directory in my project root with some executable scripts that help me author, publish and deploy my site. Here are some examples.

#!/usr/bin/env zsh
# File: tasks/scss
# Handy to watch for scss file changes while designing.
sass --scss --watch _sass:_site/resource --style compact --no-cache

#!/usr/bin/env zsh
# File: tasks/jekyll
# Create the site and tidy it.
setopt rmstarsilent
rm -rf _site/* && \
  sass --style compressed _sass/site.scss:resource/site.css && \
  bundle exec jekyll && \
  find _site -name "*.html" -exec tidy -config $(pwd)/tidy.conf {} \;

#!/usr/bin/env zsh
# File: tasks/deploy
# Build and deploy to mini.
./tasks/jekyll && \
  rsync -avz --delete _site/ mini:/Library/WebServer/hosts/metaskills.net && \
  mini sudo apachectl restart

The first one, task/scss allowed me to author my site's CSS using SASS and its newer SCSS syntax, more on that later. This task simply let me publish my site and then style it later by easily saving the source SCSS file and avoiding a long running jekyll command. The second task/jekyll is my own long winded jekyll command. Basically it will compile my SCSS file, execute my bundled jekyll binary, then post process all my HTML files with tidy. I spent a long time coming up with an appropriate tidy.conf compatible with HTML5 standards. Perhaps you may find it useful. Lastly, the task/deploy is just a simple wrapper to my own jekyll command and a final rsync command to my Mac Mini web server. I restart the web server just in case I have made changes to my .htaccess file.

HTML5

A personal tech blog is a great platform to try out new things, including the semantic goodness that HTML5 offers. Frankly, it was way past time for me to learn and I found Mark Pilgrim's - Dive Into HTML5 really helpful on the topic. The final structure of the blog came out nicely. A post is contained in its own <article> element with <header> and <footer> elements in between. A great way to organize comments and new elements like the <time> so you can associate a machine readable publication date to your post.

<article id="post">
  <header>
    <time pubdate datetime="2010-12-27T00:00:00-05:00">
      <span class="day">27</span>
      <span class="month">Dec</span>
      <span class="year">2010</span>
    </time>
    <h1>Let It Go - Moving From Mephisto To Jekyll</h1>
  </header>
  ...
  <footer id="disqus_thread">
    ...
  </footer>
</article>

Presentation - SASS

This was the second time I have used SASS from the start of a project. The first time was almost a year ago and I must say, it has gotten much nicer with the newer SCSS format. While writing my lengthy SCSS file, I found the following linear organization method very helpful.

Font Declarations - With Mixins For Usage.
Global Variables - Grids, Colors...
Vendor Mixins - Gradients, Shadows, Transformations...
Component Mixins - Links, Navigation, Fancy Images, Comments...
Layout Mixins - Posts, Footer, Excerpts, Etc
Final Layout - Site Structure (using mixins)

I think this is important, because it outlines a radical shift from writing raw CSS and really flexes SASS's @mixin and @include features. First it is important to point out that CSS falls into two categories, layout & presentation. Layout CSS is akin to using <table> tags in the old days. It is CSS that controls the structure of your site. After that, all CSS is presentation oriented. It is there to make things look pretty.

Back in the day, I would put my layout styles at the beginning of my stylesheet, then finish with abstract presentation classes. This works, but sometimes you find yourself deep into a CSS corner and having to whip out !important declarations to bail yourself out. When writing with SASS, the layout styles are now my last declarations. Why? Because you can include your earlier coded design components as a last step within the structure declarations, thereby ensuring their scope is locked down to the explicit node of your choosing. This helps you keep your layout and presentation CSS in separate and manageable chunks that have logical names while keeping your SASS file from looking like one big long procedural function. It's OO-CSS at its best!

...
section#page {
  width: $pagewidth;
  margin: 0px auto;
  min-height: 800px;
  @include cmpnt-links;
  section#content {
    float: right;
    width: $c8width;
    padding-top: $hdrheight + 20px;
    padding-bottom: 20px;
    @include layout-excerpt;
    @include layout-post;
    @include cmpnt-flash;
    @include cmpnt-blockquote;
  }
...

Presentation - Simple Grid

I have previously used CSS grids like 960.gs and the 1Kb Grid. However, for this project, I decided to flex SASS and my previous CSS knowledge of positioning and clearing for a leaner layout structure. These SCSS variables were enough to let me build out the new layout.

$cols: 10;
$colwidth: 80px;
$gutterwidth: 20px;
$c1width:  $colwidth * 1;
$c2width:  ($colwidth * 2) + ($gutterwidth * 1);
$c3width:  ($colwidth * 3) + ($gutterwidth * 2);
$c4width:  ($colwidth * 4) + ($gutterwidth * 3);
$c5width:  ($colwidth * 5) + ($gutterwidth * 4);
$c6width:  ($colwidth * 6) + ($gutterwidth * 5);
$c7width:  ($colwidth * 7) + ($gutterwidth * 6);
$c8width:  ($colwidth * 8) + ($gutterwidth * 7);
$c9width:  ($colwidth * 9) + ($gutterwidth * 8);
$c10width: ($colwidth * 10) + ($gutterwidth * 9);
$pagewidth: ($colwidth * $cols) + ($gutterwidth * ($cols - 1));

Presentation - Pseudo Generated Content

<div class="photobounding">
  <div class="tl"></div>
  <div class="tr"></div>
  <div class="bl"></div>
  <div class="br"></div>
  <div class="photoborder"></div>
  <div class="photo">
    <img src="/files/yourpicture.jpg" alt="" width="" height="" />
  </div>
</div>

We have all seen code like this! It is called divitis, the overuse of span or div elements to structure and style content. I am guilty of doing this too, in fact, that code is from my old blog's CSS convention page to make a fancy border around a photo. The new MetaSkills.net takes advantage of a pseudo generated content to style complex elements and avoid this. I would like to show you a few examples, but first, learn from the master Nicolas Gallagher (@necolas) in a series of articles he published for backgrounds/borders, speach bubbles, and GUI icons all using pseudo generated content with CSS.

Using these techniques, I was able to accomplish all the complex styles like my Apple TV navigation and photo fancy borders with simple semantic elements. The image to the right is only surrounded by one <span class="photofancy">...</span> element since you can not generate pseudo content from a <img> tag. I also used pseudo generated content to remove excessive layout elements too. My complex headers use them to add multiple backgrounds where only one element resides in the code. Learn this technique, it keeps things very clean.

Zepto.js - Something You Need To Watch For

Lastly, if you have not seen the zepto.js project, go check it out. It bills itself as the aerogel-weight mobile javascript framework. Besides being incredibly small, I think zepto has some incredible potential when used with backbone.js. I have never used jQuery and always looked at JavaScript thru the eye's of prototype since most of my JavaScript needs require rich objects first, and access to the DOM second. I'll make another post on Zepto and why I think it is so bad ass. I use it on the new MetaSkills.net to shim up my custom pygmentize theme.

In Closing

I am really happy with the new layout and using Jekyll to publish my static blog. I even think the old content carried over well once I tweaked it to the new CSS conventions. One particular old article that embodies all the design aspects of the new layout is Jack has_many :things. It is about a plugin I wrote on grouping ActiveRecord has_many associations. Hope you find some time to try out Jekyll too and let me know what you think of my new site!

Resources

You Can Let Go Now - Mephisto ends
Jekyll Project - A static blog/site generator
Migrate Mephisto Data - A gist of rails 3 model files
MetaSkills.net Source - On Github.com
Dive Into HTML5
Multiple Backgrounds & Borders With CSS2
Pure CSS speech bubbles
Pure CSS GUI Icons
Zepto.js - The aerogel-weight mobile javascript framework

TinyTds - A modern, simple and fast FreeTDS library for Ruby using DB-Library

Mon, 18 Oct 2010 00:00:00 -0400

I just finished the first cut of learning C extensions for ruby and I present The TinyTds gem. It is meant to serve the extremely common use-case of connecting, querying and iterating over results to Microsoft SQL Server databases from ruby. Even though it uses FreeTDS's DB-Library, it is NOT meant to serve as direct 1:1 mapping of that complex C API

The benefits are speed, automatic casting to ruby primitives, and proper encoding support. It converts all SQL Server datatypes to native ruby objects supporting :utc or :local time zones for time-like types. To date it is the only ruby client library that allows client encoding options, defaulting to UTF-8, while connecting to SQL Server. It also properly encodes all string and binary data. The motivation for TinyTds is to become the de-facto low level connection mode for the SQL Server adapter for ActiveRecord. For further details see the special thanks to Erik Bryn for his help, the authors/contributors of the Mysql2 gem for inspiration, and Yehuda Katz for articulating ruby's need for proper encoding support. Please read up on it here.

How To Clean A Campfire Room Of Uploads_redirect

Thu, 19 Aug 2010 00:00:00 -0400

How To Clean A Campfire Room Of Uploads

Thu, 19 Aug 2010 00:00:00 -0400

For us at work, our uploads to campfire are really transitory. Most of the time they are simple screenshots around a current topic. Every now and then vacation photos or even movies, at the end of the day, none of it has value after a certain amount of time. To us as the real value of campfire is our textual transcripts.

This morning after 4 years of campfire, we were real close to our 1GB limit of uploads. Time for a clean up. I found no easy way of automating this, so I turned to the Tinder gem. It has a nice interface to the campfire API using HTTParty as the backend. I found out that there was no easy way to delete uploads too. So after some good ole fashion DOM inspection and knowing rails application conventions, I found my own interface. Below is a little script I used to clean up our room this morning. It basically loops thru a rooms uploads, 5 at a time, and deletes them. Pausing for a quarter of a second between each so I don't freak out the new 37signals administrator :)

require 'rubygems'
require 'tinder'

class CampfireUploadCleaner

  CF_DOMAIN = 'mydomain'
  CF_ROOM   = 'My Room Name'
  CF_TOKEN  = ENV['MY_CF_TOKEN']

  def initialize
    @campfire = Tinder::Campfire.new CF_DOMAIN, :token => CF_TOKEN
  end

  def room
    @room ||= @campfire.find_room_by_name(CF_ROOM)
  end

  def connection
    room.send(:connection)
  end

  def delete_uploads
    uploads = room.send(:get,:uploads)['uploads']
    uploads.each do |upload|
      id = upload['id']
      name = upload['name']
      connection.post "/uploads/delete/#{id}?n=0"
      puts "Deleted: [#{id}] #{name}"
      sleep(0.25)
    end
  end

  def sweep_uploads
    while room.files.present?
      delete_uploads
    end
  end

end

cleaner = CampfireUploadCleaner.new

cleaner.delete_uploads  # => Deletes the top 5 uploads.
cleaner.sweep_uploads   # => Deletes all uploads.

The upload hash actually contains much more than just the id and name of the upload. There is a timestamp, filetype and other attributes. So if you wanted to extend this script, you could. I did not spent a lot of time with it, but I never figured out how to get more than the top 5 uploads too. I'm sure some param hacking would yield some good results.

Protip Exclude Your Rvm Install From Timemachine Backups_redirect

Fri, 06 Aug 2010 00:00:00 -0400

Protip: Exclude Your RVM Install From Time Machine Backups

Fri, 06 Aug 2010 00:00:00 -0400

You may not realize it, but that ~/.rvm directory is getting big and may be hurting your Time Machine backups. Especially if you are doing a lot of development. I just checked mine today.

$ du -skh ~/.rvm/gems
1012M   /Users/kencollins/.rvm/gems

Ouch! That's around 1gig of stuff that is pointless to backup just in the gems directory. I frequently blow away certain rubies and install them again using various rake tasks and just found out today that you have to pass the --gems option to rvm remove to clean those old gemsets out. Either way, that whole rvm directory can just be ignored from Time Machine backups. Here is how, there is really only one tricky part.

First, open up your System Preferences, then open up the Time Machine pref pane. From here click on the "Options...". Within this window you will be able to add a folder to be excluded from backup. Hit the "+" button to add a directory. Now here is the tricky part, your rvm directory is prefixed with a period and hence is invisible to the normal finder windows. Thankfully all Mac file windows can use a keyboard shortcut. Hit Command+Shift G. That will blind down a "Go to the folder:" window. Simply type in ~/.rvm here and hit "Go". Now choose "Exclude" from the main file window. Your done!

The Rvm Ruby Api_redirect

Fri, 30 Jul 2010 00:00:00 -0400

The RVM Ruby API - Setting Up A CI System For The SQL Server Adapter

Fri, 30 Jul 2010 00:00:00 -0400

A few weeks ago I started looking into the Ruby Version Manager (RVM) project to help me build a better testing setup for both my day job and the ActiveRecord SQL Server Adapter. In a previous article I covered details of how to get a development stack up and running for Rails with SQL Server using MacPort's. This article will cover some new additions to that goal, but first and primarily, I wanted to talk about the wonders of RVM and it's new ruby API.

So like any good agile software gardner tasked with building a continuous integration system, I wanted to do it in such a way that was completely automated using rake. My first cut at said rake task used ruby's Kernel.system to issue rvm commands down to the shell. This failed horribly! Basically no matter how hard I tried, I could not hit the rvm shell function from ruby's system command. It kept using the rvm binary which can not change the local shell environment and hence do very little magic that RVM allows. Thankfully @wayneeseguin pointed me to the new RVM ruby API and this article on how to use it for passenger. I immediately started to switch my rvm rake tasks to use the new RVM API and was just floored with how well it did. Below is a copy of that rake task. Take a look over it and read below for details and how I have used this with the SQL Server stack.

MYPROJECT_RUBIES = {
  'ruby-1.8.6-p388'   => {:alias => 'myprj186', :odbc => '0.99991'},
  'ruby-1.8.7-p299'   => {:alias => 'myprj187', :odbc => '0.99991'},
  'ruby-1.9.1-p378'   => {:alias => 'myprj191', :odbc => '0.99991'},
  'ruby-1.9.2-head'   => {:alias => 'myprj192', :odbc => '0.99992pre3'},
  'ree-1.8.7-2010.02' => {:alias => 'myprjree', :odbc => '0.99991'}
}

namespace :rvm do

  task :setup do
    unless @rvm_setup
      rvm_lib_path = "#{`echo $rvm_path`.strip}/lib"
      $LOAD_PATH.unshift(rvm_lib_path) unless $LOAD_PATH.include?(rvm_lib_path)
      require 'rvm'
      require 'tmpdir'
      @rvm_setup = true
    end
  end

  namespace :install do

    task :all => [:setup,:rubies,:odbc,:gems]

    task :rubies => :setup do
      installed_rubies = RVM.list_strings
      MYPROJECT_RUBIES.keys.each do |rubie|
        if installed_rubies.include?(rubie)
          puts "info: Rubie #{rubie} already installed."
        else
          with_my_environment_vars do
            good_msg = "info: Rubie #{rubie} installed."
            bad_msg = "Failed #{rubie} install! Check RVM logs here: #{RVM.path}/log/#{rubie}"
            puts "info: Rubie #{rubie} installation inprogress. This could take awhile..."
            RVM.install(rubie,rvm_install_options) ? puts(good_msg) : abort(bad_msg)
          end
        end
        RVM.alias_create MYPROJECT_RUBIES[rubie][:alias], "#{rubie}@myproject"
      end
    end

    task :odbc => :setup do
      rvm_each_rubie do
        odbc = "ruby-odbc-#{myproject_current_rubie_info[:odbc]}"
        RVM.chdir(Dir.tmpdir) do
          RVM.run "rm -rf #{odbc}*"
          puts "info: RubyODBC downloading #{odbc}..."
          RVM.run "curl -O http://www.ch-werner.de/rubyodbc/#{odbc}.tar.gz"
          puts "info: RubyODBC extracting clean work directory..."
          RVM.run "tar -xf #{odbc}.tar.gz"
          RVM.chdir("#{odbc}/ext") do
            puts "info: RubyODBC configuring..."
            RVM.ruby 'extconf.rb', "--with-odbc-dir=#{rvm_odbc_dir}"
            puts "info: RubyODBC make and installing for #{rvm_current_name}..."
            RVM.run "make && make install"
          end
        end
      end
    end

    task :gems => :setup do
      puts "info: Installing our app gems."
      rvm_each_rubie do
        myproject_gem_specs.each { |spec| rvm_install_gem(spec) }
      end
    end

  end

  task :remove => :setup do
    myproject_rubies.each { |rubie| RVM.remove(rubie) }
  end

end


def myproject_rubies
  MYPROJECT_RUBIES.keys.map{ |rubie| "#{rubie}@myproject" }
end

def myproject_current_rubie_info
  MYPROJECT_RUBIES[rvm_current_rubie_name]
end

def myproject_gem_specs
  [
    ['rails','2.3.8'],
    ['activerecord-sqlserver-adapter','2.3.8'],
    ['erubis','2.6.6'],
    ['haml','3.0.13'],
    ['mocha','0.9.8'],
  ]
end

def rvm_each_rubie
  myproject_rubies.each do |rubie|
    RVM.use(rubie)
    yield
  end
ensure
  RVM.reset_current!
end

def rvm_current_rubie_name
  rvm_current_name.sub('@myproject','')
end

def rvm_current_name
  RVM.current.expanded_name
end

def rvm_gem_available?(spec)
  gem, version = spec
  RVM.ruby_eval("require 'rubygems' ; print Gem.available?('#{gem}','#{version}')").stdout == 'true'
end

def rvm_install_gem(spec)
  gem, version = spec
  if rvm_gem_available?(spec)
    puts "info: Gem #{gem}-#{version} already installed in #{rvm_current_name}."
  else
    puts "info: Installing gem #{gem}-#{version} in #{rvm_current_name}..."
    puts RVM.perform_set_operation(:gem,'install',gem,'-v',version).stdout
  end
end

def for_macports?
  `uname`.strip == 'Darwin' && `which port`.present?
end

def rvm_install_options
  {}
end

def my_environment_vars
  if for_macports?
    {'CC' => '/usr/bin/gcc-4.2',
     'CFLAGS' => '-O2 -arch x86_64',
     'LDFLAGS' => '-L/opt/local/lib -arch x86_64',
     'CPPFLAGS' => '-I/opt/local/include'}
  else
    {}
  end
end

def rvm_odbc_dir
  for_macports? ? '/opt/local' : '/usr/local'
end

def set_environment_vars(vars)
  vars.each { |k,v| ENV[k] = v }
end

def with_my_environment_vars
  my_vars = my_environment_vars
  current_vars = my_vars.inject({}) { |cvars,kv| k,v = kv ; cvars[k] = ENV[k] ; cvars }
  set_environment_vars(my_vars)
  yield
ensure
  set_environment_vars(current_vars)
end

RVM Rake Task Breakdown

This is the fun part - lets start from top to bottom. I'll focus only on the parts that are centric to why the RVM ruby API is so bad ass and the rake task in general. The following section is dedicated to RVM with the SQL Server Adapter stack. First, the :setup task, this is called before every other task. It simply sets up the load path so that the RVM API file can be required. That api file is located in your rvm repo path, typically in your ~/.rvm directory. Now every other task can use the RVM module which implements method missing for many commands.

Past this my rvm namespace is broken up into three main install tasks. The primary concerns are rvm:install:rubies and rvm:install:gems each invoked by the rake task. Starting with the :rubies task, this iterates over a collection of ruby version strings first checking if RVM knows its installed then installing it otherwise.

In the :gems task, things get a little interesting. I am using a few methods (seen toward the bottom) that give this a nice little DSL of my own around RVM. The first is a block method called rvm_each_rubie. This iterates over each of my project's ruby strings, tells RVM to use that ruby, hence dynamically switching to that ruby/gemset then yielding to the block. That means that each go around I will be in an completely different ruby/gem environment, each specific to my project using RVM gemsets. This allows the inner method for iterating over the required gems for my project and asking to install them with rvm_install_gem. This method uses RVM.ruby_eval to execute a string of ruby in the context of the current ruby version and gemset. In this case, finding out if a gem is installed already. If not, I use the RVM.perform_set_operation to install the gem, again in current context. I use perform_set_operation so I can read the STDOUT back to the rake task so a user sees exactly what is going of for the gem install.

Past that there are a few other details. Most can be picked up by reading the helper methods toward the bottom. Using the RVM ruby API is a bit of a chore if you rely on documentation. Sure it has some, but nothing beats reading the code to see what is available to you. Remember you can find the API library by opening up the ~/.rvm/lib directory. I'm sure you can also find help on the RVM IRC channel.

Though it is not specific to general RVM API goodness, the :odbc task does show some great interfaces that RVM exposes for doing standard file system directory changing and running commands to the system.

Notes On RVM With The SQL Server Adapter Stack

So you use the ActiveRecord SQLServerAdapter? That means you have some underlying components installed - namely FreeTDS, unixODBC, and RubyODBC right. If your like me and believe that MacPorts is the way to go and risking a Homebrew interleaved dependency with Apple's libraries is risky, this section is for you! So you have a MacPort base and you want to compile your RVM rubies in such a way that other dependencies such as Nokogiri and RubyODBC use your /opt/local installs. By default this does not happen because unless ruby was compiled the right way, it wont be able to allow built gems to know about your /opt/local directory. So this is what I came up with.

Take a look at the my_environment_vars method. This works in-conjunction with the with_my_environment_vars block method. Basically it temporarily set's MacPort specific environment variables before installing a ruby version via RVM. The ones shown are what I have found work best for my system. I think the most important are LDFLAGS=-L/opt/local/lib -arch x86_64 and CPPFLAGS=-I/opt/local/include. Once ruby is built with those, it can easily reflect with standard 3rd party install methods that use RbConfig. To date any gem that I have had to compile, most importantly RubyODBC, does so perfectly against my /opt/local ports. This includes MySQL, Nokogiri, everything! I love it. I totally encourage anyone to use RVM and to get to know it's great ruby API for automating all sorts of things.

Resources

A Macport Rubyodbc Update_redirect

Mon, 19 Jul 2010 00:00:00 -0400

A MacPort/RubyODBC Update

Mon, 19 Jul 2010 00:00:00 -0400

Quite a while ago I wrote a soup to nuts article on getting the full multi-ruby development stack installed for those using the SQL Server adapter. The base package management system used there was MacPorts. In it I described how to edit the outdated Portfile for the rb-odbc package and exclaimed how important it was to use the +utf8 variant. I was totally wrong about that part.

This past week I started heavily exploring RVM at the advice of friend while we visited the Boston.rb user grooup. As an aside, I finally feel my unix-fu is strong enough to cover the edge cases needed to get the SQL Server stack happy with RVM. Good news, but that's next weeks blog post. Anyways I found that while updating my core MacPort's rb-odbc package that +utf8 was blowing up. I am currently using MacPorts 1.9.1 and was targeting 0.99991 of RubyODBC. Here is the updated port file. To edit your current port file run mate $(port file rb-odbc) and paste this new one in. Details afterward.

# $Id: Portfile 30250 2010-07-18 02:16:17Z ken@metaskills.net $
PortSystem    1.0
PortGroup     ruby 1.0

ruby.setup          {odbc ruby-odbc} 0.99991 extconf.rb {README doc test}
maintainers         nomaintainer
description         An extension library for ODBC from ruby.
long_description    Extension library to use ODBC data sources from Ruby. \
                    Supports Ruby 1.6.x and 1.8 on Win32 OSes and UN*X
checksums           md5 64eaf6089e7ca17eeff54c4fe052ac96
homepage            http://www.ch-werner.de/rubyodbc
master_sites        http://www.ch-werner.de/rubyodbc
categories-append   databases
platforms           darwin

configure.cmd             ${ruby.bin} -rvendor-specific -Cext extconf.rb
build.pre_args-append     -C ext
destroot.pre_args-append  -C ext

variant utf8 {
  configure.cmd             ${ruby.bin} -rvendor-specific -Cext/utf8 extconf.rb
  build.pre_args-delete     -C ext
  build.pre_args-append     -C ext/utf8
  destroot.pre_args-delete  -C ext
  destroot.pre_args-append  -C ext/utf8
}

So where did I go wrong on my old article? My first big mistake was thinking that the +utf8 port variant was needed. Not only is it NOT NEEDED, it may not work right at all. In fact, the old Portfile in MacPorts trunk technically did not even configure/make/install the utf8 version either! Honestly â€“ I spent all day learning the MacPort's Portfile syntax and tested this. Installing that variant just breaks with an error like LoadError: dlsym(0x1010cd3b0, Init_odbc): symbol not found.

So, even though my updated Portfile above now fixes that issue and supports actually building a utf8 version of RubyODBC, YOU DO NOT NEED IT! In fact the entire SQL Server stack is tested with the plain non-utf8 package and passes with flying colors. This includes passing tests where unicode columns return correctly utf8-encoded strings, among others. Though I have not tested it, I believe the utf8 version would do more damage than good. The RubyODBC documentation says it would make every string utf8 encoded. Not good for an adapter that has mixed data types. For my own personal notes, here is a diff of the Makefile below. I think the Init_odbc error was a result of a missing flag in the Makefile.

# My notes
# +CPPFLAGS = -DHAVE_LONG_LONG
# +CPPFLAGS = -DHAVE_VERSION_H
# +CPPFLAGS = -DHAVE_TYPE_SQLBIGINT
# -CPPFLAGS = -DHAVE_SQLCONFIGDATASOURCEW
# -CPPFLAGS = -DHAVE_SQLINSTALLERERRORW
# -CPPFLAGS = -DHAVE_SQLWRITEFILEDSNW
# -CPPFLAGS = -DHAVE_SQLREADFILEDSNW
--- Makefile(+utf8)  2010-07-18 12:00:24.000000000 -0400
+++ Makefile(-utf8)  2010-07-18 12:49:32.000000000 -0400
@@ -47,7 +47,7 @@
 CFLAGS   =  -fno-common -O2 -arch x86_64  -fno-common -pipe -fno-common $(cflags) -arch x86_64
 INCFLAGS = -I. -I. -I/opt/local/lib/ruby/1.8/i686-darwin10 -I.
 DEFS     =
-CPPFLAGS = -DHAVE_SQL_H -DHAVE_SQLEXT_H -DHAVE_TYPE_SQLTCHAR -DHAVE_TYPE_SQLLEN -DHAVE_TYPE_SQLULEN -DHAVE_ODBCINST_H -DHAVE_SQLCONFIGDATASOURCEW -DHAVE_SQLWRITEFILEDSNW -DHAVE_SQLREADFILEDSNW -DHAVE_SQLINSTALLERERROR -DHAVE_SQLINSTALLERERRORW -I/opt/local/include -D_XOPEN_SOURCE -D_DARWIN_C_SOURCE  -I/opt/local/include
+CPPFLAGS = -DHAVE_VERSION_H -DHAVE_SQL_H -DHAVE_SQLEXT_H -DHAVE_TYPE_SQLTCHAR -DHAVE_TYPE_SQLLEN -DHAVE_TYPE_SQLULEN -DHAVE_ODBCINST_H -DHAVE_SQLINSTALLERERROR -DHAVE_TYPE_SQLBIGINT -I/opt/local/include -D_XOPEN_SOURCE -D_DARWIN_C_SOURCE  -I/opt/local/include -DHAVE_LONG_LONG
 CXXFLAGS = $(CFLAGS)
 ldflags  = -L. -L/opt/local/lib -arch x86_64
 dldflags =
@@ -89,7 +89,7 @@
 LIBS = $(LIBRUBYARG_SHARED) -lodbcinst -lodbc  -lpthread -ldl -lobjc
 SRCS = init.c odbc.c
 OBJS = init.o odbc.o
-TARGET = odbc_utf8
+TARGET = odbc
 DLLIB = $(TARGET).bundle
 EXTSTATIC =
 STATIC_LIB =

Interactive Javascript Console With Textmate_redirect

Fri, 09 Jul 2010 00:00:00 -0400

Interactive JavaScript Console With TextMate

Fri, 09 Jul 2010 00:00:00 -0400

Last week I started reading JavaScript: The Good Parts by Douglas Crockford. It was on my list of long overdue things to do. While reading it, I wanted to be able to kick some simple JavaScript examples around. As rubyist we have it good, irb let's us fire up an interactive console anytime we want. But with JavaScript, options are limited. Sure you could install Johnson/EnvJS, Rhino or some other JavaScript engine. Maybe even load up firebug or the web inspector. But who wants to load a browser to play with JS?

Luckily if your are on a Mac, you do not have to worry about any of that. You already have a JavaScript engine installed. Where? Thanks to Safari's SquirrelFish Extreme (SFX), it is right in your system's library at this full path /System/Library/Frameworks/JavaScriptCore.framework/Versions/A/Resources/jsc. Instead of adding that to my path, I created a symlink to in my opt's bin directory (seeing how I have MacPorts). For instance.

sudo ln -s /System/Library/Frameworks/JavaScriptCore.framework/Versions/A/Resources/jsc /opt/local/bin

Assuming you just did that command in your own bin path of choice, you can now just type jsc and start typing JavaScript just as you easily as if you were in an irb prompt. Fun, but we can do better.

Creating A TextMate Bundle To Run JavaScript

So I was inspired by this article that explained how easy it is to make a TextMate bundle to run JavaScript thru JSLint and decided to make one to run the same JavaScript in an the interactive console for SFX. I wont go thru the details of how to add a simple TextMate bundle command, but the following picture and code sample shows the Run JSConsole command I made. The command resides in my own bundle namespace.

#!/usr/bin/env ruby

require ENV['TM_SUPPORT_PATH'] + '/lib/escape.rb'

def terminal_script_filepath
  %|tell application "Terminal"
      activate
      do script "jsc -i #{e_as(e_sh(ENV['TM_FILEPATH']))}"
    end tell|
end

open("|osascript", "w") { |io| io << terminal_script_filepath }

With that simple bundle command done, you can now use the Command-R keyboard shortcut to load the windows JavaScript file into a newly opened terminal window running in the SFX console. Pictures below.

Great, Now What?

My simple script just loads the whole file, it does not take into account TM's selected text. Nor does it do anything with selected files. I would love to see someone write a TextMate bundle that could preload up jQuery/Prototype or a set of files. The ultimate would be a bundle command that executed and updated markers like the Ruby bundle can do. Sadly, I do not have that much time on my hands :)

Bit Fat Legacy Schema Inspection_redirect

Wed, 07 Jul 2010 00:00:00 -0400

Bit Fat Legacy Schema Inspection

Wed, 07 Jul 2010 00:00:00 -0400

Sometime in rails 2.x the #inspect method for an ActiveRecord class was changed to show you all the column names (attributes) of that class. This is fine when things are small but if your working on a big legacy schema and you want clean terse debugging, all those column names can be noisy. I just set this initializer up today to kill it. Now the class just shows the number of columns.

module ActiveRecord
  class Base
    class << self

      def inspect
        if self == Base
          super
        elsif abstract_class?
          "#{super}(abstract)"
        elsif table_exists?
          "#{super}(#{columns.count} columns)"
        else
          "#{super}(Table doesn't exist)"
        end
      end

    end
  end
end

Custom Webrat Session Formatted_error For Rails With Nokogiri_redirect

Tue, 06 Jul 2010 00:00:00 -0400

Custom Webrat::Session formatted_error For Rails With Nokogiri

Tue, 06 Jul 2010 00:00:00 -0400

I never liked how Webrat shows the complete response body for exceptions when integration testing rails applications. For the longest time I redefined the formatted_error method to simple do nothing. Today I used Nokogiri to parse out the good bits from that page. Here is the result. Not pretty, but it's working fine so far. Got a better example? Please share!

module Webrat
  class Session
    def formatted_error
      doc = Nokogiri::HTML(response_body)
      exception_name = doc.css('head title').inner_html.squish
      exception_msg = doc.css('body h1').inner_html.squish
      exception_detail1 = "".tap do |detail|
        d = doc.css('body p')[0]
        detail << d.content.strip
        detail << d.next_sibling.content.squish
      end
      exception_detail2 = "".tap do |detail|
        d = doc.css('body p')[1]
        detail << d.content.strip
        detail << d.next_sibling.css('code').first.content.strip
      end
      app_trace = doc.css('#Application-Trace pre code').inner_html
      [exception_name, exception_msg, exception_detail1, exception_detail2, app_trace].join("\n")
    rescue
      "Could not format page exception. Perhaps try to use Nokogiri on this: \n#{response_body}"
    end
  end
end

The Alias_method_chain Of Rake Override Rake Task_redirect

Wed, 26 May 2010 00:00:00 -0400

The alias_method_chain of Rake - Override Rake Task

Wed, 26 May 2010 00:00:00 -0400

Rake is cool. It is built so that multiple tasks with the same name run in a reverse defined series. This is great, but sometimes you want to override a task with your own behavior and conditionally call the earlier task. Especially if that task is defined deep somewhere else, like in a rails gem. I have had to solve this problem before in Rake. Awhile back I hacked something up that would totally trump a predefined rake task and allow you to replace it with a new one. Lately while working on the SQL Server adapter, I had a need to method chain some core rails :db namespaced tasks. So once again I googled others work and again resorted to hacking Rake. Below is what I was left with.

Rake::TaskManager.class_eval do
  def alias_task(fq_name)
    new_name = "#{fq_name}:original"
    @tasks[new_name] = @tasks.delete(fq_name)
  end
end

def alias_task(fq_name)
  Rake.application.alias_task(fq_name)
end

def override_task(*args, &block)
  name, params, deps = Rake.application.resolve_args(args.dup)
  fq_name = Rake.application.instance_variable_get(:@scope).dup.push(name).join(':')
  alias_task(fq_name)
  Rake::Task.define_task(*args, &block)
end

It's easy to use, just require it in your Rakefile. In the example below, I was able to programmatically method chain the core rails db:test:purge and only call the original if I needed to. Very cool!

namespace :db do
  namespace :test do
    override_task :purge => :environment do
      ...
      # To invoke the original task add ":original" to its name
      Rake::Task["db:test:purge:original"].execute
      ...
    end
  end
end

Lastly, thanks to Eugene Bolshakov, John Wood, and Mark Foster whom have tackled this rake problem before. My version above was based on their work, but correctly works with namespaces which was critical for my needs.

Restful Ajax With Forgery Protection With Rails 3_redirect

Tue, 13 Apr 2010 00:00:00 -0400

RESTful AJAX with Forgery Protection (In Rails 3)

Tue, 13 Apr 2010 00:00:00 -0400

A while back ago I wrote an article about how to use Rails built-in forgery protection in your RESTful AJAX calls. Normally AJAX requests, those responding true to request.xhr? in rails, are forgery whitelisted. But sometimes, and under what conditions I am not sure, AJAX methods are subjected to forgery protection. Maybe you have the ActionDispatch::Request#forgery_whitelisted? overridden to not include AJAX requests? Either way and for whatever reason â€“ if you like to use forgery protection in your RESTful AJAX calls to rails, then here is the new implementation under Rails 3 beta2.

Forgery Protection In Rails 3

In a pre Rails 3 application all form tag helpers automatically create a hidden form field that contain the authentication token. These are passed when the form is serialized and if you viewed source, you could see them. However, Rails 3 is unobtrusive, which means you can count on them not polluting your markup with this kind of framework data. So where do the authentication params come from in a Rails 3 application? The answer is two places. First there is a new ActionView::Helpers::CsrfHelper module that allows you to use the #csrf_meta_tag helper in the head of your layout files. This helper will generate two meta tags, one for the authentication parameter name and the other for the value. The second place is in the new rails.js which reads these two meta tags for further interpolation into some generated form tags for all sorts of :remote links, buttons, etc. If your interested in reading more about these, look into the source, but I thought you might like to see some places where these are used by the framework now.

<head>
  <meta http-equiv="Content-Type" content="text/html; charset=utf-8"/>
  <%= csrf_meta_tag %>

document.observe("dom:loaded", function() {
  var authToken = $$('meta[name=csrf-token]').first().readAttribute('content'),
    authParam = $$('meta[name=csrf-param]').first().readAttribute('content'),
    formTemplate = '<form method="#{method}" action="#{action}">\
      #{realmethod}<input name="#{param}" value="#{token}" type="hidden">\
      </form>',
    realmethodTemplate = '<input name="_method" value="#{method}" type="hidden">';
  //... etc

Now Our RESTful AJAX with Forgery Protection

OK, now that bit of background information is out of the way, lets look into how we can use this for ourselves. Since Rails 3 has provided us with a good convention, we can hook into this pretty easily. This prototype example below is pretty simple and follows my OOJS approach. I have a base JS object that is basically a module with a function called authParams. I then mixin this base JS object into all my Prototype classes. We are reading the same meta tags that rails now uses and create a Prototype Hash object with the name/value of the authentication token. Then in ever AJAX request I merge these parameters in with what ever I am serializing for that request. Pretty simple eh? Did I miss anything?

// A base JS object.

var MyBase = {

  authParams: function() {
    var params = {}
    var authParam = $$('meta[name=csrf-param]').first().readAttribute('content');
    var authToken = $$('meta[name=csrf-token]').first().readAttribute('content');
    params[authParam] = authToken
    return $H(params);
  }

};

// Mixed Into Other Classes.

var SomeClass = Class.create(MyBase,{

  doAjaxRequest: function() {
    new Ajax.Request(elmnt.action,{
      parameters: this.form.serialize(true).merge(this.authParams()),
    });
  }

});

Textmate Theme Key Bindings For Xcode_redirect

Fri, 19 Mar 2010 00:00:00 -0400

TextMate Theme & Key Bindings For Xcode

Fri, 19 Mar 2010 00:00:00 -0400

I've been using Xcode for about 8 hard core months now and for all this time I have been using a theme I made that mimics the All Hallow's Eve TextMate theme with a bunch of custom key bindings that make Xcode more TextMate compatible. Here are links to download both below and a few notes I have put together. As I remember more of what I have actually done, this post will get updated. I do remember the first thing I did with Xcode for almost 3 days was just to customize the heck out of it and these files are the results.

To use these download and unzip them. Please the theme in ~/Library/Application Support/Xcode/Color Themes and place the bindings in ~/Library/Application Support/Xcode/Key Bindings. That should be it. Hare are my notes for now on what the key bindings do.

Misc Key Bindings

Command-W - Works on a file! Not the whole project window!
Command-1 - Open File List
Command-2 - Method List

TextMate Style Key Bindings

Command-T - for Open Quickly
I'll update this as I find more, it has honestly been so long I forget all the work I've done and just use this now as if it were TextMate.

Git Init Xcode Projects_redirect

Tue, 23 Feb 2010 00:00:00 -0500

Git Init XCode Projects

Tue, 23 Feb 2010 00:00:00 -0500

Here is a little ZSH function I have been using for quickly setting up new XCode apps I call tire kickers, little play and learn apps. Being able to track your learning as you go with git.

if [[ -x `which git` ]]; then

    function ginit_xcode () {
      git init
      echo "\n\n# XCode\nbuild\n*.mode1v3\n*.mode2v3\n*.nib\n*.swp\n\
*.pbxuser\n*.perspective\n*.perspectivev3\n\n# OSX\n.DS_Store\n\n\
# TextMate\n*.tm_build_errors\n\n\n" >> .gitignore
      git add .gitignore
      git commit -m "Ignore Xcode stuff."
      git add .
      git commit -m "Initial Xcode project."
    }

fi

The echo lines puts out a .gitignore file that will look something like this.

# XCode
build
*.mode1v3
*.mode2v3
*.nib
*.swp
*.pbxuser
*.perspective
*.perspectivev3

# OSX
.DS_Store

# TextMate
*.tm_build_errors

Synchronizing Core Data With Rails 3 0 0 Pre_redirect

Fri, 12 Feb 2010 00:00:00 -0500

Synchronizing Core Data With Rails (3.0.0.pre)

Fri, 12 Feb 2010 00:00:00 -0500

This is my presentation to our local @757rb/@757objc users group this past Tuesday. Hope some find it useful. Lessons learned from building HomeMarks native iPhone application to synchronize Core Data with a RESTful backend built using rails 3.0.0.pre. This covers a previous design methodology called the AJAX head pattern which decouples rails applications from the views they present which allowed an easy API foundation for the iPhone application and data sync methods.

Resources

Simple Script Console Function_redirect

Sat, 06 Feb 2010 00:00:00 -0500

Simple Script/Console Function

Sat, 06 Feb 2010 00:00:00 -0500

This is something simple I worked up today for my ZSH profile that let's me keep my simple sc alias and have it work with all versions of rails. If you did not know, all the script files in rails 3 are gone and the new all-in-one rails executable does all the heavy lifting. This little function even passes down all the arguments too.

sc () {
  if [ -f ./script/rails ]; then
    rails console $argv
  else
    ./script/console $argv
  fi
}

One other thing, the way rails uses IRB is different now. I had to change my ~/.irbrc file to look like this below to get my simple prompt and history back. IMPORTANT NOTE: In order for this to work, you have to apply this 2 line patch to your save-history.rb file. Worked like a champ for me.

# IRB history patch <http://redmine.ruby-lang.org/issues/show/1556>

require 'irb/completion'
require 'irb/ext/save-history'

IRB.conf[:USE_READLINE] = true
IRB.conf[:SAVE_HISTORY] = 500
IRB.conf[:HISTORY_FILE] = "#{ENV['HOME']}/.irb.hist"
IRB.conf[:PROMPT_MODE] = :SIMPLE

Unobtrusive Js In Rails 3 With Prototype_redirect

Fri, 29 Jan 2010 00:00:00 -0500

Unobtrusive JS In Rails 3 With Prototype

Fri, 29 Jan 2010 00:00:00 -0500

Are you bleeding on the edge of rails 3 and need to shim up some unobtrusive JavaScript to work with your link_to code that uses a destructive :method option? I did today and here is what I did to solve it. If you are unfamiliar with the problem, and what has been happening in rails 3 with UJS, check out Piotr Solnica's blog for a good run down. Or you can check out the simple code block below.

# This Ruby
link_to 'Logout', session_path, :method => :delete

# Will out put this HTML in Rails 3
# <a href="/session" data-method="delete" data-url "/session" rel="nofollow">Logoout</a>

So no more tag soup. Yea! There was much rejoicing, but I could not find any illustrated examples of what type of JavaScript to use to back this up. There are two problems at play here. First, simple link <a> tags are not forms and hence methods like post/put/delete are not going to be solved with simple query params. Second, because link tags are not forms, there is no authenticity_token hidden in the generated form, like a button_to would generate. Let's ignore the second problem for a bit and see what I came up with for using with Prototype.

var MyJsObject = {

  linkToDelete: function(event) {
    event.stop();
    var e = event.element();
    var a = e.readAttribute('href') ? e : e.up('a');
    var form = FORM({action:a.href, method:'post'},[
      INPUT({type:'hidden',name:'_method',value:'delete'}),
      INPUT({type:'hidden',name:'authenticity_token',value:authParams.get('authenticity_token')})]
    );
    form.submit();
    return false;
  }

};

document.observe('dom:loaded', function(){
  $$('a[data-method=delete]').each(function(a){ a.observe('click', MyJsObject.linkToDelete); });
});

In this example I am only covering the DELETE action. This code could be abstracted to take a "method" argument and use it. As you can see, it's just a simple iteration over the links with custom "data-method" attributes and attaching a function to it. For brevity, I am using the Builder.js methods for building DOM objects. See anything odd up there? The authentication token? You got it, destructive actions like POST/DELETE/PUT will need that token. I wrote up a great article a year or so ago that still works today titled RESTful AJAX with Forgery Protection (In Rails3) that gives me that nice global var. Check it out for the last piece to this puzzle and have fun working on the edge!

Quickie Js Prototype Wrapper For Quicktime_redirect

Mon, 25 Jan 2010 00:00:00 -0500

Quickie.js - Prototype wrapper for QuickTime

Mon, 25 Jan 2010 00:00:00 -0500

As some of you know, I am in the last steps of announcing my first iPhone application. We all know that every good iPhone application has a great marketing website with a screen cast. I myself was heavily inspired by the Tapbots Convertbot website while building my own and wanted a good way of embedding the screen cast. I think the last time I did an object/embed tag was god... around 2003 or something, seriously.

So after looking at the source of the Convertbot website, I was happy to find a nice little JavaScript wrapper for QuickTime called Quickie.js. This is a MooTools class created by Jose Prado that helps generate an object or embed tag appropriate for the requesting browser in an abstract way. Here is the link to the original Quickie.js. If you use MooTools, fine, it is real close to Prototype and has my respect, but if you use Prototype, here is my rewrite of the class.

/*

Author:
  Ken Collins <metaskills.net>

Options:
  id - (string: defaults to 'Quickie_' + unique id) The id of the Quickie object.
  width - (number: defaults to 1) The width of the Quickie object.
  height - (number: defaults to 1) The height of the Quickie object.
  container - (element) The container into which the Quickie object will be injected.
  attributes - (object) QuickTime attributes for the element. See http://www.apple.com/quicktime/tutorials/embed.html for possible attributes.

Returns:
  - (element) A new HTML object Element with browser appropriate QuickTime embed code.

Example:
  var myQuickie = new Quickie('myMovie.mov', {
    id: 'myQuicktimeMovie',
    width: 640,
    height: 480,
    container: 'qtmovie',
    attributes: {
      controller: 'true',
      autoplay: 'false'
    }
  });

Credits:
    Mootools Implementaiton: http://pradador.com/code/quickiejs/

License:
    MIT-Style License

*/

var Quickie = Class.create({

  initialize: function(path, options){
    var time = Try.these(
      function() { return Date.now() },
      function() { return +new Date }
    );
    var defaultOptions = {
      id: null,
      height: 1,
      width: 1,
      container: null,
      attributes: {}
    }
    this.path = path;
    this.instance = 'Quickie_' + time;
    this.options = Object.extend(defaultOptions, arguments[1] || {});
    this.id = this.options.id || this.instance;
    this.container = $(this.options.container);
    this.attributes = this.options.attributes;
    this.attributes.src = this.path;
    this.height = (this.attributes.controller == 'true') ? this.options.height + 16 : this.options.height;
    this.width = this.options.width;
    this._assignElement();
  },

  toElement: function() {
    return this.element;
  },

  _assignElement: function() {
    var build = (Prototype.Browser.IE) ? this._buildObjectTag() : this._buildEmbedTag();
    this.element = ((this.container) ? this.container.update('') : new Element('div')).update(build).down();
  },

  _buildObjectTag: function() {
    var build = "";
    build = '<object classid="clsid:02BF25D5-8C17-4B23-BC80-D3488ABDDC6B" codebase="http://www.apple.com/qtactivex/qtplugin.cab"';
    build += ' id="' + this.id + '"';
    build += ' width="' + this.width + '"';
    build += ' height="' + this.height + '"';
    build += '>';
    for (var attribute in this.attributes) {
      if (this.attributes[attribute]) {
        build += '<param name="' + attribute + '" value="' + this.attributes[attribute] + '" />';
      }
    }
    build += '</object>';
    return build;
  },

  _buildEmbedTag: function() {
    var build = "";
    build = '<embed ';
    build += ' id="' + this.id + '"';
    build += ' width="' + this.width + '"';
    build += ' height="' + this.height + '"';
    for (attribute in this.attributes) {
      if (this.attributes[attribute]) {
        build += ' ' + attribute + '="' + this.attributes[attribute] + '"';
      }
    }
    build += ' pluginspage="http://www.apple.com/quicktime/download/"></embed>';
    return build;
  }

});

If you ask me this is far cleaner than the original. It encapsulates factory methods and all the initialize does is just setup vars, not do ALL the work in one big procedural way. So what does this class do? It's simple, it allows you to create JavaScript objects that will represent DOM objects for QuickTime movies. This object can be passed to any method that takes expects and element to Prototype since it has a toElement method. Here is the way I am using it in my upcoming project. Just like the Tapbots website, I am creating

tags that have all the attributes I need and are the container elements. On page load, I update all those containers with generate QuickTime source code. See the example below.

/*
Example Container
<div id="mymovie_container" class="quicktime_video" rel="src=/video/mymovie.mov|width=280|height=393|loop=true|autoplay=true|video_id=mymovie"></div>
*/

document.observe('dom:loaded', function(){
  $$('.quicktime_video').each(function(container){
    var attributes = {};
    attributes.controller = 'false';
    attributes.autoplay = 'true';
    attributes.loop = 'false';
    attributes.enablejavascript = 'false';
    container.readAttribute('rel').split('|').each(function(keyvalue) {
      pair = keyvalue.split('=');
      attributes[pair[0]] = pair[1];
    });
    var qt = new Quickie(attributes['src'], {
      id: attributes['video_id'],
      width: parseInt(attributes['width']),
      height: parseInt(attributes['height']),
      container: container,
      attributes: attributes
    });
    container.update(qt);
  });
});

Resources

Quickie.js on Github

Rails Button Links In Embedded Forms_redirect

Tue, 05 Jan 2010 00:00:00 -0500

Rails Button Links In Embedded Forms

Tue, 05 Jan 2010 00:00:00 -0500

This is one I have had sitting around for almost 3 years now in my toolbox and thought I would share. Have you ever had complicated rails forms and needed simple form buttons that just took you to a simple link? Were you bitten by the button_to helper code because it generates another form inside of a form? If so, here is a simple rails view helper I made that creates simple button links for embedded forms by making an input with a javascript function. Tag soup you ask, hell yeah, but worth if if you need it.

def button_to_link(name, link, options={})
  confirm_option = options.delete(:confirm)
  popup_option = options.delete(:popup)
  link_function = popup_option ? redirect_function(link,:new_window => true) : redirect_function(link)
  link_function = "if (confirm('#{escape_javascript(confirm_option)}')) { #{link_function}; }" if confirm_option
  button_to_function name, link_function, options
end

def redirect_function(location, options={})
  location = location.is_a?(String) ? location : url_for(location)
  if options[:new_window]
    %|window.open('#{location}')|
  else
    %|{window.location.href='#{location}'}|
  end
end

Authenticated S3 GETs For Private Objects Using Paperclip

Mon, 23 Nov 2009 00:00:00 -0500

Yea I know, I am probably the last person on earth that is just getting around to using Paperclip. To be honest, most of my file upload code was written way before Paperclip or even AttachementFu was ever conceived. And frankly, I do not do much social app coding on the side - so the need never came up. But that changed recently and I wanted a really really good way of leveraging AWS::S3 storage with the best local app security while maintaining tight control over the files.

So the Paperclip wiki has a few links that already dealt with some ways of protecting your app's attachments. One mentions a method I totally love called security through obscurity. It uses a secure random token as part of the filename which combined with the original filename and the id partition makes for great random URLs. The other is a great walk thru on how to use the :url option of paperclip to point access control back to your own application for your normal biz logic.

The problem I see with both of these methods is that they do not allow you to maintain app control past the final URL handoff/redirect. It also requires that your S3 bucket is public. For instance, if you were to use the s3_permissions => :private option of Paperclip, then that URL given to you by Paperclip is pretty much worthless. I knew AWS::S3 had authenticated GETs that generated an automatically expiring URL, but saw no way of accessing its features using the abstract Paperclip::Attachment object. So this is what I did.

class MyDownload

  has_attached_file :attachment,
                    :storage => :s3, :bucket => 'mybucket',
                    :s3_credentials => {...}, :s3_protocol => 'https',
                    :s3_permissions => :private,
                    :path => lambda { |attachment| ":id_partition/#{attachment.instance.random_secret}/:filename" },
                    :processors => [:noop]

  before_validation_on_create :set_random_secret


  def attachment_url
    "#{self.class.tableize}/#{id}/#{attachment_file_name}"
  end

  def authenticated_s3_get_url(options={})
    options.reverse_merge! :expires_in => 10.minutes, :use_ssl => true
    AWS::S3::S3Object.url_for attachment.path, attachment.options[:bucket], options
  end

  private

  def set_random_secret
    self.random_secret = ActiveSupport::SecureRandom.hex(8)
  end

end

Let me walk you thru some of the highlights of that class, the general concept following is that we are going to use the best of both examples in security mentioned above. First, the secure token, that is what #set_random_secret will generate for each instance. The :path option for Paperclip uses a proc to make sure each instance uses that attribute in the string that will be later interpolated further down. You can also see how I use the id partition too. Next, I have added two public instance methods. The first is #attachment_url and it will need a bit of explaining.

Currently in Paperclip, if you use the :url option and your :storage is set to :s3, then it is ignored. This could totally be intentional. So in a typical setup where you want the file download running through your own access control, you wold have a :url option like this :url => '/:class/:id/:filename'. So this is what #attachment_url mimics, it simply gets around that shortcoming and points the download action back to your own controller. How that controller would work is beyond the scope of this article, see the resources section below for those links.

The last example method is #authenticated_s3_get_url which dips right on down to the AWS::S3 library to get the URL for the object in the bucket. AWS::S3's doc mention that it will automatically generate a secure GET url that expires in 5 minutes. However in my example, you can see where I am changing that to 10 minutes and forcing the HTTPS protocol. This would be the URL that your own controller would do the final redirect to. This URL is for your private objects in your S3 bucket and will only work for the amount of time you want it too! Meaning your app stays in complete control. Putting it all together one more time...

# A MyDownload instance.
>> dl = MyDownload.find(4)

# This is totally useless for private buckets/objects.
>> dl.attachment.url
=> "https://s3.amazonaws.com/mybucket/000/000/004/147681c16fddc1e5/private.pdf?1258989107"

# This is what you use in your own views.
>> dl.attachment_url
=> "my_downloads/4/private.pdf"

# Your controller would redirect to this secure GET.
>> dl.authenticated_s3_get_url
=> "https://s3.amazonaws.com/mybucket/000/000/004/147681c16fddc1e5/private.pdf?AWSAccessKeyId=0HJD3NS9CVWN2JV89K02&Expires=1258990967&Signature=8aWsq4o5gXfpIrRZyeETddnOeFw%3D"

What Is That Noop Processor

Good eye! Did you see that I have a processor called Noop in the has_attached_file declaration? The default processor in Paperclip is the Thumbnail processor, which no matter what calls the ImageMagick identify command to see if it can do something to the file. I did not want that or any processing, just simple attachments. So I created this simple processor that just straight returns the file object. I made a ticket on the Paperclip's issue page that hopefully would allow a :processors => false option one day that would do this as well. So maybe one day it'll be a feature.

module Paperclip
  class Noop < Processor

    def make
      file
    end

  end
end

UPDATE: Fog Example

Here is an updated #authenticated_s3_get_url that I use with Fog as the backend storage for S3. This example illustrates 2 things. First that the bucket is named as your CNAME record which is also enforced in the :fog_host. I do this so that I can generate URLs via Paperclip or Fog and always have my custom domain in place. When I gsub the "s3.amazonaws.com" out, I am left with just my CNAME domain/bucket name. Lastly, if you want to generate "https" URLs, use the #get_https_url method.

class MyDownload

  has_attached_file :attachment,
                    :storage => :fog,
                    :fog_credentials => {...},
                    :fog_directory => 'mybucket.domain.net',
                    :fog_public => false,
                    :fog_host => 'http://mybucket.domain.net',
                    :path => lambda { |attachment| ":id_partition/#{attachment.instance.random_secret}/:filename" }

  def authenticated_s3_get_url(expires = nil)
    expires ||= 10.minutes.from_now
    url = attachment.send(:directory).files.get_http_url attachment.path, expires
    url.gsub! /s3.amazonaws.com\//, ''
  end

end

Resources

Meta Programming In...

Wed, 11 Nov 2009 00:00:00 -0500

Last nights 757.rb meeting was a great success. We did a talk titled Introduction To Ruby & Rails for all the new comers that might have been interested in learning more about both - from the ground up. Since Ruby was my first language and my toolbelt only extends to JavaScript and Objective-C, I'm always interested in how other languages do the things that I love so much in Ruby. One of the biggest pluses for Ruby, to me, is the support for meta programming. In a typical rails model, you might see something like this.

class User < ActiveRecord::Base
  has_many                :articles
  named_scope             :active, :conditions => {:active => true}
  validates_uniqueness_of :email
  validates_presence_of   :email, :username, :twitter_handle
end

For the untrained, those are class level methods writing code for an model that persists objects to a database. It looks like you are typing a spec or some declaration. In fact this is code writing other code. For newcomers to the language this is voodoo magic, but if you write programs and interfaces for others, meta programming is what it is all about. In fact, I use this technique all the time instead of writing the same code over and over again.

This concept is built into the core of Ruby. In fact here is an example from my talk on the class method built into Ruby called attr_accessor which wraps a common paradigm of generating getter/setter methods for an instance variable. No magic here, the two class definitions below are equal and the interface of the object is the same. One is you do not have to write code over an over again in an vanilla class you care to have getter/setter methods for.

class MyClass
  attr_accessor :foo
end

# Same as writing

class MyClass
  def foo
    @foo
  end
  def foo=(value)
    @foo = value
  end
end

# Same results

o = MyClass.new
o.foo           # => nil
o.foo = 'bar'   # => "bar"
o               # => #<MyClass:0x100172b80 @foo="bar">

So here is where I want to learn. I know how to write Ruby well enough that I can show one way below of how to write attr_accessor if it did not exist in core Ruby. In my example below I made the method named attribute_accessor to avoid the name conflict. Can anyone show me how this would be done in PHP, Java, Anything? What is meta programming like in those languages? How about posting a code example on pastie.org and school me. I really am interested in knowing. BTW, my example only shows how the attribute_accessor takes on argument. In actuality in Ruby, the built in attr_accessor method takes multiple args and generates methods for each. For simplicity I narrowed down my examples and code to only take one argument so we can all stay focused.

module MetaSkills

  def self.included(klass)
    klass.class_eval do
      include ClassMethods
    end
  end

  module ClassMethods

    def attribute_accessor(name)
      attribute_reader(name)
      attribute_writer(name)
    end

    def attribute_reader(name)
      define_method(name) do
        instance_variable_get "@#{name}"
      end
    end

    def attribute_writer(name)
      define_method "#{name}=" do |value|
        instance_variable_set "@#{name}", value
      end
    end

  end

end

Class.send :include, MetaSkills

class MyClass
  attribute_accessor :foo
end

o = MyClass.new
o.foo           # => nil
o.foo = 'bar'   # => "bar"
o               # => #<MyClass:0x100171c80 @foo="bar">

Update: A friend was asking me about the namespaces you see in the module. It is technically possible to just open up the Class object and add my example additions. Typically in Ruby you are encouraged to create your own namespaced module and inject class/instance modules into the included class via the module's hook method self.included. The pattern is like so http://pastie.org/694019. So for instance, I could have done this http://pastie.org/694038 without taking advantages of the modules hook method.

Installing REE With The Snow Leopard SQL Server Stack

Tue, 27 Oct 2009 00:00:00 -0400

Today I noticed that Ruby Enterprise Edition 2009.10 was released and I have really been wanting to see if I could get the SQL Server adapter tested and running under it. I am really curious how the speed improvements might look and will share my results below. This article assumes that you read my previous guide titled The Ultimate OS X Snow Leopard Stack For Rails Development - x86_64, MacPorts, Ruby 1.8/1.9, SQL Server, SQLite3, MySQL & More as I will be building on top of it and referencing certain steps. So let's get down to business.

Installing REE

The guys at Phusion have done a rock solid job. My previous attempts to install REE were a bomb, but it worked perfectly this time around. Simply follow their directions and all will be OK. A snippet of my output is below. I liked how they picked a new default install directory of /opt/ruby-enterprise-1.8.7-2009.10. Wicked cool! By the way, the only issue I had during install was when they were trying to install the pg gem for me. I ignored that error for now.

$ sudo ./ruby-enterprise-1.8.7-2009.10/installer

Where would you like to install Ruby Enterprise Edition to?
(All Ruby Enterprise Edition files will be put inside that directory.)
[/opt/ruby-enterprise-1.8.7-2009.10] :

Configuring For REE

This is actually really easy. In my previous article, I talked about installing MacPorts and configure your profile. All I did this time around was add the REE path extensions after my MacPort extensions in my ZSH kit. So my path file now looks this below. If I ever want to just go back to my normal 1.8/1.9 toggle in /opt/local/bin, I just comment out the REE path extensions, source my profile and I'm back to my MacPort basics.

# MacPorts
path=(/opt/local/bin /opt/local/sbin /opt/local/apache2/bin ~/.zshkit/bin $path)
manpath=(/opt/local/share/man $manpath)
infopath=(/opt/local/share/info $infopath)

# REE
path=(/opt/ruby-enterprise-1.8.7-2009.10/bin $path)
manpath=(/opt/ruby-enterprise-1.8.7-2009.10/share/man $manpath)
infopath=(/opt/ruby-enterprise-1.8.7-2009.10/share/info $infopath)

In the last guide, I mentioned how to switch between ruby 1.8/1.9 installed by MacPorts using a simple ZSH function. You can find this in my article title MultiRuby The MacPorts Way. Testing Your Rails Apps With Ruby 1.9. I have updated this to include a condition that will noop the function if you are running REE. So to recap. I use my chruby function to switch between 1.8/1.9 installed by MacPorts and I use my profile extension to use REE. So what that profile change done... reload and/or open a new terminal window and test it out.

$ which ruby
/opt/ruby-enterprise-1.8.7-2009.10/bin/ruby
$ which gem
/opt/ruby-enterprise-1.8.7-2009.10/bin/gem

Using REE With SQL Server

Let's get the easy stuff out of the way and install the needed gems for the adapter to work. Again, always follow the TODO on the adapter page for the latest blessed versions of DBI and low level connectivity stuff. But just to recap the current needs.

$ sudo gem install dbi -v 0.4.1
$ sudo gem install dbd-odbc -v 0.2.4
$ sudo gem install activerecord-sqlserver-adapter

Now the hard part, but I have an easy solution. You can do this the hard way and recompile the UTF8 version of ruby odbc by hand or you can type one command. I tried compiling it by hand and found that I had a few bugs, even after following Jacob Riff's notes when he had a similar issue when trying to hook ruby ODBC to use unixODBC in his /usr/local . My solution was much simpler. I just coped my ruby 1.8.7 ruby odbc to the REE's vendor ruby directory. Hence:

$ sudo cp /opt/local/lib/ruby/vendor_ruby/1.8/i686-darwin10/odbc.bundle /opt/ruby-enterprise-1.8.7-2009.10/lib/ruby/vendor_ruby/1.8/i686-darwin10.0.0

Here are my notes below on where I failed on installing ruby odbc by hand under REE. Feedback welcome, but good to know that the above simple copy worked just fine.

# Installing Ruby ODBC By Hand Notes

$ ruby -Cutf8 extconf.rb
$ make -C utf8
$ sudo make -C utf8 install
  /usr/bin/install -c -m 0755 odbc_utf8.bundle /opt/ruby-enterprise-1.8.7-2009.10/lib/ruby/site_ruby/1.8/i686-darwin10.0.0
$ sudo mv /opt/ruby-enterprise-1.8.7-2009.10/lib/ruby/site_ruby/1.8/i686-darwin10.0.0/odbc_utf8.bundle /opt/ruby-enterprise-1.8.7-2009.10/lib/ruby/site_ruby/1.8/i686-darwin10.0.0/odbc.bundle

Loading development environment (Rails 2.3.4)
dlsym(0x102c6e330, Init_odbc): symbol not found - /opt/ruby-enterprise-1.8.7-2009.10/lib/ruby/site_ruby/1.8/i686-darwin10.0.0/odbc.bundle
/opt/ruby-enterprise-1.8.7-2009.10/lib/ruby/site_ruby/1.8/i686-darwin10.0.0/odbc.bundle
/opt/ruby-enterprise-1.8.7-2009.10/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:in `require'

Benchmarking REE vs MacPorts

Here is the good news, the above hacks and REE work! I have even tested the adapter and all is looking good from my end!!! Now how about those benchmarks? Here are the speed differences for running the full adapter test suite under 1.8.7 installed in MacPorts and the latest REE. Note, this is really non-scientific benchmarks, but I think they fun facts.

Ruby 1.8.7 MacPorts (patchlevel 174)
Finished in 158.574243 seconds.
2190 tests, 7395 assertions, 0 failures, 0 errors

REE 1.8.7 (2009.10)
Finished in 146.834762 seconds.
2190 tests, 7636 assertions, 0 failures, 0 errors

Installing Passenger

Since passenger was installed as part of the REE installation, all I had to do was run the apache2 installer again and paste in the new code into my httpd.conf file.

$ which passenger-install-apache2-module
/opt/ruby-enterprise-1.8.7-2009.10/bin/passenger-install-apache2-module
$ sudo passenger-install-apache2-module

The Zombie Shotgun Revisited

Sun, 11 Oct 2009 00:00:00 -0400

My how time flies. Over a year ago I created a simple bit of code that was useful for stopping ActionController routing errors from common Microsoft attacks from sending exception notification emails. Well now most people do not use exception notifications emails in favor of apps like Hoptoad. And hey, most code like this has moved to Rack middlewares.

Yesterday I noticed a rack code competition that encouraged "most useful and top quality Rack middlewares". Well the Zombie Shotgun is pretty useful to me, but I'm sure it's not top quality. That said, I did take the time to finally pick up on the rack internals and learn how to use rack-test. If you want to check out the new tested Zombie Shotgun middleware and how I tested it using Shoulda, go see the project on my github page. Also, here is my CodeRack entry.

757 Studio_redirect

Sun, 04 Oct 2009 00:00:00 -0400

757 Studio

Sun, 04 Oct 2009 00:00:00 -0400

Well it took up quite a bit of my free time and a good 2 weeks of no iPhone development, but I am very very happy that the organization of my first event has been a success so far. Check out 757studio.org for full details. Andy Hunt is headlining the event with a talk on Pragmatic Thinking and Learning. Clinton Nixon and Jamie Pinkham will be following with talks on Ruby/Rails and iPhone/Cocoa. So even though I can not make it to the West Coast for RubyConf this year, it is not so bad with a local event like this.

Thanks so much to the Pragmatic Studio and the Pragmatic Bookshelf for sponsoring the event and most important to the speakers who are giving the talks. It is my sincere hopes that the Hampton Roads area can support more talks like this and hopefully the advocacy of Ruby/Objective-C to local software developers and business will pay off, for everyone!

Some Background

I myself designed and developed the 757 Studio site. Yes, I used to be a designer back in the day, not a good one, but enough to get by. It took me a few days to get to something in PhotoShop that looked right to me. I'm digging the brick and wood masthead. I think it gives the site both a modern ".com" office look, even though some say it feels dated. On the technical side, this was my first experience using HAML and SASS.

I loved HAML so much, I doubt I'll ever go back to ERB templates in Rails again. The forced indentation and not having to manage closing tags was wonderful, it kept my views clean and forces you to use helpers like they should vs inlining a bunch of control structures. SASS, the companion to HAML is for generating your CSS. It too forces you to write a strict indentation for you CSS which works wonders. Not only will it generate deep selectors that maintain good scope, but the the layout helps you really understand where in the DOM tree you are.

If you want to see how all this came together and how I quickly hashed up a battle tested rails app for the studio's RSVP/Reservation system, the project is on my Github page. Check out the app/stylesheets/site.sass for the SASS CSS template and all the other views use HAML. Oh yea... I should also say that this was my first time playing with some webkit animations and effects. You may be able to spot them if you are on Safari/WebKit/Chrome.

Perform Occupational Maintenance_redirect

Mon, 21 Sep 2009 00:00:00 -0400

Perform Occupational Maintenance!

Mon, 21 Sep 2009 00:00:00 -0400

One of the things that bugs me about some people in the technology field is that they do not perform enough occupational maintenance. Today I just purchased Agile Coaching and Beginning Mac Programming: Develop with Objective-C and Cocoa. Both from the Pragmatic Bookshelf.

If you have not done so already, do yourself a favor and go to the Pragmatic Bookstore, check yo self, and get better at what you do! Here is my bookshelf to date.

The Ultimate Os X Snow Leopard Stack For Rails Development X86_64 Macports Ruby 1 8 1 9 Sql Server More_redirect

Sat, 05 Sep 2009 00:00:00 -0400

The Ultimate OS X Snow Leopard Stack For Rails Development - x86_64, MacPorts, Ruby 1.8/1.9, SQL Server, SQLite3, MySQL & More

Sat, 05 Sep 2009 00:00:00 -0400

This guide is all encompassing but primarily focuses on the benefits of MacPorts, second the development stack for SQL Server and lastly on anything else a rails developer might need on OS X. If you are on a Mac, possibly running Snow Leopard and x86_64 is near and dear to your heart, this article is for you. If you do not "have" to use SQL Server, you can safely skip those sections and get to the Ruby1.9/Apache2/SQLite3/MySQL stuff.

A small forward. If you are developing rails applications on a modern OS such as Snow Leopard and your needs never make it out of the shallow end of the default Ruby and SQLite3, then this how-to might not be for you. However if you find yourself needing more and you want it to be easy and optimized for your architecture, then I highly suggest MacPorts.

Even though this tutorial is based on MacPorts, there are alternatives if you have the guts. One is called Homebrew by Max Howell, which offers a very easy package management system that is closely knit to OS X's own libraries. If your smart enough to manage the sheer amount of dependencies and different ways OS X wants to pull you when using their libraries, then have at it. I failed horribly and I consider myself somewhat smart. This article also focuses on running multiple versions of ruby, specifically 1.8.7 and a flavor of 1.9.x. Again to this end there are other solutions. One recent one is called Ruby Version Manager (rvm) by Wayne Seguin. An incredible library, but something I could not use with my level of compiling source due to the fact that all the dependencies needed for SQL Server to ODBC are hard enough to get in place for one ruby installation. RVM has it's use, but if you just need two versions of ruby with a full 64-bit stack to SQL Server, read on.

Install MacPorts

If you have not done so already, go download MacPorts and run the installer and follow their installation instructions. At the time of this writing, the current version is 1.8 and has tons of new stuff for Snow Leopard. If you use bash, the default shell on OS X, then add this to your ~/.profile.

export PATH="/usr/local/bin:$PATH"
export MANPATH="/usr/local/share/man:$MANPATH"
export INFOPATH="/usr/local/share/info:$INFOPATH"

However, if you have changed your default shell to ZSH, then put this in your ~/.zshenv file. I highly recommend switching your default shell to ZSH.

path=(/opt/local/bin /opt/local/sbin /opt/local/apache2/bin $path)
manpath=(/opt/local/share/man $manpath)
infopath=(/opt/local/share/info $infopath)

Important x86_64 Arch Notice

If you are running Snow Leopard and if your processor supports 64-bit, say a Core 2 Duo (consult your manual), then I highly suggest you set the MacPorts build_arch to force it to x86_64 for all installed ports. Remember, even if you have a unibody MacBook that can not boot the 64-bit kernel yet, that does not mean you are not running 64-bit apps. If your chip supports it and your on Snow Leopard, turn it on. Here is how.

Open up your MacPorts config file located at /opt/local/etc/macports/macports.conf. Around line 59, find the section that has the build_arch and set it to x86_64. I have done this on both my unibody MacBook 13" and Nehalem MacPro and everything has installed and works perfectly. This includes, ruby, git, and everything in between. See screenshots below for proof. Lastly, as a good habit, always run sudo port selfupdate before installing ports. This makes sure you have the latest port files.

Installing Ruby 1.8.7

By now Ruby 1.8.7 is the new 1.8.6. It will be the de-facto low level version that supports the enterprise. Frankly, 1.9.2 is really where it is at, but we all have our day jobs right. Either way it is good to have this version installed. You may ask why, 1.8.7 comes on OS X by default now in Snow Leopard? Correct, but the goal here is to get a base ruby installed whose libs are known and compiled the way we want them for our low level SQL Server libraries and their dependencies. The goal here is to avoid dependency hell.

$ sudo port install ruby

$ which ruby
/opt/local/bin/ruby

$ ruby -v
ruby 1.8.7 (2009-06-12 patchlevel 174) [i686-darwin10]

Running that port install may have taken awhile, but let me show you something. Do you see that below? That's right, it's 64-bit ruby just like the default OS X installed 1.8.7. In fact, it's even a bit faster for some reason. So remember, using MacPorts this way means that everything you install that supports it, will be built for 64-bit.

Lets get RubyGems package management up and running. Thanks again to MacPorts this is really easy. Some advice. Never use MacPorts to update RubyGems! We just used it to get it installed easily and quickly and in the right scope. If you need to update RubyGems, just use it's own self update feature.

$ sudo port install rb-rubygems

Feeling good? Maybe like me, you wouldn't mind getting side tracked. If so, go ahead and install Subversion and/or Git. I personally like to install both of these since I use Git to manage my Subversion repos too.

$ sudo port install subversion
$ sudo port install git-core +svn

The SQL Server Connection Stack

So back on track. The low level connection stack for SQL Server is pretty easy. We need unixODBC and not OS X's installed iODBC. After that FreeTDS needs to be installed with the +odbc variant so that it prefers the installed unixODBC. If you did not get side tracked above, these two installs could pull in a lot of dependency installs too and take some time. When we are done the final stack for connecting Rails to SQL Server will look something like this. Ruby/Rails => ActiveRecord SQL Server Adapter => DBI Gem w/DBD-ODBC => Ruby ODBC => unixODBC => FreeTDS => SQL Server.

$ sudo port install unixODBC
$ sudo port install freetds +odbc

If all went well, you should see a configuration file notice after FreeTDS installs. This is just the port reminding us that we will ultimately have to configure FreeTDS to point to our SQL Server box. We'll get back to that later, for now, let's check that the install is looking good.

``` $ which tsql /opt/local/bin/tsql $ tsql -C Compile-time settings (established with the "configure" script) Version: freetds v0.82 freetds.conf directory: /opt/local/etc/freetds MS db-lib source compatibility: no Sybase binary compatibility: no Thread safety: yes iconv library: yes TDS version: 5.0 iODBC: no unixodbc: yes $ tsql I locale is "en_US.utf-8" locale charset is "utf-8" ```

See that last line after tsql -C? It says unixodbc is a go! That's important. If for some reason you do not see this, then I suggest reinstalling unixODBC and FreeTDS in that order again. Also note the output of tsql I and check out the locale settings. On Mac OS X, this seems to default to the utf-8 stuff we would expect. Possibly due to the default LANG env var is set to en_US.utf-8. However, on other systems this may note be right. See the configure FreeTDS section below for what to do if this is the case.

Right. That completes the UNIX'y low level parts to get us talking to SQL Server, now to put the focus back on Ruby again by installing a few gems that will be needed. Here are the ones critical to this setup. Please consult the adapter's README for update to date version information on the approved/latest gems for DBI.

$ sudo gem install dbi -v 0.4.1
$ sudo gem install dbd-odbc -v 0.2.4
$ sudo gem install activerecord-sqlserver-adapter

Almost there, we need one more piece, the Ruby ODBC bindings, so we can talk to the low level unixODBC. At the time of this writing, the MacPort portfile for rb-odbc was stuck at 0.9995 while the ruby odbc project page reports that the latest version is 0.9997. So here is a little lesson on how to edit a portfile to get our desired results.

$ port file rb-odbc
/opt/local/var/macports/sources/rsync.macports.org/release/ports/ruby/rb-odbc/Portfile

$ port file rb-odbc | xargs mate

The first command simply shows you how you can get MacPorts to revel where the port file is located. The second shows how you can pipe that result to xargs to get TextMate to open the portfile. Now that we have the port file in front of us, let's edit it so that it pulls the latest. First change all occurrences of the version to latest, in this case from 0.9995 to 0.9997. Lastly, download your target version from the ruby odbc project page and do a MD5 checksum on it. In my case, when I downloaded 0.9997, the md5 checksum is 36d21519795c3edc8bc63b1ec6682b99. Now change the odd md5 in the portfile to this new one, hit save, and close. Let's try it out.

$ sudo port install rb-odbc +utf8

So without little fuss, that should have worked. Did you notice how we have the +utf8 variant there? If your like me and have national/unicode columns in your database, this is a good thing. The adapter has tests and support for unicode as well.

Configure The SQL Server Stack

First, let's configure FreeTDS to point to our server. Assuming you have TextMate a simple mate /opt/local/etc/freetds/freetds.conf should get it open for us. FreeTDS even put in a few examples at the bottom of the file, I typically comment these out and add my own. In this example I am naming my server "my_dev_server". Note that tds version section, I personally use tds version 8.0 and have tested this against my stack on all SQL Server types with the ActiveRecord tests. Some have reported that they have errors unless the use 7.0. Your mileage may vary. Save and close the file. UPDATE: As noted above when running tsql I, if you do not see a utf-8 locale setting, you may have to add this line client charset = UTF-8 to your configuration file. Here is a thread on how to set tsql's client character set.

# A typical Sybase server
# [egServer50]
#   host = symachine.domain.com
#   port = 5000
#   tds version = 5.0

# A typical Microsoft server
# [egServer70]
#   host = ntmachine.domain.com
#   port = 1433
#   tds version = 7.0

[my_dev_server]
  host = 192.168.1.58
  port = 1433
  tds version = 8.0

It has been noted that Windows user may want to include client encodings and character sets that set UTF-8 values. If you are running on Windows, try out this style freetds conf.

[YOURSERVER]
 host             = yourserver.com
 port             = 1433
 tds version      = 7.0
 text size = 64512
 client encoding  = UTF-8
 client charset   = UTF-8

So after you have added your own server here, it is now time to configure unixODBC to use FreeTDS. Let's copy the default distribution files to the correctly named ones that unixODBC will look for.

$ sudo cp /opt/local/etc/odbc.ini.dist /opt/local/etc/odbc.ini
$ sudo cp /opt/local/etc/odbcinst.ini.dist /opt/local/etc/odbcinst.ini

Open up the newly copied /opt/local/etc/odbcinst.ini file. More than likely, this will be the only time you ever edit this file. Add a FreeTDS configuration to it like so. No personal configuration options here, this just links up the MacPort installed unixODBC and FreeTDS.

[FreeTDS]
Decscription = FreeTDS driver for SQLServer
Driver = /opt/local/lib/libtdsodbc.so
Setup = /opt/local/lib/libtdsodbc.so
FileUsage = 1

So the last thing is to create some DSNs in your /opt/local/etc/odbc.ini file. It is very likely that you could edit this file often. Perhaps you do a lot of development and DB server changes. My personal file has over 20 some DSNs here. Here is one example. The name and description of the DSN is up to you. Take note how I made the server name "my_dev_server" line up to that used in the FreeTDS conf file. That is important. Flavor this to your tastes.

[my_dev_server_dsn]
Driver=FreeTDS
Description=My Server Connection
Servername=my_dev_server
Server=my_dev_server
Port=1433
Database=killer_app

Here is an example of what your database.yml will look like. So by now you should be able to connect and work with your database. I have this working on quite a few Macs. If you found any issues, let me know, but a good place to start is running the adapter tests. That is of course if you have the stack setup right.

development:
  adapter: sqlserver
  mode: ODBC
  username: rails
  password: secret
  database: killer_app
  dsn: my_dev_server_dsn

Going The Extra Mile With Ruby 1.9.x

Here is a little trick that I put in an article called MultiRuby The MacPorts Way. Testing Your Rails Apps With Ruby 1.9. Follow the steps in this post to get 1.9.x installed and switchable between your default 1.8.7 install.

Remember, RubyGems already comes in 1.9 and that tutorial leaves your installed gems under 1.9 empty. Do the right thing and install gems on an as needed basis under 1.9. As for the ruby-odbc which is installed in the 1.8's vendor ruby directory, we will need to trick MacPorts to recompile and reinstall that for us. Here is how I did it. First make sure you in the right ruby version.

$ ruby -v
ruby 1.9.1p243 (2009-07-16 revision 24175) [i386-darwin10]

Now we need to trick MacPorts to forget about the previous install so it will build and use the extconf.rb of the ruby 1.9 installation. Here are the commands to softly remove rb-odbc from MacPorts memory. If you want, you can change rm -rf to just mv and move these files to some other place.

sudo rm -rf /opt/local/var/macports/receipts/rb-odbc
sudo rm -rf /opt/local/var/macports/software/rb-odbc
sudo rm -rf /opt/local/share/doc/rb-odbc

Now you can tell MacPorts to reinstall ruby-odbc and it should create a new odbc.bundle in your /opt/local/lib/ruby1.9/site_ruby/1.9.1/i386-darwin10/odbc.bundle.

$ sudo port install rb-odbc +utf8

Here is the proof that I have 1.9 installed, working with SQL Server adapter and 64-bit. Thanks to the chruby ZSH function described in my other post, I can easily switch between my ruby installs.

Complete The Marathon!

Installing apache2 and passenger, yes all 64-bit happy.

$ sudo port install apache2
$ sudo gem install passenger
$ sudo passenger-install-apache2-module

How about a more common setup for SQLite3? If you took the side track and installed subversion from above, then you already have SQLite3 installed via MacPorts. If not, that's easy to fix. Here is how to install it and the ruby sqlite3 gem. I have done this for both my 1.8.7 and 1.9.x install above and again I am all 64-bit happy and passing all my tets.

$ sudo port install sqlite3
$ sudo gem install sqlite3-ruby

Installing MySQL and the ruby drivers are a bit more involved. This will get it installed and your launch control setup to act as if your box is a server, ie. load MySQL when it starts up.

$ sudo port install mysql5-server
$ sudo launchctl load -w /Library/LaunchDaemons/org.macports.mysql5.plist
$ sudo -u _mysql mysql_install_db5
$ sudo /opt/local/share/mysql5/mysql/mysql.server start
 Starting MySQL
 SUCCESS!
$ sudo gem install mysql -- --with-mysql-config=/opt/local/bin/mysql_config5

Nice, fully 64-bit and passing all the ActiveRecord tests for me. Note, sometimes you will have to reset your shell for new path binaries to become usable. It's always a good idea to open a new shell post a port install If you get an unfound mysql_install_db5, try opening a new shell. Pay attention to some of the output from that command too. For instance, if you want to set a root user password. I choose not to do this for a dev DB, but I do like to limit the connections so that only local connections can be used by editing the conf file. To create yourself a my.cnf file, copy one of the templates in /opt/local/share/mysql5/mysql, I choose to copy the large conf like so sudo cp /opt/local/share/mysql5/mysql/my-large.cnf /opt/local/etc/mysql5/my.cnf. Then I edit the conf file and uncomment the link that says "skip-networking". That way no one on the local network can hit my DBs. Lastly, here are some helpful aliases that I use.

alias my_stop="sudo launchctl stop org.macports.mysql5"
alias my_start="sudo launchctl start org.macports.mysql5"
alias my_rs="my_stop ; my_start"
alias myconf="mate /opt/local/etc/mysql5/my.cnf"

Visor Terminal On Snow Leopard_redirect

Tue, 18 Aug 2009 00:00:00 -0400

Visor Terminal on Snow Leopard

Tue, 18 Aug 2009 00:00:00 -0400

This is a similar process that I had to go through back in the day when I had to hack visor terminal in Leopard. Basically the steps are pretty easy. First you just install SIMBL and the Visor.bundle as a SIMBL plugin in ~/Library/Application Support/SIMBL/Plugins/Visor.bundle. Once that is done here is the process to get this working in Snow Leopard.

First, you are going to need a copy of the Terminal.app from Leopard. I have provided a copy in the resources section below. This copy has a few key things changed in the app's Info.plist file. First I have changed the bundle identifiers and display names to VisorTerminal. This is how we are going to scope visor to use this particular app. It also allows us to set things like the LSUIElement to 1 so that this app does not show in the dock. A summary of the changes I made are below, all these are done already in the download file I provide. If you want to do these on your own copy of Leopard's Terminal.app then just right click on the app, show package contents, and edit the Info.plist file. Remember to rename the app to VisorTerminal.app.

<key>CFBundleDisplayName</key>
<string>VisorTerminal</string>
...
<key>CFBundleIdentifier</key>
<string>com.apple.VisorTerminal</string>
...
<key>CFBundleName</key>
<string>VisorTerminal</string>
...
<key>LSUIElement</key>
<string>1</string>

Now we need to edit the installed ~/Library/Application Support/SIMBL/Plugins/Visor.bundle so that it focuses on the old Leopard's terminal app (now VisorTerminal.app). Again right click on it and show the package contents, here is the complete plist below. You can see where I changed 3 places to VisorTerminal.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
  <key>CFBundleDevelopmentRegion</key>
  <string>English</string>
  <key>CFBundleExecutable</key>
  <string>Visor</string>
  <key>CFBundleIdentifier</key>
  <string>com.blacktree.visor</string>
  <key>CFBundleInfoDictionaryVersion</key>
  <string>6.0</string>
  <key>CFBundleName</key>
  <string>Visor</string>
  <key>CFBundlePackageType</key>
  <string>BNDL</string>
  <key>CFBundleSignature</key>
  <string>????</string>
  <key>CFBundleVersion</key>
  <string>Custom</string>
  <key>GoogleML</key>
  <dict>
    <key>TargetApplications</key>
    <array>
      <dict>
        <key>BundleIdentifier</key>
        <string>com.apple.VisorTerminal</string>
        <key>BundleVersionsRE</key>
        <array>
          <string>.*</string>
        </array>
        <key>ExecPattern</key>
        <string>*/VisorTerminal.app/Contents/MacOS/Terminal</string>
      </dict>
    </array>
  </dict>
  <key>NSPrincipalClass</key>
  <string>Visor</string>
  <key>SIMBLTargetApplications</key>
  <array>
    <dict>
      <key>BundleIdentifier</key>
      <string>com.apple.VisorTerminal</string>
    </dict>
  </array>
</dict>
</plist>

Now when you launch VisorTerminal, it will be hidden from the dock and be the app that Visor.bundle focuses on. I really like how this app is removed from the doc too. More so I am just stoked it is working in Snow Leopard. As a last step, yoyou can add this to your login items so that it opens up automatically. I have found that it is a bit sticky when it first launches, but a few clicks in and out on first launch fixes that. One last note, I found it is easier to edit the custom properties for this terminal window like it's default color, font size, etc, if I turn off the LSUIElement.

UPDATE: Thanks to Raptor007 for reminding us that if you want to get a head start on your terminal preferences looking the same for your VisorTerminal, you should copy ~/Library/Preferences/com.apple.Terminal.plist to ~/Library/Preferences/com.apple.VisorTerminal.plist. Thanks!

Resources

757rb Memcached Talk_redirect

Sat, 18 Jul 2009 00:00:00 -0400

757rb Memcached Talk

Sat, 18 Jul 2009 00:00:00 -0400

Earlier this week I gave a talk at our local ruby users group, 757.rb, about Memcached. Here recently I started picking it up again so that I could keep track of large sets of PK/FK changes during a big database move. In doing so, I decided to dig deep into it's internals and get a better grasp of how I would use it when I decided to do some serious fragment caching again.

The biggest thing I took away from the talk was -- First, that you can use an array of strings/objects for the key passed to various caching methods. If the object responds to #cache_key, the return of that method would be used. The default implementation of ActiveRecord's #cache_key method is to use the updated_at column for a unique timestamp. It is even smart enough to know weather the object is new or not. Thanks to Brennan Dunn for pointing that out! The second and most important thing was an article by Nahum Wild that talked about item level caching. This seemed like a strange place to start as typically I would start with a page or big sections of fragments. But as the demo app shows, this is an incredibly good place to start. Especially if you have heavy hitting pages of paginated views that talk to deep nested associations.

Resources

Pdf Writer For Ruby 1 9_redirect

Thu, 14 May 2009 00:00:00 -0400

PDF::Writer For Ruby 1.9

Thu, 14 May 2009 00:00:00 -0400

If you have legacy code written for ruby 1.8 and you want to run 1.9 and support your old PDF::Writer code, then just jump right over to my Github pdf-writer fork and get it.

$ gem install metaskills-pdf-writer

If you are interested in knowing some of the dirty details about what pitfalls are under 1.9, read on. The biggest thing for me was getting used to the character encodings. Including string literals in your code that are say UTF-8 or some other encoding will blow up on you real quick. I highly highly suggest that you read James Edward Gray II: Everything About Ruby 1.9 Character Encoding Series series. He covers all the basics. If you want to see what two commits I did for pdf-writer, see here and here. The second one is very very hackish, it basically add a marshaling support to the Mutex class which 1.9 does not have. The reason this is needed was due to pdf-writer's use of transaction simple to roll back object state as it is drawing across multiple pages. If you are like me and wish all your PDF::Writer code was in Prawn, your not alone!

Resources

Learning Objective C Iphone Development_redirect

Tue, 10 Mar 2009 00:00:00 -0400

Learning Objective-C & iPhone Development

Tue, 10 Mar 2009 00:00:00 -0400

What have I been up too lately? Well primary working my butt off in my day job and after that, I just finished a week of vacation. As an aside, I need to disconnect more often, my vacation was very relaxing. But what's more relaxing that doing things in the fresh outdoors? Well learning another programming language of course.

That is why this week I have seriously started learning Objective-C and iPhone application development. In fact I just got my approval today from Apple that I am a registered developer, see that pretty icon in the right column? So do you want to do the same? Below is a list of resources that I have found helpful to me. Let me explain something first, I am very tangential in my learning methods. Basically that means I take frequent breaks from my main course to learn deep details. For instance my main course is the iPhone SDK Development book. But I have found it very helpful to break early on and watch the Becoming Productive In Xcode videos. Knowing keyboard bindings and customizing my, dare I say it... IDE, just helps me get business done! Stay tuned for some updates.

Resources

Git Subversion User Commit Reports_redirect

Sun, 01 Feb 2009 00:00:00 -0500

Git & Subversion User Commit Reports

Sun, 01 Feb 2009 00:00:00 -0500

Want a list of the users and the number of commits they made? Git makes it really really easy, while I could not find such an easy method on Subversion. Here they are.

Git

$ git log | git shortlog -n -s

Subversion

#!/usr/bin/env ruby

require 'rubygems'
require 'activesupport'

log_xml = `svn log -q --xml`
svn_logs = XmlSimple.xml_in(log_xml)['logentry']

report_hash = svn_logs.inject({}) do |report,log|
  author = log['author'][0]
  report[author] ||= {:commit_count => 0}
  report[author][:commit_count] += 1
  report
end

commits_authors = report_hash.keys.map { |a| [report_hash[a][:commit_count], a] }
commits_authors = commits_authors.sort_by(&:first).reverse

cc_colsize = commits_authors.map(&:first).max.to_s.size
a_colsize = commits_authors.map(&:last).inject{|m,w| m.length > w.length ? m : w }.size

final_report = commits_authors.map do |ca|
  c,a = ca
  " #{c.to_s.rjust(cc_colsize)}  #{a.ljust(a_colsize)}"
end.join("\n")

puts final_report

Multiruby The Macports Way Testing Your Rails Apps With Ruby 1 9_redirect

Tue, 20 Jan 2009 00:00:00 -0500

MultiRuby The MacPorts Way. Testing Your Rails Apps With Ruby 1.9

Tue, 20 Jan 2009 00:00:00 -0500

Ruby 1.9.1, the stable release, is just around the corner and if your like me, maybe you want to start playing around with it and perhaps test a few projects using 1.9 with edge rails 2.3. If so, and your on a Mac, then perhaps this installation method might appeal to you. I'll break this article up in two parts, the first will be on installing multiple versions of ruby and how to switch between them while the other will be some things I noticed when testing ruby 1.9 with edge rails on my favorite pet project HomeMarks.

Installing Ruby 1.9 & Change Ruby Function

I love MacPorts and recommend using it even though OS X comes with ruby pre-installed. Doing so simply keeps things clean and separated in the /opt directory vs mixing things in /usr where they might be clobbered by Apple during a system update. So no to dependency hell! This article assumes you already have ruby 1.8 installed via MacPorts, if not change some of the paths in the provided code below. Now... thanks to MacPorts it is easier than ever to get ruby 1.9 preview release 2 installed in it's own separate space.

$ sudo port install ruby19

MacPort has just added these ruby 1.9 binaries to your PATH in /opt/local/bin. Nice and clean egh? From here you can start "playing" with 1.9 by just executing either ruby1.9 or irb1.9, but that gets boring quick.

erb1.9
gem1.9
irb1.9
rake1.9
rdoc1.9
ri1.9
ruby1.9
testrb1.9

Let's hack up your setup so that you can quickly switch your whole ruby install back and forth transparently. The first step is to move the corresponding ruby 1.8 binaries to the same naming convention as their 1.9 counterparts. We will then use symlinks to change which one gets executed. Step one:

$ sudo mv /opt/local/bin/erb /opt/local/bin/erb1.8
$ sudo mv /opt/local/bin/gem /opt/local/bin/gem1.8
$ sudo mv /opt/local/bin/irb /opt/local/bin/irb1.8
$ sudo mv /opt/local/bin/rake /opt/local/bin/rake1.8
$ sudo mv /opt/local/bin/rdoc /opt/local/bin/rdoc1.8
$ sudo mv /opt/local/bin/ri /opt/local/bin/ri1.8
$ sudo mv /opt/local/bin/ruby /opt/local/bin/ruby1.8
$ sudo mv /opt/local/bin/testrb /opt/local/bin/testrb1.8

So now that we have two versions of ruby installed, each in their own path suffix. So to switch back and forth without using these suffixes, we will need to create a symlink for each binary that points to the version we want to use. The users of the local ruby group 757rb came up with this ZSH function below. Place it into your .zshkit or dotfile of choice and it will create/change symlinks for your ruby, irb, rake, etc.. that each point to the version you want to use. Calling it multiple times will simply toggle your ruby versions.

chruby () {
  ree=`ruby -e "puts RUBY_DESCRIPTION.include?('Ruby Enterprise Edition')"`
  if [ $ree = "true" ]; then
    echo "NOOP: Currently using REE in your path."
  else
    v=`ruby -e "puts RUBY_VERSION.split('.')[0,2].join('.')"`
    if [ $v = "1.9" ]; then
      cv="1.8"
    else
      cv="1.9"
    fi
    rubyexes=(erb gem irb rake rdoc ri ruby testrb autotest unit_diff rails)
    for i in $rubyexes; do
      sudo unlink "/opt/local/bin/${i}"
      sudo ln -s "/opt/local/bin/${i}${cv}" "/opt/local/bin/${i}"
    done
    echo "Now Running: "`ruby -v`
  fi
}

$ chruby
Now Running: ruby 1.8.7 (2008-08-11 patchlevel 72) [i686-darwin9]
$ chruby
Now Running: ruby 1.9.1 (2008-12-01 revision 20438) [i386-darwin9]

This works amazingly well at first. The one big gotcha is that you have to remember you have two gems directories. Any gem you installed under 1.8 will have to be installed again under 1.9. There are ways around this with some clever symlinking, but I'd recommend against it. Note too, this also applies to any site/vendored libs or bundles, for instance ruby odbc.

Things To Watch Out For

If you install a gem under a certain version of ruby that places an executable in say /opt/local/bin then you are going to want to watch out for a bad she bang. For instance if you install the ZenTest gem while running under ruby1.9 suffix, then the she bang for /opt/local/bin/autotest would be incorrectly set to this #!/opt/local/bin/ruby1.9 -ws. The simple fix is to just change it to the true sym link.

Exploring Rails 2.3 & Ruby 1.9

So far my HomeMarks project does not have any major issues with ruby 1.9. I'm not surprised, it's very very slim ruby code due to the AJAX head design pattern. I did however find a few new things about the changes an app will need when moving from 2.2 to 2.3.

Your test/test_helper.rb file will need to change to from class Test::Unit::TestCase to class ActiveSupport::TestCase. I'm sure this change is related to future abstraction to choose Minitunit vs Test::Unit.
They have finally changed app/controllers/application.rb to app/controllers/application_controller.rb. Finally their hacks on that bad name are now gone. I found that I had to do this change manually on my own.
Disabling sessions for a single controller has been deprecated. Sessions are now lazy loaded. So if you don't access them, consider them off. You can still modify the session cookie options with request.session_options.

Past that, about the only interesting thing I found in Ruby 1.9 was that Test::Unit is gone and is actually loading Miniunit. So things in rails that were requiring test/unit/error are going to fail.

Autotestify Brothers and Sisters

Tue, 30 Dec 2008 00:00:00 -0500

Based on my previous article Using Autotest For Rails Plugin Development, Brennan Dunn wrote a ZSH function that helps him with his eager rails plugin work. Testing plugins is simply the most fun you will ever have. It's nice to test in isolation and easy to make a plugin test multiple rails versions, etc. Typically plugin testing is very fast to because you are not burdened with excessive tests in a big app that might use it. Just the ones needed to test the plugin. Enjoy this ZSH function.

function autotestify () {
  git clone git://github.com/metaskills/autotest_railsplugin.git
  rm -rf ./autotest_railsplugin/.git
  mv ./autotest_railsplugin ./autotest
}

Go Buy Envycasts Now!

Wed, 26 Nov 2008 00:00:00 -0500

I got a chance to meet Gregg Pollack at this years RubyConf, a great guy. In fact I seem to owe him alot. Not only did I get 3 mentions lately on the main rubyonrails.org site, but it appears I got one of my 2.2 additions mentioned in their new envycast that covers rails 2.2. Sweet!

So now it is my duty to help the Rails Envy guys to bank it Grossenbach-style. If you have not done so, go check out envycasts.com and put $9 down on one or two of their great vids! This stuff is a great resource for any ruby-rails'ist. Put it on your TV and imagine your in your own little Orlando user group.

SQL Server Adapter For Rails 2.2

Sun, 23 Nov 2008 00:00:00 -0500

So that about sums up what I've been doing for the past 4 weeks. A total rewrite of the SQL Server Adapter for Rails. On top of passing all the Rails 2.2.2 tests, which is MAJOR news, it includes tons of new features. Unicode column support, pessimistic locking, date/time column casting, DDL transactions and way more.

On top of that. It was nice being mentioned on the rubyonrails.org website not for my first, second, but a THIRD time. Thanks Gregg!

UPDATE: I just got a nod from David's twitter. Thanks!

Code-O-Lanterns: What Scares You?

Mon, 13 Oct 2008 00:00:00 -0400

While reading the The Unofficial Apple Weblog today I noticed this sweet little Jack-O-Lantern screensaver for OS X. After some feedback on the 757.rb group, I decided to make some Code-O-Lanterns.

Follow the instructions for the Jack-O-Lantern screensaver on where to put the carving files. If you are not on a Mac, I have taken some screen shots that you can use for desktop pictures. If you would like to see a code o lantern that is not up... just ask for one, providing it has a logo!

Code-O-Lanterns Files

Plug It In Plug It In

Sat, 11 Oct 2008 00:00:00 -0400

I've been busy putting together some plugins form misc work. If you did not catch my latest article Jack has_many :things where I covered GroupeScope, by all means check it out. I've also just finished up a pretty solid backport of NamedScope that can be found on my Github page. It bills itself as a well tested complete back port for rails 1.2.6 and 2.0.4.

One thing that has been a real help while I've been developing these plugins is this autotest class I put together specifically for developing rails plugins. I even got mention on rubyonrails.org. Sweet! More to come.

Resources

The NamedScope plugin.
The GroupedScope plugin.
The RailsPlugin class for Autotest.
Vilcus plug dactyloadapter. Developed specially for people who enjoy closing electrical circuits with their own fingers.

Jack Has_many Things_redirect

Sun, 28 Sep 2008 00:00:00 -0400

Jack has_many :things

Sun, 28 Sep 2008 00:00:00 -0400

I am Jack's sofa, stereo and wardrobe... I make Jack's life complete. I reside in a ActiveRecord table called "things" and Jack is the only one that has the key. This is Jack's life, and it's ending one minute at a time.

As rails developers, we have done this simple relationship over and over again. I'm sure the has_many association is by far the most common in app/db design. It gives a single resource quick and easy access to others, but as your application grows, and depression sets in, we have to open up.

It's cheaper than a movie, and there's free coffee.

I am talking about groups. Not the underground ones that carve us out of wood, but ones where we share with those around us. This is healing. The time has come to for our objects to do the same, but how?

The problem is that the ActiveRecord has_many association is scoped to an individual. No matter what conditions are tacked on, they are always pwned by the proxy owner. The things they own have ended up owning you! Sure we could model some groupable schema and go through it, ActiveRecord is beautiful that way. But what about our hard work in all those existing has_many associations and scopes?

I felt like destroying something beautiful.

The solution is on everyone's face, it is on the tip of everyone's tongue. I just gave it a name. It is called GroupedScope and it will fundamentally change the constructed SQL for the has_many associations you want to share. The best part, it will leave those associations untouched for continued use. GroupedScope even works with your existing association extensions and any scopes. Let's have a session.

First we need to install the gem. So bundle up in your Gemfile.

gem 'grouped_scope', '~> 3.1.0'

Now let's open up our people schema and the Person model, so Jack can share. First we add a group_id column to the people table.

class PeopleCanChooseAGroup < ActiveRecord::Migration
  def up
    add_column :people, :group_id, :integer
  end
  def down
    remove_column :people, :group_id
  end
end

Next we declare grouped_scope on a few associations.

class Person < ActiveRecord::Base
  has_many :things
  has_many :acquaintances
  has_many :problems
  grouped_scope :things, :problems
end

@jack = Person.find_by_name('Jack')
@bob  = Person.find_by_name('Bob')

@jack.update_attribute :group_id, 1
@bob.update_attribute :group_id, 1

So now every Person object in our app is now ready to share their :things and their :problems. I have also just arbitrarily put Jack and Bob into the same group. Declaring grouped_scope in the model generates a new group instance method that will allow us to either reflect on the group or delegate to the associations we have declared as now haveing grouped scope.

@jack.group   # => [#<Person id: 1, name: "Jack", group_id: 1>, #<Person id: 2, name: "Bob", group_id: 1>]

We all started seeing things differently.

The object returned by the group method, is an instance of GroupedScope::SelfGrouping. It is far cooler than you think. It looks and acts as an enumerable array, but in reality it is a ActiveRecord::Relation object that can delegate to generated grouped association reflections which mimic their originals. Essentially giving the group access to all associated objects of its members, in this case their :things and :problems. Did I loose you?

@jack.problems.size   # => 6
@bob.problems.size    # => 2

@bob.group.problems        # => [#<Problem...>,#<Problem...>,#<Problem...>,#<Problem...>,....]
@bob.group.problems.size   # => 8
@jack.group.problems.size  # => 8

Without going into the detail of Jack's and Bob's problems, we can see that within the group, they are all shared. This is what the GroupedScope gem is really all about. It allows existing has_many associations to be called on the group which changes the SQL generated to be owned by the group, essentially from id = 1 to id IN (SELECT id FROM people WHERE group_id = 1).

The way it accomplishes this is pretty sweet. GroupedScope creates reflections that use a custom association scope method which uses a little Arel magic to build predicate conditions. Since it copies all your existing association reflection options, it can be really smart by maintaining all the logic in the existing association. So options like :class_name, :foreign_key, :though, and :extend will just work! It also lets you chain scopes existing scopes or use your custom association extensions on the original associations. Here is very contrived example:

class Person < ActiveRecord::Base
  has_many :mental_issues, :class_name => "MentalState", :foreign_key => :name do
    def dangerous
      where(:snap_tolerance => 10)
    end
  end
  grouped_scope :mental_issues
end

class MentalState < ActiveRecord::Base
  scope :treatable_by, lambda { |doctor| where(:doctor_id = doctor.id) }
end

@jack.group.mental_issues.dangerous.treatable_by(@doctor) # => [#<MentalState...>,#<MentalState...>]

This is probably one of those "cry for help" things.

The GroupedScope gem is never quite done. It is however well tested and can do exactly all that I have outlined in ActiveRecord 3.1.0. I even have older gem versions that track our legacy 2-3-stable git branch.

Also, you may have noticed that GroupedScope does not try to solve what your group business logic may look like. This is intentional and left to the end user to implement a Group object. This object would be the primary key owner of the group_id column in your models that declare grouped scope. You would also need to declare some sort of belongs_to association to your custom group model too.

Lastly, if we you see something you might like in features or notice a bug, open a issue on our Github project page.

I'd like to thank the Academy.

Using Autotest For Rails Plugin Development_redirect

Fri, 19 Sep 2008 00:00:00 -0400

Using Autotest For Rails Plugin Development

Fri, 19 Sep 2008 00:00:00 -0400

I love autotest. I have event posted before how to extend the idea of autotest sounds to a red/green playlist but now that I am taking more time to extract some of my work to plugins, I really wanted autotest to come with me. The problem is that the default autotest mappings do not play with rails conventions, the biggest being that test files for a lib match the name of the lib with _test.rb at the END of the filename.

Today I posted a new project to my Github account, called Autotest::Railsplugin. This project should be a great start for fleshing out all the typical mappings, ignore, libs, etc. that autotest needs to know about when developing a rails plugin. I invite you to download it and give it a try and provide feedback on missing details.

To Install

Download
Rename the directory to "autotest"
Place inside of your pluginâ€™s root directory.
From your plugin root, you can now use autotest.

$ autotest
loading autotest/railsplugin

By default, autotest will scan your present working directory for an autotest folder that has a discover.rb in it. So it should find and display the loading autotest/railsplugin which means it found the plugin class just fine. If this is not the case make sure you have the directory named correctly. If you have a conflicting autotest directory for some reason you can always force the plugin class, again assuming you have it in the right, place using:

$ autotest -railsplugin

Resources

Going Horizontal On The Multicade_redirect

Sun, 14 Sep 2008 00:00:00 -0400

Going Horizontal On The Multicade

Sun, 14 Sep 2008 00:00:00 -0400

A few months ago I "invested" in a upright Galaga multicade. This is no costly refurb/restoration unit, but rather a customized new build from a great company called Arcadeshop Amusement. This place does two great things. First they will manufacturer/build you a new cabinet of your liking with all the parts professionally installed. This includes the display, JAMMA harness, coin door assembly, control panel, and all artwork. I chose a classic Galaga/Ms.Pacman/Midway cabinet with a 19" CRT display. My only deviation from the Galaga theme was a multicade control panel that had a 4-way joystick and two buttons for the left or right handed.

The second thing about this great company is that sell one of the coolest Multicade JAMMA PCBs I have seen. This board is programmable with a standard serial cable to run game ROM's in their approved list and that included such greats that I wanted like Galaga, Donkey Kong, Ms.Pacman, and Phoenix. For the record... I am not going after Billy Mitchell's record. I just wanted a nice low-tech diversion :)

So these games held me over for a time, but I really wanted to be a bit more flexible with the option to play some horizontal games. To that end, I invited my friend and fellow 757.rb'er Geoff Parsons over to help me rotate my CRT. Before we got to that jun part, I had to manufacture my own horizontal cross members to hold the display. Thankfully a trip to LOWES steel part section and a new hack saw yielded a very decent support system, see here. The icing on the cake after we remounted the CRT was to replace the 4-way joystick with a new 4/8-way super stick from HAPP Controls, a new 19-in-one JAMMA PCB from Lizard Lick, and a new monitor bezel for the horizontal display. Now with this PCB, I am enjoying games like Super Mario Bros, Gradius, and Defender.

For those that may ask, why did I not go with some sort of MAME system. Sometimes options are good... but too many are distracting. I really like the idea of having as much as my multicade mimic the old arcade cabinet as possible, and hacking on another computer to do that was not in the cards. I love just slapping in a JAMMA board and having the set of games I like. Low-tech fun!

In Hell Oo For Homemarks_redirect

Mon, 18 Aug 2008 00:00:00 -0400

Hell'OO HomeMarks

Mon, 18 Aug 2008 00:00:00 -0400

Well HomeMarks v2.0 is done and ready for the public. You can download it from my Github project page. It has a real simple rake app:bootstrap task that I came up with over the weekend will have you running a local copy in only a few seconds, give it a try. Sometime over the next few days, I'll move over the live site to this code base too. If you have not yet kept up on the implementation mantra I set down for the HomeMarks v2 project, you might want to read an older post as well as this excerpt from the project README.

HomeMarks was built using the Ruby on Rails framework with a heavy emphasis on object oriented JavaScript to make AJAX requests to a RESTful back-end. Unlike most Rails applications it does not use any inline JavaScript helpers nor does it rely on RJS (Remote JavaScript) for dynamic page updates. Instead it is nearly 100% unobtrusive JavaScript which uses simple HEAD or JSON responses to communicate to the objects on the page. This has yielded very slim controller code which is decoupled from the views and easily testable in isolation at a functional level.

So what do I think of this implementation and writing all this object oriented JavaScript? Well - its been real hard, I've been in JS Hell-OO for the better part of a few weeks as I came closer to finishing the project. Coming from a self inflicted addiction to unit/functional testing in rails, it is very hard to get into the habit of opening the browser to test view code. A big part of me hates writing all this JS without one bit of unit testing. I've heard that Prototype v2 will have unit testing, that would be awesome!

So, bitching aside, I am very happy that my JS-fu is stronger. The end result besides all those buckets of water up the hill is that I can now really appreciate the ruby object model and how prototype strives to make JS class construction follow its example. Kudos really goes to Sam Stephenson for bridging the two object models. As I learned more about how prototype's classes work, I actually learned quite a bit about ruby too. Using that knowledge I gave a talk at the Hampton Roads Ruby Users Group (757rb) on the topic. You can download the PDF version of the slides from the files section of the Google group page. Look for the file titled OO JavaScript Class Construction

Stop Exception Notifications With The Zombieshotgun_redirect

Sun, 06 Jul 2008 00:00:00 -0400

Stop Exception Notifications With The ZombieShotgun

Sun, 06 Jul 2008 00:00:00 -0400

I am all about knowing how to survive a zombie invasion â€“ as much as I am a firm believer of using the right tool for a given job. It can not be argued that killing zombies with a shotgun to the head is as natural a fit as peanut butter to chocolate. They simply just go together.

Now real zombies may not be a daily nuisance, but computer zombies are a daily pain in the butt to network administrators as well as software engineers alike. If you have ever deployed a rails application into production that used some sort of exception notification, then you may at some time seen some zombie attacks throw a bunch of exceptions. My solution a few years back was to build my own ZombieShotgun module, see below.

The idea is simple, include the module and add this line to the very top of your filter chain before_filter :shoot_zombies. Just like in real life, if rails detects a zombie attack, it will issue a 404 not found error in the beautiful rails syntax head :not_found. I love it when code models the real world! Please note that there are a ton of better ways to accomplish user agent filtering, most notably, in your web server config... but that does not mean this is not a fun module to use.

module ZombieShotgun

  ZOMBIE_AGENTS       = ['Microsoft Office Protocol Discovery','Microsoft Data Access Internet Publishing Provider Protocol Discovery','FrontPage']
  ZOMBIE_ATTACK_DIRS  = ['_vti_bin','MSOffice','verify-VCNstrict','notified-VCNstrict']

  protected

  def shoot_zombies
    head :not_found if zombie_attack?
  end

  def zombie_attack?
    zombie_attack_on_directory? || zombie_agent_attack?
  end

  def zombie_attack_on_directory?
    attack = request.path.from(1)
    attack_dir = attack.index('/').nil? ? attack : attack.to(attack.index('/')-1)
    ZOMBIE_ATTACK_DIRS.include?(attack_dir)
  end

  def zombie_agent_attack?
    ua = request.env['HTTP_USER_AGENT']
    !ua.blank? && ZOMBIE_AGENTS.any? { |za| za =~ /#{ua}/ }
  end


end

Restful Ajax With Forgery Protection_redirect

Wed, 18 Jun 2008 00:00:00 -0400

RESTful AJAX with Forgery Protection

Wed, 18 Jun 2008 00:00:00 -0400

Writing the new HomeMarks has been a great exercise. I've learned that the AJAX Head ~~Design Pattern~~ Implementation is more akin to developing a service-oriented application (SOA) since I have moved all client-side coupling from the controllers, like RJS, and only respond with HEAD or JSON data. Today I learned about Halcyon which is self described as a JSON web application framework built on Rack. If you take a look at their example, it looks a lot like HomeMarks v2, pretty cool!

Getting To The Point

Last week I finally got around to doing the controller CRUD for the user's column objects. The core homemarks/app.js JavaScript class contains the one (and only one) AJAX dispatcher/handler for the entire application. One problem that I ran into was how to continue to use forgery protection in rails to protect from CSRF attacks while relying solely on ad-hoc JavaScript requests to a RESTful back-end. I could have turned off protect_from_forgery but that did not feel right. I wanted RESTful AJAX with forgery protection.

My Solution

Since the HomeMarks v2 app has one AJAX dispatcher/handler for the entire app, all I had to do was to get the authenticity token into each and every request. It took a bit of digging in the rails source to find the methods I wanted but here both the ERB snippet I am using in my layout and the helper method found in my application_helper.rb file.

<script type="text/javascript">
  <%= auth_params_js_var %>
</script>

def auth_params_js_var
  unless RAILS_ENV == 'test'
    %|var authParams = $H({#{request_forgery_protection_token}:#{form_authenticity_token.inspect}});|
  end
end

What does this do? It creates a JS variable called authParams at the window level (right term?) that all other functions can use. That variable returns a JavaScript hash that will have a single key/value pair for the rails authenticity token, see below. Basically at this point all you have to do is merge this hash with your parameters option for your AJAX object. You can reference the HomeMarks app.js file in the resources section if you want to see how I did this on my app, code snippet below too.

doAjaxRequest: function(elmnt,finishMethod) {
  // ...
  var method = elmnt.method || 'post';
  new Ajax.Request(elmnt.action,{
    onComplete: function(request){
      this.completeAjaxRequest(request);
      if (finishMethod) { finishMethod.call(this,request) };
    }.bind(this),
    parameters: parameters.merge(authParams),
    method: method
  });
}

Resources

The Latest HomeMarks app.js Class. See doAjaxRequest() function.

Coulda Shoulda Woulda_redirect

Fri, 30 May 2008 00:00:00 -0400

Coulda Shoulda Woulda

Fri, 30 May 2008 00:00:00 -0400

It has been about 6 months now since I started using the Shoulda testing plugin as my BDD/TDD tool of choice. Unlike a lot of other people, I did not flock to the RSpec bandwaggon. Personally I think RSpec is horribly bloated a sledgehammer for a simple issue, the need to have test code organized with nested setups and context blocks.

If you are new to Shoulda, I highly urge you to take a look at the Thoughbot project page for it. If you have the time take a look at some of their other projects, like Paperclip. These guys are really smart. If you really want to see how shoulda can sing, check out Tammer Saleh's presentation of it at the MountainWest RubyConf 2008. In that presentation he covers how you can extend Shoulda and write your own macros that generate additional tests. This is what my blog post is about.

Sample Shoulda Macro For Functional Require Login

Here is a simple module that you might find for any restful authentication system. This would be included in your main test_helper.rb file where it would add the normal login_as(user) method. The good part I want to point out is the should_require_login(*actions) macro method that shows off a neat example of how you could use Shoulda in your functional tests at the class level.

module AuthSystem
  module TestHelper

    def self.included(receiver)
      receiver.extend ClassMethods
      receiver.send :include, InstanceMethods
    end

    module ClassMethods

      def should_require_login(*actions)
        actions.each do |action|
          should "Require login for '#{action}' action" do
            get(action)
            assert_redirected_to(login_url)
          end
        end
      end

    end

    module InstanceMethods

      def login_as(user)
        if u = users(user)
          @request.session[:user_id] = u.id
        end
      end

    end

  end
end

Here is how it would look in the controller functional test. These are all contrived examples, but I think it illustrates how Shoulda can be used and in general how you can make your own macros that test at a higher level.

class UsersControllerTest < ActionController::TestCase

  should_require_login :edit, :update, :etc

end

The Ajax Head Br Design Pattern_redirect

Sat, 24 May 2008 00:00:00 -0400

The "AJAX Head" Design Pattern

Sat, 24 May 2008 00:00:00 -0400

This is the first of a few articles covering the total rewrite of the HomeMarks.com code base as I upgrade it to Rails 2.1. The "AJAX Head" pattern is the moniker I have assigned to methodology that has come about during said rewrite and the design decisions I choose early on. The rules were simple.

Total Unobtrusive JavaScript (No Tag Soup)
No RJS Controller Responses

Rails makes it very easy to generate JavaScript with Ruby. The generator methods are extensive and cover both the Prototype and Scriptaculous libraries. The first version of HomeMarks used the generators exclusively to augment both the views and controller responses. In fact, I used a technique that is covered in the latest Advanced Rails Recipes (way before it came out) under the "Replace In-View Raw JavaScript" chapter. Basically I created helper methods that generated JavaScript shared by both the views and controllers. It was a great technique but things got messy quick. So life was good back then, I had a lot of Ruby code and a very dynamic site, but something did not seem right...

What Is The AJAX Head Pattern?

It is an experiment into a vision to see what happens when you make the decision to totally go unobtrusive. Not just in your views but the controllers too! Imagine it this way — your controllers are slim APIs. They should do nothing but render a page on a GET request and when it comes to a POST/PUT/DELETE they should do nothing more than just say YES or NO (with errors).

Now lets think about that. What does that mean for an application that uses exclusive AJAX calls for dynamic page updates? It means that your visual client (the browser) will need to know ALOT less from the controller about what to do. It's great, the browser knows what is on the page, from the GET request, and what it needs to do with it. The client only needs to tell the remote resource store what it did. In this pattern the controller is relegated to nothing more than acting on model data and letting the client know if it succeeded or failed. So in short.

The AJAX head design pattern forces the view and controller to work in isolation with the most minimal coupling possible. Kind of like a web service.

Tools and Code Needed

A application that does just about everything with AJAX.
Fluency with ActionController::Base#head method and HTTP status codes.
Site-wide exception rescues for ActiveRecord::RecordInvalid.
Global client-side AJAX request and response handlers.
Global client-side error handlers.

This may seem like a daunting list, but its pretty simple when you break it down. Let's do each one at at time with some of the current code from HomeMarks. The HomeMarks app is my first check on the list. When I did this app just about EVERYTHING was done via an AJAX call to update the page. Now on to the others.

Head And Status Codes

The ActionController::Base#head method and HTTP status codes are the second item on the list. The head method is very useful when you just want to respond with HTTP headers and no body content. This is typical behavior when an app responds to external web services requests. This is important to note, I chose to rely on head responses as a way to meet my design decision of not using any RJS responses from the controller.

The basic usage of the head method is simple, you just pass a symbol to the head method that is the lowercase underscore equivalent to an HTTP status code. For instance, if you wanted to respond with success, choose a status code from the 200 range, the easiest being 'OK', which would look like head(:ok). If you wanted to respond to bad authentication, you might use the 401 code like so, head(:unauthorized). Here is an example from HomeMarks restful user signup action.

class UsersController < ApplicationController
  # ...
  def create
    User.create!(params[:user])
    head :ok
  end
  # ...
end

There is actually a number of things going on in those 2 lines of code above. If the user is created, the AJAX request will get a simple 'OK' message via the head method. Obviously the client will need to know what to do on its own. But if errors happen what to do? This brings us to...

Invalid Object Handling

When Rails 2 came out, it gave us a very useful tool for our controllers, the ActionController#rescue_from method. It allows you to delegate exception handling to a method either per controller or site wide for a specific exception class. So enter the RenderInvalidRecord module, show below.

module RenderInvalidRecord

  def self.included(base)
    base.rescue_from(ActiveRecord::RecordInvalid, :with => :render_invalid_record)
  end

  protected

  def render_invalid_record(exception)
    record = exception.record
    respond_to do |format|
      format.html { render :action => (record.new_record? ? 'new' : 'edit') }
      format.js   { render :json => record.errors.full_messages, :status => :unprocessable_entity, :content_type => 'application/json' }
      format.xml  { render :xml => record.errors.full_messages,  :status => :unprocessable_entity }
      format.json { render :json => record.errors.full_messages, :status => :unprocessable_entity }
    end
  end

end

Once included in your application.rb file, every ActiveRecord::RecordInvalid exception will be handled by the render_invalid_record method. I use this pattern often and it is the primary reason I always use the bang methods that end with an exclamation point, such as create! or save!. This module above was inspired by Joshua Peek's own module of the same name but has been edited here to fit my needs. Both the "xml" and "html" response formats are not used by HomeMarks but are included here to show how this module can work for any request type.

Now, how this fits into the AJAX Head pattern. The design decision of not having RJS responses means one of two things needs to come from the response. First, if the response is a success, meaning in the 200 range, then it is assumed that the client only needs to know that all is 'OK'. Second, if the request has errors, then the response needs to communicate them. I choose JSON for the format of these errors since all AJAX request objects from Prototype automatically include a application/json accept header. So that is why I changed both the "js" and the "json" response type in the RenderInvalidRecord module to both serialize the objects full error messages into a JSON array with an HTTP status code of 422 "Unprocessable Entity" which seemed to make the most sense to me. This pretty much wraps up the server side part of the pattern. Now onto the client side. Everything from here on is personal taste on how your client will react to an "good" or "bad" response object.

Client-Side AJAX Handlers

I can not stress enough how much you have to bake your own functions to handle the client side reaction to both good and bad AJAX responses. The client's ability to know what to do with errors will be tightly coupled to AJAX handlers and the design choices of your site. In the examples below I have broken up methods in the same JavaScript classes to show you the parts for a specific behavior in a more concise manner. Also please note that I am using Prototype module and class organization heavily in HomeMarks. This allows me to build a Base JavaScript module and classes that mimic Rails models which inherit/mixin Base methods. I also use the Scriptaculous Builder class for creating DOM fragments too. Now back to business.

The first part of the AJAX head pattern was to use unobtrusive JavaScript. Because of this design choice, there are no remote_form_for methods in the HomeMarks view code. All forms are simple form_for methods. Below is a sample of the HomeMarksSite JavaScript class that creates AJAX handlers for all site forms. The below example is slimmed down to only show the initialization of this behavior for the signup form. But no matter which form is given this behavior the process is the same.

The form's submit event is handled by startAjaxForm(). It does the following:

Stops the normal submit behavior.
Adds a loading spinner in the #form_loading span.
Creates a new AJAX.Request using the forms own action. It also binds a function to delegate the logic of completing the form when the request is finished. More on that later in #2 below.
Finally it disables the form.

Once a response comes back from the server, the delegateCompleteAjaxForm() function will be called with both the form object and the AJAX request object. The logic of this function includes:

Figuring out the "mood" of the request using getRequestMood() function which is part of my base module.
Sends the form and mood to the completeAjaxForm() function. This function will replace the loading spinner with an image that reflects a good or bad response.
If the mood is good, meaning the response status code was within a 200 range, then the function will delegate to a function specific for that forms success using a simple case statement.
If this mood is bad, meaning the response status code not within a 200 range, the form will be enabled and the response JSON errors will be displayed. Currently I am using a modal class for this. Again more on that later. You may also note that my base module has a function called messagesToList() that process the response into a DOM <ul> list.

var HomeMarksBase = {
  getRequestMood: function(request) {
    return (request.status >= 200 && request.status < 300) ? 'good' : 'bad';
  },
  messagesToList: function(request) {
    var messages = request.responseJSON;
    var messageList = UL();
    messages.each(function(message){
      messageList.appendChild( LI(message.escapeHTML()) );
    });
    return messageList;
  }
};
var HomeMarksSite = Class.create(HomeMarksBase,{
  initialize: function() {
    this.signupForm = $('signup_form');
    this.initEvents();
  },
  startAjaxForm: function(event) {
    event.stop();
    var form = event.findElement('form');
    var options = Object.extend({imgSrc:'loading_invert.gif'}, arguments[1] || {});
    var loadArea = form.down('#form_loading');
    var imgTag = IMG({src:('/images/'+options.imgSrc)});
    $(loadArea).update(imgTag);
    new Ajax.Request(form.action,{
      onComplete: function(request){ this.delegateCompleteAjaxForm(form,request) }.bind(this),
      parameters: form.serialize(true)
    });
    form.disable();
  },
  delegateCompleteAjaxForm: function(form,request) {
    var mood = this.getRequestMood(request);
    this.completeAjaxForm(form,{mood:mood});
    if (mood == 'good') {
      switch (form) {
        case this.supportForm : this.completeSupportForm(request); break;
      }
    }
    else {
      form.enable();
      if (!request.responseText.blank()) {
        var flashHTML = DIV([H2('Errors On Form!'),this.messagesToList(request)]);
        this.flashModal('bad',flashHTML);
      };
    };
  },
  completeAjaxForm: function(form) {
    var options = Object.extend({mood:'good'}, arguments[1] || {});
    var loadArea = form.down('#form_loading');
    var completeId = 'complete_ajax_form_' + form.id;
    var imgSrc = '/images/'+options.mood+'.png';
    var moodHtml = SPAN({id:completeId,className:'m0 p0'}, [IMG({src:imgSrc})]);
    $(loadArea).update(moodHtml);
    setTimeout(function() { $(completeId).fade(); },3000);
  },
  completeSignupForm: function(request) {
    var message = "Thank your for signing up for your own HomeMarks page. An email has been sent to " +
      "your address along with a link to activate your account. If you have not done so already, please " +
      "take a moment to read the HomeMarks documentation.";
    var flashHTML = DIV([H2('Signup Complete:'),P(message)]);
    this.flash('good',flashHTML);
  },
  initEvents: function() {
    if (this.signupForm) { this.signupForm.observe('submit', this.startAjaxForm.bindAsEventListener(this)); };
  }
});

Client-Side Error Handling

Currently I have 3 client-side options that I can use to present errors back to the user from the JSON array in AJAX response. The first I started out with was using the alert() function. That got boring real quick, but I left my implementation below which is part of the base class, called messagesToAlert().

The second slightly more interesting way was to create a flash message that used the same XHTML/CSS already present in my layout file. This was real easy to implement. Basically using the ERB fragment below in my layout, I guaranteed that I had 3 unique DOM elements for each of my flash message styles, good/bad/indif. Now all I had to do was to create an accessor in my HomeMarksSite class. I choose this.flashes. I then created a clearFlashes() function and then a flash() function that would take the mood and the HTML to embed, viola â€“ I can now call this.flash.('good',someHTML) and I get the same type of flash one would have seen if I set it in the controller and rendered. Note, this is what I used on the the signup form success above.

<% [:good,:bad,:indif].each do |key| %>
  <div id="flash_<%= key %>" class="flash_message" style="display:<%= flash[key].blank? ? 'none' : 'block' %>;">
      <span><%= flash[key] %></span>
    </div>
<% end %>

var HomeMarksBase = {
  messagesToAlert: function(request) {
    var messages = request.responseJSON;
    var alertText = messages.join(".\n");
    if (alertText.endsWith('.')) { alert(alertText); } else { alert(alertText+'.'); };
  },
  messagesToList: function(request) {
    var messages = request.responseJSON;
    var messageList = UL();
    messages.each(function(message){
      messageList.appendChild( LI(message.escapeHTML()) );
    });
    return messageList;
  }
};
var HomeMarksSite = Class.create(HomeMarksBase,{
  initialize: function() {
    this.flashes = $$('div.flash_message');
  },
  clearFlashes: function() {
    this.flashes.invoke('hide');
    this.flashes.invoke('update','');
  },
  flash: function(mood,html) {
    this.clearFlashes();
    var moodFlash = this.flashes.find(function(e){ if (e.id == 'flash_'+mood) {return true}; });
    moodFlash.update(html);
    moodFlash.show();
    $('site_wrapper').scrollTo();
  }
});

Lastly, and probably the coolest, is the HomeMarksModal JavaScript class I created. This is my new default way of errors to a user. It can be seen in all my video examples on this page. To the right is another example of the login form that uses the same handlers described above. The HomeMarksModal JavaScript class is a bit large to paste inline in this article, but if you want to see the latest version, you can always get it from the master branch of the homemarks_core project on Github.com. When you bind an instance of this object to the window/document, it will automatically crate the modal HTML using Builder. If you want to use it, make sure to get the CSS and images the project too. Below is a function that I put in my site class that allows me to call this.flashModal('good',someHTML)).

var HomeMarksSite = Class.create(HomeMarksBase,{
  flashModal: function(mood,html) {
    var moodColor = this.flashMoodColors.get(mood);
    HmModal.show(html,{color:moodColor});
  }
});

Wrapping It Up

This article turned out a lot longer than I had wanted. The AJAX head pattern is pretty simple and it is really fun to see a two line user signup action yield such interactive results. Something also feels good about not putting view code in the controller, yes you can look at inline RJS as view code. Sure it requires that you do a lot more JS work, but there are benefits.

Image you have team of programmers, one person can stay in model/controller land while the other stays on the view/javascript. The model/controller person would write their own tests (PLEASE TAKE A LOOK AT MINE), while the view/javascript person might even take the initiative to use selenium to test the JavaScript. At the time of this writing, I have not done any selenium tests since the plugin for rails is not compatible for 2.0. That last benefit I could think of for this design pattern is that you could easily update view code without restarting your rails app. Just update the JavaScripts and your good to go.

In closing, please let me know what you think. Perhaps you have a better name for the design pattern? Perhaps someone else has a name for it already?

Resources

Original application_helper.rb from HomeMarks v1. An overboard example of shared view/controller tagsoup/rjs.
The latest Advanced Rails Recipes book. See the chapter on "Replace In-View Raw JavaScript". I discourage this technique used in mass.
Rails API for the ActionController::Base#head method.
Rails API for the HTTP status codes.
Rails API for the ActionController#rescue_from method.
The latest RenderInvalidRecord module from the HomeMarks core.
The latest HomeMarksModal JavaScript class from the HomeMarks core.

Using Rubyamp Textmate Bundle With Opt Ruby_redirect

Fri, 25 Apr 2008 00:00:00 -0400

Using RubyAMP TextMate Bundle With /opt Ruby

Fri, 25 Apr 2008 00:00:00 -0400

I've been a TextMate user for a long time now and I'm still finding new things to do with it. Here recently I wanted to use the RubyAMP TextMate Bundle and was a little miffed to find that it was pointing to my OS X system ruby. The error message looked something like this when it went looking for my ruby/gems.

No such file to load -- appscript (LoadError) from
/System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/rubygems/custom_require.rb:27:in `gem_original_requireâ€™
/System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/rubygems/custom_require.rb:27:in `requireâ€™
...

My problem is that I use ruby installed from MacPorts (yes I have Leopard). I do this because I put a lot of crap and extra dependencies into my opt direcotry and I do not trust Apple to NOT blow away the crazy dependencey hell I would have ended up creating in the standard bin directories if I chose not to use MacPorts. I would have thought that my TM_RUBY environment variable being set correctly to /opt/local/bin/ruby in my TextMate preferences would have given RubyAMP enough info to find my correct gem environment. Obviously not... and it took me quite a bit of digging around to learn what else I needed to do. For starters, here is more than you ever wanted to know about how TextMate gets the $PATH information. You can skip reading that and do these simple steps.

Open /Developer/Applications/Utilities/Property List Editor.app
Click on "New Root", now expand that node in the list view below.
Click on "New Child", name it PATH
The child row for PATH should be a String type
Enter your PATH info here, should mimic your .bash_profile, without $PATH
Save this file to your Desktop as environment.plist

Now do this in the console, we need to move that file to an invisible .MacOSX folder in your home directory. In all likelyhood this folder does not exists, nor does the environment.plist file inside of it, if so, please do your own work to make sure that you do not overwrite existing information. Now:

$ mkdir ~/.MacOSX
$ mv ~/Desktop/environment.plist ~/.MacOSX

With the file in place, you now have to log out and back in. To be on the safe side, just reboot. Now your OSX apps, including TextMate and it's bundles will have the correct PATH information to get to your binaries. See below for an example of what my environment.plist file looks like. Note too that my full PATH info in that file is this string /opt/local/bin:/opt/local/sbin:/opt/local/lib/mysql5/bin:/opt/local/apache2/bin. Notice that path info after the apache2 path? This is where you should add the default path info for a Mac. Unlike the .bash_profile, you can not specify $PATH and have it expanded from within a plist file. So to be on the safe side, when I added this, I put the standard mac path info at the end of my own additions. That standard path info is /usr/local/bin:/bin:/sbin:/usr/bin:/usr/sbin

How To Use Passenger Mod_rails With Rails Edge 2 1_redirect

Tue, 22 Apr 2008 00:00:00 -0400

How to use Passenger (mod_rails) with rails edge 2.1

Tue, 22 Apr 2008 00:00:00 -0400

If you are like me and have been using passenger, then you may have run into an issue when working with rails edge. I mean the REAL rails edge on Git, not that fancy rake task which I think is still pointing to a subversion snapshot. Let me take an aside on how to freeze rails edge to a project that you are managing in Git. This method is akin to using svn:externals. As a cop out, here are 3 links that you should read to learn how.

OK... now that you know how to freeze the real rails edge to your git project, and if you have been using passenger, you may now have seen this error below. The basic problem is that passenger bypasses the work that the rails boot.rb does and in doing so, it only accounts for setting RAILS_ROOT during the ApplicationSpawner process and not the FrameworkSpawner process. In the latest rails, ActionPack is now relying on RAILS_ROOT to be set by calling Rails.root (shortcut method to that constant) when loading. So my fix was to add the RAILS_ROOT to the FrameworkSpawner class.

Framework that failed to load: Vendor directory: /Users/foo/project/vendor/rails
Error message: Anonymous modules have no name to be referenced by
Exception class: ArgumentError

Resources

Now On Passenger Mod_rails_redirect

Sun, 13 Apr 2008 00:00:00 -0400

Now on Passenger (mod_rails)

Sun, 13 Apr 2008 00:00:00 -0400

Well this is working out well so far. I'm really liking the Passenger (mod_rails for Apache) extension. Right now I have this Mephisto site running it and it seems to be doing really well. Also, most people do not do this, but I run a full development stack Apache/MongrelCluster to mimic production boxes the best way I can. Now I am running mod_rails on all my development hosts.

Some Things I Like

I do not have to fuss with OS X launchd startup scripts for my mongrels, just Apache.
Typically in a high volume site that runs mongrel behind an Apache proxy balancer, will get a large timeout and proxy error even if the mongrels are immediately available. Passenger has a nice way to restart the app, just touch the tmp/restart.txt file.

Some Things I'm Waiting For

The RailsEnv can not be set per virtual host. You have to set RAILS_ENV = 'development' in each app if you want to run mixed virtual hosts with different environments.
Normally I would pass environment variables to the console when issuing mongrel cluster starts. I would really love to see Passenger support apache environment variables.

Autotest Playlist For Red Green Feedback_redirect

Sun, 06 Apr 2008 00:00:00 -0400

Autotest Infinite Loop On Failure Error_redirect

Sun, 06 Apr 2008 00:00:00 -0400

Autotest Playlist For Red/Green Feedback

Sun, 06 Apr 2008 00:00:00 -0400

Here is how to get a playlist of sounds that will be hooked to both your autotest :red and :green callbacks. Basically this gives you a folder of sounds that are played one after another, in a loop, as your tests pass or fail. See this move below for a quick example.

Step 1: Install QTPlay from MacPorts

$ sudo port install qtplay

Step 2: Download My Autotest Playlist Files

$ cd ~
$ curl -O http://cdn.metaskills.net/downloads/autotest_playlist.tar
$ tar -xf autotest_playlist.tar
$ mv autotest_playlist .autotest_playlist && rm autotest_playlist.tar

Step 3: Modify Your .autotest File

User your favorite editor to open your ~/.autotest file. And add this code below to it. Now you should be setup to have your own playlist of sounds that play using the qtplay binary installed above. If you want to play around with the sounds, then open ~/.autotest_playlist/sounds and start swapping out sounds. By the way, I made the :initialize and :quit sounds using the new Alex voice in Leopard. I hijacked some text to speech and then ran an audio tool called WavePad to speed change, flange, and chorus the voice a little bit.

require '~/.autotest_playlist/playlist'

Credits:

The idea for this came from FoxWorks. Shouts out "Thank You".

Autotest Infinite Loop On Failure & Error

Sun, 06 Apr 2008 00:00:00 -0400

I just had an issue pop up today that seemed to be an issue for a few others. It seemed that all of a sudden that my autotest was stuck in an infinite loop after a failure or error. At first I thought it was related to some additions to my ~/.autotest file but after commenting out the whole lot of additions there, I realized it was something else. Here was my fix. Basically I think these errors are always related to a file that has changed during your test run. Now we just have to find out what that files are. Here are the steps I took to find out.

Step 1: Gather Changed FIle Info

To find out what files are changing. To do this add the -v option when you start autotest. This will cause it to run in verbose mode. Now after you have failed a test and YOU KNOW YOU DID NOT SAVE ANYTHING watch what happens below your listing of test, assertions, failures, and errors. There will be an array dumped that will contain the files change that have caused autotest to start another test cycle. In my case here is what I saw.

1) Failure:
test_truth(BookmarkTest)
[test/unit/bookmark_test.rb:6
test/unit/bookmark_test.rb:5]:
<false> is not true.

1 tests, 1 assertions, 1 failures, 0 errors
[["config/uuid.state", Sun Apr 06 13:58:55 -0400 2008]]
Dunno! config/uuid.state

Ah ha... there it goes. I was using a UUID state file and it appears that file being written into the config directory is the culprit. By default autotest does not ignore that directory.

Step 2: Fix The Problem

We have two options here. You can either add an autotest exception in you ~/.autotest file or you can bail out. My option is to bail out because I should not be adding a file at run time to the config directory. It would seem a better place would be the tmp directory. However if you think that adding an exception to your ~/.autotest file would be more appropriate, here is the syntax:

Autotest.add_hook :initialize do |autotest|
  ['.svn','.hg','.git'].each { |exception| autotest.add_exception(exception) }
end

Macbook With 4gb Of Memory_redirect

Thu, 03 Apr 2008 00:00:00 -0400

How To Stop Delivery Of An Email Within An Actionmailer Method_redirect

Thu, 03 Apr 2008 00:00:00 -0400

MacBook With 4GB Of Memory

Thu, 03 Apr 2008 00:00:00 -0400

My MacBook is about a year old now and I've read in quite a few places that they could be upgraded to around 3.3GB of memory for this specific model. Unlike the latest MacBook which can "as advertised" accommodate 4GB of memory, the theory was that the Santa Rosa chipset can address all 4GB. It could very well be that technically my MacBook is only using ~3.3GB of that memory but the performance has been significant for a very cheap stick of RAM. I highly recommend Crucial.

I think this upgrade will have to hold me for some time. In general it has really helped my uptime. Odd as that is I think working with only 2GB of memory caused a lot of IO swap and memory corruption. Especially when stopping and starting so many different mongrels for local Rails development. As of this writing I have not found an issue that I have had to reboot for... uptime 10 days and going strong.

To keep my uptime going strong, I just purchased an app called iFreeMem today after reading about it on TUAW. Because I use PhotoShop ever now and then it really helped having this utility free up some memory that it would not release. So far it is working great!

How To Stop Delivery Of An Email Within An ActionMailer Method

Thu, 03 Apr 2008 00:00:00 -0400

OK, so you want to keep your code placement REALLY organized. You have read about my persnicketyness and now you want to practice the best in concern placement and keep those controllers of yours really slim. Like me, you may want to try and keep controller feature additions to very specific one liners of code. Organizing your controller code to do just that with ActiveRecord models or even your own custom classes is a pretty easy task, but how do you keep things simple when dealing with controller actions that have to send email AND you want that single email link of code to be responsible for everything in it's own encapsulated way.

The answer is to push the logic back to the model again. Your controllers should not be concerned with the logic that deals with your own business rules for sending email. The problem with ActionMailer is that most people just use the dynamic MyMailerClass.deliver_ methods that utilize method_missing to instantiate a mailer classs, and send an email in one fell swoop. I like this usage too, but the challenge is how to tell an instance of a the ActionMailer::Base class that it needs to stop delivery within a method definition that is all set to deliver? The answer is Ruby singleton method magic.

module ActionMailer
  class Base

    # A simple way to short circuit the delivery of an email from within
    # deliver_* methods defined in ActionMailer::Base subclases.
    def do_not_deliver!
      def self.deliver! ; false ; end
    end

  end
end

I suggest placing this code into your lib/core_ext/action_mailer.rb file. It will allow you to write mailer methods that can short circuit themselves to stop delivery. For instance:

class MyMailerClass < ActionMailer::Base
  def user_notification(user)
    # ...
    do_not_deliver! if user.email.blank?
  end
end
# In your controller code.
MyMailerClass.deliver_user_notification(@user)

Now you do not have to worry about placing delivery concerns in the controller and even worse duplicate that code when you have to use the same mailer method in multiple places. This works by letting the instance of the ActionMailer::Base class define it's own deliver! method which trumps the delier! method called by that object normally defined in ActionMailer::Base.

Resources

Don T Be A Plinko Programmer_redirect

Wed, 26 Mar 2008 00:00:00 -0400

Don't Be A Plinko Programmer

Wed, 26 Mar 2008 00:00:00 -0400

One of the things that I have really grown persnickety about is the placement of code. For example, I am a huge advocate that controllers in a rails project should read like a mini Domain Specific Language (DSL) and that as much logic as possible be delegated to the models. In my opinion the best way to do that in a Rails project is to learn the proper usage of ActiveRecord Association Extensions. You can check out the Rails API on this page and scroll down to the section called "Association Extensions" if you read the official docs. In short:

The proxy objects that control the access to associations can be extended through anonymous modules. This is especially beneficial for adding new finders, creators, and other factory-type methods that are only used as part of this association.

Now this brings me to the topic of my article, what is a Plinko Programmer. If you have no class and don't even know what Plinko is, Wikipedia has a great write up on it. A Plinko Programmer is someone that writes code which smells in a few ways. For instance they use excessive arguments in their methods and unnecessarily pass objects around as arguments. They like to bake their own Enumerable methods vs using the ones readily available. They even like to create large class level methods, or even worse controller actions, that really should be factory methods in 2 or more classes. The analogy is akin to much of the Java code I have rewritten in Rails. Plinko code is long and full of if/else conditions, it's just like the game. You drop an object in the top and just watch it "by chance" work it's way thru the method/function. It is a nasty way to write code and if for anything else it is illegible and hard to test.

What Is The Right Way?

Here is a short example of how to use Association Extensions. This is a simple example, but when you get used to really using association extensions you will begin to see just how much of your code really belongs there. Let's assume you have a simple invoice and items class.

class Invoice < ActiveRecord::Base
  has_many :items, :class_name => 'InvoiceItem', :foreign_key => 'invoice_id'
end
class InvoiceItem < ActiveRecord::Base
  belongs_to :invoice
  def total
    # Some complex stuff
  end
end

Now let's say that you want to have a clean little method for getting the total of the Invoice object. Resist the temptation to simple add an instance method to the Invoice class. Although it is logical to have @invoice.total it is better to add it too the association extension. Why? Well think about it, what are you doing? The answer is that you are working with a "collection" of InvoiceItems. It turns out that this is the first part of what the association extension is for, an easy way to work with a collection that has the benefits of knowing how to proxy to methods that can reflect back down to the original caller. It's hard to explain but I'll just leave you with my persnickety code example. Your general rule should be if you are working with the collection in part or in total, then the association extension is the place for it. Keep in mind that so far I have only talked about has_may association extensions, you can do these for one-to-on belongs_to and has_one associations as well.

class Invoice < ActiveRecord::Base
  has_many :items, :class_name => 'InvoiceItem', :foreign_key => 'invoice_id' do
    def total
      proxy_target.map(&:total).sum
    end
  end
end
class InvoiceItem < ActiveRecord::Base
  belongs_to :invoice
  def total
    # Some complex stuff
  end
end
# Would yeild code like:
@invoice.items.total

P.S. Here lately I've been creating an app/concerns directory where I put modules that encompass mixed in behavior in so many ways for top level models. Typically these modules/concerns shared instance and class methods with two or more primary classes. They have become an excellent home for association extensions since many large applications will define the same association from different models in the object. To keep the code from duplicating in those different models it is better to do something like this

# This file "invoice_item_concerns.rb" would reside in app/concerns
module InvoiceItemConcerns
  module AssociationExtensions
    def total
      proxy_target.map(&:total).sum
    end
  end
end

class Invoice < ActiveRecord::Base
  has_many :items, :class_name => 'InvoiceItem', :foreign_key => 'invoice_id', :extend => InvoiceItemConcerns::AssociationExtensions
end

class PackingSlip < ActiveRecord::Base
  has_many :shipments, :class_name => 'InvoiceItem', :foreign_key => 'packing_slip_id', :extend => InvoiceItemConcerns::AssociationExtensions
end

class InvoiceItem < ActiveRecord::Base
  belongs_to :invoice
  def total
    # Some complex stuff
  end
end

# Would yeild code like:
@invoice.items.total
@packing_slip.shipments.total

Metaskills Net Reborn On Mephisto_redirect

Sat, 22 Mar 2008 00:00:00 -0400

MetaSkills.net Reborn on Mephisto

Sat, 22 Mar 2008 00:00:00 -0400

Well after a year of neglect, the MetaSkills.net blog has been reborn on Mephisto. Previously I was using Drupal and it finally got to a point where I was so deep into ruby that I did not even have the gumption to open up a PHP session to publish anything. The sad part is that I told myself that this PHP disdain would help me get off my butt and move to Mephisto. You know, eat my own dog food â€“ obviously procrastination won out â€“ but not forever. For the past week I worked hard to get the Meta theme for Drupal converted to Mephisto. You can use this theme yourself if you want, the source is available on my github and I am making updates often. Heck... feel free to fork the project and make some changes or let me know if you want me to incorporate them into mine.

Here are a few things that I liked about rewriting the Meta theme for Mephisto. First, unlike Drupal, the administration of your Mephisto blog is not inline, but all tucked away in a private admin section. When you make a theme in Drupal, you are burdened to to the task of coding all the admin CSS for the inline admin features. The Meta theme had over 500 lines of CSS just for the Drupal administration. That is all gone and I love keeping theme code focused on doing nothing but presenting the "user" experience.

Also, when I first wrote the Meta theme I was just a JavaScript beginner. Nowadays I am pretty good at it and have moved away from the simple script like functions that pass arguments around into a full OO style that spawns from smart classes and a persistent state. The Meta theme is now 100% on Prototype and all the interactive features are tightly knit into the MetaTools, MetaSearch, and MetaContent classes. Take a look at the meta.js source if you want. Lastly, if you are on an old Drupal blog looking to get over to Mephisto, maybe this migration script will help.

Migration from Drupal to Mephisto

This script is what I used in the rails console to populate data from my old Drupal 4.7 blog to Mephisto. This code is untested on the schema of higher versions of Drupal.

# My Drupal 4.7 to Mephisto Script
# ----------------------------------------------------------------
# Things I did before
#   * Delete all rows from drupal node table where type != 'blog'
#   * Remove the type column from the drupal node table
#
# Things I did afterward
#   * Found all comments in the Mephisto contents table that were
#     from me and added user_id 1 to them.
#
# Uncomment these if you are debugging the script and want to
# start off on a clean mephisto install
# ----------------------------------------------------------------
# Article.find(:all).each(&:destroy)
# CachedPage.delete_all
# AssignedSection.delete_all
# Tagging.delete_all
# Tag.delete_all

@mysite = "http://www.metaskills.net/"
@home = Section.find_by_name('Home')

class DrupalArticle < ActiveRecord::Base
  establish_connection :drupal
  set_table_name  :node
  set_primary_key :nid
  has_many :comments, :class_name => 'DrupalComment', :foreign_key => 'nid'
  has_one :version, :class_name => 'DrupalArticleVerson', :foreign_key => 'nid'
  def created_at ; Time.at(self[:created]) ; end
end
class DrupalArticleVerson < ActiveRecord::Base
  establish_connection :drupal
  set_table_name  :node_revisions
  set_primary_key :vid
end
class DrupalComment < ActiveRecord::Base
  establish_connection :drupal
  set_table_name  :comments
  set_primary_key :cid
  belongs_to :article, :class_name => 'DrupalArticle', :foreign_key => 'nid'
end

DrupalArticle.find(:all).each do |da|
  # Creating the article
  na = Article.new
  na.site_id      = 1
  na.created_at   = da.created_at
  na.published_at = da.created_at
  na.updated_at   = Time.now
  na.title        = da.title
  na.body         = da.version.body
  na.excerpt      = da.version.teaser
  na.updater_id   = 1
  na.user_id      = 1
  na.save!
  na.sections << @home
  # Creating comments for this article
  da.comments.each do |dac|
    nac = na.comments.build
    nac.site_id       = 1
    nac.created_at    = Time.at(dac.timestamp)
    nac.published_at  = Time.at(dac.timestamp)
    nac.updated_at    = Time.at(dac.timestamp)
    nac.author        = dac.name
    nac.author_url    = dac.homepage
    nac.author_email  = dac.mail
    nac.author_ip     = dac.hostname
    nac.user_agent    = "Mozilla/5.0 (Macintosh; U; PPC Mac OS X; en) AppleWebKit/523.12.2 (KHTML, like Gecko) Version/3.0.4 Safari/523.12.2"
    nac.referrer      = @mysite
    nac.approved      = true
    nac.title         = dac.subject
    nac.body          = dac.comment
    nac.save!
  end
end

Rake Task To Read And Write The Activerecord Schema Version Table_redirect

Wed, 31 Jan 2007 00:00:00 -0500

Rake task to read and write the ActiveRecord schema version table.

Wed, 31 Jan 2007 00:00:00 -0500

After attending Rails Edge in Reston, Virginia I decided to move some common tasks from my ~/.irbrc file and put them into Rake. I thought I would share a task that reads and writes the ActiveRecord schema table. Sometimes in migrations this is either good to know or manually change. Simply copy this in a foo.task file in your project/lib/tasks directory and use rake -T to see the description and usage. I have placed these tasks in the db:version namespace.

namespace :db do
  namespace :version do

    desc "Read the current version of the database."
    task :read => :environment do
      puts "The #{RAILS_ENV}' database version is: #{ActiveRecord::Migrator.current_version}"
    end

    desc "Manually set the schema version to a specific target version with VERSION=x"
    task :write => :environment do
      if ENV['VERSION']
        if ActiveRecord::Base.connection.update("UPDATE #{ActiveRecord::Migrator.schema_info_table_name} SET version = #{ENV['VERSION'].to_i}")
          puts "SUCCESS: The '#{RAILS_ENV}' database version is now: #{ActiveRecord::Migrator.current_version}"
        else
          puts "FAILED: The '#{RAILS_ENV}' database version is still: #{ActiveRecord::Migrator.current_version}"
        end
      else
        puts 'You must specify a VERSION=n argument to this command. Use rake db:version:read to get the current version.'
      end
    end

  end
end

Web Service Woes

Fri, 29 Dec 2006 00:00:00 -0500

Only now do I really get this image from DHH's blog. I am really beginning to despise SOAP for some reason, but I guess I will "get it" as I learn more about it -- as it seems to be the only way to do business with the big players out there. For a glimpse of some of my hell, check out this Ruby + SOAP4R + WSDL Hell article.

The bad news aside, I was really happy to find someuseful tools today from Todd Ditchendorf Scandalous Software site. The list is pretty extensive so go check it out. Not on the list that I was happy to see were:

Check XML and XHTML documents for Well-Formedness and Validity while editing them in TextMate with support for DTD, W3C XML Schema, RELAX NG, Schematron and XInclude.

SOAP Client is a free Cocoa-based developer tool for Mac OS X Tiger that allows you access and debug WSDL & SOAP-based Web Services from the comfort of your desktop.

HomeMarks, My First Ruby On Rails Pet Project

Thu, 21 Dec 2006 00:00:00 -0500

I've heard that the best web applications are ones that usually start as a simple tool that meets the needs of its creator, in this case, myself. I have always been close to Web Developers that code their own HTML start pages with links to the projects they are working on. So when the time came for me to really get serious with my programing career using RAILS, this was the first app that I thought about building. From the site's copy...

HomeMarks is a unique application allowing everyone from novice to elite the ability to create their own bookmark start page using easy drag and drop methods for sorting Columns, Boxes and Bookmarks. Our custom bookmarklet tool allows you to add links to your HomeMarks start page while surfing and/or researching online. If you're a bookmark junkie with literally hundreds of saved links to your favorite Internet haunts, HomeMarks is not for you. However, if youâ€™re looking for a clean, simple homepage to organize your most recent or relevant links in an environment tailored to your needs and interests, give HomeMarks a try. After all, HomeMarks is where the heart is.

So it is not like the world needs another AJAX homepage, there are plenty out there. My problem is that they seem to go to far and build to much into them, Netvibes is a great "poor" example. I do not want XML Feeds, Images, News Headlines, or anything else in my start page. I just want links and only about 100 or so at maximum. For me bookmarks are very transitory objects. They need to be direct links to the things that I am working on, or referencing at that exact time. Once they are not, they get throw away and I can almost certainly find them in the 1st page of a Google result if I really needed to. So this is what HomeMarks is for me, simple software, using build less principals. If you choose to use it, I hope you enjoy using it. Note, only for Firefox or Safari/WebKit.

Happy Halloween from TextMate

Tue, 31 Oct 2006 00:00:00 -0500

Upon downloaded the Cutting-Edge version (1.5.4 r1311) of TextMate yesterday, I was happy to find a halloween surprise. How cool can one program be - thanks Allan! Especially to some of your latest work on extending the dialog features for bundle developers.

Custom Rake Task To Unload Fixtures_redirect

Mon, 02 Oct 2006 00:00:00 -0400

Custom Rake Task To Unload Fixtures

Mon, 02 Oct 2006 00:00:00 -0400

I made a revised version of a rake task that I have used quite often, for unloading existing DB tables into fixture data and thought I would share. This rake task is the in correct name space and adds a "rake db:fixtures:unload" command to your rails project when you put this in "lib/tasks/foo.rake". It can take an optional TABLES variable or if none is present the whole array of DB tables are converted. I find this rake task helpful when dealing with LARGE databases.

namespace :db do
  namespace :fixtures do
    desc 'Create YAML test fixtures from data in specifed tables. Set table names by TABLES=foos,bars,etc'
    task :unload => :environment do
      sql  = "SELECT * FROM %s"
      tables = ["schema_info"]
      ActiveRecord::Base.establish_connection
      (ENV['TABLES'] ? ENV['TABLES'].split(/,/) : ActiveRecord::Base.connection.tables).each do |table_name|
        i = "000"
        File.open("#{RAILS_ROOT}/test/fixtures/#{table_name}.yml", 'w') do |file|
          data = ActiveRecord::Base.connection.select_all(sql % table_name)
          file.write data.inject({}) { |hash, record|
            hash["#{table_name}_#{i.succ!}"] = record
            hash
          }.to_yaml
        end
      end
    end
  end
end

Flying Light Configuring Drupal And Lighttpd_redirect

Fri, 30 Jun 2006 00:00:00 -0400

Flying Light - Configuring Drupal and LightTPD

Fri, 30 Jun 2006 00:00:00 -0400

So the WebSvr mini here at my home-based NOC (named ActionMoniker.com) is now running LightTPD as the web server. The benefits are that I can now run my PHP-based Drupal blog in FastCGI mode while also allowing virtual hosting under the same server/IP for my RAILS projects. The end result has turned out quite well and I am actually loving the speed improvements and the simple configuration for LightTPD, which has a more natural feeling for me when it comes to configuring a web server.

Below is a code snippet of my current configuration file for this Drupal host. Please be aware, that this is not a full LightTPD configuration file which would likely have further restrictions on how your web server operates and secures itself. That said, this snippet incorporates the following configurations which are mostly necessaries for any Drupal blog using the Drupal supplied .htaccess file for Apache.

A vhost default so that both the www and the root domain point to the same directory.
URL access rules that deny access to common Drupal install and app files.
URL rewrite rules for my mint statistics, multi site setup in a /meta-theme-for-drupal directory which, and the standard drupal rewrites for clean ULRs and a few system functions. It is important that these rewrites be in the correct order with the root drupal install being last.
URL expires directives that help browser caching for all page assets that are core Drupal and found in the Meta Theme.
A compress cache directory, server error log, and access log using the web server root variable set at the tops of the vhost config. I like that LightTPD can set variables.
Finally a LightTPD declaration for spawning FastCGI process for PHP.

simple-vhost.server-root      = "/Library/WebServer/hosts/"
simple-vhost.default-host     = "default"

$HTTP["host"] =~ "^(metaskills.net)|(www.metaskills.net)$" {
  simple-vhost.default-host     = "metaskills.net"
  url.access-deny = (
    ".engine", ".inc", ".install", ".module", ".sh", ".sql", ".pgsql",
    ".theme", ".tpl.php", ".xtmpl", ".code-style.pl", ".Repository", ".Root"
    )
 url.rewrite-once = (
   # hard-coded rewrites so the drupal catch-alls do not get them.
   "^/mint/$" => "/mint/index.php",
   "^/mint/\?(.*)" => "/mint/index.php?$1",
   # drupal rules for (meta-theme-for-durpal) multi-site, these MUST BE FIRST.
   "^/meta-theme-for-drupal/system/test/(.*)$"  => "/meta-theme-for-drupal/index.php?q=system/test/$1",
   "^/meta-theme-for-drupal/search/node/(.*)$"  => "/meta-theme-for-drupal/index.php?q=search/node/$1",
   "^/meta-theme-for-drupal/([^.?]*)\?(.*)$"    => "/meta-theme-for-drupal/index.php?q=$1&$2",
   "^/meta-theme-for-drupal/([^.?]*)$"          => "/meta-theme-for-drupal/index.php?q=$1",
   "^/meta-theme-for-drupal$"                   => "/meta-theme-for-drupal/index.php",
   # drupal rules for default site behavior.
   "^/system/test/(.*)$"        => "/index.php?q=system/test/$1",
   "^/search/node/(.*)$"        => "/index.php?q=search/node/$1",
   "^/([^.?]*)\?(.*)$"          => "/index.php?q=$1&$2",
   "^/([^.?]*)$"                => "/index.php?q=$1"
   )
  expire.url = (
    "/favicon.ico"              => "access 3 days",
    "/files/"                   => "access 3 days",
    "/misc/"                    => "access 3 days",
    "/themes/meta/css/"         => "access 3 days",
    "/themes/meta/images/"      => "access 3 days",
    "/themes/meta/js/"          => "access 3 days",
    "/themes/meta/meta-black/"  => "access 3 days",
    "/themes/meta/meta-grey/"   => "access 3 days",
    "/themes/meta/meta-paper/"  => "access 3 days",
    "/themes/meta/meta-pink/"   => "access 3 days"
    )
  compress.cache-dir            = webserver-root + "tmp/cache_metaskills/"
  server.errorlog               = webserver-root + "logs/metaskills.error.log"
  accesslog.filename            = webserver-root + "logs/metaskills.access.log"
  fastcgi.server                = ( ".php" =>
                                    ( "localhost" =>
                                      (
                                        "socket" => "/tmp/php-fastcgi.socket",
                                        "bin-path" => "/opt/local/bin/php-fcgi",
                                        "max-procs" => 4,
                                        "bin-environment" => (
                                          "PHP_FCGI_CHILDREN" => "16",
                                          "PHP_FCGI_MAX_REQUESTS" => "10000"
                                        ),
                                        "bin-copy-environment" => (
                                          "PATH", "SHELL", "USER"
                                        ),
                                        "broken-scriptfilename" => "enable"
                                      )
                                    )
                                  )
}

Obviously this configuration is dependant on php-fcgi being installed and If you followed along in my past Soup to Nuts Recipe for Ruby on RAILS on OS X, here is a few crib notes I kept on how to recompile a few things so that you can have your whole installation of LightTPD, Ruby, PHP, and MySQL all running from the Darwin Ports /opt directory.

Reinstall LightTPD with SSL

If you do not plan on using SSL with Lighty, this step is optional.

$ sudo port uninstall lighttpd
$ sudo port install lighttpd +ssl

MySQL 5 Installation in Darwin Ports

This part is definitely not for the week. If you are total noobie, stick with the compiled packages offered by MySQL AB because doing this installation may break some of the tools you might be familiar with. Most can be fixed, but you will definitely loose your fancy start/stop button in the System Preferences.

I installed MySQL 5 in this manner so that I could have PHP from Darwin Ports totally hooked into the MySQL installation by DP as well. I do not suggest you do this if you are new to this sort of stuff, again, these are just some quick notes I made. OK...

First, add /opt/local/lib/mysql5/bin to the $PATH environment variable in your .bash_profile. When you do this you are linking to the REAL path for the MySQL binaries and not the Darwin Ports suffix5 sym links that it creates in/opt/local/bin

Now issue these command. This will install mysql5 with Darwin Ports and the server startup scripts. The second command will load that startup script into your launchd list. and the third will install the default MySQL tables for permissions, etc.

$ sudo port install mysql5 +server
$ sudo launchctl load -w /Library/LaunchDaemons/org.darwinports.mysql5.plist
$ sudo -u mysql /opt/local/lib/mysql5/bin/mysql_install_db

So that you do not have to do the following steps, restart you computer which should let MySQL use the startup script you enabled in the step above. In this way you can bypass these first two commands below. The other two commands are some notes of mine on the usage of the wrapper LaunchDaemon installed by Darwin Ports for MySQL. For easy reading, check out the support pages for UNIX Post-Installation Procedures offered on the MySQL Reference Manual. Remember, I am pretty sure that you do not have to do these steps if you just restart your computer.

$ sudo -u mysql /opt/local/lib/mysql5/bin/mysqld_safe &
$ sudo /opt/local/lib/mysql5/bin/mysqladmin shutdown
$ sudo -u mysql /opt/local/etc/LaunchDaemons/org.darwinports.mysql5/mysql5.wrapper start
$ sudo -u mysql /opt/local/etc/LaunchDaemons/org.darwinports.mysql5/mysql5.wrapper stop

Here we are going to create the my.cnf file in one of the places this installation of MySQL likes to find it from the example files installed by MySQL. In this example I used the medium example file, but if you are expecting more traffic and have more RAM, use the my-huge.cnf file instead. Next we are creating a sym link to the my.cnf in the standard place the normal MySQL AB binaries install them on OS X, in the etc directory. In this way you can still use MySQL Administrator buy be aware that I had to use 127.0.0.1 instead of localhost. Not sure why.

$ sudo -u mysql cp /opt/local/share/mysql5/mysql/my-medium.cnf /opt/local/var/db/mysql5/my.cnf
$ sudo ln -s /opt/local/var/db/mysql5/my.cnf /etc/my.cnf

Now to fix the MySQL gem that was installed the first time around. This install type uses the mysql-config as described in the fine manual.

$ sudo gem uninstall mysql
$ sudo gem install mysql -- --with-mysql-config=/opt/local/lib/mysql5/bin/mysql_config

Installing PHP5 using Darwin Ports MySQL

Now you are ready to install PHP5 with its FastCGI support and Darwin Ports dependencies on the MySQL previously installed. The second command copies the php.ini installed by DP into the place the Darwin Ports PHP likes to find it.

$ sudo port install php5 +fastcgi +mysql5
$ sudo cp /opt/local/etc/php.ini-recommended /opt/local/etc/php.ini

Now add this line of code to the php.ini file somewhere at the end of the section called "Paths and Directories" that is about mid way down. See also this web page for more info on this.

cgi.fix_pathinfo = 1

Quake Style Terminal Window On Os X_redirect

Tue, 13 Jun 2006 00:00:00 -0400

Quake Style Terminal Window on OS X

Tue, 13 Jun 2006 00:00:00 -0400

Well this is far beyond cool but highly functional, a Quake like terminal implementation of Terminal.app that is a HotKey away from within any application. A friend turned me on to this after it showed up on the Monday morning Apple links post from arstechnica.com. Although I have never thought of this idea, it seems to have been a popular request for quite some time and after a public request, the author of QuickSilver stepped up to the challenge and coded this little goodie using the application enhancer method called SIMBL which was created by Mike Solomon, the creator of the PithHelmet plugin for Safari.

So if you are itching to try this out here is what you do.

Download and Install SIMBL
Download Visor and place it in the /Library/Application Support/SIMBL/Plugins folder.
Logout and back into your account (maybe restart)
Launch Terminal.app (close it if you want, also see my AppleScript below)
Set your Visor HotKey preferences in the new menu.

Now, when you launch any other program and hit your HotKey (mine is Command ~) a terminal application will slide down from the top of your screen just like in Quake. The only down side is that if you are used to your iTerm display, you will have to set the preferences of Terminal.app to mimic that. I set mine with a black background, ProFont 10px, and 80% visibility.

Lastly, since you do have to have the Terminal application running for this to work and I was too lazy to open this al the time, I simple created a small AppleScript which can be compiled into an application and placed into your Login Item in the System Preferences. Here is a screenshot of the code. Notice that it tells window 2 to close. This is because once you have SIMBL and Visor installed, an invisible window opens for Terminal that you cannot see. So the window that you can see (window 2) is closed by AppleScript.

Back In Black_redirect

Tue, 06 Jun 2006 00:00:00 -0400

Back in Black

Tue, 06 Jun 2006 00:00:00 -0400

I have not been a black Mac owner since I got rid of my Lombard (aka the Bronze Keyboard) way back in 2001. Before that, I was the proud owner of a 3400c which was also black and one of the first PowerPC laptops offered from Apple. That PowerBook cost me close to $4,000 if memory serves and I swear some of the debt is still hanging around today! So after being with Apple for so long I have very glad that good Mac hardware has gotten cheaper over the past 4 years and to see the return of a the darker colored books. Its about time too and the price is just right.

So this past Sunday I drove from Portsmouth all the way to the Apple Store at Short Pump in Richmond and purchased myself a brand new MacBook, the black one of course. The 12" G4 I have been working on lately was showing its age. Start ups were really slow, database queries on local MySQL development kept grinding to a halt and I'm pretty sure that power management unit was next to go.

Now that it is two days later and I have spent the past two days reinstalling RAILS using my own soup to nuts recipe and putting back every piece of software on this new machine, I thought I would share a few of my experiences to those considering the upgrade to an Intel Mac and perhaps their own MacBook.

Migration

Do not use the migration assistant. I repeat, DO NOT USE THE MIGRATION ASSISTANT. I can not tell you why but doing such will kill the speed of your new Mac. When I did this and even after I waited for the spotlight index to rebuild itself, the computer was still way slower than it should have been. I don't blame Apple for this, it would probably be impossible for the to weed out every prefs and configuration file that might conflict and I have always been a huge advocate of doing it the manual way for any drastic hardware change, but I was curious. If anyone has any insight on why the Migration Assistant would botch so bad, I would love to hear it.

More RAM

Simply but, do it! There is no way to run your new Mac on only 512MB. It should be enough but, shortly after the system boots, it slows down real fast because the first few apps that need Rosetta will likely kill all the remaining RAM you have left. I highly suggest maxing the MacBook to 2 Gigabytes by ordering from TheChipMerchant.com which sell these for only $96 a stick and they have a lifetime warranty.

The MacBook's memory slots are easily accessible, but pulling those leavers scared the hell out of me. I swear I thought they were going to break.

The Screen

I cannot say anything bad about it. I love it! The size of the MacBook screen is perfect for me. I need a small computer for portability and this 13" wide screen is the perfect blend between compact and working real estate. This screen has to be the brightest I've seen and in no way does the glossy cover bother me at all.

My Own Soup To Nuts Recipe For Ruby On Rails On Os X_redirect

Mon, 29 May 2006 00:00:00 -0400

My Own Soup to Nuts Recipe for Ruby on Rails on OS X

Mon, 29 May 2006 00:00:00 -0400

Tim Toady tells us that (There Is More Than One Way To Do It) and I am sure this is not the first and not likely the last blog post you will ever see that tells you the best way to install RAILS on OS X. Some people would just rather than go the simple path but I implore you, do it this way. Gaining the experience of compiling your own software will help you down the road when it comes time for doing more advanced things, especially deploying your RAILS app. When you are done, you will have a rock solid deployment environment for RAILS that will include:

Ruby and RAILS
MySQL Community Edition
FastCGI & LightPPD

Step 1 - Install Apple Developer Tools

The Apple Developer Tools are needed to install Darwin Ports which opens up the door for a bunch of other goodies we will need.

Go to Apple's Developer Connection site.
Click on "Join Now", create an account and log into the site.
Go to the "Downloads" section.
In the right box, click on "Developer Tools"
Scroll to the "Xcode Tools 2.3" section and download the large disk image.

This is obviously going to take awhile, if you want to skip ahead and start on some of the following steps, by all means do so. Once you are done downloading the Xcode Tools and have the disk image mounted, install the package named "XcodeTools.mpkg". You do not have to worry about the others, especially the WebObjects package :)

Once you are finished with the install of the Developer Tools, you will have a new folder at the root of your hard drive called "Developer", pictured below on an OS X Server install. For the most part you can largely ignore this new folder.

Step 2 - Install MySQL & Setup Your First DB

The Ruby on Rails framework comes by default with the MySQL adapters for ActiveRecord to talk to MySQL servers. I highly recommend you start development with MySQL, its a good database and if you have not read the recent news regarding MySQL AB and Apple working more closely then I will tell you now that it seems like good time to revisit MySQL as your primary database choice. If you believe the press, MySQL AB seems more committed than ever to deploy its development tools to the Mac platform. Many would say that these are not really needed when you use RoR's database migrations and a nice white board.

I have previously used their Administrator and Query Browser only to find them full of bugs and crashing often. I'm now happy to say that most of these bugs have disappeared and I can now use these applications with the current versions of MySQL Community Edition without much ado. I was very happy to hear that MySQL AB has a new tool in the line up (currently in beta) called Workbench. This is their first Entity Relationship Modeling application that brings in many of the features available for editing tables from their Administrator application with the GUI development of ER diagramming. ER tools have been very sparse on the Mac and I am very happy to see this one coming. Either way, do your self a favor and download those three tools, at the least, we will need the Administrator tool to setup some database accounts on your new MySQL installation. You may also find Query Browser helpful if you find yourself debugging ActiveRecord methods. Here is our install and setup process.

Go to the MySQL 5 Community Edition download page.
Scroll down to the section title "Mac OS X downloads" and pick the "Max" package that best matches your system.
Once downloaded, install the MySQL package, the StartupItem package, and the PreferencesPane. The PreferencePane is just a simple switch to turn MySQL on and off.
Download and install the MySQL Administrator.
Download and install the Query Browser. (optional)

That's it, MySQL is installed and ready once you restart your system. If you are still downloading the Xcode Tools, you will have to wait to do the following steps which will secure your MySQL installation.

So after a fresh restart from installing MySQL, let's secure your installation by assigning some passwords. To do this launch the MySQL Administrator and log into your MySQL server using the server hostname of localhost and the username of root. You do not need a password, it is blank for root@localhost by default, this is what we are going to fix. It is not a big security bug as long as you are the only one on your machine, it is just a good idea to secure this account.

Click on "Accounts" at the top and delete the account that has no name. You optionally may want to set a password for the root at localhost. From this point on you can optionally create an account specific to any databases you may be using for developing. When creating a user, you have to create a host (typically localhost) at the same time by using that + host icon to the right of the new user icon in the bottom right. Click on "Catalogs" to create a database and then come back to "Accounts", expand the arrow beside the user you want to have access to this new database, select the host for that user, click on "Schema Privileges", click on the name of the newly created database schema and then assign the privileges from the available list to the assigned list. Typically I grant all privileges in development environments.

In the next step, installing DarwinPorts, we are going to cover some $PATH additions that will let our system know where it can find MySQL.

Step 3 - Install DarwinPorts & All Needed Packages

DarwinPorts is your key to a TON of open source software for your Mac. This can include PHP, Ruby, Apache, and Subversion to smaller utilities like, wget, furl, and pine... the list goes on to over 2,000 ports currently available. It is generally accepted that Mac OS X and Darwin Ports is a good thing, but some disagree. Meanwhile, using DP for all of your open source software can help you keep installed packages in one easy to understand place, by default in the /opt directory at the root of your drive. This can save you lots of headaches from the dreaded dependency hell that has plagued most of us whom have not used a good package management system. That is not to say that dependencies will never happen, but when the do, knowing where DP places things can help you resolve them quickly. Let's get to installing our new package management system.

Download DarwinPorts installer from their site.
Install the package.

That's about it, DarwinPorts is installed and ready to use. I do however like to clean things up a little bit. If you have followed my previous article Getting on good terms with the OS X shell then you already know how to customize your .bash_profile, so lets fire up TextMate and do just that right now. Although DP does install a .profile file that will customized your $PATH environments, I think it's a good idea that you put this back in your control. If your new to this path stuff, they are basically just a list of directories that lets your computer know where software is installed. Keeping it that simple lets just put these lines anywhere in your .bash_profile.

# My PATH Environment Variables
export PATH="/opt/local/bin:/opt/local/sbin:/usr/local/mysql/bin:$PATH"
export MANPATH="/opt/local/share/man:$MANPATH"
export INFOPATH="/opt/local/share/info:$INFOPATH"

You can see by the example we are simply adding other path options before your systems default, denoted by the $PATH. Using colons is how you separate the list and this is how we have added the MySQL installation into your system's path environment and if you ever need more, make sure to separate them too by colons and typically try to keep your default $PATH at the end. One last note, that /opt/local/sbin is in the code box above so that LightTTPD will be available to RAILS after we get it installed, more info below.

Now let's start using DarwinPorts, if you want to take a small course on the generals about how to use DP, I suggest taking a look at Joe Auty's Getting Started with DarwinPorts, otherwise, just follow along. Remember that building and installing software from scratch is a CPU intensive task for your computer. It is not unusual for an install to take anywhere from 15 to 30 minutes and maybe longer, so be patient.

Fire up a terminal app and use this command, it will bring our fresh DP install up to date. If you had your terminal open before you added your path variables, remember to close it and open a new one so that the changes will take affect. All remaining installs will be issued in the Terminal window.

$sudo port -d selfupdate

Now let's install the latest release of Ruby and its dependencies. This will take awhile!

$ sudo port install ruby

Now install the Ruby Gems package management system.

$ sudo port install rb-rubygems

Now the FastCGI Development Kit and the Ruby interface to FastCGI. These will be needed when deploying your application into a production mode (or development when using ./script/server) using LightTPD also known as Lighty.

$ sudo port install fcgi
$ sudo port install rb-fcgi

Here we will now go ahead and install Lighttpd and its dependencies. The LightTPD executable is located in /opt/local/sbin and this is why we placed it in our path environment. For future reference the default config file for LightTPD is in /opt/local/etc/lighttpd. You may want to make friends with that file when you start thinking about deployment. The RAILS team makes it easy when you use ./script/server by loading in a RAILs specific LightTPD config file, so you do not have to think about this in the default development mode.

$ sudo port install pcre
$ sudo port install lighttpd

Step 4 - Final Installs (RAILs and MySQL Bindings)

Now to the fun part we have installed Ruby and Ruby Gems using Darwin Ports in our previous step. Now we get to install RAILS with a two part command, the first one liner will install RAILS with all the dependencies using Ruby Gems. The second command, again using Ruby Gems, we will install the latest MySQL bindings. This should speed things up a bit, you can read more about this here along with the performance gains it offers. Please note that when you run this second command, you will likely get a prompt for which platform you would like to install on. I selected option 3 which was mysql 2.7 (ruby).

$ sudo gem install rails --include-dependencies
$ sudo gem install mysql -- --with-mysql-dir=/usr/local/mysql

Just like DarwinPorts, Ruby Gems is a package management system. If you get some time or if you find yourself in a pinch, I find that the command reference here offers a lot of insight and help into how to use Ruby Gems.

Pragmatic Studio Alumni Member_redirect

Fri, 19 May 2006 00:00:00 -0400

Pragmatic Studio Alumni Member

Fri, 19 May 2006 00:00:00 -0400

Well, I just got back this past weekend from the Pragmatic Studio in Boston and it was well worth the time and effort. If you have not yet used the RoR framework for web applications, stay tuned to my next post. By next week I will lay out an extensive tutorial on how to install the framework with all the proper goodies on your Mac. So do not wimp out and go for that easy package install and dare to impress your friends as we build our own software installs with Darwin Ports.

Remember, if you are new to RoR, I highly advise that you take the time to browse over to the Pragmatic Bookshelf and buy yourself some PDF books. I suggest starting with Chris Pine's How To Program and then moving on to the 2nd edition of Agile Web Development with RAILs mixed with brief dives into the Ruby language with the famous Pickaxe book.

Getting On Good Terms With The Os X Shell_redirect

Sun, 19 Mar 2006 00:00:00 -0500

Getting On Good Terms With The OS X Shell

Sun, 19 Mar 2006 00:00:00 -0500

I will be the first to admit that I am really just learning how to tap into the power of my shell environment and to be honest, I've spent way to many hours reading man pages and figuring out how to do some really neat things that help my automate my workflow and system administration. Mostly these are just basic tasks like my Simple MySQL Backup and Deleting Invisible Resource Files scripts. But in all seriousness, when you get right down to using a UNIX-based operating system, you cannot escape using the shell environment. This is a good thing, its your friend, and getting your feet wet sooner than later is a good idea.

Getting A Better Terminal Application

Go download iTerm from the sourceforge project page and install it. That default Terminal.app that comes with the Mac has got to go, it is dreadful looking and the features are really slack. Fortunately iTerm is a native Cocoa app that supports both the PowerPC and the new Intel Macs. The best feature of iTerm is that it supports tabbed windows, because you can never have to many terminal windows open. Here are some preferences I recommend after installing iTerm.

Tab Position Top
Selection copies text to clipboard & Focus follows mouse
The "Foreground" and ANSI Colors pictured below, most are defaults.
A dark background image with little detail so as not to interfere with your foreground text.
In there "Terminal" tab of the profiles, set the "Type" to "xterm-color".

Customizing Your Bash Profile

After you have a nice terminal program, its time to tweak your environment a bit. Here is a great set of customization commands that will change the way your shell interface is displayed when using the bash terminal, this is the standard shell environment on newer versions of OS X. Using these environment enhancements you will get a solid visual on the following information which is also colored coded to appear well on a dark background.

The current user you are logged in as (green) for you (red) for root/others.
The current machine name and working directory.
A new entry line which is not obscured by the items up above.

# User colors (root colors in /etc/bashrc)
export TERM="xterm-color"
C1="\[\033[1;32m\]"
C2="\[\033[1;30m\]"
C3="\[\033[0m\]"
C4="\[\033[0;36m\]"
export PS1="${C2}(${C1}\u${C2}@${C4}\h${C2}) - (${C4}\A${C2}) - (${C4}\w${C2})\n${C2}-${C1}=>>${C3}"

# Root colors (user colors in ~/.bash_profile)
export TERM="xterm-color"
C1="\[\033[0;31m\]"
C2="\[\033[1;30m\]"
C3="\[\033[0m\]"
C4="\[\033[0;36m\]"
export PS1="${C2}(${C1}\u${C2}@${C4}\h${C2}) - (${C4}\A${C2}) - (${C4}\w${C2})\n${C2}-${C1}=>>${C3}"

Here is how to install these, they go into two separate places. The first customization commands go into your user home directory in a file called .bash_profile and if you have never modified or used this file, you will have to make a new one. The period that precedes the name is technically not allowed by most text editors since it makes the file invisible. I suggested using TextMate to create or modify this and the file below. Remember, the first file .bash_profile needs to go into your home directory. The more you use your terminal, the more you will start to modify this file. It too is your friend.

The second set of customization commands go into the root bashrc file. Again, using TextMate, you can find this file in the /etc directory. If you are new to using TextMate to get to files in invisible directories, do this. When you have the "Open..." dialog window in view in TextMate, hold down Command-Shift+G. This will open a sub dialog window called "Go to the folder:" with a text field to type the directory structure into. Simply enter /etc and hit the "Go" button. This will take you right to the /etc directory and in there you should see the bashrc file. Open it add the lines from the second code sample box above and save it. From this point onward your terminal window in iTerm should look something like this.

I never setup a server without putting these customization settings in place. It just makes me feel more at home on the command line. In some later articles, I'll start to cover some really sweet shell scripts for more workflow automation. Let me know if you like this or if you have any problems setting them up on your system.

Learn To Program In Ruby And Basic Sql_redirect

Sun, 05 Mar 2006 00:00:00 -0500

Learn To Program in Ruby and Basic SQL

Sun, 05 Mar 2006 00:00:00 -0500

I've been learning to or "trying to learn" Ruby on Rails for a few months now and things have always kept me from finishing the book that I purchased from those great publishers at the pragmatic bookshelf. My problem has been that sometimes other work has gotten in the way, but mostly it was because I did not have the core understanding of the basics for building web applications. Especially in the areas of object oriented programming and database languages. For me this was a big problem, I'm typically a fundamentalist when it comes to learning and applying knowledge. Knowing the details helps me understand the big picture and more importantly the confidence to know what I am doing is correct. So rather than learning super high level code, I decided to revisit the 3-foot section of the pool again by reading these two books.

Teach Yourself SQL in 10 Minutes by Ben Forta

Just so you know, you are not going to learn SQL in 10 minutes! This book is chopped up into "10 minute" exercises that take you into the language step by step. If you are new to databases, this book is great because it teaches you the core SQL syntax that most database management systems use to extract and input data into the database itself. It even covers idioms that are unique for each database language including SQL Server, PostgreSQL, MySQL, and Oracle.

By the time I picked up Teach Yourself SQL in 10 Minutes, I had already learned to tread water in a huge Microsoft SQL Server database that I work in every day. So hacking stored procedures and building some basic queries were in my capabilities, but after reading Ben's book, I was able to do a lot more. It really pays off to know the basics and I'll let you in on a little secret. If you really want to work with databases and plan on sticking with it, I suggest using SQL 4X Manager J 3 by InterServices New Media. It will allow you to be database agnostic and its features are so rich that it will continue to be a valuable tool long after you have mastered SQL. This application is written in pure JAVA and it uses ODBC connections to support every database I have come across. I simple love its console support, the ability to edit and run stored procedures, create views, and generate entity relationship models. Trust me, it's worth the price!

Learn to Program by Chris Pine

If you are new to programming, this is a great book to pick up first. It is a modest 150 pages long and baby steps you into the concepts shared by most languages, using my favorite language, Ruby. That's right, I buy into that hype around this language because to me Ruby is like Apple Computer, its different! Here is an quote from its creator, Yukihiro Matsumoto, in the de facto Ruby resource guide, the Pragmatic Programmers' Guide to Programming Ruby.

I believe that the purpose of life is, at least in part, to be happy. Based on this belief, Ruby is designed to make programming not only easy but also fun. It allows you to concentrate on the creative side of programming, with less stress.

So if you believe that JAVA is better thank Ruby or maybe PHP or any other language, please do not start to flame me. I think each language has its place. You can have yours and I'll have mine. I for one have always valued the underdog more. That said, Learn to Program by Chris Pine is the place to start.

Like most other programming books, this one is rich in examples and test scripts. The thing that I thought most unique about it was that it forced you to utilize the concepts you learned by asking you to write a program that did X, Y, and Z. It did this without giving you the answers which really forces you out of the academia of learning into real practice. So if you pick up this book, or if decide to use the free tutorials on Chris' website, check back here often and I'll keep posting updates on what code I came up with. Perhaps others can share how the faced the challenges since I could not find once single resource on the web that did this. Here are a few to start this off.

A Few Things To Try (bottles of beer program)

bottles = 99

while bottles.to_i != 0
  puts bottles.to_s + ' bottles of beer on the wall. ' +
    bottles.to_s + ' bottles of beer!'
  bottles = bottles.to_i-1
  puts 'You take one down and pass it around... you have ' +
    bottles.to_s + ' bottles of beer on the wall.'
  puts ''
end

A Few Things To Try (roman numeral method)

def romanize integer
  mdiv = integer/1000
  m = 'M'*mdiv
  d = 'D'*(integer%1000/500)
  c = 'C'*(integer%500/100)
  l = 'L'*(integer%100/50)
  x = 'X'*(integer%50/10)
  v = 'V'*(integer%10/5)
  i = 'I'*(integer%5/1)
  m+d+c+l+x+v+i
end

puts romanize(420)
puts romanize(9)
puts romanize(1)
puts romanize(278)
puts romanize(2156)

A Few Things To Try (how old are you, spank)

puts 'What year were you born in? (number please)'
year = gets.chomp.to_i
puts 'What month were you born on? (number please)'
month = gets.chomp.to_i
puts 'What day were you born on? (number please)'
day = gets.chomp.to_i

bday        = Time.mktime(year.to_i, month.to_i, day.to_i)
time        = Time.new
diftime     = time - bday
diftoyears  = (diftime/60/60/24/365).to_i/1

puts
diftoyears.times do
  puts 'SPANK!'
end

If Microsoft Designed The Ipod Packaging_redirect

Thu, 02 Mar 2006 00:00:00 -0500

If Microsoft Designed The iPod Packaging

Thu, 02 Mar 2006 00:00:00 -0500

iPods are part of our digital lifestyles and I have owned one ever since they first came out! Actually, I had mine one day before they came out since I was able to convince my local Apple store to release my prepaid item before the market date. Since then, I never looked back to the CD as a precious piece of material associated with music. Right after getting home with my new iPod, I promptly encoded all 300+ of my CDs onto my Quicksilver G4, looted Amazon for images, and tagged every album with image art. After that, all my CDs turned into gifts for my friends and as I have upgraded from computer to computer, that 20 gigabytes of music just follows me around every where I go.

Since I used to do package design, I found this little parity of Microsoft's attempt to correctly design the iPod packaging hilarious. Every time I watch it I think of all those years I've laughed at large agencies and their inability to think as cleanly as Apple has prooved both technology and the design of it's presentation can work. Enjoy, and watch this video please note, that I am not the author.

Shell Script To Delete All Invisible _ Resource Files_redirect

Fri, 24 Feb 2006 00:00:00 -0500

Shell Script To Delete All Invisible ._ Resource Files

Fri, 24 Feb 2006 00:00:00 -0500

If you have ever accessed your website using a network protocol such as the Apple Filing Protocol (AFP), Samba (SMB), Web-based Distributed Authoring and Versioning (WebDAV), or Network File System (NFS) using your Mac â€“ I am sure you have run into this problem before â€“ dreaded invisible resource fork files. These are the files that begin with a ._ and they are normally not seen from within the finder. My understanding is that these files are not even created on your local Mac hard drive since the HFS+ file system is smart enough to keep your data and resource forks together in one single file. But even if you are accessing your website from one Mac HFS+ volume to another Mac HFS+ volume, these files will still be created by programs like DreamWeaver and TextMate because the various protocols to access that remote share and/or the file systems themselves will need to split them to cope. So ultimately when ever you use these other types of file systems or network protocols, you will eventually be creating lost of invisible resource fork files.

Hey! ._ Files Are Not Really Bad

No they are not. By nature your Mac should deal with these just fine be it on you local drive or on a network volume. That said, I think it is important to point out what these invisible resource fork and other invisible files actually do. The most common one on any Mac is the .DS_Store file. This stores preferences on window settings. If you open a window, and choose icon view and then move them around, the .DS_Store file stores remembers all those settings. When you open that folder again, you see it the way it was left, thanks to that .DS_Store file. I have never had a problem with these files on a remote volume. They can live :)

The files that are prefixed with ._ are resource fork files that have a one to one match with another file that you can see. For example, let's say you have a file called index.php on your web server and you have made some changes to it with your editor such as TextMate. Once you save that file, TextMate will create another file called ._index.php on your web server. Why? Well Textmate, just like the Finder, likes to save your preferences. The most common is what line number your cursor was on. Great feature, I hate scrolling back to that line I just edited. However convenient this is, I have had ._ files cause technical and visual problems on my web server. Most of the time these problems happen when a PHP application like Drupal or Expression Engine needs to scan directories for files to include or enable in its application. The reason these files can be detrimental to your web application can be numerous and if you are reading this article, you probably don't care, so let's just get rid of them.

The Killer Resource Script

This shell script consists of three commands. The first is the UNIX find command. It is told to look for all files in a directory that have a name which starts with ._ and then print them to your window. The second part of the command is called xargs and basically this takes each line form the first command and applies the command immediately following it. In this case, remove, as indicated by the rm at the end. The -t after xargs is optional, it is useful if you are running this from your terminal since it echos the command to you before it is executed. The final shell script looks like this.

#! /bin/bash
find /Volumes/Your_NetworkVolume -name '._*' -print | xargs -t rm

To customize this script you will need to change "Your_NetworkVolume" where it appears above to the name of your web server share as it appears on your desktop.

I find it useful to put this script in my AppleScript menu. If your apple script menu is not on menu bar at the top of your screen, you can turn it on using the "AppleScript Utility" program located in your (/Applications/AppleScript) directory. Just check the box labeled "Show Script Menu in menu bar" and unless you want a really long list of extra pre installed example scripts by Apple, I suggest keeping "Show Library scripts" unchecked.

Then using TextMate, create a file named something like this "DELETE ._ Files-on-MySvr.sh" and save it in your home (~/Library/Scripts) folder. While the file is still open in TextMate we need to make it executable by going to the menu (Automation Â» Run Command Â» Unix Shell Â» Make Script Executable). Once that is done you can close the script and you should see it appear in your scripts menu like this.

This way, you can get in the habit of running this command when you are done working on the web server. Of course, you could setup this script on the web server itself and then execute it via the crontab every so often. It's really up to you.

One last thing, in some cases you may have to delete invisible files on your web server that have spaces or special characters that might cause the script above to fail. Now why you would have files on a web server that are non web friendsly is completely against my understanding, but if that is the case then you will have to use this modified version. It includes what I call a shielded version of the -print and xargs commands so they can understand how to escape or deal with the spaces and special characters.

#! /bin/bash
find /Volumes/Your_NetworkVolume -name '._*' -print0 | xargs -t0 rm

How To Control Browser Caching With Apache 2_redirect

Sun, 19 Feb 2006 00:00:00 -0500

How To Control Browser Caching with Apache 2

Sun, 19 Feb 2006 00:00:00 -0500

If you are like me, an up and coming network geek, you probably thought that browsers automatically cached a site's assets and media as you went through it and looked at each page. I was very surprised to find out that this was not the case. In fact, every time I went from page to page on my newly created MetaSkills.net blog all of the CSS, PNG, JavaScript and other media files that were common to those pages were being downloaded at each and every request. WOW! This was bad news for me and, if left untouched, it would have easily caused all of my bandwidth to be taken up when 50+ users came and started clicking around.

This tutorial documents how I learned about browser caching and how you can force a browser to maintain representations of your site's assets using Apache 2 and its configuration directives. These directives will allow you to have either general or fine grained control over how a browser caches each file/type on your site. If you want to skip ahead to the good part and just apply them, feel free to do so.

Useful Tools For This Tutorial

Before we go any further, let's cover two really useful tools that will help you figure out what is going on with your website and to see if your web server is controlling browser caching as it should.

About The tail Command

Tail is a program on Unix-like systems that is used to display the last few lines of a text file, most commonly used on log files. The command syntax is real simple and for the purposes of this tutorial you will only need to concern yourself with the specific command in the code block shown below; the path can be either an absolute string (from the server root) or relative to the directory you are working in. This command will let you see live additions as they are being written to your Apache 2 log file. To exit from this program, simply press Control-C on your keyboard.

$ tail -f /the/path/to/your/logfile.log

If you have ever used the OS X Server Admin tool, you may have noticed that each service listed in the left hand side has a corresponding "log" tab on the right that allows you to see that latest log entries for that particular service. The log viewing feature even has a nifty search box. What is really happening here is that the Server Admin GUI is using the same Unix tail command mentioned above to show you the last several dozen lines in that file. I normaly like using the GUI tools that the Mac has to offer, but I am not using the default Apache 1.3 that comes with OS X, rather the evaluation version of Apache 2 located in the /opt/apache2 directory. This means the GUI is not going to allow me to see the running log file of Apache2. Here is where using the underlying Unix tools on OS X comes in handy.

The tail command can be found on any OS X Server or Client installation; it is there by default. You can use this command from the server itself or you can SSH into the server with your laptop or other personal Mac and execute it from Terminal. Remember, this command is not useful to you if you do not have direct or SSH access to your web server where your log file is being written. If this is the case you will have to download it via your normal means and examine it in your favorite text editor.

About HTTP Headers & The Firefox Extension

HTTP headers are transparent to the end user and most people don't even care that they exist at all. Despite this, they are included at the beginning of every response that comes from your server and they contain important information about that response. The most common information includes the date, type of server sending the HTTP response, and encoding of the HTTP response. If you have the gumption, go read the 40+ pages on the field definitions defined by the RFC 2616 for HTTP headers. Otherwise, let's just focus on caching and seeing those HTTP headers which are normally invisible.

This is where Firefox comes in handy. Many times I will use Firefox during the development of a website simply because it has many extensions that have been specifically written to ease the technical aspects of making a website. If you don't have Firefox already, you can download Firefox here. Once you have that installed and running, you can get the Live HTTP Headers extension from the developers site at mozdev.org.

Using the Live HTTP Headers extension is really easy. All you have to do is open the window by going to the "Tools" menu in Firefox and clicking on "Live HTTP headers", this will open a blank window. When you use Firefox to go to a page or file, the HTTP header(s) will appear in that new window. Below is an example:

If you are more comfortable using the Unix command line, there is one more useful tool that I would recommend called furl. It does the same thing as the Live HTTP Headers Firefox extension but only from the command line interface (CLI). It does not come on Mac OS X Server or Client by default but can be installed using the Darwin Ports system. I highly suggest using this vs. the Firefox extension since it is a leaner utility, but I'll leave it up to you whether you are more comfortable with a GUI or the command line.

Does My Site Control Browser Caching ?

Examining Your Apache Log Files (using tail) - Let's start with the easy way of finding out. Simply open your Terminal program in the utilities folder and tail the log file for your host by typing the command described above. While leaving the terminal window open, navigate to your site with your browser. You should see additions to your log file appear live in your terminal window and it would look something like this picture below.

As you browse around while looking at this window, you should NOT see yourself logging requests for CSS, Images, GIFFs, or JPEGs over and over again. If you are the only person on your website at that time, this should be fairly apparent.

But, if you are not the only person on your site, it may get kind of noisy looking at all the log file entries whizzing by in the terminal window. In this case, you may find it useful to "pipe" an extra command to the tail program so that all you see is your requests and not the requests of others. To do this, simply make a note of your IP address and replace the 10.0.1.4 IP address in the command below with your own.

$ tail -f /the/path/to/your/logfile.log | grep 10.0.1.4

This combined command will now use the GREP command in conjunction with the tail program to search and display log entries with the matching pattern, in this case lines that contain your IP address. Remember to use Control-C to exit from the tail program when needed.

Examining Your HTTP Headers - The other way of finding out if your site is taking advantage of browser caching is to examine the HTTP headers on any given server response, be it a whole page or a single file. You can do this using the Live HTTP Headers Firefox extension mentioned above or by using the Unix furl program if it is installed on your system. A common HTTP header response will look something like this.

HTTP/1.1 200 OK
Date: Sun, 19 Feb 2006 16:42:05 GMT
Server: Apache/2.0.55 (Unix) DAV/2 PHP/5.1.1
Last-Modified: Sun, 08 Jan 2006 16:17:25 GMT
ETag: "73049-defc-37c52f40"
Accept-Ranges: bytes
Content-Length: 57084
Cache-Control: max-age=7200
Expires: Sun, 19 Feb 2006 18:42:05 GMT
Connection: close
Content-Type: image/png

What we are looking for in the header are the "Cache-Control" and "Expires" fields. These fields control how long the browser will cache this media or page asset from your server. Having a low value like "1" in "Cache-Control" can be just as bad as no value at all.

If you have those fields present, you've got a head start on the situation. If not, you may need to make sure Apache 2 is loading the mod_expires and mod_headers modules. Most installations and builds of Apache 2 include these modules since they are pretty essential. If you are using the Apache 2 in OS X Server's opt directory, this is the case and you only need to make sure that they are turned on by opening the http.conf file and making sure the following two lines do not have an # sign in front of them.

LoadModule expires_module modules/mod_expires.so
LoadModule headers_module modules/mod_headers.so

When these modules are loaded and working, we can then start to use the correct Apache 2 directives to control how browsers cache everything from pages to images across the whole site or in a specific directory.

Using Apache 2 To Control Browser Caching

Now that we are here, I can assume that you have both the mod_expires and mod_headers modules compiled and loaded into your Apache 2 installation. I can also assume that you have examined a few headers from your server's responses and determined that the "Cache-Control" and "Expires" fields are either not set or they are configured at such a low value to be ineffective. Lastly, you may have bypassed some header examinations and just determined that browser caching is not happening by tailing your Apache 2 log file and monitoring redundant requests. Either way, let's get to fixing Apache 2 to control browser caching that is right for you and your particular site.

Here is a code snippet of an Apache 2 directive that we will be using. This directive can be modified to suite your tastes or it can just be used "as is" for most users. The directives here can be placed into the <Directory> directive of your virtual host in http.conf or it can be placed loosely in a .htaccess file in the root of your website.

<IfModule mod_expires.c>
  ExpiresActive On
  ExpiresDefault "access plus 1 seconds"
  ExpiresByType text/html "access plus 1 seconds"
  ExpiresByType image/gif "access plus 120 minutes"
  ExpiresByType image/jpeg "access plus 120 minutes"
  ExpiresByType image/png "access plus 120 minutes"
  ExpiresByType text/css "access plus 60 minutes"
  ExpiresByType text/javascript "access plus 60 minutes"
  ExpiresByType application/x-javascript "access plus 60 minutes"
  ExpiresByType text/xml "access plus 60 minutes"
</IfModule>

Let's examine what is happening here in brief. If you are interested full documentation of the expires directive can be found on Apache's website along with different syntax formats than the ones used here. I like this format since it is inheritably legible. This directive will do the following:

Set the default expiration of content in the browser cache to 1 second past the time of accessing that content. This is good for setting a catchall or default if you fail to explicitly define a content type in the following directives.
Set the expiration of text/html pages to 1 second. My content management system Drupal does this already in its .htaccess file, but I include it here if you wish to change it. I think this is a good setting since technically most html pages are small and I like to err on the side of caution and always want my page content to be fresh. For instance, I may make changes to my global template and want it to be visible immediately.
Set the expiration of standard images like GIFF, JPEG, and PNG to 2 hours.
Set the expiration of CSS and JavaScript to 1 hour.
Set the expiration of XML files such as RSS feeds to 1 hour.

References

If you are interested in knowing ALL about caching including proxy caching and CDNs which I covered in my last post, please read Mark Nottingham's article "Caching Tutorial for Web Authors and Webmasters". Also included below are links to Apache's website manuals for each of the modules discussed here.

A Review Of Digg Com Traffic And Cdns_redirect

Thu, 16 Feb 2006 00:00:00 -0500

A Review of Digg.com Traffic and CDNs

Thu, 16 Feb 2006 00:00:00 -0500

Thanks to all those that read my recent post about networking 3 Mac Mini's Hopefully it can help you create a network that is as close as possible to the administrator's second home, the NOC. As an aside form the posts I had planned, I did want to share some of the statistics that the digg.com exposure generated and some "simple" helpful tips to those interested in surviving high traffic/bandwidth peaks. I'll cover more "technical" ways of setting up your server(s) to handle this too, but that is for my next post.

Bandwidth Chart (7 Days)

The story made it to the front page of digg about one week ago on Monday, February 6th. It was posted to Digg about 36 hours earlier on Saturday evening and within 1 hour of making it to the home page, using approximately 60 diggs to get it there, the bandwidth capacity here at ActionMoniker.com really took a dive. Below is a 7 day chart that goes from Sunday, February 5th to Saturday, February 11th.

The first big spike in the chart was the initial onslaught of traffic on Monday afternoon from the digg.com homepage. That small gap you see there before the next big spike is where the WebSvr Mini had to be rebooted. In fact, I even upgraded the RAM to one gigabyte. You can see another spike towards the end that correlates to the secondary traffic generated from the blogosphere, basically when other bloggers and news portals got around to posting the information about my article and then linking back to my site. Neat statistics, but let's take a look at why that WebSvr Mini crapped out.

We All Need Cache

Apache seemed to do well with the quantity of requests coming at it from Digg. That said, my post did have one HUGE draw back, the 10 large images on the article's page that totaled 124 Kilobytes in size. My assumption is that when the server started taking requests in mass my 2.6 Megabit connection was not fast enough to serve those images in a timely manner. This caused a cascading effect on Apache by keeping memory intensive threads open for longer and longer until the server could not take it any more.

This is where a good cache comes in handy. Not browser or proxy caches, those are for my next post too, but rather what are called gateway caches which are more commonly called Content Delivery Networks (CDNs). These are distributed caching servers throughout the internet that typically help deliver content to a client by geotargeting a visitors IP address and then finding a server close to that IP with the requested content. The best example of commercial CDN is Akamai, I think Apple bought or invested in these guys about 3 years ago, but I digress. The point is that CDNs can help distribute the load off of your local server by copying your content be it a whole web page or just certain media types and serving them up to your visitor's browser.

The most popular CDN that is freely used by many digg.com readers is a company called Coral. Many times a Digg or Slashdot article will take down an unsuspecting server (know as the slashdot effect) forcing prospective visitors to view your content from a cached version on a sever very far away from your own. Often readers on both Slashdot and Digg try to "coralize" your article by linking to it from a comment thread using the Coral network as a cache. This is easily done by adding .nyud.net:8090 to the end of any hostname in a URL. For instance http://www.metaskills.net/article.php would become http://www.metaskills.net.nyud.net:8090/article.php but this is really only helpful to prospective readers if the coralized link was used as the initial link on Digg or Slashdot.

In my case it was not and I did not like the idea of users having to read my article on somebody else's server nor relying on them finding a coralized link to my site buried in a comment thread. If that had happened, I would have lost traffic statistics, user comments, RSS subscriptions, lateral page visitation, and many visitors altogether. I had to get the server running again and cope with my limited bandwidth. Wow, I just said 2.6 megabits was limited :)

Just Cache Images or Large Media

When I rebooted the WebSvr Mini, I immediately changed the the <img src=""> tags of the 10 images in the content area that had the largest file sizes to appear from the Coral network. This was done by changing the image source tags from <img src="/images/mac-mini-network.jpg"> to a fully qualified domain source using the coral server which ended up looking something like this <img src="http://www.metaskills.net.nyud.net:8090/images/mac-mini-network.jpg"> thus ensuring any future visits to my article received those images from 1 of any of the the 260 Coral network servers. In this way the largest aspect of my page was completely offloaded leaving my local Apache to handle the page source and other media request in a timely fashion. Best of all, this was completely transparent to the end user and no features were lost in their browsing experience.

I highly suggest that if you know ahead of time that an article of yours will be Slashdoted or Dugg, that you coralize as many of the images or large media files on your web server as possible. Your options on what that media content type is may vary, but choosing the ones with the largest file size first is usually the best idea. It can save your site from going down or to a creeping crawl.

Do You Digg It ?

Well I do and to date we have had 34,000 reads to the Mac Mini Network article. With a success like that it was time to sport some gear. I had already changed my default home page from my customized Google with Slashdot articles to digg.com and now all I needed was a hat to tell the world. For the past 5 months I have been sporting a /. hat and matching retro LED digital watch from ThinkGeek.com and it was time for a change. I thought for sure that there would be a vendor out there selling "I got Dugg" or "Digg It" or even a logo embroidered hat, but I could not find one at all! I would have to say that this company has sure missed out on some sweet merchandising opportunities. So I will have to stay with my /. hat.

Above is a comp I put together at CafePress.com. I even mailed it to Digg.com and asked what they thought. They kindly asked that I refrain from using their mark on resale items until their lawyers finished helping them with their copyrights and licensing agreements. Maybe I will go back to /.

Mini Network With A Big Xserve Style_redirect

Sun, 05 Feb 2006 00:00:00 -0500

Mini Network with a Big XServe Style

Sun, 05 Feb 2006 00:00:00 -0500

Or, how to be the biggest network geek you can in only 1 square foot of space. Starting last year after reading Ward Mundy's How-To Bonanza, I knew I needed to use the Mac Mini as the core hardware component for my home network. Mostly because it looks good and small form fits easily on my desk. Good as that might be, the Mac Minis do have drawbacks and, if you're serious about your hosting choices, these do have to be overcome; this is what my post is all about.

So like most people that create networks I did not realize that the Mac Mini includes 3 high speed network interfaces and that with a little bit of work and the right architecture they can be used to operate in much the same manner one would see in a high-end network operations centers. I manage one such NOC and I wanted my home network to function like most companies who do serious business online. These features typically include:

Internal LAN Network Interfaces - (10.0.1.0/24 Addresses)
Internal Perimeter Network (DMZ) Behind a NAT or Port Forwarding Firewall
Both Internal (LAN) and External (Public) DNS Services
IP Over Firewire Connections - (192.168.1.0/25 Addresses)
Hard Drive Redundancy and Backup Using RsyncX - (typically RAID 1 or 5)

Starting At The End

Below is a simple diagram of my final network. I had to lose a few features from the list above or even change their meaning a bit, but I did end up with a network that utilizes most of them.

Public Interfaces

In the diagram above, the (green) represents public 100baseT ethernet interfaces while (orange) and (red) represent an internal network. Due to many questions regarding this layout, I want to stress that the WebSvr and generic Mini (mail and DNS) are publicly available on the 100baseT connections through my Linksys switch. The AirPort is for local LAN only, see below for details.

Internal Local Area Network ( LAN )

First thing on the list that is the most important feature. A local area network that allows connections to each server without going outside to the public internet and thus taking up valuable bandwidth to those public interfaces. It's also good for security! The AirPort Extreme I had did this quite well by setting it to share a network connection using DHCP and NAT on the 10.0.1.0/24 subnet. The AirPort internet connection is setup to DHCP to my ISP from its WAN port. Each Mac Mini's AirPort network interface is manually set on its own IP address that never changes. This will need to be done so you can assign it an internal DNS name and reverse PTR record.

Internal DNS Services ( WebSvr Mini )

Now that we have an internal LAN we can use this to host our own DNS so that machines on the local network including my powerbooks and other systems will use the AirPort interfaces for services. For instance, if I browse to my website from home, I am not using the public interface and bandwidth, but rather the IP with the AirPort address. The WebSvr Mini was my best choice for hosting DNS and I have BIND 9 configured to only listen for queries on the local 127.0.0.1 IP and its own 10.0.1.x AirPort IP address. BIND is also configured to forward all unknown queries to my ISP. It's very important to set the basestation's internet tab be set to use the WebSvr Mini's 10.0.1.x AirPort IP address for DNS, rather than your ISP. In this way, all clients that are not manually configured to do so will use the WebSvr mini for DNS and hence the AirPort interfaces on each server for services. I hope I have not lost you yet. When using internal DNS the right way, all local traffic such as Apache, Apple Remote Desktop, MySQL, SSH, etc. are all done locally and fast. So do not forget to tell Apache it can listen on that port.

IP Over Firewire Database Connection ( DbSvr Mini )

The DdSvr Mini is using Apple's IP over firewire capabilities to connect to the WebSvr Mini. I have a patch cable connected to the WebSvr Mini with static IP addresses on both ends using the 192.168.1.0/24 subnet. For convenience each of the 192. addresses ends with the same number as their respective 10. AirPort addresses. This is the 3rd network interface that most people are not aware of. It has limitations but used wisely it can be very useful and fast. In the set up above I am able to keep all database traffic entirely off of my internal switch and localized to that one patch cable. Unless of course, I decide to access the DbSvr Mini using its LAN/AirPort address. Either way, this machine is totally off the public network and quite secure.

Other Services ( Mail and External DNS )

That's what the other unlabeled Mini is for. I have it configured to for mail and public DNS. Postfix is happily listening on its public and private addresses while DNS is only listening on the public IP. This server also does not have an external drive (see below) since I am not worried about its performance since the overhead is quite low for those two services.

Firewall Services

In this setup I did loose the ability to have private addresses for my two public Minis. Hard core network engineers would point out that this would be another layer of security by having a single firewall/router filter and translate packets to these two boxes. I agree and it makes for a nice portable network too. But, I am more than happy running the ipfw service built into OS X Server that is running on each Mac Mini. It is a great stateful firewall and it's not really a configuration nightmare to maintain each one. All in all, it's what I call the RONCO network, set it and forget it. Now on to fixing some of the deficiencies for the Mac Mini.

Improving Hard Drive Performance

One of the first drawbacks to the mac mini is its poor hard drive speed. This can be a real big hit on performance when it comes to hosting a descent web or database server. Thankfully there are alternatives including upgrading the internal drive with a faster form factory laptop drive. However, these can be expensive and if you are like me and have a few Barracuda 7200 RPM 3.5 inch external drives laying around, you could save a lot by purchasing an inexpensive firewire enclosure. I decided on the miniStack from newertechnology since it had the best cooling features and good Mac Mini style. More importantly it sells with a driveless option and comes with firewire cables. A deal for only $69.00 for the entire kit.

I purchased (2) of these, one for each Mini who's performance I was really concerned about, one for the web server and one for the database server. Each of those Macs boot directly from that external drive connected via firewire, the internal drive is used as a back up. Currently I have the DbSvr Mini setup as an RsyncX server and every night each of the 3 servers backs up to its internal hard drive. That's a separate post in itself.

Improving AirPort Signal Strength

I was really surprised to find out how poor the Mac Mini's AirPort signal strength was after you stack something on top of it. My extreme basestation was only a few feet away and the interference from the drive enclosures was so bad that I could not connect two of them that were in the middle of my stack. Thankfully, I was no stranger to taking apart my Mac Mini. In fact, I had installed AirPort cards on all of them since it was cheaper for me to do so and I had an idea! So be warned, the following steps will void your warranty - but if you want to get good AirPort strength and this was a must for me, here is some helpful instructions.

First though, the unordered list below is a parts list with links to Mac-Pro.com. If you are wanting to install an AirPort Extreme card in a Mac Mini, these are the parts you will need. Secondly, my AirPort signal strength "fix" below is by no means a comprehensive step by step for a first timer into the Mac Mini. I highly recommend PBFixIt.com's Mac Mini Take Apart Guide for that.

OK, now to the fun part, Improving your AirPort signal strength. You will need to get a good clean work area setup and take apart your Mac Mini down to the mezzanine board where your AirPort card is installed. Do not actually take that out, we are just going to reroute the cable. Just unplug the AirPort extreme antenna.

The most logical place to run the AirPort antenna externally is through the security hole on the back right side. The only other choice would have us drilling holes in the vents and the hood that funnels air out the back of the Mac Mini. Least invasive is good, so I took a dremel and widened that hole up. I made a circular opening the same diameter as the existing width of the security hole.

Now that the hole is wide enough, route the AirPort antenna back through this new opening. Make sure not to cut the antenna cord on the metal shielding.

After you connect the AirPort antenna back to the AirPort extreme card, make sure that you leave the cable lying next to the slot the drive and fan assembly goes back into.

Here is a picture of the how the antenna cable routes through the small opening that is designed to the left drive and fan assembly slot. This is where Apple intended the AirPort antenna to go. We are just taking a detour.

Now we are ready to put the cover back on the Mini, but since we have the AirPort antenna going out the security slot, we will need to cut the metal anchor off. I sliced it right at the bottom, marked in the larger photo.

I have found it extremely useful to take your putty knife and push on the shielding on the back side of the Mini when you are snapping the cover back on. Otherwise you will just bend the clips when you are pushing it together. And just in case your asking, yes, a putty knife is the standard tool for taking apart a Mac Mini.

Here is a final picture of the back of the Minis after they are all back together. Here you can see the AirPort antennas sitting comfortably out the back. No signal interference now!

This was the reason I started my network project. I have 3 servers right on my home office desk with all the comforts of a full fledged NOC in less than 1 square foot of space.

That's it! If you have any questions, please let me know. Or perhaps, maybe you have some ideas on what I could work on for version two of the network?

Cooking With Drupal_redirect

Sat, 28 Jan 2006 00:00:00 -0500

Cooking with Drupal

Sat, 28 Jan 2006 00:00:00 -0500

One of my few New Year's resolutions was to blog often in the year 2006, in fact, I promised myself I was going to write at least one post every day. Well it's been two weeks since my last update and the world has missed some pretty good brain dumps from yours truly (sorry about that) but I have been keeping good notes and I do have great news about today!

Over the past few weeks I have been cooking up a storm in my digital kitchen with one of the best content management systems around, Drupal. This is the same CMS that runs the metaskills.net site and it has a good history for not just being a Blog or Forum software but an entire community platform, thus Drupal's tag line, Community Plumbing.

How Can You Say Drupal Is The Best CMS ?

OK, I would like to take back my exclamation about Drupal being the best CMS. A lot of people make absolute statements like this and they are always never wholly true. Choosing the "best" CMS is a relative concept and only possible when one starts backwards and defines their own end requirements and vision purpose of the system needed. For instance, if all you wanted was a personal blog, I would highly recommend WordPress as the right CMS for you. In that same vein, if you are new to the internet and tend to stay in the 3-foot section, a hosted non open source blogging system like TypePad, LiveJournal, or even Google's Blogger.com would be a good choice. These are great for beginners that have no interest in the back end setup work or maintenance typically associated with a professional CMS.

On the other hand, if you do real business on the internet, a SaaS might not do. That's OK because your still in for a good swim as there are numerous choices to be made. If you are a small publishing company like iPodLounge, you could use pMachine's Expression Engine. Its tag-based and well documented system is very easy to use, customize, and control permissions to content and features. If you are a software developer and need workflow, project management, CRM, bug and issue tracking Atlassian's JIRA or Confluence could be right for you. Moving on into the open source software realm, options include Zope or Plone for the Python programmers to Mambo or even Drupal for the PHP enthusiasts. Many of the features in all of these system blur together and it really does matter on many personal and business factors to agree on the "best" CMS for your needs.

That Smells Good, What Ya Cooking ?

Ah yes, that! Well now that I have avoided a flame war by sidestepping the "best" CMS argument let's get back into my favorite CMS at the moment, Drupal. When I first launched metaskills.net I used the current 4.6 version. Since then the Drupal team has been hard at work on version 4.7 (currently beta4) and to help give back to this great open source software movement, I decided to learn how Drupal works on the back end and contribute my blog theme. Doing so meant I had to actually learn a little PHP and JavaScript and the correct Drupal API and publishing hooks, both old and new. I have to say that I am very impressed with the Drupal system. Its power and extensibility is quite unique and its community demonstrates the best practices of what open source software is all about.

Here is the new Meta Theme for Drupal, this is a demonstration site that will serve as a working reference and online documentation for the theme's project on drupal.org. Here is an abstract list of some of the new features.

Total CSS over write of all Drupal's 4.7 features
The world's best style sheet switcher with DOM2 support and cookies
Custom image gallery theme with dynamic lightbox.js support
PNG support for IE 6 and lower
Multiple code syntax highlighting options
Custom and dynamic maintenance page
Dynamic comment depth integrated with CSS rules

Please feel free to browse this site and comment and ask question. If you see any problems, let me know as I will be actively supporting this them for upcoming Drupal releases.

The Best Font For Mac Programmers Profont_redirect

Wed, 11 Jan 2006 00:00:00 -0500

The Best Font for Mac Programmers, ProFont

Wed, 11 Jan 2006 00:00:00 -0500

After being exposed to using TextMate for all my normal text editing needs, I soon began to do more things than just simple GREP-based search and replace functions. I actually started to look at code more often. Within a few short months, I had gone away from my established norm of the happenstantial viewing of simple CSS, HTML and PlainText files to a ful-time lookey lou of languages such as Ruby, XML, PHP, SQL or any number of configuration files for Apache, PHP, Postfix, Bind, and so on. It's been an exciting time for me over the past 6 months and it feels like open hunting season on any hard to find UNIX application files that are deep in the brush of my Mac OS X. Belive me, treasure are rich once you start digging.

So What's Great About ProFont?

On the left, you can see ProFont compared to the original version of Apple's Monaco font face. This picture was taken from the ProFont distribution readme file. It really shows the differences in how it treats zeros, apostrophes, lowercase L, upper I, colons, and semi colons. This stuff makes it really important if you are looking at code all the time. Profont even shines in the Terminal window too. Size matters too, ProFont works well at 9pt if your eyes are holding up that well, mine are not, so I have to stick at 10pt.

Where Do I Get ProFont?

Right here. I've packaged it for Mac and Windows. If your a Linux or Atari junky and want ProFont, go check out an indepentdant maintainer for the font and download those verions from his website.

TextMate, by Programmers, for Efficiency Experts

Thu, 22 Dec 2005 00:00:00 -0500

About a year ago a fellow co-worker of mine was raving that I should learn VIM from the command line interface (CLI) so that I could use it for all my text editing needs. My co-worker at the time was (and probably still is) the Systems Administrator and Lead Application Developer for a company that sold cheap discount software. During my tenure there I served as the Marketing Manager and as such, many of my projects to completely overhaul their search engine and online marketing strategies called for aligning their internal tools and processes. While doing said technical revamps and specification writing, I guess I was exposed as more of a geek than your typical marketer. Hence the recommendation to go further down the rabbit hole. DISCLAIMER: before I go any further -- I would like to say, yes, that link up there is a shameless plug and is meant only to help in the organic search strategies for PlanetCDROM.com. OK, I feel better.

Down the Rabbit Hole?

So did I do it? Am I using VIM and did I even code this XHTML in such an arcane system? The answer is no, but I will admit that I love the CLI from a UNIX operating system. In fact, I will be writing another installment on how to tweak and customize your shell environment.

The main reason I did not learn VIM is because it goes against my technical MO which is also the main theme of this website -- that working at a higher level is better than keeping your head in the trenches. Sure tried and true pays off sometimes, but technology moves fast and there evolves better ways. VIM stays to far down the path of the harder, at least for me anyway. I typically like to get into the details only enough, so that I can figure out a better way of doing things. VIM would have made a good choice to learn if I was forced to work in a Windows environment where my only escape to the UNIX-style world was shell access to a Fedora Core operating system, but hey, that's not the case. I like to advocate working and hosting on Macs and that's a convention all to itself.

Die VIM, Die! Enter TextMate.

OK, now I'm going to say it and say it only once. Using TextMate from the Mac OS X is pure "Convention over Configuration" one of the founding principles of agile software development. I know many don't like the hype of this methodology, especially when used to discuss Ruby on Rails application development but I think these are the same people that like working harder, not smarter and I'm sure they said the same thing about PHP six years ago and also think JAVA is the only way of programming. So we can ignore them, I choose to. So here are a few of the marketing reasons about why I choose TextMate as my editor over BBEdit on the Mac OS.

Column Typing - I love clean organized code and this image snippet shows that editing clean organized code is really easy in TextMate which keeps you from performing the same change across multiple lines.

Collapsible Foldings - Use these to hide method definitions, class files, tables, DIVs, or any of your markup that you want out of view. I have also found this especially helpful when editing or viewing very large XML files. HINT: You can old down the option button to fully collapse and expand the foldings. Option-collapse to hide everything and then open up the folding like normal and it will easy you in tag by tag.

Syntax Highlighting - Not just for the whole document, but nested markup too. TextMate does this better than any program I've seen and it does it really well for a huge amount of different file types, even shell scripts. This task was a huge chore in BBEdit. HINT: When you download TextMate, don't stop there. Install Subversion on your system and download the trunk files to get all the bundles. This not only gives you more syntax highlighting, but more actions too.

Tabbed Navigation & Project Drawers - This was another chore in BBEdit and by now most applications are figuring out that tabbed navigation is a must for every program. HINT: One of the things I really like about using the project drawer, which comes up automatically if you drag a folder onto TextMate, is that you can create custom environment variables for the project that talk directly to your shell environment. These could be custom paths, etc.

That's just a small excerpt from their their website on why TextMate is so great. There are many more and I have only really started to scratch the surface on using it. Here are a few more that directly relate to how I use it.

Man oh Man.

TextMate links directly with your shell environment. That means you can run scripts and code directly from the editor. It even has an HTML output window. I am constantly reading the man pages for all the UNIX software that comes on OS X. Rather than doing this in the shell, I select the command or application name from within TextMate and it pops up the manual page in a nice searchable HTML window. For instance, rather than using man dig or man rsync from the terminal, just select it in TextMate and hit Control-H.

It's all Hallow's Eve

What can be cooler than a built in theme called "All Hallow's Eve". This theme is the typical black background with high contrast code and markup that you see in many of the screenshots for TextMate. Many VIM'ers loved this setting for legibility and its built right into Textmate.

What's the Difference?

Diffs are great and tell you a lot. I really only discovered them because TextMate offers such a simple way of using them. In this example I can show the differences in the configuration of a 2621 Cisco router. This really helps sort things out.

Lastly, I would love to hear why some other people have moved to using TextMate or perhaps you may have an example of another editor that is the end all for geeks like us?

How To Setup a Simple MySQL Backup Script

Mon, 19 Dec 2005 00:00:00 -0500

Lately, I've been loving all things that can be solved by using RsyncX. It's my de facto backup utility and I just keep finding more and more tasks for it as each day goes by. It has become the hammer for all my system's needs. So when it came time for me to implement a nice little backup routine for the MySQL databases hosted here at ActionMoniker.com, it was the first tool I considered. My requirements were simple, I needed a SQL script of selected databases in logically named folders for each DB with time-stamps in the file name. Then step and repeat on a regular basis.

To use RsyncX for the job, I would have to copy the literal file system for the databases, typically these are located in /usr/local/mysql/data/dbname on your macintosh file system. Doing such would have been a technically valid method, but this solution would not have given me a portable backup file since RsyncX would copy the directory in whole, which includes separate files for each table in the database. Not to mention permission issues for restoring the files.

Perhaps MySQL Administrator?

So instead of square pegging RsyncX I turned to the one of the greatest inventions for the Open Source database community, the MySQL Administrator application. This wonderful program brings many features of administrating MySQL databases into a nice tidy graphical user interface that most users can understand by simply pointing and clicking around. It is very easy to understand and it is akin to the "Enterprise Manager" for Microsoft SQL Server running on Windows. The things you can accomplished with MySQL Administrator are past the scope of this article, but I do want to point out that MySQL Administrator does offer a manual and automatic backup solution for you by using your Mac's "user" crontab. If you are unfamiliar with cron, it is your Mac's (UNIX-Style) built-in job scheduling system.

If you have ever tried using MySQL Administrator's backup routine, DO NOT DO IT! The program fails miserably for most users since it uses your local system account, not the DB user account associated with the database you are tying to backup. If you are reading this article because you have tried using MySQL Administrator's backup system, you may have seen a system message like this:

From: kencollins@ken-pb.local (Cron Daemon)
To: kencollins@ken-pb.local
Date: Mon, 19 Dec 2005 23:04:01 -0500 (EST)
Subject: Cron <kencollins@ken-pb> "/Applications/MySQL Administrator.app/Contents/Resources/mabackup" --directory="/Users/kencollins/Desktop" --connection="localhost" --prefix="MetaDB" MetaDB

Before I get to that simple solution and if you are one of the unlucky that tried backups with MySQL Administrator, let's digress for a moment and clean up the work it did to your system. First, if you have not already done so, install Webmin . Many of my tutorials will rely on it. This is a powerful tool that runs as a bundle of PERL modules accessible via a web browser on your local machine. It allows you to administrate many underlying system events and programs on your system. DISCLAIMER: It can be somewhat hazardous if you are not familiar with what you are doing. That said, the documentation on their website has a very straight forward help file on installing the latest version. Now to the cleanup:

Step 1 - removing the MySQL Administrator cron entry. Open Webmin and navigate to "System -> Scheduled Cron Jobs", directly beside "yourusername" you should see a command similar to the subject line in my example error above. Click on it to edit the job, and hit "delete" at the bottom of the page.
Step 2 - removing any junk emails or system notices. Open Webmin and navigate to "Servers -> Read User Mail", click on your username, then delete any messages similar to the ones above. You should have one message each time MySQL Administrator failed to do what you wanted.

The Simple Solution

OK, now that our system has less junk processes, to the fun part, a simple and working MySQL backup. For this task I created a really simple script. Let's take a look at it below.

#!/bin/bash
timestamp="$(date -u +%Y-%m-%d_%a)"
/usr/local/mysql/bin/mysqldump -uXXXX -pXXXX metadb > /Backups/MetaDB_${timestamp}.sql

There are two parts to this script, the first line sets a variable called "timestamp" in the friendliest format I can think of (YYYY-MM-DD_Day), for example 2005-12-19_Mon. The second line of this script uses the mysqldump utility, gives it a username, password, and your database name. Replace XXXX with the username and password of your database and remember, if you are calling this script from a remote machine, it must have remote access to the database name, directly after your password. The last half of the second line writes your database backup file in a directory of your choosing with a filename using the ${timestamp} variable of the system time from line one. In this example, this will generate a file called "MetaDB_2005-12-19_Mon.sql".

To run this script, save it to a text file like MyDB-Backup.command or .sh and make it executable. My favorite way of doing this is to use TextMate's built-in UNIX Shell automation scripts, but if you do not have such a fancy text editor, simple use the Terminal and enter this command on the file chmod +x yourfilename.sh Lastly, you can schedule this job to run at any interval using the system's crontab. This can be added using Webmin or by opening up the crontab, located at /etc/crontab and then follow the example in the header of the file.