When running Autospec, noise abounds!

/Users/peterfitzgibbons/.rvm/rubies/ruby-1.9.1-p378/bin/ruby /Users/peterfitzgibbons/ruby-projects/wizardry/vendor/gems/rspec-1.3.0/bin/spec --autospec /Users/peterfitzgibbons/ruby-projects/wizardry/spec/gosu_star_catcher_spec.rb -O spec/spec.opts
Profiling enabled.
/Users/peterfitzgibbons/ruby-projects/wizardry
./Users/peterfitzgibbons/ruby-projects/wizardry
./Users/peterfitzgibbons/ruby-projects/wizardry
.

Top 10 slowest examples:
0.2427560 GosuStarCatcher when initialized should have set dimensions
0.1411460 GosuStarCatcher when initialized should have a background image
0.1077080 GosuStarCatcher when drawn should have background

Finished in 0.526593 seconds

3 examples, 0 failures
2010-03-28 21:47:13.642 ruby[72302:903] *** __NSAutoreleaseNoPool(): Object 0x1005200f0 of class NSCFNumber autoreleased with no pool in place - just leaking
2010-03-28 21:47:13.645 ruby[72302:903] *** __NSAutoreleaseNoPool(): Object 0x100567690 of class NSConcreteValue autoreleased with no pool in place - just leaking
2010-03-28 21:47:13.646 ruby[72302:903] *** __NSAutoreleaseNoPool(): Object 0x11236f080 of class NSCFNumber autoreleased with no pool in place - just leaking
2010-03-28 21:47:13.647 ruby[72302:903] *** __NSAutoreleaseNoPool(): Object 0x1005300c0 of class NSConcreteValue autoreleased with no pool in place - just leaking
2010-03-28 21:47:13.649 ruby[72302:903] *** __NSAutoreleaseNoPool(): Object 0x100538b40 of class NSCFDictionary autoreleased with no pool in place - just leaking
2010-03-28 21:47:13.667 ruby[72302:903] *** attempt to pop an unknown autorelease pool (0x100db0a00)
2010-03-28 21:47:13.668 ruby[72302:903] *** __NSAutoreleaseNoPool(): Object 0x101976320 of class NSCFNumber autoreleased with no pool in place - just leaking
2010-03-28 21:47:13.668 ruby[72302:903] *** __NSAutoreleaseNoPool(): Object 0x1019f1d30 of class NSConcreteValue autoreleased with no pool in place - just leaking
2010-03-28 21:47:13.688 ruby[72302:903] *** __NSAutoreleaseNoPool(): Object 0x11236f080 of class NSCFNumber autoreleased with no pool in place - just leaking
2010-03-28 21:47:13.689 ruby[72302:903] *** __NSAutoreleaseNoPool(): Object 0x100589980 of class NSConcreteValue autoreleased with no pool in place - just leaking
2010-03-28 21:47:13.693 ruby[72302:903] *** __NSAutoreleaseNoPool(): Object 0x1019c1710 of class NSCFDictionary autoreleased with no pool in place - just leaking
2010-03-28 21:47:13.706 ruby[72302:903] *** attempt to pop an unknown autorelease pool (0x100db1a00)

Solution?  Run autospec in terminal window-x with a file redirect of stdout :

autospec 2>>autospec.log

If you have issues, tail the log.  Most of the time, your useful info will be written to &1 (stdout) because autospec is trappping all exceptions and wrapping.

Yeay, nice output!

/Users/peterfitzgibbons/.rvm/rubies/ruby-1.9.1-p378/bin/ruby /Users/peterfitzgibbons/ruby-projects/wizardry/vendor/gems/rspec-1.3.0/bin/spec --autospec /Users/peterfitzgibbons/ruby-projects/wizardry/spec/gosu_star_catcher_spec.rb -O spec/spec.opts
Profiling enabled.
/Users/peterfitzgibbons/ruby-projects/wizardry
./Users/peterfitzgibbons/ruby-projects/wizardry
./Users/peterfitzgibbons/ruby-projects/wizardry
.

Top 10 slowest examples:
0.2427560 GosuStarCatcher when initialized should have set dimensions
0.1411460 GosuStarCatcher when initialized should have a background image
0.1077080 GosuStarCatcher when drawn should have background

Finished in 0.526593 seconds

3 examples, 0 failures

Github Tag for Iteration-01

Observations :

• Most Gosu objects are opaque due to the Ruby wrapper of C++ Gosu library
• Heavy use of mocking and expectations will be required to properly Spec the game

Additions to environment:

• mocha gem : additional mocking tools above Rspec’s internal mocker

It’s on GitHub http://github.com/pjfitzgibbons/Gosu-Star-Catcher .

I’m using a basic Ruby library layout.. hand created.

Here are the components involved at the current moment :

rvm 0.1.23 by Wayne E. Seguin (wayneeseguin@gmail.com) [http://rvm.beginrescueend.com/]

ruby 1.9.1p378

*** LOCAL GEMS ***

archive-tar-minitar (0.5.2)
autotest (4.2.9)
autotest-fsevent (0.2.2)
bundler (0.9.11)
columnize (0.3.1)
ffi (0.6.3)
gosu (0.7.18)
linecache19 (0.5.11)
nice-ffi (0.3)
rake (0.8.7)
rsdl (0.1.4)
rspec (1.3.0)
ruby-debug-base19 (0.11.23)
ruby-debug19 (0.11.6)
ruby-sdl-ffi (0.2)
ruby_core_source (0.1.4)
rubygame (2.6.2)
rubygems-update (1.3.6)
sys-uname (0.8.4)

Editor is Textmate

So, I’ve reviewed several libraries.  Currently I’m trying Gosu for Ruby.

I will blog of my experience in following the Gosu Ruby Tutorial

Ruby Gems we used to setup the environment.

• rspec
• ZenTest
• redgreen (for windows users)

heredocs — we talked about how to produce heredocs   You can either use multi-line double-quoted, or <<-eot

text = <<-eot
This is the heredoc text
It can even include evaluations #{Time.now} !!
eot

More about heredocs : http://blog.jayfields.com/2006/12/ruby-multiline-strings-here-doc-or.html

A windows-implementation of textmate : http://www.e-texteditor.com/

The Ruby Way — Hal Fulton

http://www.google.com/url?sa=t&source=web&ct=res&cd=1&ved=0CAwQFjAA&url=http%3A%2F%2Fwww.amazon.com%2FRuby-Way-Second-Techniques-Programming%2Fdp%2F0672328844&ei=OuLwSsj_PIa1tgf59N26Cw&usg=AFQjCNHhpmT3C35Xiianfordqb21_DsWIA&sig2=Pb631QnepV_sAD8wmRuPTA

The Rails Way — Obie Fernandez

http://www.google.com/url?sa=t&source=web&ct=res&cd=3&ved=0CBYQFjAC&url=http%3A%2F%2Fwww.amazon.com%2FRails-Way-Obie-Fernandez%2Fdp%2F0321445619&ei=V-LwSunCEYeVtgfWib26Cw&usg=AFQjCNGpPcvEJDxIOGEwWZZIp9HXnbPC0A&sig2=0ofdwEJfrMOm7U3bR7c5OQ

The Rails Way Blog therailsway.com

Enumerable Module

Rails Boot

Here is a condensation of the Rails boot process from Rails Guts.  As Mike Gunderloy suggests, it will be really helpful to have a blank rails app open with

 rake gems:freeze 

invoked :

1. script/server (vendor/rails/railties/lib/commands/server.rb) requires environment.rb from the local path (./)
2. in environment.rb

    Rails::Initializer.run do |config| 

3. Initializer.run has a default parameter

    configuration = Configuration.new 

   (that’s Rails::Configuration in railties/lib/initializer.rb)

4. Configuration#initialize is long, including

    self.database_configuration_file = default_database_configuration_file 

5. Configuration#default_database_configuration_file is :

   def default_database_configuration_file
    File.join(root_path, 'config', 'database.yml')
    end
   

6. Notice that the default command on Rails::Initializer#run was :process, so Initializer#process is now called, which includes #initialize_database …

   def initialize_database
    if configuration.frameworks.include?(:active_record)
    ActiveRecord::Base.configurations = configuration.database_configuration
    ActiveRecord::Base.establish_connection
    end
    end
   

   1. Configuration#database_configuration is where the database.yml is loaded and returned to ActiveRecord::Base.configurations :

      def database_configuration
       require 'erb'
       YAML::load(ERB.new(IO.read(database_configuration_file)).result)
       end
      

7. Now were getting somewhere!!  AR::Base#establish_connection(nil) :
   1. because the call is with spec = nil, #establish_connection is re-executed with RAILS_ENV
   2. this time through, spec = RAILS_ENV (ie, :development), so #establish_connection is called again, this time with the database.yml hash-key of RAILS_ENV (from Configuration#database_configuration)
   3. 3rd time’s a charm!  Now, with spec set to config hash, require the adapter gem,

       gem "activerecord-#{spec[:adapter]}-adapter" 

      and execute the adapter-method (which is expected to be def’d from the adapter’s gem) :

       adapter_method = "#{spec[:adapter]}_connection" 

The adapter-method is the entry-point of our new adapter, so in my next post I’ll pick-apart the #establish_connection process and prepare for our multi-connection pooling adapter’s version.

Inspiration and Perspiration

Thomas Edison told us how to invent.  So let’s get to it.  The Adapter we’ll build will assist in handling the problem of database-failover in Rails 2.3.2 applications

Inspired – Seamless Database Pool

Seamless Database Pool gem provides us with a template to follow.  The SDP is focused purely on connection logic.  The true db adapter is used within the pool connections.  Our Adapter, like SDP, will simply serve as a proxy to handle the event when an ActiveRecord method (#select, #create, #update, #delete) encounters a DatabaseConnectionError.

Perspired – Activerecord Failover Adapter

Ok, we’re going to have to work for it this time.  Event though we have SDP as a template, there is A LOT of logic in there that is really not useful to us.  SDP is a load-balancing pool based upon a weighted one-write/many-read database cluster / replication group.

Requirements

Ahh, the real meat of our cookout.

1. Configure Master + Slaves(multiple) in database.yml just like SDP
2. Use Master Always.  Well, until failover.
3. Use periodic timeouts to attempt re-connect to master
4. Retry db queries on alternate connection
5. Rollback transactions upon connection failure… then retry them?
   1. Re-try the transaction in the alternate connection
6. Allow #establish_connection to target master/slave (especially needed for targeted migrations)
7. Allow db:migrate to target master/slave through use of RAILS_ENV

In my next post, we’ll start with #1 Configure Master + Slavees in database.yml

Here’s the steps and notes that I have for the move.

System Layout

The system of servers/services comes together like this :

Git / Redmine

Git and Redmine run on server ‘redmine-server’, which is a CentOS 5 Intel box.  OpenSSH is installed, Redmine runs on Ruby 1.8.7, yum package install.
User on this server is ‘git’

Cerberus continuous integration

Cerberus runs on an OSX box named ‘ci-server’, Leopard.
Ruby on this box is installed using ruby-switcher, with 1.8.7p173 and 1.9.1p129r23412.
My customization of that ruby-switcher is here ruby_switcher.sh.
User on this server is ‘cerberus’

SSH setup Git -> Cerberus

The Git repository lives on a different server, so SSH is employed to protect passwords-cleartext over the wire.

There is an additional constraint in that Git hooks are “non-interactive”, so ssh keys must be configured to not challenge.  There are [security implications] to this.  The option I have chosen is passwordless “backup keys” for specific purposes.

git@redmine-server $ ssh-keygen -t rsa -C git_ci_server@redmine_server
Generating public/private rsa key pair.
Enter file in which to save the key (/home/git/.ssh/id_rsa): \
  .ssh/id_rsa_git_ci_server
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in .ssh/id_rsa_git_ci_server.
Your public key has been saved in .ssh/id_rsa_git_ci_server.pub.
The key fingerprint is:
42:62:16:4c:00:74:7c:a9:96:90:5a:75:5d:08:c5:f9 git_ci_server@redmine_server

Move the key to the target server ci-server

[git@redmine-server]$ scp .ssh/id_rsa_git_ci_server.pub \
  cerberus@ci-server:id_rsa_git_ci_server.redmine-server

Password:

id_rsa_git_ci_server.pub                     100%  410     0.4KB/s   00:00

[git@redmine-server]$ ssh deployer@web-ci
Password:

Last login: Mon Jun 15 09:20:00 2009 from redmine-server

[cerberus@ci-server]$ cat id_rsa_git_ci_server.redmine-server
ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAtOVHRlhLmUI/9XglQs6u9F1reUwAq2U0clJI2l
j5kAX5XkVW7FoiiGo4BfAqP3j69ynMBnLx/eUtcYcxHUKZfikwQbXnUdAUd8mIXoNhOfVZ/gqr
i0yS0XHpkGU2lJve3soqEIyYu2gxf5QPP18ibJeyFZ5QRTPUATfi1c6Pb12tsr4O9H+b5T08cU
ejvQP5ISyYH3ysPZwLQ3d1Ods9ZItm7h9/eoZQxgXpVLxUkQ3NO+JoktKlruGajTRgm5xP7NBd
gUZsC2vs84/mW64RWSzbl8fA5oKUhKQIDt7QSOGx2UVGYAoarFGTsvps5g9WnXTJ37XRT+ORHI
cpmO90mQ== git_ci_server@redmine_server
[cerberus@ci-server]$ cat id_rsa_git_ci_server.redmine-server >> .ssh/authorized_keys
[cerberus@ci-server]$ exit

SSH Backup-Key

From ‘man ssh’ :

 -i identity_file
    Selects a file from which the identity (private key) for RSA or DSA authentication is read.  The default is
    ~/.ssh/identity for protocol version 1, and ~/.ssh/id_rsa and ~/.ssh/id_dsa for protocol version 2.  Identity files
    may also be specified on a per-host basis in the configuration file.  It is possible to have multiple -i options
    (and multiple identities specified in configuration files).

This Git server does not have ssh-agent running, so using -i will cause the indicated file to be the only identity used in the ssh session.

Now test the login using the backup-key switch of ’ssh’

[git@redmine-server]$ ssh -i .ssh/id_rsa_git_ci_server cerberus@ci-server
Last login: Mon Jun 15 10:53:38 2009 from redmine-server
[cerberus@ci-server]$

The backup-key switch will be used in the Git post-receive hook for executing Cerberus build upon Git Push

Git Post-Receive

Simple.  Use ssh to call a build script on ci-server, which we’ll build after the break…

#!/bin/sh
#
# An example hook script for the post-receive event
#
# This script is run after receive-pack has accepted a pack and the
# repository has been updated.  It is passed arguments in through stdin
# in the form
#  <oldrev> <newrev> <refname>
# For example:
#  aa453216d1b3e49e7f6f98441fa56946ddcd6a20 68f7abf4e6f922807889f52bc043ecd31b79f814 refs/heads/master
#
# see contrib/hooks/ for an sample, or uncomment the next line (on debian)
#

ssh -i ~/.ssh/id_rsa_git_ci_server cerberus@ci-server \
 bin/cerberus-build racc-webapp

NOTE: the use of the -i switch

This hook is applied to each repository that will be monitored by Cerberus Continuous Integration.  It must be made executable so Git will execute it!

[git@redmine-server]$ cp post-receieve repos/my_app/.git/hooks/post-receive
[git@redmine-server]$ chmod +x repos/my_app/.git/hooks/post-receive

Cerberus configuration

Using the ruby-switcher, install the ruby of your choice (’install_ruby_187p173′).  The use it (’use_ruby_187′).

Install cerberus gem (note we are using local gem storage to accomodate ruby-switcher):

[cerberus@ci-server]$ gem install cerberus

Configure cerberus globals, especially email server config (for SMTP send).

I also configured the builder/rake to handle new migrations (Rails 2.3) and Rspec (rake spec)

[cerberus@ci-server]$ cat .cerberus/config.yml
publisher:
#  active: mail jabber rss campfire
  mail:
    address: smtp.test.com
    port: 25
    domain: test.com
    authentication: :none
    sender: Cerberus CI <cerberus@test.com>
#    address: smtp.gmail.com
#    port: 587
#    domain: gmail.com
#    authentication: plain
#    user_name: someuser
#    password: somepassword
#    on_event: all
#  jabber:
#    jid: somemailbox@gmail.com/cerberus
#    password: somepassword
#  twitter:
#    login: twitter_username
#    password: twitter_password
#  irc:
#    nick: cerb
#    server: irc.freenode.net
#    channel: cerberus
#  campfire:
#    url: http://someemail:password@cerberustool.campfirenow.com/room/51660
#  rss:
#    file: /usr/www/rss.xml
builder:
  rake:
    task: db:migrate spec
#changeset_url: POINT_TO_YOUR_TRAC/changeset/
#hook:
#  rcov:
#    on_event: successful, setup #by default - run hook for any state
#    action: 'export CERBERUS_HOME=/home/anatol && sudo chown www-data -R /home/anatol/cerberus && rcov' #Add here any hook you want

Configure the cerberus project. Note how the config points back to the repository

[cerberus@ci-server]$ cerberus add git@redmine-server:my_app/.git \
  APPLICATION_NAME=my-app SCM=git

Edit the configuration of the project to add project-level mailing addresses.  The BUILD WILL FAIL WITHOUT THIS!!!

[cerberus@ci-server]$ cat .cerberus/config/my-app.yml
---
publisher:
  mail:
    recipients: someuser@test.com
scm:
  url: git@redmine-server:repos/my_app/.git
  type: git

An initial build ‘cerberus build my-app’ is required to set the sources.  This one will likely fail, as it is necessary to configure non-SCM files like ‘database.yml’.

You should receive email from “Cerberus CI” declaring your failure.

You have completed config.  Now FIX YOUR BROKEN CHECKIN FTW!!

I could go into great detail… but I’m beat.  Creating the dev environment from the ground up was a lot of work.  I also needed to get some project details under way.

Here is what I needed to get Features and Specs to load up the library properly

Cucumber

Create a cucumber.yml.  This is necessary to set the default profile, which I wanted to get a little more pretty printing during the cucumber command.

In a step definition (grid_steps.rb), I needed the $LOAD_PATH to point to the lib, so I added the following to the top of support/env.rb

$:.unshift File.expand_path(File.dirname(__FILE__) + "/../../lib")

Also support RSpec expectations/matchers within the steps, so

require 'spec/expectations'

RSpec

Update the $LOAD_PATH here from a different location in the tree than features, from ./spec

$:.unshift File.expand_path(File.dirname(__FILE__) + "/../lib")

Autotest integration

In order to get cucumber integrated with autotest, add this to cucumber.yml

autotest: --format pretty features
autotest-all: --format pretty features

Also autospec requires an environment switch, which I have added to ./bashrc for convenience

export AUTOFEATURE=true

Amber to Red to Green to Amber to Red to Green…

With all the prettyness set up with cucumber and RSpec, we now have a new color in the Red-Green cycle.

Cucumber Features’ Given/When/Then lines are displayed in Amber until each is properly trapped by a defined Given/When/Then in a steps definition file (./features/step_definitions/*.rb).

RSpec specs are displayed in amber if the spec is not defined (name only) like this

it "this spec is not defined yet"

Well, that’s about all I can handle for now.

I will make an attempt to trap the Amber-Red-Green cycle in git commits during my work tomorrow.

• Ubuntu 8.10 Desktop as VirtualBox Guest on Vista Home Premium on Compaq C771.  NAT, 1gb ram, 16gb vhd, shared folders
• Ruby 1.8 + dependencies from Apt
• Rubygems 1.3.1 installed from .tgz (Ubuntu 8.10 has trouble with Apt’s rubygems install)
• Rails, RSpec, RSpec-Rails, Cucumber gems installed
• NetBeans 6.5.1 / All Modules.  NBGit installed for Git integration.