Hedgehog's Hiatus

Git Submodules: Automatic init and update on fetch and pull

Tue, 28 Jun 2011 18:52:52 -0700

NOTE:

You still have to be careful to alwyas push changes made in the submodule/sub-project before you commit and push changes in the super-project.

Git 1.7.5 and later

#!/usr/bin/env bash
git clone --recursive -o name repository destination 
pushd destination
  git fetch --recurse-submodules=yes
  git checkout name branch
popd

Once cloned and abranch is checked out the usual update workflow is:

#!/usr/bin/env bash
pushd destination
  git pull --recurse-submodules=yes

  # Or

  git fetch --recurse-submodules=yes
popd

To make these the default behavior in a project, so you can use git pull and git fetch without the recurse option being given:

#!/usr/bin/env bash
git config fetch.recurseSubmodules yes

Git pre-1.7.5

The usual clone workflow for Git earlier than 1.7.5:

#!/usr/bin/env bash
git clone -o name repository destination
pushd destination
  git fetch
  git checkout name branch
  git submodule init
  git submodule update 
popd

Once cloned the usual update workflow is :

#!/usr/bin/env bash
pushd destination
  git pull name branch
  git submodule update
popd

For earlier versions of Git:

Create post-merge hook

Create an alias

git config alias.pullall '!f(){ git pull "$@" && git submodule init && git submodule update --recursive; }; f'

Hope that helps.

Permalink | Leave a comment »

Autoload and How-Not-To Require in Ruby 1.9.2.p0|p136

Thu, 03 Mar 2011 14:24:00 -0800

A little known gotcha, the cause of which can be obscure… You can oftentimes see suggested, even when discussing best practices, snippets like the following:

$LOAD_PATH.unshift(File.dirname(__FILE__) + '/../lib') unless \
  $LOAD_PATH.include?(File.dirname(__FILE__) + '/../lib')

If you intend to use Ruby’s [autoload], then do not do that. If you do you’ll run into this issue.

Rather, if you have to keep backward compatibility, insert File.expand_path, or use some of methods from the Pathname standard library. The Pathname has the advantage of being more declarative. But you should take care to not feed ‘~’ Pathname, or just move on to Ruby 1.9.2….

Example (incl. 1.8.7):

require 'pathname'
lp = Pathname.new(__FILE__).dirname.realpath.parent + 'lib'
$LOAD_PATH.unshift(lp) unless $LOAD_PATH.include?(lp)

If you don’t have to worry about 1.8.7:

Example (1.9.2):

require_relative '../lib/mylibrary'

Permalink | Leave a comment »

Upgrade VirtualBox Guest Additions in a Vargrant Base Ubuntu Box

Thu, 24 Feb 2011 23:44:00 -0800

[update:03-01-2011] Add OSE caveat [update:03-10-2011] Removed install virtualbox-ose-guest-dkms

This describes how to update Vagrant’s lucid64.box base box to the latest Guest Additions for VirtualBox 4.0.4. The only wrinkle was that the base box did not have the ppa:debfx/virtualbox repository configured.

Start The Old Vagrant VM

Make a backup and extract to a temporary folder:

$ mkdir tmp
$ cp lucid64.box tmp/lucid64.box
$ vagrant box add lucid64_4.0.4 lucid64.box
$ cd tmp
$ vagrant init lucid64_4.0.4
$ vagrant up

You should see message like:

[default] The guest additions on this VM do not match the install version of
VirtualBox! This may cause things such as forwarded ports, shared
folders, and more to not work properly. If any of those things fail on
this machine, please update the guest additions and repackage the
box.

It is this we aim to eliminate….

Update VirtualBox’s Guest Additions

$ vagrant ssh
vagrantup:~$ sudo apt-get update
vagrantup:~$ sudo apt-get install python-software-properties
vagrantup:~$ sudo add-apt-repository ppa:debfx/virtualbox
vagrantup:~$ cd /opt
vagrantup:~$ sudo wget -c http://download.virtualbox.org/virtualbox/4.0.4/VBoxGuestAdditions_4.0.4.iso \
                       -O VBoxGuestAdditions_4.0.4.iso
vagrantup:~$ sudo mount VBoxGuestAdditions_4.0.4.iso -o loop /mnt
vagrantup:~$ cd /mnt
vagrantup:~$ sudo sh VBoxLinuxAdditions.run --nox11
vagrantup:~$ cd /opt
vagrantup:~$ sudo rm *.iso
vagrantup:~$ exit

According to Brett Alton you can ignore the last error about the windows system driver’s failure to install, it should be harmless in non-GUI contexts.

Now check the Guest additions ‘just work’:

$ vagrant halt
$ vagrant up
[default] VM already created. Booting if its not already running...
[default] Running any VM customizations...
[default] Clearing any previously set forwarded ports...
[default] Forwarding ports...
[default] -- ssh: 22 => 2222 (adapter 1)
[default] Cleaning previously set shared folders...
[default] Creating shared folders metadata...
[default] Booting VM...
[default] Waiting for VM to boot. This can take a few minutes.
[default] VM booted and ready for use!
[default] Mounting shared folders...
[default] -- v-root: /vagrant

Excellent. No more warniings.

Package The New VM

$ vagrant halt
$ vagrant package
$ mv package.box lucid64_4.0.4.box

Done.

Caveats

An OSE version of guest additions seems to be only available via a Ubuntu package (virtualbox-ose-guest-x11) that has an X11 dependency. As of The Ruby gem virtualbox-0.8.3: Running a non-OSE Guest on a OSE Host (both VitualBox v4.0.4), with non-OSE Guest Additions installed, leads to this issue

Permalink | Leave a comment »

Chef issue 1526

Mon, 14 Feb 2011 17:21:00 -0800

A quick post to address this Chef issue. These issues (and more) can be resolved by using this fork of Bundler that targets the Chef use case. I’ll cover this in another post, but fro now note that this bundler fork is only intended for dealing with Chef Cookbook. Hopefully it’ll get to the point where the Bundler team accept (or provide) the requisite features.However, Bundler’s specs (and code base) make extending its behavior a nightmare – c.f Chef’s sweetness.

Anyway, the issues…

A Cookbook is installed only if dependencies are present in the Gemfile.
Installed dependencies are only updated if they have changed.
Only cookbooks specified in a gemfile are downloaded.
If you don’t specify dependencies, it gives LOTS of warning about that missing cookbook
Cookbooks are downloaded individually by branch, tag, or commit hash.
Cookbooks can be downloaded from any Git Repository (e.g. https://github.com/cookbooks )
Cookbooks can be downloaded using Git clone recognized URI’s

Outstanding:

Make pending specs pass (Bundler’s spec suite is a first class nightmare)
Make bundle update and bundle show change into a folder pointed to by --lockfile-path

The Gemfile

This where to list (just) the Cookbooks you want to download. You can think of a Gemfile as describing an application stack. Defining and is as simple as listing the Cookbooks repos in the Gemfile – to get the stack configured and running is Chef’s domain of expertise. You can download from just about any Git repository. Most git clone URI syntax should work (but not path/file names with spaces – i.e. it uses an unencoded file:// URI). The Gemfile corresponding to Opscode’s build-a-rails-stack example is shown in this gist.

Downloading the Cookbooks

The transript of downloading the build-a-rails-stack example’s cookbooks is here.
Use --gemfile to point at any file containing Gemfile data. Download the Chef Cookbooks to any folder pointed to by --install-path, if the folder does not exist it will be created. There will also be a lockfile created: Gemfile.lock

Update the Cookbooks

The transript of updating the build-a-rails-stack example’s cookbooks is here.
If there is no update nothing is downloaded, per the issue request. If you use Git commit hashes, then you can ensure your Cookbook(s) never change, e.g. when you are in production. Currently you need to change into the folder containing the lockfile (Gemfile.lock), created by bundle install ... in the folder the --gemfile ... option pointed to.

Show/List the Cookbooks and their Git hash.

The transript of the build-a-rails-stack example is here.
As with updating, you need to change into the lockfile folder.

Missing a Cookbook dependency

Remove the couchdb entry from the Gemfile. This is required by Chef. As we can see in this gist, show, update and install behave in ways that are not unreasonable for Chef users.

Permalink | Leave a comment »

Carving and Bundling Chef Cookbooks - Alpha 2.

Mon, 13 Dec 2010 22:37:00 -0800

My Bundler fork can now provide the minimum functionality required to deliver collections of cookbooks from any Git servers.

The Chef branch of my Bundler fork now can install cookbooks without decorating the folder name with a Git hash. See below for some caveats.
The cc- prefix has been removed from the repo names.
Version tags, based on the contents of metadata.json, have been added to each repo.
Some tweaks to the cc.gemspec.
cc.gemspec has been added to each version tag. Infact this file was inserted to each commit that contained the file metadata.json.

Example 1: Rails Quick Start Gemfile Revisited.

The environement is Ruby 1.9.2 via RVM, and assumes you’ve followed Opscode’s getting started. I’ll show the new Gemfile, built as described in the last post, that downloads the Ruby-on-Rails quick start cookbooks

Getting setup.

$ rvm gemset create test-bundler-chef
'test-bundler-chef' gemset created (/home/hedge/.rvm/gems/ruby-1.9.2-p0@test-bundler-chef).
$ gem list

*** LOCAL GEMS ***

rake (0.8.7)
$ git clone git://github.com/hedgehog/bundler.git
<snip>
$ pushd bundler
$ git checkout chef
<snip>
$ rake install
<snip>
$ gem list

*** LOCAL GEMS ***

bundler (1.0.7)
rake (0.8.7)

Acquire Cookbooks

Build the Gemfile as described previously. This time, rather than tie our cookbooks to the latest in the live branch, we can pull specific versions of each cookbook – pulling by Git reference hashes is also possible. I worked out the version by inspecting all the metadata.json under each cookbook’s folder of Opscode’s rails-quick-start repo. Note that without the hash decoration, or at least with the way I removed it, Bundler is better able to work out several dependencies that it missed previously (c.f. the following with the rails-quick-start.gemfile here):

# Useage:
# bundle install --gemfile /your/stack/library/rails-quick-start.gemfile \
#                --install-path /your/chef-repo/cookbooks
# 

gem "apache2",
    :git => "git://github.com/cookbooks/apache2.git", 
    :tag => '0.12.3', :git_decorate => false
gem "application",
    :git => "git://github.com/cookbooks/application.git", 
    :tag => '0.7.0', :git_decorate => false
  gem "unicorn",
      :git => "git://github.com/cookbooks/unicorn.git", 
      :tag => '0.1.3', :git_decorate => false
  gem "passenger_enterprise",
      :git => "git://github.com/cookbooks/passenger_enterprise.git", 
      :tag => '0.4.2', :git_decorate => false
  gem "passenger_apache2",
      :git => "git://github.com/cookbooks/passenger_apache2.git", 
      :tag => '0.16.3', :git_decorate => false
    gem "rails",
        :git => "git://github.com/cookbooks/rails.git", 
        :tag => '0.9.2', :git_decorate => false
    gem "packages",
        :git => "git://github.com/cookbooks/packages.git", 
        :tag => '0.9.2', :git_decorate => false
    gem "ruby",
        :git => "git://github.com/cookbooks/ruby.git", 
        :tag => '0.9.0', :git_decorate => false
  gem "ruby_enterprise",
      :git => "git://github.com/cookbooks/ruby_enterprise.git", 
      :tag => '0.2.3', :git_decorate => false
  gem "runit",
      :git => "git://github.com/cookbooks/runit.git", 
      :tag => '0.14.1', :git_decorate => false
  gem "build-essential",
      :git => "git://github.com/cookbooks/build-essential.git", 
      :tag => '0.7.0', :git_decorate => false
  gem "nginx",
      :git => "git://github.com/cookbooks/nginx.git", 
      :tag => '0.14.3', :git_decorate => false
  gem "rubygems",
      :git => "git://github.com/cookbooks/rubygems.git", 
      :tag => '0.2.0', :git_decorate => false
gem "apt",
    :git => "git://github.com/cookbooks/apt.git", 
    :tag => '0.9.0', :git_decorate => false
gem "aws",
    :git => "git://github.com/cookbooks/aws.git", 
    :tag => '0.10.1', :git_decorate => false
gem "database",
    :git => "git://github.com/cookbooks/database.git", 
    :tag => '0.7.1', :git_decorate => false
  gem "xfs",
      :git => "git://github.com/cookbooks/xfs.git", 
      :tag => '0.1.0', :git_decorate => false
  gem "mysql",
      :git => "git://github.com/cookbooks/mysql.git", 
      :tag => '0.24.1', :git_decorate => false
    gem "openssl",
        :git => "git://github.com/cookbooks/openssl.git", 
        :tag => '0.1.0', :git_decorate => false
gem "git",
    :git => "git://github.com/cookbooks/git.git", 
    :tag => '0.9.0', :git_decorate => false
gem "haproxy",
    :git => "git://github.com/cookbooks/haproxy.git", 
    :tag => '0.8.1', :git_decorate => false
gem "radiant",
    :git => "git://github.com/cookbooks/radiant.git", 
    :tag => '0.11.3', :git_decorate => false
  gem "sqlite",
      :git => "git://github.com/cookbooks/sqlite.git", 
      :tag => '0.7.0', :git_decorate => false

Note: Bundler’s gem building from Git repo’s is still broken…

$ bundle install --gemfile /tmp/rails-quick-start.gemfile \
                 --install-path /tmp/rails-quick-start
Updating git://github.com/cookbooks/application.git
Updating git://github.com/cookbooks/unicorn.git
Updating git://github.com/cookbooks/passenger_enterprise.git
Updating git://github.com/cookbooks/passenger_apache2.git
Updating git://github.com/cookbooks/rails.git
Updating git://github.com/cookbooks/packages.git
Updating git://github.com/cookbooks/ruby.git
Updating git://github.com/cookbooks/ruby_enterprise.git
Updating git://github.com/cookbooks/runit.git
Updating git://github.com/cookbooks/build-essential.git
Updating git://github.com/cookbooks/nginx.git
Updating git://github.com/cookbooks/rubygems.git
Updating git://github.com/cookbooks/apt.git
Updating git://github.com/cookbooks/aws.git
Updating git://github.com/cookbooks/database.git
Updating git://github.com/cookbooks/xfs.git
Updating git://github.com/cookbooks/mysql.git
Updating git://github.com/cookbooks/openssl.git
Updating git://github.com/cookbooks/git.git
Updating git://github.com/cookbooks/haproxy.git
Updating git://github.com/cookbooks/radiant.git
Updating git://github.com/cookbooks/sqlite.git
Could not find apache2-0.12.3 in any of the sources

Copy the rails-quick-start.gemfile to /path/to/chef-repo/rails-quick-start/, point Bundler to the cookbooks folder, and run:

$ bundle install --gemfile /path/to/chef-repo/rails-quick-start.gemfile \
                 --install-path /path/to/chef-repo/rails-quick-start/cookbooks

$ ls /path/to/chef-repo/rails-quick-start/cookbooks
apache2               database  passenger_apache2     runit
apache2-0173e252e689  git       passenger_enterprise  sqlite
application           haproxy   radiant               unicorn
apt                   mysql     rails                 xfs
aws                   nginx     ruby
build-essential       openssl   ruby_enterprise
cache                 packages  rubygems

Now you should be good to continue with the rest of the Opscode rails-quick-start how-to – modulo version tagging issues. This at least means my Bundler fork can now provide the minimum functionality required to deliver collections of cookbooks from any Git servers.

Note: The folder apache2-0173e252e689 above is generated after the repository contents are installed – I’m looking into this. However, Bundler has no unit-specs so trying to work out exactly how different parts are expected to behave is nigh impossible. But I’ll stop this topic now, before I get up a full head of steam….

Some TODOs

bundle install from Git repos should not complain about gemspec missing from sources.
bundle update accept switches --gemfile and --install-path.
bundle check accept switch --install-path
bundle package accept switches --gemfile and --install-path

Permalink | Leave a comment »

Carving and Bundling Chef's Cookbooks - Alpha

Wed, 08 Dec 2010 00:44:00 -0800

This is the introductory post about the repositories in the GitHub Cookbooks account. It (belatedly) addresses the topic “Centralized cookbook-library repos vs distributed cookbook repos”, discussed in this thread.

I also describe unpleasant side effects of using Bundler 1.0.7 that are being worked on.
I’m in the process of trying to make Bundler accomodate Chef’s use case, hence the Alpha tag on this post. The reason why I’m describing something that is clearly incomplete is in response to Indirect’s question about the pain points of using Bundler (as of <= 1.0.7, and possibly higher versions).

The Github Cookbooks account repo’s are trial repo’s and should not yet be forked since they WILL be deleted and replaced, or abandoned if feedback (hint) and experience dictates.

The objective is to carve individiual Chef cookbooks from the monolithic cookbook-library repositories. The two cookbook-library repositories are those of Opscode and 37Signals. Doing this requires adopting some dependency management mechanism, since one reason for these cookbook ‘silos’ was as a quasi-dependency-management solution (see the mail list discussion).

Now, some details.

Chef cookbooks on Github:

Each Cookbooks repo is a pure Git repo and initializing the repo using Niev’s git-flow does not restrict you to using Niev’s git-flow. An initial description of what this workflow might look like in the context of Chef wil be in another post if interest in these repos warrants this. Anyway, it is nothing radically different from what is described here. Each repository name has a cc- prefix to disambiguate it from the actual application name, the Opscode/37 Signals repos, and to help when searching. There are currently no wiki or gh-pages added, however it might be useful to add a common howto page to each repo describing any best-practice workflows.
Please comment below or track-back to your blog-post about your suggestions/thoughts.

Bundler:

Bundler is just one way of gathering Chef cookbooks into the cookbooks folder of the the local chef-repo – from there they are uploaded to the Chef server via knife. Bundler’s Gemfile allows you to set out the cookbooks you wish to gather to make up your application stack, and eventually specify different groups of that stack in one simple file. The rational for choosing Bundler was: Grabbing Chef cookbooks for an application stack should be as fine-grained and simple as:

$ cat <<-EOT >/local/stacks/wordpress.gemfile
    gem "cc-wordpress", :git => "git://github.com/cookbooks/cc-wordpress.git",
                        :ref => 'eb0203133d14587e90f4' 
    gem "cc-mysql",     :git => "git://github.com/cookbooks/cc-mysql.git",
    gem "cc-apache2",   :git => "git://github.com/cookbooks/cc-apache2.git"
    gem "cc-php",       :git => "git://github.com/cookbooks/cc-php.git", 
                        :branch => "qa"
    gem "cc-openssl",   :git =&> "git://github.com/cookbooks/cc-openssl.git"
  EOT
    
$ bundle --gemfile /local/stacks/wordpress.gemfile \
         --install-path /local/chef-repo/cookbooks/

To get the above to work required adding the install-path feature to my Bundler fork, and shaving a herd of Yaks I bumped into. If you’d like this feature added to Bundler, please find and vote on, or open a Bundler issue.

Now to the possibly contentious part.

Each cookbook’s metadata.json file is parsed in the gemspec file, cc.gemspec. This gemspec has generic contents, so it has a generic name to convey that point. Nonetheless, gem build cc.gemspec will generate a gem file with correct name – in case you have dreamed up some way of doing interesting things with Chef cookbook gems…. The cc.gemspec is parsed by Bundler. Consequently, the version requirements in the cc.gemspec are checked by Bundler inspecting the cc.gemspec in each repo.

Potential benefits:

Cookbook development can be more atomic, and each cookbook now has a dedicated issues forum.
Cookbook changes/branches/tags/forks are more visible and cookbook commit history has been preserved
A single repo houses both Opscode (live and qa branches) and 37 Signals (37s branch) cookbooks. Cookbook styles (37 Signals c.f. Opscode) may converge more easily (merging commits between branches)
The extracted repositories can be bundle installed into a local Chef repo using my Chef branch of Bundler
Repositories use Nvie’s Git workflow branching ideas in setting up live and qa branches. Comments welcome on branch names production, development and testing
Each cookbooks dependency/version requirements are in the cc.gemspec for Bundler to resolve
Additional control/flexibility is obtained via Git forks, refs, tags and branches. Cookbook deployment can be made from your fork’s latest master commit down to the level of a reference hash. You are no longer restricted to defining dependency by version numbers in the metadata file. This is more useful in dev and testing phases.
Additional flexibility will be obtained from Bundlers groups, elaborated here. Specifically, when Bundler supports different versions (Git hashes) between different groups.
Different Cookbook ‘styles’ (see here) coexist as a different branch, e.g. the 37s branches for 37 Signals cookbooks.
Additional opportunities for people to participate as maintainers/collaborators of cookbooks they have a special interest in.
Tagged cookbooks are, via GitHub download, easily accessible outside of Git

Potential costs

More knowledgeable collaborators required to be active to review changes
Cookbook styles (37 signals cf Opscode) may fragment further, see here for some discussion.
Roles have to be explicit about which cookbook they are using. This involves more typing since Bundler delivers the cookbook decorated with a hash (I am working to eliminate this).

Example: The base role from Opscode rails-quick-start:

Using Opscode’s rails-quick-start repo the role description is:

  name "base"
description "Base role applied to all nodes."
run_list(
  "recipe[apt]",
  "recipe[git]",
  "recipe[build-essential]",
  "recipe[ruby]"
)

Which with Bundler hash decoration would have to become this

  name "base"
description "Base role applied to all nodes."
run_list(
  "recipe[cc-apt-0cb9519bf687]",
  "recipe[cc-git-730475fa9847]",
  "recipe[cc-build-essential-b458e6200289]",
  "recipe[cc-ruby-e42796a7e807]"
)

Furthermore this would have to be edited everytime you updated your cookbooks with Bundler… ouch.

The Yeast

In the thread “Centralized cookbook-library repos vs distributed cookbook repos” Adam Jacob commented:

our job is to provide you guys the primitives you need to be able to do things the way you want to do them… it’s just me trying to suss-out where the primitives are.

This is a fantastic attitude, and I think that this usecase indicates that the Chef primitives here are the version and dependency data, parsed from metadata.json to the cc.gemspec. Hopefully they can stay.
Additional primitives are

the Git forks/branches/references you pull from, but that is an additional layer of flexibility outside of Chef’s scope, but within Bundler’s scope.
Chef environments c.f. Bundler groups. When (if?) Bundler allows different versions between groups we will have the ability to bundle install different Cookbook versions into different Chef environments – assuming Chef > 0.9 accomodates this (see this thread for a description of how this might work). Until then, the notion of a group of Cookbooks is not fully exploited. It is not yet clear to me if Chef’s upcoming environments feature will accomodate Bundler mapping (installing) its groups to Chef environments. A the time of writing the Chef documents suggest an environment’s cookbooks are specified in a file.
Hopefully this will be answered in the positive in a Beta post on this topic.

Adam also summarises, correctly in my view, that there are three things teased out in that thread. I’ve inserted how I see Bundler+Git fitting in to the three:

“How do you discover cookbooks you want to use?”
- cc- is the naming convention in Google/Github searches for cookbooks people have contributed.
- You feed Bundler a Gemfile you’ve been given describing the application stack you want installed and configured.
- Any Git server becomes one (not the) publishing/distribution platform for cookbooks.
- Github downloads provide non-git access (most useful when version tags are added to each repository. The following cookbook would be extracted with folder name cookbooks-cc-activemq-6f442c7

1
2
3

$ cd /tmp
$ wget --no-check-certificate -o qa-latest.tgz https://github.com/cookbooks/cc-activemq/tarball/qa-latest

“How do you track them over time, and potentially make site-specific changes, and track those over time?”
- A classical Git repo per cookbook, either upstream or, equivalently, someone’s public|private fork, allows you to track site specific changes over time. Hopefully, just until they get accepted back into the upstream repo.
- Cookbook tracking is explicitly controlled in the Gemfile, and can be at the level of Git forks, branches, tags, or tree hashes.
“How do you track and resolve the dependencies one cookbook has on another?”
- According to a cookbooks metadata and the Git fork/branch/tag/hash you pull from, as parsed from the cc.gemspec and Gemfile, and resolved by Bundler.

Example: Grabbing the rails-quick-start cookbooks

The environement is Ruby 1.9.2 via RVM. And assumes you’ve followed Opscode’s getting started.

I’ll demonstrate grabbing the Ruby-on-Rails quick start cookbooks, without the Amazon EC2 details. See here for full detail on the EC2 instructions.

Acquire Cookbooks

What follows is one way to describe application stacks. The Gemfile created is an alternative to
setting up cookbooks in a dedicated repository like: rails-quick-start.

Gemfile construction steps

Add one cookbook (:branch => ‘live’) listed in the rails-quick-start’s cookbooks folder.
Running Bundler’s install during construction of Gemfile:

$ cd /path/to/rails-quick-start
$ bundle install --gemfile rails-quick-start.gemfile 
                 --install-path /tmp/rqs/ckbks

Adding missing dependencies (indented) or, if none, adding the next cookbook.
- Not done here, but you could change the :branch to a :ref once the application stack is confirmed as working.
Running Bundler’s install to point to the finished cookbook repo:

1
2
3

$ bundle install --gemfile rails-quick-start.gemfile 
                 --install-path ~/chef/chef-repo/rails-quick-start/cookbooks

The finished Gemfile, rails-quick-start.gemfile, looks like this:

# Useage:
# bundle install --gemfile /your/stack/library/rails-quick-start.gemfile \
#                --install-path /your/chef-repo/cookbooks
# 

gem "cc-apache2",
    :git => "git://github.com/cookbooks/cc-apache2.git", 
    :branch => 'live'
gem "cc-application",
    :git => "git://github.com/cookbooks/cc-application.git", 
    :branch => 'live'
  gem "cc-unicorn",
      :git => "git://github.com/cookbooks/cc-unicorn.git", 
      :branch => 'live'
  gem "cc-passenger_enterprise",
      :git => "git://github.com/cookbooks/cc-passenger_enterprise.git", 
      :branch => 'live'
  gem "cc-passenger_apache2",
      :git => "git://github.com/cookbooks/cc-passenger_apache2.git", 
      :branch => 'live'
  gem "cc-ruby_enterprise",
      :git => "git://github.com/cookbooks/cc-ruby_enterprise.git", 
      :branch => 'live'
  gem "cc-runit",
      :git => "git://github.com/cookbooks/cc-runit.git", 
      :branch => 'live'
  gem "cc-build-essential",
      :git => "git://github.com/cookbooks/cc-build-essential.git", 
      :branch => 'live'
gem "cc-apt",
    :git => "git://github.com/cookbooks/cc-apt.git", 
    :branch => 'live'
gem "cc-aws",
    :git => "git://github.com/cookbooks/cc-aws.git", 
    :branch => 'live'
gem "cc-database",
    :git => "git://github.com/cookbooks/cc-database.git", 
    :branch => 'live'
gem "cc-git",
    :git => "git://github.com/cookbooks/cc-git.git", 
    :branch => 'live'
gem "cc-haproxy",
    :git => "git://github.com/cookbooks/cc-haproxy.git", 
    :branch => 'live'
gem "cc-mysql",
    :git => "git://github.com/cookbooks/cc-mysql.git", 
    :branch => 'live'
gem "cc-nginx",
    :git => "git://github.com/cookbooks/cc-nginx.git", 
    :branch => 'live'
gem "cc-openssl",
    :git => "git://github.com/cookbooks/cc-openssl.git", 
    :branch => 'live'
gem "cc-packages",
    :git => "git://github.com/cookbooks/cc-packages.git", 
    :branch => 'live'
gem "cc-radiant",
    :git => "git://github.com/cookbooks/cc-radiant.git", 
    :branch => 'live'
gem "cc-rails",
    :git => "git://github.com/cookbooks/cc-rails.git", 
    :branch => 'live'
gem "cc-ruby",
    :git => "git://github.com/cookbooks/cc-ruby.git", 
    :branch => 'live'
gem "cc-rubygems",
    :git => "git://github.com/cookbooks/cc-rubygems.git", 
    :branch => 'live'
gem "cc-sqlite",
    :git => "git://github.com/cookbooks/cc-sqlite.git", 
    :branch => 'live'
gem "cc-xfs",
    :git => "git://github.com/cookbooks/cc-xfs.git", 
    :branch => 'live'

Download to your cookbooks folder

$ bundle install --gemfile /path/to/rails-quick-start.gemfile \
                 --install-path /path/to/rails-quick-start/cookbooks
    
$ ls /path/to/rails-quick-start/cookbooks
cache                            cc-packages-35e85e0ab6b2
cc-apache2-490a410a8a38          cc-passenger_apache2-ef5688ef1e51
cc-application-144f665d1e52      cc-passenger_enterprise-ce23f03ac342
cc-apt-0cb9519bf687              cc-radiant-38c2de357868
cc-aws-502ce059e3e2              cc-rails-564b986e7bd7
cc-build-essential-b458e6200289  cc-ruby-e42796a7e807
cc-database-1dcc49afc3ad         cc-ruby_enterprise-662be867cbeb
cc-git-730475fa9847              cc-rubygems-85617ddffa08
cc-haproxy-bfb669c013e2          cc-runit-eb70fb557938
cc-mysql-a6034592d159            cc-sqlite-c24a6b87b12e
cc-nginx-bfa943be18fd            cc-unicorn-254419ebec0c
cc-openssl-b3fec75435cc          cc-xfs-b01c94e4daf7

Kitchen sinks

At the moment Opscode and 37 Signals have single repo’s with multiple cookboooks. You could replicate their current respositories by creating a superproject with each of the repositories at Cookbooks as a submodule. In fact, you could have one repo with: live, qa, and 37s branches. Each branch corresponding to the branches in the individual repos.
As they say in the Math’s books ‘We leave this as an exercise for the reader’.

WIP Caveats

The current repository structure can be used, with some pain in the case of Bundler.
When downloaded via bundler, changes to recipe names in role files would be required, so would only be useful in production settings – where you want to pin a cookbook down.

Note 1:

Currently some work is still needed on Bundler for it to be used painlessly. – The hash decoration of the folder names needs to be eliminated, or accept an alias – bundle update and bundle check should accept --gemfile and --install-path – bundle could accept a --with counterpart to --without

Note 2:

There is a bug in how Bundler handles Gem specification files in Git repositories. Consequently, you’ll see this error once dependencies have been resolved for a gem:

Source does not contain any versions of 'cc- (>= 0, runtime)'

In which case add the next cookbook in step (3). This has the side effect of not generating the lock file which, when the bug is fixed, the lock file will be a very handy way of finding the hash – in case you want to pin your cookbooks that precisely.

Note 3:

Another limiation of Bundler 1.0 is that you cannot have different versions in different groups. This has been mentioned on the maillist, and may change, but is not explicitly listed as an upcoming feature in the issues tracker.

Permalink | Leave a comment »