Mind Junk

Why Care About Vagrant

2013-09-12T10:59:00+00:00

I have become a big fan of Vagrant and Puppet. They really change how you think about application development. If you are scratching your head about what these tools are here is a quick overview.

Vagrant

Vagrant is a tool that plugs into VirtualBox* and with a simple configuration can create, suspend and destory machines. For instance the vagrant box that I am developing this blog on right now was created using a configuration that looks like this

Vagrant.configure("2") do |config|

  config.vm.box = "precise64"

  config.vm.provision :puppet do |puppet|
    puppet.manifests_path = "puppet/manifests"
    puppet.module_path = "puppet/modules"
    puppet.manifest_file = "init.pp"
    puppet.options = "--verbose --debug"
  end
end

Ignore the stuff about puppet because we are going to talk about that next. The config.vm.box= line is saying that when I ask vagrant to create the machine for this project use Ubuntu 12.04. After the machine comes up it dynamically maps in the projects source code into a folder on that machine. This allows one to run linux commands in that directory on the guest machine vagrant created and it will affect the code on the host machine. I don’t even have to be on a Linux host machine for this to work. Vagrant can run on Windows and Mac OS X as well.

*It plugs into various virtualization frameworks including both vmware and virtualbox. It even can build boxes on remote virtualization systems including aws, openstack and rackspace.

Puppet

By running vagrant up you get an almost instant development box to play with. But to do anything interesting you are going to need a whole lot of packages and enviroment settings that are normally manually placed on the developers machine. This is where puppet come in. Using a declarative ruby based language you can state expectations on what this new machine must have for you to do your job. Vagrant kicks off puppet on the way up and by the time you are ready to ssh into the machine it is has everything you need to start your application.

Here is the puppet manifest file for the vagrant machine this blog was built on:

package{"curl":
    ensure=>"present"
}
package{"libxslt-dev":
    ensure=>"present"
}
package{"libxml2-dev":
    ensure=>"present"
}

@exec { '/usr/bin/sudo apt-get update':
   tag => update
}
Exec <| tag == update |> -> Package <| |>

exec { 'install-rvm':
  command => "/bin/su -l --command='/usr/bin/curl -L https://get.rvm.io | bash -s stable' vagrant",
  creates => "/home/vagrant/.rvm",
  require => Package['curl']
}

exec {'install-ruby':
  command =>"/bin/su -l --command='rvm install $(cat /vagrant/.ruby-version)' vagrant",
  require => Exec['install-rvm']
}

exec {'bundle-install':
  command =>"/bin/su -l --command='cd /vagrant;bundle install' vagrant",
  require => Exec['install-ruby']
}

The package directives are things that I need puppet to apt-get install. Then we have a couple exec directives. These setup rvm, install the version of ruby I need and then bundle install the gems for my project. After puppet is done running I should be able to execute jekyll build on my blog without any errors. This is a really simple case but hopefully you can see the utility.

At work we have vagrant and puppet configurations that will spawn an ubuntu box, install mysql, rvm, phantomjs and ruby; configure all required users and directories; set up the machine to be able to capistrano deploy and finally start the application server and database. These are all things developers normally do on their host machines on their first day at a job. By installing vagrant and going to the break room for a cup of coffee a developer can have a “ready to code” development virtual machine by the time she gets back!

Why Git

2011-09-10T08:57:00+00:00

This post is meant to capture why we transitioned to Git from Subversion and what we learned along the way.

Code Reviews

The main reason I started to consider switching version control systems was we had a work flow issue that was causing members of our team to be non productive for days at a time. As a team we had instituted mandatory code reviews to try to maintain a high level of code quality. Our software was deployed into the pentagon network so it was important to know what was being built and how it was being built.

The work flow with subversion was:

Tasking came into a senior developer.
The senior developer assigns the task to a team member.
Team member emails a patch with the requested changes.
A senior developer reviews the patch.
If accepted the team member commits his changes to Subversion.

The sticking point was step 4 “A senior developer reviews the patch.” If the senior developer was busy with her own tasking or not in the office the team member could not continue to make changes for other tasks. This waiting period could last for hours to days. We tried to allow commits prior to review but we were finding that changes were slipping through the cracks unreviewed.

Distributed Version Control

Git separates the concept of committing changes and sharing those changes with a remote repository. Committing is a local action that adds your changeset to your complete clone of the remote repository. When you are ready to share one or more commits you execute a push.

Because commits were local actions it allowed team members to commit changes prior to a code review and to continue there work even in the absence of a senior developer. As time permitted, patches would be reviewed and the team members would then be permitted to push changes to the remote repository.

Local commits fixed a very specific work flow issue we were having with code reviews and improved the productivity of our team.

Initial Commit

When a new project was created a developer would would work on that project till it got to a point where it was ready to be put in Subversion. At that point she would create a new folder in Subversion, checkout that folder, add all her local files to the fresh checkout and commit her changes to the remote repository.

The result of this was every project had this mystery commit with the message of “initial commit.” There would be no information how it got to that state, it just was.

By executing a git init command, you can turn any folder in your filesystem into a Git repository. This project will be unattached from any remote repository ready for you to begin making local commits. At the time you are ready to share your project with the team you can create a remote git repository, attach your local git repository to the remote repository and push all your commits to the remote repository.

Being able to instantly create a repository in any folder greatly minimized the amount of history lost from the “initial commit.”

Missing Files

Ever forget to commit a file that is part of a change? This leads to those awkward commits in Subversion with a message like “Oops forgot to commit Xyz.java.” In Git local commits that have not been pushed to a remote repository can be modified without consequences. If you forgot a file amend your previous commit prior to pushing your changes to a remote repository.

Pitfalls

Most developers do not know Git; staff will require training. I highly recommend the book Pro Git.
Window’s support is not as strong as Subversion but it is improving every day. Msysgit and TortoiseGit are the definitive windows clients.
It is overly complex to get a Windows git server running. I recommend sticking with Linux for the server. Its a difference between minutes to setup to hours or days.
Sometimes a developer will forget to push changes prior to deployment day; this is a training issue. Check the logs prior to deployment to verify expected changes.
By default git allows developers to rewrite history and modify changes on the server if they execute push with a force option. This can be turned off with a configuration change.
There are sometimes more than one way to do things in Git. If this is the case then there isn’t a universally agreed upon way. I recommend picking a work flow and sticking with it. This is especially bad with rebase vs merge.

Markdown to PDF

2011-08-13T19:46:00+00:00

I wrote a Gradle script to convert a markdown file to a PDF. Despite being 38 lines it was quite a little challenge to discover which libraries needed to be strung together and how to use them. Hopefully this can help someone in the future.

Kinect Is Amazing

2011-08-12T10:59:00+00:00

This demo out of siggraph is stunning. They use the kinect to create 3D models of entire rooms.

Domain Moved

2011-08-11T13:17:00+00:00

I love new toys. I decided to move my domain from pointing at blogger to pointing at github pages. So, that means mindjunk.org has gone from wordpress on dreamhost to vpslink to blogger to github pages powered by octopress. I decided not to try to bring over my old content from blogger.

I am playing alot with Amazon EC2. I spawned up a micro instance thats operating out of the free tier to act as my private git server. Took about 20 minutes from account creation to cloning a project. I work with git servers alot at work so it may take you longer to get going. The basic steps are:

sign up for a amazon web services account
create a new micro instance. Defaults are fine. (Make sure its the one with the star)
login using your new ssh key
sudo yum update
sudo yum install git
sudo adduser git
sudo su -l git
mkdir ~/.ssh
touch ~/.ssh/authorized_keys
chmod 500 ~/.ssh
chmod 500 ~/.ssh/authorized_keys
add your public key to authorized_keys
mkdir ~/repos
git init –bare ~/repos/test.git

Octopress Setup

2011-08-02T11:40:00+00:00

As you can see I got Octopress working! Tried to do it twice so I could capture the steps. This is by no means a definitive guide but this is exactly the order of commands I ran. I am sure some steps could be removed or reordered. Enjoy!