Rails Machine Blog

Return from the Ruby Hoedown

Wed, 02 Sep 2009 13:54:00 GMT

Jesse Newland and I are back from the 2009 Ruby Hoedown, the South’s regional Ruby conference. We both had a blast! The talks were all enjoyable and very informative, and we had an awesome time hanging out with folks like John Nunemaker, Steve Smith, Matt Todd, and many, many others.

It was great seeing all of you there who did come by and say “Hi” at some point this weekend—we hope you enjoyed the conference (and your spiffy lanyards!) as much as we did.

The talks that really stuck out out in my mind were those given by Ben Mabey (Writing Software, Not Code With Cucumber), Leon Gersing (couldn’t find a link to his slides, but it was on Appcelerator Titanium), and Jim Weirich (Source Control for People Who Don’t Like Source Control). The talk on threading in Ruby 1.9 was also really interesting—it’s definitely something we’ll revisit when more people are using 1.9 (ie. right around the release of Rails 3.0). The lightning talks were particularly amusing, both for the fact that they’re very impromptu and also because the speakers were given props to play with during their presentations. Anyone who was there can tell you all about the guy with the kazoo. All of the talks were very well done, and I’d like to thank everyone who did speak this year for doing a great job.

If you’d like to read up on specifics of the talks themselves, take a look at one of the first recaps I’ve seen posted or at the hashtag on twitter.

While Jesse and I had a blast at the conference, I think I can speak for both of us when I say that I’m glad to be back home, working on your rails machines to make sure you get the best experience possible.

Hoedown Bound

Thu, 27 Aug 2009 11:33:00 GMT

Jesse Newland and I are currently on our way up to Nashville for this year’s Ruby Hoedown, the South’s regional Ruby conference. We’re both looking forward to good talks, demos, and of course hacking on some awesome stuff. We’ll definitely be at the Smash-Up Derby, as well as a few other things in the area. For a list of things to do while in Nashville, we recommend taking a look at this list compiled by Rick Bradley.

If you’re a customer, are interested in hosting at Rails Machine, or just want to ask us some questions, come find us (or hit us @railsmachine) and we’ll grab a beer and chat. And remember, our “ask us anything” support is always available in person!

We hope we’ll see you there, but if we don’t, we hope you have a good time at the Hoedown!

Free Scout Monitoring for Customers!

Thu, 25 Jun 2009 15:07:00 GMT

We are excited to announce that Scout monitoring is now available for free (via a $14 off coupon) to all Rails Machine customers. Created by our pals at Highgroove Studios, Scout was recently updated with a slew of awesome new features including deep Rails instrumentation, triggers, and a more robust agent. With Scout’s extensible plugin system, you can monitor any data that you want and track any trends over time. We’re big fans of the plugin system for catching tricky issues that elude basic system health metrics. There are a number of community-supported plugins available for your own use.

If you are a customer of our Ruby on Rails managed hosting and not currently on Scout, we will be in touch and take care of it without worry. If you have a traditional account, then please request a coupon code via the support request form. This coupon code will provide $14 off any of Scout’s affordable plans ($7/server) if you need more than the Basic account. We even have a new plugin for Moonshine for automating Scout’s installation.

Ruby BigDecimal Security Vulnerability (CVE-2009-1904)

Wed, 10 Jun 2009 10:35:00 GMT

If you have a server at Rails Machine, please head over to the Rails Machine Status Blog for our detailed take on the Ruby BigDecimal security vulnerability that was announced late last night.

If you’re not already subscribed to our Status Blog, now is an excellent time to do so. We post post news and updates about system, security, and network issues affecting multiple Rails Machine customers over there, and generally reserve this blog for other posts, such as job postings and software releases. Here’s the RSS URL for the Status Blog:

http://status.railsmachine.com/rss

Rails Machine is Hiring

Thu, 04 Jun 2009 15:31:00 GMT

Love helping people with their Rails deployments? Come work for The Machine! Rails Machine is hiring a part-time or full-time Ruby on Rails Support Engineer in Atlanta, GA or the surrounding area.

As a Ruby on Rails Support Engineer, you would work with the existing Rails Machine team to:

Support existing customers’ Ruby on Rails deployments
Deploy new customers’ Ruby on Rails applications
Write documentation covering common problems
Write software to automate anything you do more than twice

In addition to the above work, which can all be performed from the comfort of your home office, your back patio, our coworking space, or anywhere you can find the internets, the following work will need to be occasionally performed onsite at our two Atlanta data centers.

Unbox, rack, cable, and configure new servers
Perform basic server maintenance (replace hard drives, etc)

You should be comfortable with all of these things:

Linux
Ruby
Rails
Passenger
Mongrel
Apache
MySQL
Capistrano
Git
Subversion

You should be excited to learn about these things:

Xen
KVM
Nagios
LVS
Puppet
Moonshine
Sinatra
TDD/BDD with RSpec

Rails Machine is a small business, so this will be anything but a boring job. We all wear many hats, but have a damn good time doing it. Oh, and if you like strange beers, that helps too, as we’re running out of names for our servers.

If this sounds like fun to you, please send your resume, a link to your GitHub account, and a note explaining why you’d be perfect for the job to Jesse Newland. Thanks!

Moonshine Plugins: Yo dawg, I put a plugin in your plugin

Wed, 06 May 2009 09:44:00 GMT

We hope by now that you’ve had a chance to check out Moonshine and maybe even give it a go on your own server. We’ve been using Moonshine to deploy our customer’s servers for a while now, and we’ve started extracting little pieces of manifests into Moonshine plugins. I’m going to show you how to add Moonshine plugins to your manifest, and give you the lowdown on writing your own.

Installing and using plugins

Moonshine plugins are installed just like any other Rails plugin:

script/plugin install git://github.com/railsmachine/moonshine_ssh.git

Then add a few lines to your application’s manifest- app/manifest/application_manifest.rb, by default.

configure( :ssh => { :allow_users => ['rails'] } )
plugin :ssh
recipe :ssh

Not all plugins will need to be configured before they’re used and the recipe name doesn’t need to be the same as the plugin name- in fact, a plugin could provide as many recipes as you like. The next time you deploy your application, Moonshine will run the recipes you specified from the installed plugins.

Writing your own

Now here’s how you can really shine while you ‘shine. If you’ve added functionality in your manifest that you think others could use, create a new Moonshine plugin for it and put it up on github. Here’s how we do it.

Generate the plugin stub

Start by running the plugin generator from your application directory. Just switch out “ssh” for whatever it is you’re plugin-ifying.

script/generate moonshine_plugin ssh

This creates the following structure in the vendor/plugins directory of your app:

  |--moonshine_ssh
     |-- README.rdoc
     |-- moonshine 
     |  `-- init.rb
     |-- lib
     |  `-- ssh.rb
     |-- spec
         |-- spec_helper.rb
         |-- ssh_spec.rb

Add recipes

The lib/ directory is where we’ll put puppet recipes. For our SSH plugin we add this:

def ssh(options = {})
  package 'ssh', :ensure => :installed
  service 'ssh', :enable => true, :ensure => :running

  file '/etc/ssh/sshd_config',
    :mode => '644',
    :content => template(File.join(File.dirname(__FILE__), '..', 'templates', 'sshd_config'), binding),
    :require => package('ssh'),
    :notify => service('ssh')
end

This is straightforward- we’re making sure that SSH is installed, that the service is running and will start on system boot. We’ll create a templates/ directory with an ERB template called sshd_config. It will be rendered and saved as /etc/ssh/sshd_config. The SSH service will be notified to restart when it changes. The template will have lines like this sprinkled throughout:

LoginGraceTime <%= options[:login_grace_time] || 30 %>
PermitRootLogin <%= options[:permit_root_login] || 'no' %>

The options come from our configure line in the first example. If you recall, we defined a property called :ssh, which we set to a hash. This hash will be passed to the ssh recipe by Moonshine. This little convention helps keep the custom configuration of your server separate from its list of parts.

Do the safety dance

Now we’ve got a working plugin that can manage your SSH settings. Just one thing- what if someone configured ssh like this?

configure(:ssh => {:port => 'two'})

This clearly isn’t valid and we don’t want them to be locked out, so let’s test the file before we update it.

def ssh(options = {})
  package 'ssh', :ensure => :installed
  service 'ssh', :enable => true, :ensure => :running

  file '/etc/ssh/sshd_config.new',
    :mode => '644',
    :content => template(File.join(File.dirname(__FILE__), '..', 'templates', 'sshd_config'), binding),
    :require => package('ssh'),
    :notify => exec('update_sshd_config')

  exec 'cp /etc/ssh/sshd_config.new /etc/ssh/sshd_config',
    :alias => 'update_sshd_config'
    :onlyif => '/usr/sbin/sshd -t -f /etc/ssh/sshd_config.new',
    :refreshonly => true, # do nothing until notified
    :require => file('/etc/ssh/sshd_config.new'),
    :notify => service('ssh')
end

There you have it. We upload the new configuration to a temporary location and replace the previous one only if the syntax check returns 0. This plugin is ready to be put online for all to enjoy.

The first plugins

So far, we’ve released plugins for managing iptables, SSH, and god. Look for more in the coming days. If you’ve written one, let us know about it in the comments!

Update: Check out the list of plugins and add your own on the plugin wiki page.

RailsConf 2009: Vegas Edition

Mon, 04 May 2009 13:22:00 GMT

If you’re in Vegas for RailsConf, we’d love to meet you! Jesse Newland will hanging out in the Caboose Conf room during the day, hacking on Moonshine. Feel free to drop by if you’d like to learn more about Moonshine, chat about Rails deployment, or anything else. Remember: our Ask Us Anything Support is always valid in person.

Oh, and if you’re a Rails Machine customer, Jesse would love to buy you a beer. Just @jnewland him. Mmmm beer.

Moonshine: What burns blue makes your blues go away

Wed, 18 Mar 2009 11:25:00 GMT

We’re happy to announce the release of Moonshine, the system we’re using to make the deployment and configuration management of our new customers’ Rails applications a no brainier. Moonshine combines all of the good parts of Puppet, Shadow Puppet and Capistrano into an amazingly simple solution for deploying your Rails application.

Look at all the choices I’m not making

One of the things that separates Moonshine from other solutions like Chef and Sprinkle is that out of the box, Moonshine comes with recipes for the same Ubuntu/Ruby Enterprise Edition/Apache/Passenger/MySQL stack that’s in production use at Rails Machine. We’re open sourcing this stack as a part of Moonshine for a couple reasons:

To make it easier for Rails developers to deploy an application to any host, not just Rails Machine.
To allow our customers to have a say in what they want in a Rails stack. If there’s something missing or a choice we’ve made that you don’t agree with, fork Moonshine, make your changes, and send us a pull request.

Since Rails Machine was founded in 2006, we’ve been dedicated to making the deployment of Rails applications easier for our customers. With the release of Moonshine, we’re taking that one step further by making Rails application deployment and configuration management easier for everyone!

So, how do I get started?

Moonshine is distributed as a Rails plugin. To get started with Moonshine, install this plugin and run the included generator:

ruby script/plugin install git://github.com/railsmachine/moonshine.git
ruby script/generate moonshine

You’ll notice a couple new files:

app/manifests/application_manifest.rb
config/moonshine.yml

The first file is the Moonshine::Manifest for your application. A Moonshine Manifest is a Ruby class that contains ShadowPuppet recipes – essentially just instance methods – that install packages, create configuration files, and run system commands. The manifest that’s been generated for your application is actually subclass of Moonshine::Manifest::Rails, which contains the entire Rails Machine production stack as recipes.

The second file is a hash serialized to YAML that contains the configuration for your application’s deployment. You’ll notice that this file contains variables you may be used to configuring in Capistrano – the location of your source code repository, the user to execute commands on the server as, and the deploy_to location on your server. Configure these just as you would in Capistrano – this configuration hash will be available to both Capistrano and Moonshine.

If your application doesn’t have any requirements in addition to the gems specified in your config/environment.rb (you have specified all of your gem requirements using config.gem calls, right? If not, stop reading and do this now), then you’re ready to deploy your app!

Deploying your Application with Moonshine

Once you’ve installed the Moonshine plugin, generated a manifest, configured Moonshine, and committed all of these changes to your repo, you’re ready to deploy! To work with Moonshine, your server needs to satisfy these requirements:

Ubuntu 8.10
Has a user with sudo privileges that can access your applications’ repository. (Moonshine uses rails as the default user – this is configurable in config/moonshine.yml)

This server doesn’t need Ruby, MySQL, or anything installed on it. That’s what Moonshine is for!

Once your server has been provisioned, capify your application and replace the stock config/deploy.rb with this one-liner:

server "myubuntuserver.com", :app, :web, :db, :primary => true

Once that’s in place, run the following command:

cap deploy:setup deploy

First, in the deploy:setup task, Moonshine will install Ruby Enterprise Edition, Git, and Shadow Puppet and setup your deploy_to directory. What happens next, during the deploy task, is what makes Moonshine so awesome.

On the first deploy of your application, all requirements defined by recipes are installed, configuration files are created, and services started. In repeated tests on a Ubuntu 8.10 server running in VMWare, this first deploy takes between 10-15 minutes to install everything needed for a Rails application. You read that right – less than 15 minutes after running cap deploy:setup deploy on a stock Ubuntu 8.10 server, your Rails app will be up and running.

Idempotent Deployment

On each and every deploy of your Rails application, your Moonshine Manifest is applied and verified. This means no more failed deploys because of a missing gem or package, and more importantly, you’ll never have to SSH to your server to install a package again.

For example, say you have a Rails app in production using Moonshine and you’ve added file uploading using Paperclip. After adding a config.gems 'paperclip' line to config/environment.rb, run rake moonshine:gems to cache your gem dependencies, commit the changes to your repo, and then deploy. Moonshine will automatically install Paperclip and ImageMagick (!!!) during the deploy. Done. Awesome, right?

Brew Your Own

So the default Rails stack we’ve included is just the tip of the iceberg. As we use Moonshine more and more at Rails Machine, we’ll be adding built in support for things like Memcached to the recipes in Moonshine::Manifest::Rails. And the plugin support in Moonshine makes it ridiculously easy for you to create reusable of recipes to share between apps, or even to include Moonshine recipes as a part of your plugin! If none of you beat me to it, I’ll be forking Thinking Sphinx and pouring some Moonshine on it pretty soon.

Use the Source, Luke

So what are you waiting for? The source and RDocs are up on GitHub – go nuts.

Moonshine at Rails Machine

Moonshine is currently available for all customers of our Application Management services at Rails Machine. If you’re an existing customer on a CentOS server and are interested in using Moonshine without the expert management, 24/7 monitoring, and proactive support that comes with our Application Management services, you will be able to redeploy to a new Ubuntu-based system very soon. Stay tuned!

Video and Slides from Acts as Conference

Fri, 13 Feb 2009 16:57:00 GMT

On Saturday, February 7th 2009, Dan Benjamin, Rails Machine Evangelist, gave the closing keynote at this year’s Acts as Conference in Orlando Florida.

Both the video and slideshow are available for your viewing pleasure below.

Slideshow

Distraction, Attention, and Simplicity (Acts As Conference Keynote)

View more presentations from danbenjamin. (tags: dan benjamin)

Video

Rails Deployment and Automation with ShadowPuppet and Capistrano

Tue, 10 Feb 2009 16:41:00 GMT

While our Moonshine is still distilling, I’d like to present some examples of using existing automation tools to deploy Rails applications easily.

Taskify

When Jamis Buck released Switchtower (now known as Capistrano) around 3 years ago, automated Rails deployment become standardized. It was a historic moment in Rails history. Since then, Capistrano has been used for a number of tasks beyond code deployment related to system configuration and automation. The railsmachine gem provides tasks for configuring an application stack. However, our gem has always assumed that you were deploying against a golden image (or foilball) and only addressed configuration. Other Capistrano-based projects took the additional step of automating calls to package installers to setup an empty machine. All of these projects provided a quick way to deploy a Rails application on a new server. Despite their utility, the approach has a number of shortcomings: increased code complexity, not idempotent, difficult to model complex dependencies, only executable via cap, difficult to manage existing systems, etc. When all you have is Capistrano, everything looks like a cap task.

Automate

With the rise of the clouds, automation tools are getting a lot attention recently. Unbeknownst to most developers, sysadmins and operations folks have been using an automation tool called Puppet for the past few years. Puppet is written in Ruby and provides its own external DSL and services for managing infrastructure. In addition, the new, Puppet-inspired Chef is helping spread the gospel of automation to Rails developers. More awareness is a good thing. Automation is a best practice that all Rails developers should add to their development process.

Cheese and Crackers

By using using Puppet with Capistrano, we can use the right tools for the right job. The issue of how to partition the work between the two tools has been been recently discussed by Andrew Shafer of Reductive Labs. Although I think we can push more onto Puppet then Andrew suggests, we both agree when Adam Jacob writes: Capistrano and Puppet go together like cheese and crackers.

One of our current goals is to easily integrate server automation into every Rails deployment. The challenge is that existing tools are just frameworks and have no knowledge or opinion on deploying Rails applications. Some developers will take the time to learn the tools and others won’t. This is understandable since spending time learning about automation frameworks is time not spent developing revenue-generating features.

shadow_rails

To jump start the process, I’ve written an example called shadow_rails that can be used as a starting point for introducing automation into your existing deployment process. The code is written using ShadowPuppet to install packages and configure the system. The tricky part is that Ruby-based automation tools require ‘ruby’. Therefore, there is a little bootstrapping script to install Ruby Enterprise Edition from source or the Ubuntu 8.10 ruby packages. In addition, shadow_rails installs Apache, MySQL, Passenger, and Rails. It configures Apache, creates your database, and sets up the Capistrano directories. From the comfort of your development box, you can go from nothingness to a deployed app in one command: cap deploy:bootstrap deploy:migrations.

This project is meant to help you get going with using automation and not the final word in deployment. To get started, read the README in the GitHub repository. Clone and browse around the code to get familiar with the syntax. Fork the code and try to make it work for your application. There are even a few rspec examples written using ShadowFacter for those interested in exploring behavior driven automation.

You’ll need a fresh install of Ubuntu 8.10 with an admin user named ‘rails’ and ssh installed. I highly recommend using a virtualization tool like VMware Fusion on OS X or VMware Player on Ubuntu for testing. Using snapshots and rollbacks, testing your manifest is super easy for both clean installs and existing ones as the manifest evolves.

Feedback

Please let me know what you think and ping me with any questions. If you are a Rails Machine customer and would like to try it on a new virtual machine, please submit a ticket.