unethical blogger

About the Lookout Hackers Blog

2012-05-22T00:00:00-07:00

I've neglected cross-posting some of the articles I've written over the past month or so for the newly inaugurated Lookout Hackers blog. The generally idea of the Hackers blog is to give Lookout Engineering a bit clearer of a voice and an avenue to publish blog posts on things that are nitty-gritty and technical.

A couple of the posts I've written which may interest you are:

Integration Testing with Foreman

At Lookout we find ourselves building more and more APIs and backend services these days. Naturally we would like to be certain that everything will work fine and dandy once it has been deployed. The reality of building out a service-oriented architecture is that you not only have to expect failure to happen, you have to plan and test for it.
Continuous Deployment for Ruby Gems

With the above workflow and these conventions in place, we’ve reaped a couple of benefits, the most obvious one has been the severely reduced time it takes for new gems to be created and incorporated into production systems. At a higher level, we’ve gained more confidence in our gems with this emphasis on testing, traceability and code review baked into the process from the start.

The blog is still fairly young, and we're getting more developers used to writing blog posts for it. You can subscribe to the atom feed here or just follow @LookoutEng on Twitter.

Introducing Blimpy, a cloud thing

2012-05-21T00:00:00-07:00

DEATH TO VIRTUALBOX

Phew, okay. With that out of the way, let me tell you about Blimpy; a tool that I have written to make working with "cloud" machines easy as pie.

As much as I love my Thinkpad, it's not really suited to heavy virtualization. Despite having a delightful 8GB handy, it still only has a couple of teeny tiny CPU cores. What it does have though, is a nice internet connection to the CLOUD.

Blimpy was built to help me take advantage of the plethora of machines that can be spun up on AWS and other public cloud providers.

My immediate usecase for Blimpy is that I've needed to spin up a machines rapidly to perform end-to-end testing of Puppet modules such as puppet-jenkins

Below is a screencast of using Blimpy to validate the module:

This was done with a simple Blimpfile:

 Blimpy.fleet do |fleet|
   fleet.add(:aws) do |ship|
     ship.name = 'puppet-jenkins'
     ship.ports = [22, 8080]
     ship.livery = :cwd
     ship.flavor = 'm1.large'
   end
 end

Blimpy is structured to make it easy to spin up multiple machines at once, in this case however I'm only going to spin up a single machine (defaulted to Ubuntu 10.04 64-bit in US West 2), of the "m1.large" flavor. Jenkins is a hungry application, so I'm not going to use the default "t1.micro" flavor.

The Blimpfile also specifies that port 8080 should be opened, and will create a special Blimpy security group in AWS that properly allows ports 22 and 8080 through.

The project is still fairly young, but reasonably well tested (in my opinion) Ruby code, all of which can be found on GitHub

Today's Meatspace Hack

2012-05-12T00:00:00-07:00

I always delight in finding clever little solutions to real world problems that I have, the image below is a good example.

I walked inside from our front door and noticed that there was a bit of cool air coming through the door frame. Unlike our last apartment, which simply had piss-poor insulation, this door frame has proper insulation around the edges, but the door was not pressing against it.

A little piece of cardboard moving box, combined with some slicing from my leatherman, and tada! When the door is locked it will press against the frame, keeping our place a little warmer at night.

(The piece is pulled out a little from the lock for the photo op)

Breaking up with Dane

2012-05-07T00:00:00-07:00

For a long time I've been a staunch supporter and happy customer of Sonic.net, the scrappy Bay Area internet service provider that has recently started kicking ass, laying fiber, and embarrassing bigger ISPs.

As of this past Sunday, only half of that is true. After a few happy years of Sonic.net DSL, I had to leave Sonic.net. I think they're taking the break-up okay.

Backstory

A couple years ago I moved to sunny Berkeley, California. The rent in San Francisco was just too damn high, and I discovered that almost everything I loved about the Haight district was to be found in Berkeley. The "internet situation" in Berkeley leaves much to be desired though, unlike Sebastapol or certain areas of San Francisco, my favorite ISP isn't really paying attention to Berkeley.

It's not just my favorite ISP either, neither AT&T nor Comcast seems to give a damn about my cozy little earthquake death trap.

What's a guy to do?

I suffered for a couple years with sub-6Mbps speeds, but with my most recent move, my distance from the nearest CO would have put me in the low-4Mbps range which is simply unbearable.

Sure, things like an IPSec VPN service, "unsupported" IPv6 tunneling, stellar customer support, an engaging and receptive CEO, plans for building a next-generation fiber-to-the-home network, and many others, they're great and all. That still doesn't make up for the fact that my wife can't watch streaming bootlegs of "Toddler's and Tiaras" while I work on serious business.

It's not me, it's you.

Something new

Interestingly enough Sonic.net CEO Dane Jasper gave me the idea to look into a wireless broadband provider with his post: "I hate wireless." After some digging, I discovered Unwired Ltd, and Berkeley-based company offering wireless broadband, if you're lucky enough to have line-of-sight.

Turns out, I'm lucky enough!

When I found out Unwired had (great) line of sight to the top of my new house in Berkeley, I asked for "the most." I didn't care so much about the cost, I cared about getting the most damn speed I could possible achieve with the microwave link. "Most" ends up costing about $130/month, about $40 more than my last Sonic.net bill was, with 14Mbps down/8Mbps up (burstable, I know the caveat here).

I will continue to cheer Sonic.net on, with their plans to lay fiber in select parts of the bay area. If they ever decide to drop fiber in Berkeley, which would be very surprising, then I will gladly sign up. Until then, I will be trying out this newfangled wireless broadband.

You broke my heart Sonic.

My bio according to Thomson

2012-04-10T00:00:00-07:00

For an upcoming internal tech-talk that I will be giving titled "Amber: Smalltalk in your browser, or, Tyler is a hypocrite."

Thomson, an esteemed colleague, tech talk organizer, and by far the hardest working Filipino I know, sent out the following bio attached to the meeting invite:

A decorated veteran of two wars (the Korean War and the War on Drugs), R. Tyler Croy is an Infrastructure engineer at Lookout, bringing with him decades of obscure programming language experience. Previously at Apture (and before that, Slide), Croy has now avoided working for Google as the result of an acqui-hire twice (and counting). At Lookout, he is responsible for ensuring the fidelity of our server infrastructure, and as such possesses an extensive duct tape library under his desk. Croy is a graduate of the school of hard-knocks and speaks five languages (four of which exist solely on the internet). He is originally from Texas (which explains his casual, laconic southern drawl), and enjoys mint juleps, stealing road dots off of the highway, and yelling at folks to get off his lawn. He currently lives in Vallejo with his wife, two cats, and pet charizard.

He knows me so well.

Live Coding with ffmpeg and justin.tv

2012-04-04T00:00:00-07:00

Recently I've been experimenting with operating a live coding stream of my desktop. What this means in practice is that I focus on a single project or set of tasks for an hour or two, while streaming everything I do to my channel on justin.tv.

Here's a boring demonstration:

Watch live video from Live coding with @agentdero on Justin.tv

I'll save my thoughts on how the experiment is going for a later blog post, in this post I just wanted to share the how.

First I use this ~/bin/screenstream script:

#!/bin/sh -xe

INFO=$(xwininfo -frame)
API_KEY="YOUR_API_KEY_GOES_HERE"

WIN_GEO=$(echo $INFO | grep -oEe 'geometry [0-9]+x[0-9]+' | grep -oEe '[0-9]+x[0-9]+')
WIN_XY=$(echo $INFO | grep -oEe 'Corners:\s+\+[0-9]+\+[0-9]+' | grep -oEe '[0-9]+\+[0-9]+' | sed -e 's/\+/,/' )
FPS="15"

INRES='1680x1010'
OUTRES='1280x720'

ffmpeg -f x11grab -s "$INRES" -r "$FPS" -i :0.0+$WIN_XY \
    -f alsa -ac 2 -i default -vcodec libx264  -s "$OUTRES"  \
    -acodec libmp3lame -ab 128k -ar 44100 -threads 0 \
    -f flv "rtmp://live.justin.tv/app/$API_KEY"

There's a couple important things to mention about this:

The xwininfo invocation just gives me a cross-hairs cursor so I can select which window area I want to record. This doesn't technically restrict ffmpeg(1) to just that window, rather it just grabs the offset from the window and uses those.
The segment: -f alsa -ac 2 -i default pertains to the audio input. According to arecord -L, the pulse audio sink I should use is called "default", on some machines it might be called "pulse", your mileage may vary.
I use the pavucontrol (GUI) tool to direct my audio out to the pulseaudio input, this allows me to share what I'm listening to with whoever is watching the stream.
You'll notice the $INRES and $OUTRES parameters, I am techically live-resizing the video on the fly, which takes up a lot of CPU power, you may or may not want to do this depending on your machine speed, size of your screen and the amount of desktop space you want to stream.

That's about all there is too it, I wish I could either take credit for the script or at least attribute it, but I can do neither because I found it on a forum some where and then quickly lost the link. Whoops.

You funny guy

2012-03-15T00:00:00-07:00

Earlier this week, my only grandfather died.

He had lived a rather long time, and was ready for a rest, so nothing to really be sad about.

I did want to share some anecdotes about my last visit with him:

Anecdote 1

While my wife and I were hanging out in the hospital room with him, a day nurse came in to take his temperature and blood pressure, standard fare. After gingerly wrapping his left arm and pumping up the arm-squeezer-contraption, the machine fails to make a reading.

The nurse tries again, same result.

She switches to his right arm, tries again and fails to get a blood pressure reading again.

My grandpa deadpans "maybe I'm dead" making the nurse visibly uncomfortable, while shooting my wife and I a wry smile while she's looking away nervously.

Anecdote 2

Grandpa wasn't terribly pleased with the food and level of service at the hospital. He was incredulous after being given diet chocolate pudding, remarking "what the hell is the point?"

On another occasion, after waiting a rather long time to receive a bottle of grape juice from a nurse, he snarked "..did you have to stomp them yourself?"

He made up for his smart-ass remarks with his typical flirting and sweet talking. Having been married three times (outlived two wives, divorced the third), my grandpa had more game on his deathbed than I have had at any point in time.

Anecdote 3

With his mobility drastically reduced, he relied on family members to bring the cup of soda or juice to him when he was thirsty.

After not being able to perform the drink-duty to his satisfaction, he said "as a grandson, you've done great; as a nurse, well..." He then just asked one of my aunts to perform the duty.

Grandpa was a funny guy.

Vagrant plugin for Jenkins in action

2012-03-13T00:00:00-07:00

Since the Jenkins update center is taking a little bit longer to propogate the release of the Vagrant plugin for Jenkins, I figured I'd create a little demonstration for you.

The demo is from a running development environment I have locally, but once the v0.1.2 version of the plugin is available in the update center, all of this will work for you as well.

Introducing the Vagrant plugin for Jenkins

2012-03-13T00:00:00-07:00

Look! A screencast of the plugin in action!

The impossible has happened. Not only have I written a plugin for Jenkins, I've released it. An event I've long avoided has finally come to pass, mostly thanks to the fantastic Ruby plugin development support developed largley by Charles Lowell.

The Vagrant plugin is still fairly new, and "beta" tested. Thus far I've modeled it after the fantastic Android Emulator plugin in that it will bring up a Vagrant VM for the duration of the job, and then destroy it afterwards.

As you can see, the Vagrant plugin adds a couple new build steps to the configure page. One thing to note is that the provisioning is operated as a separate step, instead of bundled in with the boot of the Vagrant machine. This is purposeful, to allow any additional set up of the environment prior to running Puppet or Chef.

The plugin does of course expect that you have a functional VirtualBox environment on the slave you're running the job on, if you don't, Vagrant will explode with a fantastic stack-trace and probably burn your house down.

In the near future

I'm thinking about bundling vagrant-snap with the plugin to allow fancy snapshotting support, but I'm not sure of a good definite usecase for that just yet.

If there's anything you'd like to see, please file a request on the GitHub issues page.

You can find the plugin:

in your Jenkins "Manage Plugins" page :)
on GitHub
on the Jenkins wiki

Enjoy!

NOTE: Be sure to grab version 0.1.2 or later!

Master Logician

2012-02-18T00:00:00-08:00

Shortly before departing for FOSDEM, I dropped my car at the mechanic's to fix an engine issue. Some number of weeks later, I finally was able to pick the vehicle up today, fully repaired and ready to undergo further damage at the hands of yours truly.

Earlier this week I called the shop, after not having heard from them for a couple weeks, wondering what the status of the repair was. The mechanic's response was something along the lines of:

"Oh yeah, we tried to call you but the number was busy or something every time. Anyways, your car needs a new flux capac.."

"Well, I was out of the country" I thought, my line had effectively been disconnected, so I suppose that's what the mechanic had experienced when he tried to contact me. Something still seemed a little fishy to me.

Before hanging up the phone, I asked him what phone number he had for me, just to make sure they could contact me again after they made a few further repairs:

"Uh let's see here, 475-7.."

If you're a bay area resident you can see the problem, the area code is 415, not 475.

Simple enough of a mistake , but what really bothered me about the mistake was that it seems (to me) so obvious or clear to resolve. In fact one simple hypothesis ("this is probably a wrong number") and an experiment ("what if I call 415-etc") would have saved over a week of time spent "in limbo."

Like most city garages, this place is certainly limited for space to store cars that are either waiting to be fixed or waiting to be picked up. I've also been to this mechanic three or four times now, so they've previously been able to reach me. On top of that, area code 475 is in Connecticut of all places; this mechanic is good, but not good enough to justify that kind of commute.

A little skepticism would have cut the amount of time the car spent in the shop from 3 weeks to about 4 days.

I'm clearly obsessing a little too much over this minor inconvenience.

Sometimes I need to be reminded that perhaps everybody doesn't think like me, doesn't question or care about every little detail in the same way that I might.

FOSDEM: Slides from my talk

2012-02-10T00:00:00-08:00

I was fortunate enough to give a talk in the "Configuration and Systems Management dev room" this year at FOSDEM.

After some initial frustrations with LibreOffice Impress, I decided to take a slightly unorthodox route in creating my presentation slides. Unfortunately this route means that I do not have any accompanying notes to offer you alongside the slide deck.

Image courtesy of Nigel Kersten

If a video becomes available, i'll make sure to post that, but as it stands now, I'm not 100% sure what I said!

Slides (PDF) (rotate the slides in your PDF viewer)

Here's a photo of one of my originals, which I had to bring with me just in case. That's real printer paper with real felt-tip pen, yeah.

Image courtesy of Nigel Kersten

Giving the talk was a blast, the whole event was fantastic, I can't recommend it enough. I look forward to coming back to Brussels next year for FOSDEM, and I hope to see you there!

FOSDEM: Smalltalk Pairing

2012-02-09T00:00:00-08:00

As I previously mentioned, I've been learning Smalltalk lately, in an attempt the understand the language that inspired two of my other favorite languages: Objective-C and Ruby.

This past weekend at FOSDEM, I was fortunate enough to attend a few sessions in the "Smalltalk devroom", which from my understanding made its debut appearance at this year's conference. I'll save my thoughts on Amber.js for another time, but in this post I wanted to talk about the Smalltalk Workshop which was held at the end of the day.

One of my major stumbling blocks with Smalltalk has been general unfamiliarity with the development environment, the workshop was the perfect opportunity to resolve some of this. The structure was to pair one experienced Smalltalker with one noob, and for both to work through a pre-planned exercise with an existing image and application set up.

Haxx haxx haxx (Photo courtesy of Adriaan van Os)

While my partner (Norbert) and I did not complete all the exercises, we did spend a good amount of time discussing and working through the "Smalltalk way", or at least Norbert's Smalltalk way, of solving particular problems, refactoring and method structure.

One of the things that struck me as we worked through the exercises was how small the ideal method body is for most things in Smalltalk. I don't think there was a single method that was longer than 10 lines, except for some test methods which certainly had a bad code smell.

Below is a sample of one of the longest methods I wrote in the entire workshop:

vote: aVote for: aUser
  author ~= aUser ifTrue: [
    self votes
      detect: [ :vote | vote author = aUser and: [ vote direction = aVote direction]]
      ifNone: [ self votes add: aVote ]]

I'm quite happy with the way things turned out, while I didn't anything I didn't know already about Seaside, I did learn a lot about using the development environment effectively and test-driven development "the Smalltalk way" which as it turns out is quite impressive.

I'm looking forward to learning more about the ways of the Pharo, maybe next year I'll count as one of the experienced Smalltalkers.

Puppet/XML: A Brand New Synergy

2012-02-08T00:00:00-08:00

During on of the last talks of the last day of FOSDEM, some person asked in one of the tracks (paraphrased):

"Why do we need tools like Puppet or Chef when we have XML?"

This question was relayed to me second-hand, so I apologize if I'm butchering it. The question itself is so mind-boggling that I can only assume one of three things about the asker:

He/she is a rather talented troll
He/she has some crucial misunderstandings about configuration management, in which case I'm hoping somebody politely explained things.
He/she understands CM but has a serious fetish for XML.

Regardless of their intention, I have them to thank for inspiring what might be the innovation breakthrough of 2012:

Puppet/XML

I've long found Puppet's DSL syntax to be too sparse, easy to type and semantically boring, take for example this block:

class users {
    group {
        "tyler" :
            ensure => present;
    }
    user {
        "tyler" :
            require => Group["tyler"],
            ensure  => present;
    }
}

How terrible! Let's turn that into Puppet/XML!

<?xml version="1.0"?>
<puppet>
    <classes>
        <class name="users">
            <resources>
                <group name="tyler">
                    <ensure>present</ensure>
                </group>
                <user name="tyler">
                    <requires>
                        <require>Group["tyler"]</require>
                    </requires>
                    <ensure>present</ensure>
                </user>
            </resources>
        </class>
    </classes>
</puppet>

Not only is that easier to understand, it's more enterprise ready!

Currently Puppet/XML is in pre-alpha and in this repository on GitHub. Thus far it's only been tested with puppet apply (i.e. standalone mode) under Vagrant, an example run is below:

[default] Running Puppet with /tmp/vagrant-puppet/manifests/base.ppx...
stdin: is not a tty
Thanks for using Puppet/XML - A brand new synergy

info: Retrieving plugin
info: Applying configuration version '1328719113'
notice: /Stage[main]/Users/User[tyler]/ensure: created
notice: /Stage[main]//Node[default]/Group[puppet]/ensure: created
info: Creating state file /var/lib/puppet/state/state.yaml
notice: Finished catalog run in 0.62 seconds

Stay tuned for new innovations and more middleware that your enterprise can leverage to help meet your business objectives.

Disclaimer: Unfortunately the code currently relies on transcribing the Puppet/XML to Puppet's "native" DSL, this means more complex things like conditionals and function calls are not yet supported. After spending about an hour and a half trying to generate the appropriate Puppet::AST structures, I gave up and went the route which guaranteed lulz sooner.

Happy 1st Birthday Jenkins!

2012-02-02T00:00:00-08:00

It's officially been a year since the first release of Jenkins went public, and my what a year.

I'm writing this quick post while waiting at the airport to go to FOSDEM to talk about Jenkins and some of the work that we've been doing. This comes two weeks after going to SCALE10x to advocate and talk to people about Jenkins there.

I am immensely proud of the work everybody in the Jenkins community has done, and happy to be a part of one of the most welcoming groups of open source enthusiasts that I've seen yet.

Looking forward to way year number two brings for the project!

Speaking Sunday at FOSDEM

2012-01-31T00:00:00-08:00

I meant to post this earlier, but the schedule for the "Configuration and Systems Management" devroom at FOSDEM 2012 has been posted.

My talk will be Sunday at 13:00 in Room K.3.601.

If you're looking for a meaty technical discussion about Puppet this talk will likely not be for you.

I'm aiming to tell a story more than anything about how our use of Puppet came to be and has evolved in building out open source project infrastructure.

I hope my talk ends up as light-hearted and entertaining as I'm trying to make it, in the off chance it doesn't, feel free to heckle and throw fruit.

Real talk (about Smalltalk)

2012-01-14T00:00:00-08:00

Somehow I found myself reading an interesting article on Smalltalk yesterday which led to a couple of comments in IRC:

tyler:  man, I am getting that "I should really learn smalltalk"
        feeling again
tyler:  everytime I get close I find myself downloading a squeak
        VM and then going "LOLWTF IS THIS SHIT"

I said this before I left the office and somewhere in the back of my mind I kept thinking "why can't I crack Smalltalk?" This continued through most of the evening until 9 pm (21:00) rolled around and I decided that I wouldn't let that commie Alan Kay defeat me.

After finding this fantastic series of screencasts I set about learning some Smalltalk. Not enough to build anything major, but enough to really understand Smalltalk's core concepts.

In exploring the world of Squeak, Pharo and Amber I believe I have experienced the full spectrum of emotions that my circuits provide, all the way from unfettered rage to mild bemusement. Despite all of this, around 2 a.m. I found myself lying awake in bed wanting to marvelling at the design of the language and wanting to learn more.

A couple of things still bug the hell out of me:

The VM concept while novel is a completely jarring experience for anybody who has done any programming in any other environment ever.
I hate mice. I also have no trackpad on my laptop which means if i'm not at my desk using a mouse-heavy program becomes painful (quite literally). Both Squeak and Pharo have this sick mouse fetish that makes me want to （╯°□°）╯︵┻━┻
GNU/Smalltalk feels closer to development environments that I'm familiar with, but VisualGST is incredibly buggy and I think part of what makes Smalltalk special is the integrated "live environment."

The things that I do really like however:

The integrated "live environment" is something out of programmer fairy tales. Inspect/debug/edit anything and everything? Yes please!
Everything is an object which receives messages, the syntax for conditionals helps drive it home. A boolean object can receive an ifTrue or an ifFalse message which takes a block argument. This means there is no real for if statements, it's all implemented with the basic Smalltalk primitives:
```
  1 < 3
      ifTrue:  [Transcript show: 'Truthy!']
      ifFalse: [Transcript show: 'Untruthy!']
```
How about a for loop? Surely the language must have a for loop! No way José! Instead an Array object just takes a do message with a block, fancy!
```
  #('tom' 'dick' 'harry') do:
      [ :each | Transcript show: ('Hello ', each); cr]
```
No magic, that I've seen thus far. One of my biggest criticisms of Ruby comes out of the Rails tradition of magic methods by using method_missing to magically generate methods on-demand, leaving a system that's abominably difficult to inspect and debug. Thus far I've not found any slight-of-hand in Smalltalk, which is a good thing.

I can really see where Objective-C pulled from Smalltalk having now experienced some Smalltalk myself. That said, it's rather unfortunate that C and some additional custom bits have creeped into Objective-C making it less Smalltalky than it once was.

I am definitely going to continue to tinker with Smalltalk, likely with Pharo as my VM of choice. My only concern is that I will grow to resent the other languages I use regularly which feel dumber than a language invented almost 40 years ago.

Update: This is the original post that pushed me over the edge on tinkering with Smalltalk.

And now a word about the Jenkins DNS debacle

2011-12-29T00:00:00-08:00

Now that things have settled down, I'd like to address what happened during the jenkins-ci.org DNS outage I discussed here.

Long story short, some poor assumptions caused a lot of trouble while switching registrars away from GoDaddy.

First let me explain the motivations for moving from GoDaddy: aside from the SOPA issue, I've always hated dealing with GoDaddy. The have had an absolutely shit product for a long time, but as a long time customer, momentum was quite a limiting factor. Before my involvement with the Jenkins infrastructure, I would have only used GoDaddy once or twice a year.

Since becoming the driving force behind a lot of the Jenkins back-end, I've had to deal with them once or twice a month, and everytime I would be bombarded with upsells, ads and other shit that I wanted no part in. With the renewal of jenkins-ci.org coming up in January, it makes sense to switch registrars before I renew the domain for 5 years (which I will be doing).

The SOPA boycott is just gravy on top.

As for the "DNS debacle", here's a quick bulleted list of what happened:

The Negative

Jenkins used GoDaddy's DNS service, when the GoDaddy side of the domain transfer was complete they canceled the DNS service and seemingly zeroed out all our DNS records.
The jenkins-ci.org nameservers, which are generally copied over when switching registrars, were set to GoDaddy's nameservers prior to the transfer (mistake on my part).
What I had assumed would automatically take effect once the transfer was complete on Namecheap's side, did not take effect. In fact, somehow the data I had entered in was obliterated when GoDaddy's nameservers were copied over.
I'm not good at writing zone files, and the most subtle mistakes were easy to slip past those who I asked to review them. Zone files are the devil.

Instead of focusing too much on the colossal screw-up, I'd like to focus more on what positive has come out of the situation.

The Positive

jenkins-ci.org is now with a registrar that isn't commonly accepted to be "freakin' terrible"
To mitigate the downtime people were experiencing, I quickly set up what I presumed would be a temporary nameserver on one of our machines colocated with the OSUOSL.
Instead of taking that nameserver down, I've actually promoted it to become ns1.jenkins-ci.org which is wholly powered by our Puppet manifests. Now anybody in the community can submit a pull request to add records to our zone file and we can approve and deploy it all from GitHub via Puppet.
While working on DNS, I noticed that iptables(8) wasn't properly configured on some of our hosts. We've since bundled the firewall module from Puppet Labs in our repository and set up the appropriate rules on all hosts to lock them down.

I could vent about the idiocy of the DNS system, how much I abhor GoDaddy, or a myriad of other issues that had to be resolved in these past two days, but that wouldn't be very useful.

Despite the amount of time consumed in the affair, I think the project comes out ahead and stronger, and at the end of the day that's what is really important to me.

Pulling Jenkins' strings with Puppet

2011-12-28T00:00:00-08:00

A couple months ago I created this puppet-jenkins module while experimenting with using Puppet to script or otherwise control more and more of my daily sysadmin-life.

After a weekend of hacking I figured I was mostly finished, that was until Puppet Labs engineer Jeff McCune picked up the module and added a number of rspec tests, CentOS/RHEL support and generally tidied up the place. Pretty neat I thought!

Once again through the power of GitHub a weekend project has taken a life of its own now that multiple people are interested in the concept and have forked the repo accordingly.

With the encouragement of a number of people, I've also published the module to Puppet Forge which allows for easy integration with thousands of Puppet installations.

If you'd like to use the Jenkins puppet module from Puppet Forge, you can use:

% -> puppet-module install rtyler-jenkins

Once the module is included you can slurp in the Jenkins module with this in your manifests:

class { 'jenkins': ; }  # logically equivalent to "include jenkins"

You can also use the module to handle plugin installation for you with the install-jenkins-plugin type:

install-jenkins-plugin {
    "git-plugin" :
        name    => "git,
        version => "1.1.11";
}

(Omit the "version" if you just want to pull the latest)

If you're interested in contributing, the GitHub project is here.

Add this to the ever increasing list of projects that were planted on GitHub and flourished afterward.

How is Ruby built?

2011-12-26T00:00:00-08:00

Note: This is a rant, I'm not actually going to explain anything of use

For a rather long time, I've used clang on my Linux machine as the default value of the $CC environment variable. I've simply found it to be a faster and more friendly compiler to work with than the default gcc.

When I first came to Ruby, I tried building Ruby 1.8.7 with clang and it failed utterly. After bitching on Twitter, as I'm prone to doing, Joe Damato pointed out that Ruby 1.8 isn't really valid C. Instead it is a hodge podge of macros, C code, assembly and tears.

I later found out that Ruby 1.9 seems to cope with being built with clang properly, so I built a couple of 1.9 variants with RVM for my personal projects.

Everything was all fine and good until I tried building a native extension for a 1.9 Ruby:

make
compiling em.cpp
cc1plus: warning: command line option ‘-Wdeclaration-after-statement’ is valid for C/ObjC but not for C++ [enabled by default]
cc1plus: error: unrecognized command line option ‘-Wshorten-64-to-32’
cc1plus: warning: command line option ‘-Wimplicit-function-declaration’ is valid for C/ObjC but not for C++ [enabled by default]
make: *** [em.o] Error 1

I lost hours trying to figure out what the hell was going on here. I was convinced for hours that EventMachine (the gem being built here) had a bug in it. Then I was convinced that RVM had a bug in it, but I finally landed on Ruby's build system itself.

I should point out that the warning shorten-64-to-32 only exists in patched versions of compilers shipped with Mac OS X. It certainly doesn't exist on any compiler on Linux, so where the hell did this flag come from?

Somehow, and arriving at this "somehow" took me a number of miserable hours, the difference between using clang(1) and gcc(1) to build Ruby resulted in a single nonsense CXXFLAGS value which didn't come back to bite me until I tried to build a native extension.

How the shit does Ruby get built!?

Identi.ca is dead to me

2011-12-23T00:00:00-08:00

There was a time when I was a big fan and user of open source microblogging site Identi.ca. I regret to inform you that this time has passed and I can no longer in good conscience support the platform.

Identi.ca has always had issues, to be expected of an open source web property, there are bound to some rough edges running around. Things took a turn for the worse when the team decided to perform a major upgrade which took the site offline for an entire weekend. The major upgrade in mid-September introduced a number of issues:

Entirely new UI which improved things in some areas (more AJAX) but introduced whole new UX flows, heavily padded rounded UI elements and new hiding places for elements (Groups) that I used to use heavily (why the christ would you put the Groups that I admin behind three clicks and not have them in the stupid sidebar? Idiocy).
Took the XMPP bot offline for interacting with Identi.ca. This was often the only way I interacted with the site!
Data integrity issues
A number of stability issues and bugs

I would go on, but my mouth is already foaming at an unhealthy rate so I better stop before I get kicked out of this coffee shop.

Anyways, it is of my opinion that the folks behind made a classic blunder: the major re-write (joining Netscape 5, Perl 6, Python 3, and numerous other over-promising projects that falter in the "real world"). Eschewing the evolutionary approach to a project means large changes in data structure or user experience can and will be made before anybody has seen them operate with "real world" data and users.

I'm not sure where this leaves me with regards to using Identi.ca, I want to like the site so badly but every tiem I've tried to use it in the past three months I end up terribly frustrated.

Extra bonus rant: If you ever try to use the mobile site, you'll have a taste of how silly Identi.ca can be sometimes. If the site doesn't detect that your browser is mobile (which happens on every device for me), then you have to load the full non-mobile home page, at the bottom of which is a "Switch to mobile desktop layout" link. The link doesn't have an href, it uses fucking JavaScript, clicking this link will reload the page with a special mobile cookie set. There is no m.identi.ca site at all.

To use Identi.ca from a mobile browser: your browser must run JavaScripts, you must load the full non-mobile document, click that link, set the cookie then hope that your browser keeps that cookie around forever.

Uh oh, my mouth is foaming up again, better stop here.