Gluttonous

Microsoft to Aquire Powerset

kev — Wed, 02 Jul 2008 02:55:00 -0400

There’s details here. Sounds like they’re keeping us intact, and there’s something to be said about having an additional 10k machines behind you.

We’ll see what happens.

Holy God, Powerset Launches

kev — Mon, 12 May 2008 02:21:00 -0400

We’re out, searching Wikipedia and FreeBase. Proud doesn’t begin to describe my feelings about what’s been created. I think it speaks for itself, and I’ve been using it instead of Wikipedia for the last few months.

Some people even seem to really get it. And that’s an amazing high.

Huge thanks are not only due to the Powerset team (I love working with these people), but also to all of the open source projects we’re making use of. Hadoop. HBase. Thrift. Ruby. Merb. Rails. god. Mongrel. Mootools. RabbitMQ. The ActiveMessaging Project. Memcache and MemcacheDB. Erlang. Fuzed. YAWS. Countless others. All those people who take the time to answer our questions, and respond to our bugs, and consider our patches, and write interesting articles, and make our code better. You guys rock, and we couldn’t do it without you. So much love is heading in your direction right now. Thank you, thank you, thank you.

I’m exausted, and going to bed. Good night, and good luck. And try the app. I’m stoked.

TechCrunch's Twitter Article Translated from Arrington-Speak

kev — Wed, 23 Apr 2008 12:19:00 -0400

With apologies to Gruber.

Original text from TechCrunch.

It doesn’t really matter if Twitter’s Chief Architect Blaine Cook was fired or resigned. The important thing is that he’s gone now, and this gives Twitter the opportunity to hire someone (or a team) who may actually be able to scale the nearly two year old service and keep it live.

I haven’t done any research or spoken to anyone about it, so don’t ask, but I have found a scapegoat.

Cook was directly responsible for scaling Twitter, and he very much failed in his job.

Twitter is down sometimes, and I’m angry about it.

A year ago he spoke at the Silicon Valley Ruby Conference about scaling Rails applications. His presentation suggested Twitter’s problems were behind them, but in fact some of their biggest stumbles hadn’t occurred yet. Note in particular slide 9 of that presentation, where Cook says about scaling Rails apps like Twitter: “It’s Easy. Really.”

I found a posted slide from a public conference during my “investigative reporting phase. Oh, and Twitter is on Rails, and I can blame Rails. It’s Easy. Really.

Whether Twitter’s woes were all on Cook’s shoulders or not, he should not have been boasting about solving the problem last year.

I’m high as a kite.

Meanwhile, Twitter has made at least three key hires this year on the technical side. Lee Mighdoll joined as VP Engineering and Operations in January. And this week they hired two scaling experts - John Kalucki and Steve Jenson (”known for his work scaling Blogger and Blogspot”).

And they hired someone, yada yada yada, sprinkle random facts in so nobody notices my complete incompetence. Perfect!

Really Simple Git Deployment with Vlad

kev — Sat, 05 Apr 2008 19:13:00 -0500

Just drop this in your Rakefile. This is slightly modified from something I’m using in production.

Disect! Enjoy! Explanation (read: spoilers) after the jump.

##############################################################################
# Deploy
##############################################################################

begin
  require 'rake_remote_task'

  APP_NAME = "someapp"
  DEPLOY_ROOT = "/usr/local/share/applications/#{APP_NAME}"
  ON_DEPLOY_RESTART = ["someappd"]

  role :app_server, "myserver.com"

  def archive
    commit = `git-rev-list --max-count=1 --abbrev=10 --abbrev-commit HEAD`.chomp
    file = "#{APP_NAME}-#{commit}.tar.gz"
  end

  def restart_daemons
    ON_DEPLOY_RESTART.each do |app|
      run "sudo god restart #{app}"
    end
  end

  namespace :deploy do
    task :build do
      sh "git archive --format=tar HEAD | gzip > #{archive}"
    end

    remote_task :push => :build do
      rsync archive, "/tmp"
    end

    desc "Install a release from the latest commit"
    remote_task :install => :push do
      date_stamp = Time.now.strftime("%Y%m%d")
      last_release = run("ls #{DEPLOY_ROOT}/rels | sort -r | head -n 1").chomp

      if last_release =~ /#{date_stamp}\-(\d+)/
        serial = $1.to_i + 1
      else
        serial = 0 
      end

      rel = ("%d-%02d" % [date_stamp, serial])
      rel_dir = "#{DEPLOY_ROOT}/rels/#{rel}"

      run "sudo mkdir -p #{rel_dir}"
      run "sudo tar -xzvf /tmp/#{archive} -C #{rel_dir} && rm -rf /tmp/#{archive}"
      run "sudo ln -s -f -T #{rel_dir} #{DEPLOY_ROOT}/current"
      restart_daemons
    end

    desc "Rollback to the previous release"
    remote_task :rollback do
      current_link = run("ls -alF #{DEPLOY_ROOT} | awk '/current -> .*/ { print $NF }'").chomp
      current = File.basename(current_link)
      releases = run("ls #{DEPLOY_ROOT}/rels | sort -r").split("\n")
      previous = releases.find {|rel| current > rel}
      raise "No previous release" if previous.nil?
      run "sudo ln -s -f -T #{DEPLOY_ROOT}/rels/#{previous} #{DEPLOY_ROOT}/current"
      restart_daemons
      puts "Moved to #{previous}"
    end

    desc "Rollforward to the next release"
    remote_task :rollforward do
      current_link = run("ls -alF #{DEPLOY_ROOT} | awk '/current -> .*/ { print $NF }'").chomp
      current = File.basename(current_link)
      releases = run("ls #{DEPLOY_ROOT}/rels | sort -r").split("\n")
      next_rel = releases.find {|rel| current < rel}
      raise "No next release" if next_rel.nil?
      run "sudo ln -s -f -T #{DEPLOY_ROOT}/rels/#{next_rel} #{DEPLOY_ROOT}/current"
      restart_daemons
      puts "Moved to #{next_rel}"
    end
  end
rescue LoadError => e
  puts "NOTE: Install vlad to get Kevin's awesome deployment tasks"
end

postgresql gem on Leopard stock gem system

kev — Sat, 22 Dec 2007 20:24:00 -0500

This fails horribly.

The solution is to make sure you’re only building for your architecture:

Odysseus:ext kev$ sudo -s
bash-3.2# ARCHFLAGS='-arch i386' gem install postgres
Building native extensions.  This could take a while...
Successfully installed postgres-0.7.9.2007.12.22
Installing ri documentation for postgres-0.7.9.2007.12.22...
Installing RDoc documentation for postgres-0.7.9.2007.12.22...

Rubinius Runs Mongrel

kev — Mon, 10 Dec 2007 04:40:00 -0500

Reposted from my message to rubinius-dev. Congrats to the whole Rubinius team. This was entirely a group effort, and one hell of an achievement.

Here's the first Mongrel handler running on Rubinius:

http://pastie.caboo.se/paste/asset/126441/Picture_4.png

From this code:

$:.unshift "/Users/kev/code/mongrel/mongrel-1.1.1/lib"

puts "Requiring mongrel"
require 'mongrel'

class HelloHandler < Mongrel::HttpHandler
 def process(request, response)
   response.start(200) do |head, out|
     head["Content-Type"] = "text/html"
     out.write "Hello World! I'm running on Rubinius!"
   end
 end
end

server = Mongrel::HttpServer.new("0.0.0.0", 3000)
puts "Started Server"
server.register("/hello", HelloHandler.new)
puts "Registered handler"
t = server.run
t.join

***THE CATCH (as this may be viewed by many people)***

This isn't completely complete. rb_global_variable was #define'd out
to do nothing (so no garbage collection on the global vars), and there
was a slight modification from the trunk to make global aliasing
ignore the fact that the globals just weren't there. Mongrel's
http11.c was also _slightly (very very slightly)_ modified to use the
rb_str_get_char_* methods we've decided to move to from RSTRING()->ptr
and RSTRING()->len, and I haven't gotten around to defining ALLOC_N
yet, so it was changed to a simple malloc. That's it though.

And it seems to run. And I feel like I need to run around the block.

It's in 9976301ba.

WOOOOOOOOOOOOOOOOOOOOO!

Secret Incantations

kev — Mon, 10 Dec 2007 01:03:00 -0500

Install the do_postgres gem against postgresql82 on MacPorts:

Make sure that /opt/local/lib/postgresql82/bin/ is in your path. You need pg_config easily accessible. Then run:

sudo gem install do_postgres -- --with-pgsql-include-dir=/opt/local/include/postgresql82/ --with-pgsql-lib=/opt/local/lib/postgresql82/

Autotest with Rspec on Merb with a Leopard install using the supplied Ruby (whew)

This will break because it can’t find the “spec command”. It searches the configured bin directory, which with the supplied ruby is /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/bin.

ln -s /usr/bin/spec /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/bin/spec

Depth First Search

kev — Tue, 27 Nov 2007 14:24:00 -0500

Because I hadn’t implemented DFS in Ruby before, and it’s just so damn easy.

Update: Phillip rightly pointed out in the comments that with the yield at the end, it’s actually post-order traversal, not depth first search per se.

class TreeNode
  attr_reader :name

  def initialize(name)
    @name = name
    @children = []
  end

  def add_node(node)
    @children << node
  end

  def each_depth_first
    @children.each do |child|
      child.each_depth_first do |c|
        yield c
      end
    end

    yield self
  end
end

# root  -  a  -  b
#   \       \ 
#    e - f   c  -  d
#     \
#      g

root = TreeNode.new("root")
root.add_node( a = TreeNode.new("a"))
a.add_node( b = TreeNode.new("b"))
a.add_node( c = TreeNode.new("c"))
c.add_node( d = TreeNode.new("d"))
root.add_node(e = TreeNode.new("e"))
e.add_node(f = TreeNode.new("f"))
e.add_node(g = TreeNode.new("g"))

root.each_depth_first do |child|
  puts child.name
end

# produces:
# b
# d
# c
# a
# f
# g
# e
# root

This is too entertaining not to share

kev — Thu, 08 Nov 2007 21:08:00 -0500

I’ve apparently “hacked” someone’s unborn children. Or something.

And, among the lists of names, includes someone who goes by the name of Kevin Clark. And, if you were to take it even one step further, and INVESTIGATE this person, you would also come to the conclusion that he is a computer hacker who resides (or has resided) in San Francisco before. Another coincidence, I suppose?

Update: She says in one of her latest comments that she has a gun now. It’s amusing, but please do not poke/provoke her. Look, but do not touch.

Update 2: She’s still accusing me of multiple felonies, but seems to have leveled off. I don’t think she’s going to track me down at this point. Woo personal safety. Oh, and technorati is apparently involved now:

I believe Mr. Clark is somehow routing fake websites through technorati via powerset and is doing something illegal.

13 New SD Ruby Podcasts Up

kev — Sat, 03 Nov 2007 10:04:00 -0500

They’re all up at once. Wow.

Mine felt good, but it’s long. Rather long. 50 minutes fairly non-stop, ~600 megs long. Find some time before watching.

Episode 036: The Return of Kevin Clark
Kevin Clark takes a break from Powerset to give a full-throttle talk
on using Merb as a JSON-RPC service, god, gem2rpm, and heckle.

Episode 035: ActiveRecord Backup & MimetypeFu
Matt Aimonetti demonstrates his newest plugins: ActiveRecord Backup
and MimetypeFu.

Episode 034: Intro to JRuby
Brian Chapados shows how to install and work with the latest JRuby
release.

Episode 033: Life on Edge
If you’re a Rails junkie, you’ll want to develop on Edge Rails. Matt
Clark explains how to get started and shares some of the challenges
of working on Edge.

Episode 032: Capistrano
Rob Kaufman takes on Capistrano 2. What is it? How does it work?
What’s changed since version 1?

Episode 031: Seaside
Roger Whitney explores Seaside, the web application framework based
on Smalltalk.

Episode 030: Tuneshelf
Dominic Damian talks about his experiences building Tuneshelf, a web
application that allows music fans to keep track of their favorite
music albums.

Episode 029: Big Stinking Piles (of data)
What do you do when third-party data vendors don’t speak REST? Rob
Kaufman discuss real-world techniques for importing and exporting
data. (This talk was also given at RailsConf 2007.)

Episode 028: Simple Sidebar Plugin
Ryan Felton shows how to use Simple Sidebar plugin to DRY up sidebar
content in applications.

Episode 027: Headliner and Styler
Patrick Crowley talks about his newest plugins: Headliner and Styler.

Episode 026: ActsAsSolr
Rob Kaufman shows how easy it is to integrate Solr powered search
into your Rails application using the ActsAsSolr plugin.

Episode 025: Ajax CSS Star Rating with ActsAsRateable
Ryan Felton shows off how to build an Ajax-powered, CSS star rater
using the ActsAsRateable plugin and Komodo Media’s CSS Star Rating
Redux technique.

Episode 024: Using Ruby + Amazon SQS to build backdoors
Brian Chapados talks about using Ruby and Amazon’s Simple Que Service
web service to build backdoors into systems.