Blog.

Lift

Tue, 23 Aug 2011 16:27:22 -0700

Great news today! I am joining my long-time friend and colleague, Tony Stubblebine, to co-found a new company, Lift.

We’re working in a space we love – human potential – in a unique partnership with Obvious, founded by Evan Williams, Biz Stone, and Jason Goldman. There will be more to share in the coming months. If you’re interested, please let us know by signing up for the upcoming beta.

Snarl - A JavaScript Bridge for JRuby

Sun, 08 Nov 2009 19:36:52 -0800

If you would like to use JavaScript directly from Ruby, check out Snarl. I just pushed its first public release to Gemcutter. Snarl is JRuby-only, as it uses Mozilla’s Rhino JavaScript engine to do its work.

First, make sure you’re using Gemcutter:

gem install gemcutter
gem tumble

Then, install and use:

gem install snarl
irb
> context = Snarl::JavascriptContext.new
> context.eval(“1+1”) # => 2
> context.eval(“var increment = function(i) { return i+1 };”)
> context.eval(“increment(3)”) # => 4
> context.load(“/path/to/js/lib.js”) # allows use of lib.js
> context.put(“foo”, “bar”)
> context.get(“foo”) # => “bar”

Golden Gate Ruby Conference: CloudKit Video and Slides

Thu, 04 Jun 2009 22:09:31 -0700

In April of this year, I had the pleasure of presenting CloudKit at the first Golden Gate Ruby Conference here in San Francisco. The videos are now online thanks to Pivotal Labs. If you’re interested in CloudKit, REST, Discovery, or the Open Stack, please take a moment to check it out.

Slides are also available on SlideShare and a brief article on Composable REST Services is available in the wrap-up PDF.

CloudKit 0.11.2 Released

Tue, 05 May 2009 17:55:24 -0700

A patch release for CloudKit, version 0.11.2, has been released. This is a recommended upgrade for those using the 0.11.x series of gems.

From the release notes:

Added Location header for 201s
Fixed JSON response for DELETE operations
Updated MD5 dependency for Ruby 1.9 (Andreas Haller)
Updated spec setup for RSpec 1.2 (Andreas Haller)
Updated gem dependencies for Rack, OpenID, and JSON
Other minor fixes

The next planned release is 0.12 with JSONQuery support (already available client-side in the JQuery plugin). If you are interested in working on this, see the jsonquery branch on GitHub.

Thank you to everyone for the feedback and patches.

JSONQuery, New API in CloudKit jQuery Plugin

Wed, 08 Apr 2009 23:56:30 -0700

Upcoming JSONQuery support was recently announced for CloudKit and there is now good news to share about the progress.

A No-Dependency JSONQuery Library for JavaScript

Thanks to Kris Zyp and Jason Smith, a no-dependency JavaScript library for JSONQuery is available on GitHub. I am maintaining my own fork of this library with bug fixes and a large set of tests. If you’ve been curious about JSONQuery but wanted some concrete examples of usage, there are now over 300 lines of tests and examples to browse on my GitHub fork.

JSONQuery in the CloudKit jQuery Plugin

The latest version of the jQuery plugin for CloudKit now supports JSONQuery and includes an improved API. The original version of the plugin used TaffyDB. While TaffyDB is a quality library, the promise of a standard querying mechanism for JSON via JSONQuery was too powerful to ignore.

jQuery.data Inside

Under the hood, jquery.cloudkit.js uses jQuery.data as its local in-browser store now that TaffyDB is no longer required. jQuery.data is nothing new, but for those unfamiliar, the quick summary is that it provides a key/value store that can be attached to any DOM element.

This is useful in the case of CloudKit where a top-level store for each resource collection is attached to the window element.

The New API

The basics of setting up your local in-browser database mapped to a CloudKit service are the same; you can still point the library at the metadata URL and it will configure and load itself. Each RESTful collection in CloudKit is still exposed as its own store. Beyond this, the API has been altered to provide a better interface, delegating work to the resources themselves instead of filtering all work through the outer collection. This new philosophy more closely matches that of CloudKit::Resource on the server.

Here are some examples from the README:

JSONQuery in CloudKit

On the server side, things are moving quickly thanks to Johnson, a JavaScript bridge for Ruby. Johnson vendors SpiderMonkey to perform its work which means the exact same query API that is available in your browser is also available in CloudKit. Interested hackers can build a current gem for Johnson and use it with the jsonquery branch of CloudKit on GitHub today. These updates will be merged into the master branch for the next release.

CloudKit 0.11.1 Released

Tue, 24 Mar 2009 21:07:43 -0700

CloudKit 0.11.1 has just been released. This is a maintenance update for the 0.11.x series of gems. It is a recommended upgrade, containing the following changes (pulled straight from the change log):

Added a block option for configuring OpenID bypassed routes (Devlin Daley)
Added write locks for Tokyo Tyrant Tables
Added Tokyo Tyrant Table example
Fixed POST method tunneling bug (Saimon Moore)
Fixed escaping of nested JSON Objects and Arrays

You can track CloudKit in these places:

The Site
GitHub
The Mailing List
Twitter
#cloudkit on Freenode

Thanks to all who provided feedback and patches for this release.

Video for Rack Middleware Presentation Available

Wed, 18 Mar 2009 18:01:18 -0700

The Confreaks crew is lightning fast. The video from my Rack Middleware presentation at MountainWest is now available, just days after the event.

Rack Middleware Slides from MountainWest RubyConf 2009

Tue, 17 Mar 2009 01:20:33 -0700

MountainWest RubyConf was an incredibly good experience this year. There was a high signal-to-noise ratio with great speakers stacked up on both days. I consider myself the least of these speakers, but nevertheless have posted my slides for anyone who may be interested.

In a World of Middleware, Who Needs Monolithic Applications?

Some of the slides won’t convey all of the intended meaning without the audio, but that should be reconciled once Confreaks posts the videos on their site. I’ll link again when those appear for those who are interested.

Beyond the main conference track, the people in attendance really made the event special. There were many great conversations between talks and especially at the Engine Yard HackFest, which was amazingly good. At $100, the conference was a steal and one that I’ll never forget.

JSON Meets Tokyo Cabinet (and More!): CloudKit 0.11.0 Released

Mon, 09 Mar 2009 01:30:53 -0700

CloudKit 0.11.0 is out with exciting new features!

Tokyo Cabinet

CloudKit now supports Tokyo Cabinet as its storage system. If you haven’t checked out Tokyo Cabinet, take a moment to read Ilya Grigorik’s overview or check out its main site. You will find it to be an extremely fast and compelling storage system.

While Tokyo Cabinet is impressive, it comes with the cost of a binary dependency. Although it took only a moment to install it on my Mac using John Mettraux’s simple instructions, I didn't want to sacrifice CloudKit’s install-and-go development experience. So, I added a pure Ruby store that mimics the Tokyo Table API (as exposed to CloudKit via Rufus::Tokyo). This is the default store used by CloudKit until you specify otherwise. See the README for an example of using the native Tokyo Table backend when you’re ready.

Making the switch to Tokyo Cabinet paves the way for the final significant piece of CloudKit’s architecture – JSONQuery support. With super-fast storage, a discoverable API, built-in Open Web auth (using OpenID and OAuth), and fully searchable data with JSONQuery, CloudKit will cover a wide range of use cases. Not only this, but it fits in the Rack stack, so it can be used alongside your other favorite frameworks and middleware.

CloudKit::Resource

Unsatisfied with “alongside,” CloudKit has made the move to “inside” too. Prior to this release, the only interface to CloudKit was through its HTTP/REST web service. With the 0.11 release, you can also use it in a manner similar to DataMapper or ActiveRecord with one small difference; instead of having to map arbitrary models to SQL, CloudKit provides a single Resource model that represents a JSON document and all of its metadata (URI, ETag, Last-Modified date, etc.). There are more details in the CloudKit docs, but here are a few examples:

JQuery Plugin

Still early in development, a working jQuery plugin is available for CloudKit. The plugin provides an in-browser queryable JSON database that automatically loads and configures itself using the metadata provided by CloudKit. For every resource collection you are hosting with CloudKit, the plugin will expose a local mirror that amounts to a queryable JSON array (implemented using TaffyDB patched for asynchronous operation). Updating your local data is simple and is synchronized with the matching web service automatically.

Here are some examples from the README:

Clouder: A Ruby API for CloudKit

For across-the-network access to CloudKit using Ruby, Demetrius Nunes has put together a library called Clouder. This use case matches the way many are using CouchDB as an ORM backend. If you’re interested in using CloudKit in this manner, be sure to fork Demetrius’s project on GitHub and contribute.

IRC

Beyond the existing site, GitHub repository, Twitter, and mailing list, there is now #cloudkit on Freenode for interested hackers. Feel free to drop in and participate in the next release or ask questions about using CloudKit in your own projects.

CloudKit 0.10.1 Released

Tue, 27 Jan 2009 21:58:53 -0800

A new 0.10.1 CloudKit gem has been released. This is the first release with patches from the growing community on GitHub. As always, a big thank you to everyone who sent feedback or code.

To install:

gem install cloudkit

To update:

gem update cloudkit

From the change log:

Updated oauth and sequel gem dependencies
Fixed 410 responses for stale PUT operations
Fixed MySQL content encoding (Harry Weppner)
Various fixes for filter_merge!, rekey!, and excluding (Cameron Walters)

You can track CloudKit in these places:

A World of Middleware

Thu, 22 Jan 2009 17:14:28 -0800

What if every endpoint (or “origin” in HTTP-speak) was a 404 and everything else was middleware? In Rack-land, this can be a reality. CloudKit, for example, detects if it is the last thing in the stack, inserting a default developer page at the root and a 404 everywhere else. If it is not the last item in the stack, requests for resources that it has not claimed are passed downstream.

If you consider that no web framework or piece of middleware is all-knowing, then this behavior seems to be more flexible than that of the pre-Rack view of the world where the URI space was “everything” by default.

This Gist from Jesse Newland shows a quick hack to make Sinatra behave as middleware for Rack:

There is no reason that Rails and other frameworks should not behave similarly, allowing themselves to be yet another component in a chain leading to the eventual 404 at the end of the stack. I struggle to see the distinction between middleware and “web apps” as anything other than arbitrary.

This concept and a few others provide the idea behind my upcoming talk “In a World of Middleware, Who Needs Monolithic Applications?” at MountainWest RubyConf in March of this year. If you haven’t registered, take a moment to browse the list of speakers and see if there is a way to get away from work for a long weekend.

CloudKit 0.10.0 Released: Rack 0.9, Batch URI Resolution

Fri, 09 Jan 2009 16:45:36 -0800

A new 0.10.0 CloudKit gem is out.

To install:

gem install cloudkit

To update:

gem update cloudkit

This was originally going to be 0.9.2 with support for this week’s Rack 0.9 release. However, the new batch URI resolution in the REST API was too valuable to hold back for the 1.0 release so it was pulled in and the minor version number was incremented to reflect its presence. This release contains no breaking API changes.

The exact details of the new feature are in the updated REST API document and cURL tutorial on the site. I will attempt to summarize with an embedded cURL log:

As always, thank you for the feedback on GitHub, Twitter, the mailing list, and elsewhere.

OAuth, Phishing, and Twitter

Mon, 05 Jan 2009 02:33:16 -0800

In the midst of what might seem like a crisis when viewed from within the web’s echo chamber, it can be difficult to sort fact from fiction, threat from hysteria. Such was the case last weekend when three terms that typically evoke passionate responses were combined in a single meme – OAuth, Phishing, and Twitter.

The source of concern was a scam spreading on Twitter via fake direct messages. These messages mimicked Twitter’s own opt-in email notifications that go out when users of the service receive a direct message. You can read the details on Twitter’s blog, released in a responsible and timely fashion during the incident. Engineers no doubt worked overtime to mitigate this while the rest of us enjoyed holiday leftovers and more relaxing weekends.

It didn’t take long for some to try and link the incident to Twitter’s lack of OAuth support in their API. Some, such as Chris Messina, clearly understood the distinction but others seemed to blur the line. This post is an attempt to demonstrate the orthogonality of OAuth and phishing with regard to this specific incident.

Phishing Defined

Let’s start with a precise definition of phishing, taken from Wikipedia and quoted on Twitter’s own blog covering the incident:

“Phishing is the criminally fraudulent process of attempting to acquire sensitive information such as usernames, passwords and credit card details by masquerading as a trustworthy entity in an electronic communication.”

If your inbox looks like mine, phishing attempts are nothing new. Links typically contain misleading subdomains, misspelled top level domains, and other tactics in an attempt to establish enough trust to get a click, after which a fake site mirroring the original attempts to collect the sensitive information.

This was the tactic employed by the scammers over the weekend in their attack against Twitter users and it is almost completely unrelated to OAuth or lack thereof in Twitter’s API. The end-to-end flow of receiving an email, clicking a link, and logging into what looks like a legitimate site does not change when using OAuth. This exact attack is equally possible for sites using OAuth.

Authentication vs. Authorization

OAuth provides a method of secure API authorization where as the act of logging into a site is the act of authentication. For sites that use OAuth, users normally must authenticate prior to authorizing third party API access. These are two separate steps. The difference between authentication and authorization isn’t mere pedantry; it is critical to the understanding of OAuth and its place along side complementary technologies such as session-based logins or OpenID.

Remember that authentication answers more of the “Who am I?” question, sometimes merely validating a claim such as “I have control of this URL,” as is the case with OpenID. Authorization handles the “I, having been authenticated, am authorized to perform this action” part of the equation.

OAuth allows scoped API access (a la valet keys) to user data. The user remains in control, having the ability to disable a particular third party’s access to data while leaving others intact. Service providers can take this as far as they want, scoping each OAuth token to specific actions rather than an all-or-nothing approach. Users under this system have no reason to give their passwords to any third parties and sites can message strongly against it without worries of curtailing API usage and its related marketing benefits.

Tying (or rather, not tying) this back to Twitter, the phishing scam was an attack to gain authentication credentials for Twitter users and had nothing to do with authorization, the domain of OAuth.

CloudKit 0.9.1 Released

Wed, 31 Dec 2008 02:40:35 -0800

A new CloudKit 0.9.1 gem has been released. Thank you to everyone who sent feedback via the mailing list, GitHub, Twitter, and personal email.

To install:

gem install cloudkit

To update:

gem update cloudkit

From the change log:

Fixed Rack::Lint/rackup errors related to Content-Type headers
Patched Rack to support StringIO#string in Rack::Lint::InputWrapper
Fixed server_url encoding in OpenIDStore
Added sqlite3-ruby dependency in gemspec
Updated documentation

Release: CloudKit 0.9 - An Open Web JSON Appliance

Mon, 22 Dec 2008 06:07:11 -0800

How many lines of code does it take to deploy a fully discoverable, RESTful, GET-optimized, auto-versioned, JSON API?

Two. First, install the gem:

CloudKit is Rack middleware, so let’s create a rackup file called config.ru like this:

Fire it up with Thin, for example:

You now have a running JSON container managing collections of “notes” and “projects.” For the full REST API spec, see the CloudKit site. Don’t miss the curl tutorial while you’re there.

Let’s say we want to add both OpenID and OAuth support to this API. How many more lines of code?

Zero. Simply change “expose” to “contain” and you’re ready to go:

CloudKit is built around HTTP and JSON for the purpose of building efficient APIs quickly. It’s a bit like CouchDB with baked-in Open Web auth plus the entire spectrum of Rack middleware at its disposal. The automatic version history for each JSON document is provided as an aid for decentralized or occasionally-connected clients, allowing a progressive diff/merge against history to “catch up” in the case of conflicts.

Thanks to Rack, you can run CloudKit on its own or alongside other Rack-based apps or middleware components such as Rails, Merb, or Sinatra. Any requests outside of the named collection scopes or authentication endpoints are passed along to the next piece in the stack.

If you’re interested in hacking, the source is on GitHub.

Rack::Config 0.9 Released

Thu, 27 Nov 2008 23:54:49 -0800

Smaller than its own README, Rack::Config is a straightforward method of configuring a stack of Rack middleware that needs to share information.

To install:

Example config.ru:

"JSCocoa bridges Cocoa to JavascriptCore (WebKit’s JS engine). It allows you to call C code,..."

Thu, 09 Oct 2008 12:24:18 -0700

“JSCocoa bridges Cocoa to JavascriptCore (WebKit’s JS engine). It allows you to call C code, ObjC code, use C structs, and build Javascript classes inheriting from ObjC classes.”

- JSCocoa — A bridge from JavascriptCore to Cocoa

"However, the NDA has created too much of a burden on developers, authors and others interested in..."

Wed, 01 Oct 2008 10:54:51 -0700

“However, the NDA has created too much of a burden on developers, authors and others interested in helping further the iPhone’s success, so we are dropping it for released software.”

- iPhone Developer Program

"SquirrelFish Extreme runs JavaScript quite a bit faster than the original SquirrelFish. It’s..."

Mon, 08 Sep 2008 00:27:50 -0700

“SquirrelFish Extreme runs JavaScript quite a bit faster than the original SquirrelFish. It’s available in Mac and Windows nightlies and you can take it for a spin yourself.”

- [webkit-dev] SquirrelFish Extreme?

Peace, Hope & Surveilance (via get directly down)

Mon, 01 Sep 2008 01:06:49 -0700

Peace, Hope & Surveilance (via get directly down)