Kris Jordan Blog

Micro PHP's Missing Tenet

Tue, 10 Jan 2012 01:52:07 -0500

Ed Finkler's post on "MicroPHP Manifesto" made the the rounds last week. It expresses a widely felt desire lurking in the minds of PHP developers. Especially those who have spent time working with libraries from the Node.js or clojure communities. While the manifesto did a good job covering motivations and aspirations, it missed a key tenet of "micro", and stopped short of covering a few key practical opportunities and obstacles with PHP.

"I believe functions are the smallest unit of functionality"

The missing tenet from the Micro PHP Manifesto is: I believe functions are the smallest unit of functionality.

Not classes. Not methods. Functions. This belief fits in naturally with, and is a precursor to, most every other tenet of the manifesto.

"I like building small things with simple purposes." There is no smaller or simpler reusable unit of functionality than a function.

"I like building small things that work together to solve larger problems." Functions and their higher-order friends compose with mathematical simplicity and beauty.

"I want code that is easily verifiable." Pure functions are the easiest unit of functionality to test.

"I need to justify every piece of code I add to a project." Do we really need to subclass Controller to add a route to our app? Nope, not if you're using Slim, a great HTTP micro framework (full disclosure [1], my office mate wrote it). An anonymous function will do just fine, thank you. Senseless subclassing is what got us into this mess to begin with! [2]

When the traditional PHP framework programmer looks at a problem, he asks himself these questions, in order of preference:

Can I add a method to an existing class?
Can I add a subclass and override a method?
Or should I start a new base class?

When the Micro PHP framework programmer looks at a problem, she asks herself these questions:

Can I use an anonymous function?
Can I use a named function?
Can I add a method to an existing class?
Can I add a subclass and override a method?
Or should I start a new base class?

Namespaced functions and function values being legitimate options in 5.3 enables lighter weight libraries that play together nicely. Combined with jQuery pushing web developers' collective comfort with a more functional style of programming, the micro PHP movement is in a great position to take off.

But, for the love of Rasmus, PHP has avoided solving two fundamental problems that would radically inspire even smaller, even simpler, even more single-purposed, micro PHP libraries.

Damnit PHP, why can't you just...

1. Allow functions to be imported like classes

User defined functions are second rate compared to user defined classes in the PHP language. You can import classes from namespaces but you can't import functions. Read that again. You can import namespaced classes but not namespaced functions. PHP itself doesn't fully believe that "functions are the smallest unit of functionality". Oh, sweet irony. Most of PHP's core functionality (even array manipulation!) is exposed as functions (in a global namespace).

PHP discourages writing plain old functions by making them more painful to reuse than classes. Currently, the best way to write useable "plain old functions" is to write methods on a class you'll only ever invoke statically (i.e., underscore.php's __::each). If PHP respected functions they wouldn't need to be wrapped in bastardized classes to be useful.

2. Indoctrinate a symbol loading algorithm

So it isn't possible to import namespaced functions directly. But it gets worse. Even using what is available (use namespace; namespace\func();), if the function has not yet been defined, there is no way to autoload that function's source like PHP will do for classes. But it gets worse. The class loading algorithm is user defined. This is another historical impediment to "micro" component compatibility in PHP, because every framework has its own class loader [3]. There is hope in PSR-0, but it won't ship with 5.4.

PHP needs to plant a hard foot in the ground and indoctrinate one built-in, automatic symbol loading algorithm that will load classes and functions and constants. The chaos has gone on long enough. If Python and node.js can be bold enough to do it, so can PHP. The flexibility has stunted our growth.

Please, if you are listening PHP Core Team, for the love of the fat blue elephant in the room, do whatever it takes and break whatever you need to get these fundamental language utilities in the next major point release. Or even the mythical 6.0. The temporary pain will be worth the massive, revitalizing shock it will give to a community of PHP developers who would like to share components and code in a simple and standard way.

How should micro libraries share plain functions until PHP fixes importing and symbol loading?

Step 1. Use a PSR-0 class loader There are plenty. The Symfony2 ClassLoader's is good and PSR-0 compliant. It's MIT. Potencier writes great code. Use it.

Step 2. Write plain-old functions as static methods on a bastard, wrapper class. Ugh. Step 2 is the worst, but distributing plain-old functions puts consumers back in require_once hell. If all you are trying to write and share are plain-old functions, use static methods.

Step 3. Write to your local PHP Core politician. Demand importable functions and a native, standard symbol loader for PHP.next. Support this PHP RFC for function autoloading, and this one for PSR-0 inclusion.

Finally, will an isaacs please step up? PHP needs a modern package manager.

If someone really wants to blow oxygen on the Micro PHP fire they should start with a new package manager for PHP. PEAR is far too bureaucratic and, frankly, embarrassing in the face of a modern package manager like node.js's npm. PEAR has been around since before PHP5's release in 2004 and there are 179 modules hosted at pear.php.net for 5.0+. (And only 19 for poor PEAR2 and PHP 5.3) Contrast with npm which has been around since August 2010 and hosts 6,157 packages.

npm hosts over 30 times more packages than PEAR!?! How? You don't have to pass a bureaucratic, written proposal process to publish to npm. Anyone can publish any package they want in under 10 minutes. So do all npm packages have the high quality of PEAR packages? No. Do they all follow the same coding standards like PEAR packages? No. Are these real problems to fear in practice? No. The best packages are naturally high quality and obtain the most attention and dependencies. PEAR is selection by committee, npm is natural selection. The Micro PHP movement really needs a package manager with fewer, smaller rules for publishing and sharing. PEAR may be for sharing small packages with simple purposes, but it is certainly not Micro PHP.

(Update: some promising movements in this direction are Packagist and Phark.)

tl;dr

Functions are the smallest unit of functionality. This is a key missing tenet of the Micro PHP Manifesto. It's a practical precursor to many of the others. This isn't to say object-oriented libraries and designs don't have their place in PHP, they most certainly do, but that if you can solve your design problem with functions, choose functions. Prefer small, simple functions over complex, stateful class hierarchies.

Footnotes

[1] Even fuller disclosure: I've written a "Full Stack" PHP Framework. A lot of what Ed said is wrong with Zend and Symfony was wrong with Recess, which I wrote three years ago. It's intention was to be light weight. In terms of CLOC, it was, under 10k for a RESTful HTTP layer, ORM, "Views", & Annotations. In terms of code written by end-users, it was, comparatively, too. Recess was the first PHP framework to use Annotations. Declarative-style programming requires less code. I don't use it for new projects anymore. It's still too heavy. It's not line counts or ugly code, though. It caries the PHP 5.2 OOP-only baggage, too [2].

[2] Ed's post stops short of tracing the legacy leading to the current state of popular frameworks being un-micro. Here's the story. When all you know are objects...all you see are classes. And methods. And private variables. And design patterns. And Java. When all you know is Java all you see is Java.

PHP did not become be one of the most popular kids in school without having a "hey look, me too" mindset. PHP grew its popularity in the early 2000s by copying everything it could from Java's object model in the move from PHP4 to PHP5. From visibility controls to reflection.

When your language is inspired by Java, your libraries and frameworks will be too. You can't look at Recess' annotations without smelling Java. You can't look at Doctrine without smelling Java. You can't look at Symfony components without smelling Java. This isn't to say Symfony is bad, it's not. It's really damned good, but it's manifesto is different. It's manifesto was written by a Gang of Four. It's lineage predates PHP 5.3. Popular frameworks and libraries originating before PHP 5.3's release carry the burden of pursuing best practice object-oriented patterns at the cost of ignoring perfect opportunities where standalone or anonymous functions would suffice.

[3] PEAR/Zend-style class naming and directory structuring became somewhat of a norm before 5.3, but it sucked. Class names were combined with namespaces which lead to instantiating classes with names like Zend_Db_Table_Definition. Only in recent history, since 5.3 namespaces, has the community embraced a "standard" class loading algorithm with PSR-0. (Design by committee retained PEAR-style support, which is unfortunate, but I digress.) In theory this will make its way into the Standard PHP Library and be bundled with PHP, which will be great.

multimethod.js

Thu, 15 Dec 2011 17:40:00 -0500

What is a multimethod?

The quickest way to understand is by looking at a simple, familiar example implemented with a multimethod. So let's write a recursive Fibonacci function.

> var fib = multimethod()
                .when( 0, 0 )
                .when( 1, 1 )
                .default( function(n) {
                    return fib(n-1) + fib(n-2);
                });
> fib(20);
6765

Multimethods are a functional programming "control structure" enabling you to dynamically build-up and manipulate the dispatching behavior of a polymorphic function. Inspired by Clojure's multimethods, multimethod.js provides a functional alternative to classical, prototype based polymorphism. The multimethod.js library is MIT licensed, is found on GitHub and npm with npm install multimethod, and its annotated source code is available here.

Motivating Examples

Let's use the node.js REPL to build a few multimethods and see what they are capable of doing. In this first example we'll create a mulimethod that calculates the area of shapes instantiated with object literals.

> var multimethod = require('multimethod');
> var area = multimethod()
                .dispatch(function(o) {
                    return o.shape;
                })
                .when("square", function(o) {
                    return Math.pow(o.side, 2);
                });
> var aSquare = { "shape":"square", "side": 2 };
> area( aSquare );
4

> var aCircle = { "shape":"circle", "radius": 5 };
> area( aCircle );
undefined

> area.default(function(o) { 
    throw "Unknown shape: " + o.shape;
  });
> area( aCircle );
Unknown Shape: circle

> area.when("circle", function(o) {
    return Math.PI * Math.pow(o.radius, 2);
  });
> area( aCircle );
78.53981633974483
> area( aSquare );
4

> area.remove("circle");
> area( aCircle );
Unknown Shape: circle

Notice how dispatch returns the value we'll match to a "method" registered with when. You can introduce, overwrite, and remove new methods dynamically at runtime. Fallback behavior can be established with a default function called when no methods match the dispatched value.

> var hitPoints = multimethod()
                    .dispatch(function(player){ return player.powerUp; })
                    .when( {"type":"star"} , Infinity)
                    .default(5);

> var starPower = { "type":"star" },
>     mario = { "powerUp": starPower };
> hitPoints(mario);
Infinity

> mario.powerUp = null;
> hitPoints(mario);
5

> var godModeCheat = function() { return starPower; };
> hitPoints.dispatch(godModeCheat);
> mario.powerUp;
null
> hitPoints(mario);
Infinity

In this last example notice how we are matching against an object. Matching is done using deep equality so objects and arrays are valid method matching criteria. Also notice how we can completely override our dispatch function to change the logic with which a multimethod evaluates its arguments for dispatch, or, in this case, ignores them!

Multimethod Dispatch Algorithm

User calls multimethod with argument anArgument.
Multimethod calls its dispatch function with anArgument. The returned value is stored in dispatchValue.
Multimethod iterates through each 'method' registered with when and performs an equality test on the dispatchValue and each method's match value. If a match is found, set matchFunction to the method's function and go to step 5.
If no method match found, set matchFunction to the multimethod's default function.
Multimethod calls matchFunction with anArgument. The returned value is returned to the user who called the multimethod.

A More Detailed Walkthrough

The Basics

A multimethod is instantiated with the multimethod function.

var stopLightColor = multimethod();

A multimethod has methods. A method is has two parts, its match value and its implementation function. Methods are added using when.

stopLightColor.when("go",    function() { return "green"; })
              .when("stop",  function() { return "red"; });

You can call a multimethod just like any other function. It will dispatch based on the argument(s) passed in, invoke the matched method, and return the results of the method call.

console.log( stopLightColor("go") ); // "green"

When no method matches control can fallback to a default method.

stopLightColor.default( function() { return "unknown"; } );
console.log( stopLightColor("yield") ); // prints "unknown"

A multimethod can handle new cases dynamically at run time.

stopLightColor.when("yield", function() { return "yellow"; });

There is a shorter way for a method to return a plain value. Rather than passing an implementation function to when, pass the value.

stopLightColor.when("yield", "yellow");
console.log( stopLightColor("yield") ); // prints "yellow"

A method can be removed dynamically at run time, too.

stopLightColor.remove("go");
console.log( stopLightColor("go") ); // prints "unknown"

Dispatch Function

Each multimethod call first invokes a dispatch function whose return value is used to match the correct method to call. The dispatch function is passed the arguments the multimethod is invoked with and returns a value to match against.

The default dispatch function is an identity function. The basic stopLightColor examples could have been created with an explicit dispatch function.

var stopLightColor = multimethod()
      .dispatch(function(state){
         return state;
      })
      .when('go', 'green');
console.log( stopLightColor('go') ); // green

The power of the multimethod paradigm is the ability to dispatch with a user-defined function. This gives a multimethod its "polymorphic" powers. Unlike classical, object-oriented polymorphism where the compiler dispatches based on the type hierarchy, a multimethod can dispatch on any criteria.

var contacts = [
  {"name":"Jack", "service":"Twitter","handle": "@jack"},
  {"name":"Diane","service":"Email",  "address":"d@g.com"},
  {"name":"John", "service":"Phone",  "number": "919-919-9191"}
];

var sendMessage = multimethod()
     .dispatch(function(contact, msg) {
       return contact.service;
     })
     .when("Twitter", function(contact, msg) {
       console.log("Tweet @"+contact.handle+":"+msg);
     })
     .when("Email", function(contact, msg) {
       console.log("Emailing "+contact.address+":"+msg);
     })
     .default(function(contact, msg) {
       console.log("Could not message " + contact.name);
     });

// Blast a message
contacts.forEach( function(contact) {
  sendMessage(contact, "Hello, world."); 
});

Plucking a single property from an object is so commonly used as a dispatch function, like in the example above, there is a shortcut for this pattern. The following dispatch call is equivalent to above.

sendMessage.dispatch( 'service' );

A multimethod's dispatch is usually specified when constructed.

var sendMessage = multimethod('service');

Just like methods can be added and removed from a multimethod at run time, the dispatch function can also be redefined at run time. Ponder the implications of that for a minute. It is really powerful and really dangerous. Don't shoot your eye out.

Deep Equality Matching

A method's match value is compared to dispatch's return value using the underscore.js isEqual function. Deep equality method matching enables concise expressivity. Contrast this with a traditional switch statement that is limited by JavaScript's === equality behavior.

var greatPairs = multimethod()
      .when( ["Salt", "Pepper"], "Shakers" )
      .when( [{"name":"Bonnie"}, {"name":"Clyde"}], "Robbers" );
console.log( greatPairs( ["Salt", "Pepper"] ) ); // Shakers

API Recap

Constructor: multimethod( [fn | string] ): No arg constructor uses an identity function for dispatch. Single arg constructor is a shortcut for calling dispatch with the same argument.
dispatch(fn | string): Sets the multimethod's dispatch function. String values are transformed into a pluck function which projects a single property value from the first argurment.
when(match, fn | value): Add a method to be invoked when the dispatch return value matches 'match'. If a non-function value is provided it will be returned directly. Calling when with the same match value twice will override the previously registered method.
remove(match): Remove a method by it's match value.
default(fn | value): Catch-all case when no method match is found.

Installation

Install with npm for use in node.js based projects.

npm install multimethod
node
> var multimethod = require('multimethod');

For in-browser use you will need to grab underscore.js and multimethod.js:

underscore.js
- Development: http://documentcloud.github.com/underscore/underscore.js
- Minified: http://documentcloud.github.com/underscore/underscore-min.js
multimethod.js
- Development: https://raw.github.com/KrisJordan/multimethod-js/master/multimethod.js
- Minified: https://raw.github.com/KrisJordan/multimethod-js/master/multimethod-min.js

How-to Contribute

Submit bugs and feature requests on GitHub Issues page.
Fork the repository and submit pull requests. Pull requests that update the test suite for coverage on changes will be brought in quickly.

Fixing WebKit's Accept Header

Thu, 10 Mar 2011 21:25:56 -0500

WebKit has been on the forefront of web browser innovation for years, but their HTTP Accept header has lagged behind. (tl;dr it is fixed now.) The Accept header is how the web browser says to the web server "these are the content-types I can understand and the order in which I prefer them." For more detail, wrote about it in more detail on the HiFi CMS Blog.

The Accept header allows a single object or resource to have a single URI with multiple representations, such as HTML, XML, or JSON. It is a fundamental component of the HTTP spec and REST style web services. Unfortunately, today's web services rarely use content-negotiation, because browsers have historically chosen bad Accept headers. WebKit's was the last serious offender:

application/xml,
application/xhtml+xml,
text/html;q=0.9,
text/plain;q=0.8,
image/png,
*/*;q=0.5

Parsed and prioritized:

1. application/xml
2. application/xhtml+xml
3. image/png
4. text/html
5. text/plain
6. */*

WebKit's Accept header preferred XML over HTML prior to my fix landing in trunk today. Web services could not offer both HTML and XML for a single resource and use content negotiation because WebKit based browsers (Safari, Chrome, et. al.) would be served the XML data not the HTML web page. Developers have had to hack around the issue. Twitter's API, for example, uses a common work around that changes the URI by appending a format extension.

http://api.twitter.com/version/statuses/friends_timeline.format
format ::= json | xml | rss | atom

The goal of the fix for WebKit's Accept header was straightforward: prefer HTML over XML. The ultimate solution was to minimize risk and take Firefox's lead. Here is the Accept header, already in order of preference, WebKit-based browsers will be using in the future:

text/html,
application/xhtml+xml,
application/xml;q=0.9,
*.*;q=0.8

Contributing to Open Source Software

The fix may have been committed today, but the initial bug report was filed in July 2009, almost two years ago! What took such a simple, but important, fix so long to be made? I would like to say it was the need for five other people to chime in on the ticket, someone on the WebKit team to notice, or a duplicate bug request to be filed 15,000 tickets later. But no, all the WebKit team needed was for someone, anyone, to submit a patch. So I checked out the subversion repository, made a patch, submitted it, and boom, I have contributed code to WebKit and fixed its Accept header woes. Very cool, but I feel horrible for not doing so with the original bug report in 2009; content negotiation would be more usable today.

Lesson learned: no matter how big or small an open source project is, when you find a bug and know the fix just do it. Do not assume code to be sacred or untouchable; just fix it and let the committers make the call. Bug reports are helpful, but patches and pull requests are gold. Discussions get read, code gets committed. Code speaks louder than words.

Refactoring Cache Tables using Memoize

Sun, 06 Mar 2011 20:51:33 -0500

Have you ever written code that feels like this?

var byKey = function (key) {
	... do slow, expensive stuff ...
	return { value: “something expensive” };
};

And later optimized it to feel like this?

var lookup = {};
var byKey = function (key) {
    if ( lookup[key] === undefined ) {
        ... do slow, expensive stuff ...
        lookup[key] = { name: “something expensive” };
    }
    return lookup[key];
};

I certainly have. The lookup table is a trusty pattern for caching the results of expensive calls. With four additional lines of code you can dramatically improve the performance of an often called function. Sprinkle these lines around your most expensive, referentially transparent functions and you have a nicely optimized program, right?

With a functional language this pattern is considered harmful; an anti-pattern to refactor. It may be four lines of code, but it is redundant when used over and over. Your function’s intentions become conflated with performance concerns. It would be best to keep our intentions and optimizations separate.

Can we simply write “cache this function’s return values”? It would feel like this,

var byKey = cache( function(key) {
    ... do expensive stuff ...
    return { name: “something expensive” };
} );

That feels much better. Although, not knowing what the cache function actually does is a little off-putting. Let’s implement it.

If you are unfamiliar with anonymous functions and closures in JavaScript this is about to get slightly hairy. Understanding those concepts will make writing JavaScript much more interesting, and little blog posts like this more palatable. I suggest Crockford, but there are plenty of good on-line resources, too.

We need our cache function to take any function we throw at it and return a new function. The returned function must accomplish what our imperative lookup table code did: remember the arguments of previous invocations and cache their return values.

var cache = function( expensiveFn ) {
    var lookup = { };
    return function(key) {
        if(lookup[key] === undefined) {
            lookup[key] = expensiveFn(key);
        }
        return lookup[key];
    };
};

Let's walk through this. We have defined cache as a function that takes expensiveFn as a parameter. Within this function we setup the variable lookup to be our hash/cache table. Then we return a function that encapsulates the lookup table pattern used earlier. Do you see how the structure has been extracted and the state hidden within the closure?

In computer science this function is conventionally named memoize. It is often demonstrated using a recursive function like factorial. Factorial is a more beautiful application of memoization than basic key-value caching, but this is an anti-patterm you are more likely to find in real code.

An Exercise for the Reader

The most obvious shortcoming to cache, our implementation of memoize, is that it does not generalize to functions of more than one argument. If you have a few minutes, pop open your browser’s JavaScript console in Firebug or Webkit Inspector and try writing it. Hints: 1) The arguments keyword provides an array of all arguments a function is called with. 2) A function can be invoked with an array of arguments by using Function's apply method.

Refactor one-off caching via lookup tables with `memoize`

Plenty of libraries provide a generic memoize function, including underscore in JavaScript. The next time you catch yourself writing a naive in-memory cache lookup table around a function stop and consider memoizing it instead. Using memoize is faster to write and less bug-prone than a one-off cache, it separates intentional concerns from optimization concerns, and it keeps your code a little cleaner.

On Muted Methods, Fluent Interfaces, and Bar Game

Mon, 24 Aug 2009 23:47:02 -0400

One night a weekend back I went out to the local bar back home with my younger sister some of our friends. Back home means Rutherfordton, North Carolina. The kind of charming area that greets you at the county line with the road sign "Welcome to Rutherford County, Small Town Friendly". A friendly place, it is.

While sitting at the bar one of my sisters' friends drew a lot of attention from the gentlemen of Rutherford County. We'll call her Liz (gah, when does 30Rock start up again??). Some would come up and chat, others ask for a dance. The most peculiar case of them all was one who had been ordering drinks from across the bar and finally got up the courage to come over and, without a word, slip Liz a piece of paper with his phone number on it before disappearing into the night.

Yes, this post is about programming, bear with me.

Let's give this guy, say Dennis, the benefit of the doubt and assume that these tactics have worked before with success. On this particular night with a gorgeous girl from out of town: no dice. Dennis wouldn't get the callback.

Let's pseudo-code this scenario:

$liz = new AttractiveOutOfTownGirl();
$dennis = new SmallTownFriendlyYoungMan();
$liz->setDrink($dennis->buyDrink('Vodka Redbull'));
$liz->setDrink($dennis->buyDrink('Sex on the Beach'));
$liz->receiveNumber($dennis->getPagerNumber());
$dennis->disappearIntoTheNight();

Spot any patterns? Thrice Dennis has taken an action that results in some value: he's bought 2 drinks and retrieved his pager number. All of his methods have resulted in some value that is handed to Liz and he's getting no acknowledgement back. He interacted with muted methods. Dennis' shot at success depended on Liz having some internal, finite state machine that transitioned from "stranger" to "stranger willing to call a stranger" with every additional drink. How could this have gone better for Dennis?

Before we get there let's talk about muted methods. Procedures. Voids. Black holes in programs where input results in no output. Scary stuff! But why, what's so bad about muted functions? Let's nitpick our example. It is so choppy. Do you feel the choppiness?

Noun verb.
Noun verb.
Noun verb.
Liz set drink.
Liz set drink.
Liz give number.

When your verb is muted, there's nothing more to be said. End of story. Next line. You've got nothing to play off of, no chance to converse. When you're working with muted functions you have no segue. You're giving without receiving.

If muted functions are the problem, how could this scene look a little less ugly? Perhaps Dennis could have conversed with Liz. Interacted with her. Made use of a more fluent interface...

$dennis->saysTo($liz, "Hey girl, from out of town?") // returns $liz
       ->saysTo($dennis, "Yes, how'd you know?") // returns $dennis
       ->saysTo($liz, "Never did seen you before!") // returns $liz
       ->isStillSmiling()
       ? // If she's smiling
   $liz->setDrink($dennis->buyDrink('Vodka Redbull')) // returns $liz
       ->saysTo($dennis, "Thanks, you're kind of cute.")
       ->saysTo($liz, "That's how we're grown here. Can I call you?")
       ->saysTo($dennis, "Sure, my name is Liz and the number is...")
       ->punchesInNumber("919-928-9090") // returns $dennis
       ->says("Great, call you tomorrow!") // returns $dennis
       ->disappearIntoTheNight()
       : // If she's frowning
   $dennis->movesAlong();

When you're not dealing with muted methods you give your code the chance to come to life in a more fluid, fluent way. There's only sentences here, it's just longer, richer, and more involved. There's magical, programmatic banter happening here!

This style of object-oriented interface, where method chaining and thoughtful return values enable a pseudo-domain specific language written entirely in a general purpose language, has been called a "Fluent Interface" by Martin Fowler and Eric Evans. It's an inspiring departure from the days of the void, black hole methods like setters.

So, next time you're writing an interface for some new class, or sitting at a bar wondering how best to approach that attractive member of the opposite sex, avoid muted methods at all costs. They're black holes. They're just awkward, feedbackless, and overall uncomfortable for everyone involved.

When your functions or methods don't return ask yourself "if I were to return something here what would be useful?" Most often, it's a reference to the object itself. Or an immutable clone.

"Oh, god no, not an immutable clone! I thought this article was about bar game?" Hmm... you're right. I'll save the immutable clone wars for another post. You should subscribe to this RSS feed if immutable clones pique your interest. Or nerdy bar game.

setDrink($this->buyDrink('Vodka Redbull'));

What's on my mind...

Thu, 20 Aug 2009 22:39:50 -0400

In an hour I'll be out doing late night trivia in a bar in Chapel Hill, North Carolina. Pop trivia has never been something I excel at, but with the right company it's fun to wager a guess, or two, and let go of the problems that consume idle cycles. As of late my thought has been switching between any of the following:

Using virtual machines and shell scripts to achieve the Ultimate Small Team Software Engineering Environment (tm)
Real Software Engineering in PHP
Trees, Recursion, Permissions, and Architecture of my company's next Content Management System
How to simplify internals and increase functionality of the Recess Framework by leveraging PHP 5.3's new features like anonymous functions and namespaces
A big client integration project that deals with scary acronyms like HIPAA

This space should be more lively than the ghosttown that it has become over the last 8 months. More off the cuff writing on the topic of the hour than the more structured pieces I've been writing for New Media's Web Development Blog or the Recess PHP Blog.

To kick that off the off the cuff writing, a few words on _why. I don't know _why, I haven't read the poignant guide to Ruby (although I've come across it a number of occasions and been intrigued), nor have I used much of his huge body of open source work. None the less his recent disappearance has been sitting really funny on my stomach. Really funny. This guy is prolithic, talented, and obsessed with teaching the art and joy of programming. I didn't fully appreciate _why until watching his recent talk at Carnegie Melon [embedded below]. His excitement and creative knack for opening programming up to a younger audience is energizing. Here's to hoping an even brighter chapter to that story is about to unfold...

Recess! Framework Screencast #1

Tue, 16 Dec 2008 22:34:44 -0500

I'm excited to get the first Recess! screencast out on the interwebs over at RecessFramework.org. Special thanks to Joel Sutherland and the Raleigh Web Design firm New Media Campaigns for helping me launch the Recess site in record time!

Without further adieu... Routing in Recess!

Towards RESTful PHP - 5 Basic Tips

Tue, 02 Dec 2008 01:33:57 -0500

What is REST?

REST is an architectural style, or set of conventions, for web applications and services that centers itself around resource manipulation and the HTTP spec. Web apps have traditionally ignored the HTTP spec and moved forward using a subset of the protocol: GET and POST, 200 OKs and 404 NOT FOUNDs. As we entered a programmable web of applications with APIs the decision to ignore HTTP gave us problems we're still dealing with today. We have an internet full of applications with different interfaces (GET /user/1/delete vs. POST /user/delete {id=1}). With REST we can say /user/1 is a resource and use the HTTP DELETE verb to delete it. For more detail on REST check out wikipedia and "quick pitch".

Tip #1: Using PUT and DELETE methods

In PHP you can determine which HTTP method was used with: $_SERVER['REQUEST_METHOD']; From web browsers this will be either GET or POST. For RESTful clients applications need to support PUT and DELETE (and ideally OPTIONS, etc.) as well. Unfortunately PHP doesn't have $_PUT and $_DELETE variables like it does $_POST and $_GET. Here's how to access the content of a PUT request in PHP:

$_PUT  = array();
if($_SERVER['REQUEST_METHOD'] == 'PUT') {
    $putdata = file_get_contents('php://input');
    $exploded = explode('&', $putdata); 

    foreach($exploded as $pair) {
        $item = explode('=', $pair);
        if(count($item) == 2) {
            $_PUT[urldecode($item[0])] = urldecode($item[1]);
        }
    }
}

Tip #2: Send Custom HTTP/1.1 Headers

PHP's header function allows custom HTTP headers to be sent to the client. The HTTP/1.x header contains the response code from the server. PHP will, by default, send back a 200 OK status code which suggests that the request has succeeded even if it has die()'ed or a new resource has been created. There are two ways to change the status code of your response:

header('HTTP/1.1 404 Not Found');
/* OR */
header('Location: http://www.foo.com/bar', true, 201); // 201 CREATED

The first line is a generic way of setting the response status code. If your response requires another header, like the Location header to the resource of a '201 Created' or '301 Moved Permanently', placing the integer status code in the third parameter of header is a shortcut. It is the logical equivalent of the following example, which is easier to read at the cost of being an extra line of code.

header('HTTP/1.1 201 Created');
header('Location: http://www.foo.com/bar');

Tip #3: Send Meaningful HTTP Headers

Policy for deciding when it is appropriate to send each HTTP status code is a full post on its own and the HTTP spec leaves room for ambiguity. There are many other resources on the net which provide insights so I'll just touch on a few.

201 Created is used when a new resource has been created. It should include a Location header which specifies the URL for the resource (i.e. books/1). The inclusion of a location header does not automatically forward the client to the resource, rather, 201 Created responses should include an entity (message body) which lists the location of the resource.

202 Accepted allows the server to tell the client "yeah, we heard your order, we'll get to it soon." Think the Twitter API on a busy day. Where 201 Created implies the resource has been created before a response returns, 202 Accepted implies the request is ok and in a queue somewhere.

304 Not Modified in conjunction with caching and conditional GET requests (requests with If-Modified-Since / If-None-Match headers) allows web applications to say "the content hasn't changed, continue using the cached version" without having to re-render and send the cached content down the pipe.

401 Unauthorized should be used when attempting to access a resource which requires authentication credentials the request does not carry. This is used in conjunction with www-authentication.

500 Internal Server Error is better than OK when your PHP script dies or reaches an exception.

In the Recess! Framework I use this StatusCodes class to provide named constants for all HTTP/1.1 status codes. Example usage:

header(StatusCodes::httpHeaderFor(StatusCodes::HTTP_NOT_FOUND));

Tip #4: Don't Use $_SESSION

A truly RESTful PHP application should be entirely stateless- all requests should contain enough information to be handled without additional server side state. In practice this means storing authentication information in a cookie with a timestamp and a checksum. Additional data can also be stored in a cookie. In the event you need more than a cookie's worth of data fall back to storing it in a central database with the authentication still in the cookie. This is how Flickr approaches statelessness.

Tip #5: Test with cURL or rest-client

cURL makes it easy to execute any HTTP METHOD on a resource URL. You can pass request parameters and headers as well as inspect response headers and data. The command line tool 'curl' is standard on many *nix distros. Windows users should check out MinGW/MSYS which supports cURL. Even PHP has cURL functions which are enabled on most hosts (php/curl install page). cURL Example Usage & Common Parameters:

# curl -X PUT http://www.foo.com/bar/1 -d "some=var" -d "other=var2" -H "Accept: text/json" -I

-X [METHOD] Specify the HTTP method. -d "name=value" Set a POST/PUT field name and value. -H [HEADER] Set a header. -I Only display response's headers.

Alternatively, a free GUI to test REST interfaces is Java/Swing based rest-client. rest-client is scriptable and has support for JSON/XML.

Tip #6 - Use a RESTful PHP Framework

Frankly, developers shouldn't have to worry about many of these low-level details of REST when writing PHP apps. REST is based on conventions and conventions, by nature, involve a lot of boilerplate. This is right up a framework's alley (as Rails has shown). What options exist for PHP? CodeIgniter's routing completely ignores the HTTP METHOD so there is serious hacking that needs to be done. Cake has REST support but wasn't designed to make specifying useful response status codes a part of the framework. Konstruct appears to have a very well thought out architecture for a controllers framework built around HTTP and REST. Unfortunately it is not easily approached and lacking (intentionally) many components, like an ORM layer, developers have come to expect in a modern web framework.

(Disclaimer: Shameless Plug!) The lack of a solid, RESTful PHP framework was one of my primary motivations for creating the Recess! Framework. Recess is a full-stack, open source (MIT), RESTful PHP framework. If you're interested in writing RESTful PHP applications check it out and sign-up to be notified of its upcoming release.

PHP Class for HTTP Response Status Codes

Mon, 01 Dec 2008 19:37:35 -0500

The Recess! Framework uses the following class which provides constants and simple helper methods for HTTP Response Status Codes.

<?php
/**
 * StatusCodes provides named constants for
 * HTTP protocol status codes. Written for the
 * Recess Framework (http://www.recessframework.com/)
 * 
 * @author Kris Jordan
 * @license MIT 
 * @package recess.http
 */
class StatusCodes {
	// [Informational 1xx]
	const HTTP_CONTINUE = 100;
	const HTTP_SWITCHING_PROTOCOLS = 101;
	// [Successful 2xx]
	const HTTP_OK = 200;
	const HTTP_CREATED = 201;
	const HTTP_ACCEPTED = 202;
	const HTTP_NONAUTHORITATIVE_INFORMATION = 203;
	const HTTP_NO_CONTENT = 204;
	const HTTP_RESET_CONTENT = 205;
	const HTTP_PARTIAL_CONTENT = 206;
	// [Redirection 3xx]
	const HTTP_MULTIPLE_CHOICES = 300;
	const HTTP_MOVED_PERMANENTLY = 301;
	const HTTP_FOUND = 302;
	const HTTP_SEE_OTHER = 303;
	const HTTP_NOT_MODIFIED = 304;
	const HTTP_USE_PROXY = 305;
	const HTTP_UNUSED= 306;
	const HTTP_TEMPORARY_REDIRECT = 307;
	// [Client Error 4xx]
	const errorCodesBeginAt = 400;
	const HTTP_BAD_REQUEST = 400;
	const HTTP_UNAUTHORIZED  = 401;
	const HTTP_PAYMENT_REQUIRED = 402;
	const HTTP_FORBIDDEN = 403;
	const HTTP_NOT_FOUND = 404;
	const HTTP_METHOD_NOT_ALLOWED = 405;
	const HTTP_NOT_ACCEPTABLE = 406;
	const HTTP_PROXY_AUTHENTICATION_REQUIRED = 407;
	const HTTP_REQUEST_TIMEOUT = 408;
	const HTTP_CONFLICT = 409;
	const HTTP_GONE = 410;
	const HTTP_LENGTH_REQUIRED = 411;
	const HTTP_PRECONDITION_FAILED = 412;
	const HTTP_REQUEST_ENTITY_TOO_LARGE = 413;
	const HTTP_REQUEST_URI_TOO_LONG = 414;
	const HTTP_UNSUPPORTED_MEDIA_TYPE = 415;
	const HTTP_REQUESTED_RANGE_NOT_SATISFIABLE = 416;
	const HTTP_EXPECTATION_FAILED = 417;
	// [Server Error 5xx]
	const HTTP_INTERNAL_SERVER_ERROR = 500;
	const HTTP_NOT_IMPLEMENTED = 501;
	const HTTP_BAD_GATEWAY = 502;
	const HTTP_SERVICE_UNAVAILABLE = 503;
	const HTTP_GATEWAY_TIMEOUT = 504;
	const HTTP_VERSION_NOT_SUPPORTED = 505;
		
	private static $messages = array(
		// [Informational 1xx]
		100=>'100 Continue',
		101=>'101 Switching Protocols',
		// [Successful 2xx]
		200=>'200 OK',
		201=>'201 Created',
		202=>'202 Accepted',
		203=>'203 Non-Authoritative Information',
		204=>'204 No Content',
		205=>'205 Reset Content',
		206=>'206 Partial Content',
		// [Redirection 3xx]
		300=>'300 Multiple Choices',
		301=>'301 Moved Permanently',
		302=>'302 Found',
		303=>'303 See Other',
		304=>'304 Not Modified',
		305=>'305 Use Proxy',
		306=>'306 (Unused)',
		307=>'307 Temporary Redirect',
		// [Client Error 4xx]
		400=>'400 Bad Request',
		401=>'401 Unauthorized',
		402=>'402 Payment Required',
		403=>'403 Forbidden',
		404=>'404 Not Found',
		405=>'405 Method Not Allowed',
		406=>'406 Not Acceptable',
		407=>'407 Proxy Authentication Required',
		408=>'408 Request Timeout',
		409=>'409 Conflict',
		410=>'410 Gone',
		411=>'411 Length Required',
		412=>'412 Precondition Failed',
		413=>'413 Request Entity Too Large',
		414=>'414 Request-URI Too Long',
		415=>'415 Unsupported Media Type',
		416=>'416 Requested Range Not Satisfiable',
		417=>'417 Expectation Failed',
		// [Server Error 5xx]
		500=>'500 Internal Server Error',
		501=>'501 Not Implemented',
		502=>'502 Bad Gateway',
		503=>'503 Service Unavailable',
		504=>'504 Gateway Timeout',
		505=>'505 HTTP Version Not Supported'
	);
	
	public static function httpHeaderFor($code) {
		return 'HTTP/1.1 ' . self::$messages[$code];
	}

	public static function getMessageForCode($code) {
		return self::$messages[$code];
	}
	
	public static function isError($code) {
		return is_numeric($code) && $code >= self::HTTP_BAD_REQUEST;
	}
	
	public static function canHaveBody($code) {
		return
			// True if not in 100s
			($code < self::HTTP_CONTINUE || $code >= self::HTTP_OK)
			&& // and not 204 NO CONTENT
			$code != self::HTTP_NO_CONTENT
			&& // and not 304 NOT MODIFIED
			$code != self::HTTP_NOT_MODIFIED;
	}
}
?>

How Recess Solves Common PHP/MySQL Issues

Fri, 28 Nov 2008 16:25:25 -0500

Justin Carmony wrote a great article titled 'PHP Design - Biggest Database Oversights'. The article points out 5 naive PHP/MySQL design decisions that will come back to haunt projects if they've been made. As I'm getting closer to the public preview release of the Recess! Framework this article discusses how Recess! addresses some of the most common oversights in database development.

Re: Oversight #1 - No Data Access Layer

Quality software design and engineering is all about abstracting away details and finding the proper layers of functionality. As Justin points out, if there is no layer of abstraction which wraps data access code then low-level data access code will appear everywhere. PHP projects which interleave data connections and queries throughout their code are difficult to maintain.

The database stack in Recess! has four major components: 1) SQL statement builder, 2) Data Sources, 3) Data Sets, 4) Object-Relational Mapped Models. The notion of a 'data access layer' is taken care of by Data Sources. Data sources wrap properly around PDO and introduce new methods beyond PDO's, for example: getTables and getColumns($table). The implementation of these methods is specific to the RDBMS so Recess! also has the notion of a vendor specific driver called a DataSourceProvider. Currently Sqlite and MySql are supported.

Re: Oversight #2 - Design for Only One Database Connection

Small, greenfield projects often start out with a single database. This leads to naive data access layers which only support connections to a single database. Recess! supports multiple, named Data Sources. Instead of using a singleton pattern it uses a registry to store the Data Sources by name.

Having named Data Sources in an application gets really powerful at higher layers of abstraction, such as at the ORM layer, where we can mark-up a model with a single annotation to define which Data Source the model is persisted in.

Re: Oversight #3 - No Developer Logging

This isn't as fully fleshed out yet as I'd like but the plans and hooks are in place to be able to log queries, their stack traces, as well as their EXPLAIN'ed strategies for execution in the RDBMS. In Recess! there is a distinct difference between Development mode and Production mode. Sql logging will be a toggle option on by default in development and off by default in production.

Re: Oversight #4 - Queries Written in Procedural Processes

My dislike of queries being written in procedural code may run even deeper than Justin's! Recess! abdicates SQL string manipulation to a low-level SqlBuilder class. An instance of SqlBuilder allows SQL query strings to be built incrementally using chained method calls. Let's take a look at some examples:

$sqlBuilder = new SqlBuilder();
$sql = $sqlBuilder
            ->from('people')
            ->like('name','Kris')
            ->orderBy('age')
            ->toSql();
// $sql is now: 
// 'SELECT people.* 
//    WHERE people.name = :people_name 
//    ORDER BY people.age
$args = $sqlBuilder->getPdoArguments();
// $args is now array( 'people_name' => 'Kris' );

// Let's add another criterion
$sql = $sqlBuilder->equal('home_city', 'Charlotte')->toSql();
// $sql is now: 
//  SELECT people.* 
//      WHERE people.name LIKE :people_name 
//      AND people.home_city = :people_home_city 
//      ORDER BY people.age

$args = $sqlBuilder->getPdoArguments();
// $args is now array( 
//                   'people_name' => 'Kris', 
//                   'people_home_city' => 'Charlotte' );

SqlBuilder can do more complex things like joins, as well as inserts, updates, and deletes. Recess! users, though, will likely never use SqlBuilder because it's still too low level of an abstraction. Internally, however, having the ability to incrementally construct query strings has made the framework code quite pretty.

Here's code at the level of abstraction Recess! developers can expect to write:

class Person extends Model { }
$person = new Person();
$person->name = 'Kris';
$person->homeCity = 'Charlotte';
$people = $person->find()->orderBy('name');
foreach($people as $person) {
   echo $person->name, ' ', $person->age, '<br />';
}

This example just scrapes the surface of Recess! object-relational mapping. Relationships, CRUDS, cascading deletes, etc. are all handled as well. In a future post I'll step through the data access stack's capabilities in more detail. Needless to say, in Recess! queries won't be mixed with procedural code!

Re: Oversight #5 - No Separation of Reads and Writes

Once projects outgrow a single server RDBMS the next step is often to do a Master/Slave setup in MySQL. In a Master/Slave setup expensive and less frequent writes are channeled to the Master server while reads are channeled to the Slave server.

In Recess!, by using named data sources as described in #2, we can handle Data Sources on a per model basis. A natural extension of this is handling reads and writes independently on a per model basis. The pseudo-code below shows how this may look on a model:

/** 
 * !Source master, For: Writes
 * !Source (slave1, slave2, slave3), For: Reads
 * !HasMany books
 */
class Person extends Model {}

/**
 * !Source master
 * !BelongsTo person
 */
class Books extends Model {}

In a future post I'll describe the Recess! annotations system which will explain the constructs you're seeing in the DocComments. Recess! annotations give us a way of providing static metadata about a class and are built-in to the framework's core. Recess! annotations should be familiar to Java and .Net programmers who have used annotations.

Conclusion

So that's a quick look at how Recess! handles Justin Carmony's commonly made MySQL/PHP mistakes. These issues are abstracted away in the internals of Recess. If you're a PHP developer check out RecessFramework.com and sign-up to be notified when it is publicly released "very soon now".

Dynamic Properties in PHP and StdClass

Thu, 27 Nov 2008 00:49:15 -0500

Languages like JavaScript and Python allow object instances to have dynamic properties. As it turns out, PHP does too. Looking at the official PHP documentation on objects and classes you might be lead to believe dynamic instance properties require custom __get and __set magic methods. They don't.

Simple, Built-in Dynamic Properties

Check out the following code listing:

class DynamicProperties { }
$object = new DynamicProperties;
print isset($object->foo) ? 't' : 'f'; // f

// Set Dynamic Properties foo and fooz
$object->foo = 'bar';
$object->fooz = 'baz';

// Isset and Unset work
isset($object->foo); // true
unset($object->foo);

// Iterate through Properties and Values
foreach($object as $property => $value)  { 
     print($property . ' = ' . $value . '<br />'); 
}

// Prints:
//   fooz = baz

Using the built-in dynamic instance properties is an order of magnitude faster (30x, by my profiling) than using magic __get and __set methods. Built in dynamic property accesses happen without invoking a method call back to PHP script.

So when does it make sense to use __get and __set? If you need more complex behavior, like calculated properties, you must use __get and __set. Also, as an astute comment points out, if you would prefer not to have dynamic properties on a class you can throw errors from __get and __set.

StdClass: Anonymous Objects

Sometimes all that is necessary is a property bag to throw key value pairs into. One way is to use array, but this requires quoting all keys. Another way is to use dynamic properties on an instance of StdClass. StdClass is a sparsely documented class in PHP which has no predefined members.

$object = new StdClass;
$object->foo = 'bar';
json_encode($object);

Next I'll touch on the SPL's Countable and ArrayAccess as a means of being able to accomplish the following in PHP:

class MyClass implements Countable, ArrayAccess { ... }
$myObject = new MyClass();
// Using array access notation
$myObject[0] = 'hello';
$myObject[1] = 'world';
$myObject['foo'] = 'bar';

Thanks to the folks pointing out that you don't need to extend from StdClass in order to have dynamic properties!

How does the quality of political candidates’ websites correlate with campaign success?

Wed, 29 Oct 2008 17:11:41 -0400

VoteTheSite.com, a micro-site my good friends at New Media Campaigns have built, is conducting an on-line experiment to explore just that question. VoteTheSite.com pits congressional candidates’ websites against each other race-by-race. Races can be viewed randomly or by state. I was surprised by the range of quality congressional websites have. From top notch sites in Texas' 5th to the really poor websites in Georgia's 1st they run the quality gamut.

After the elections have taken place the votes on websites will be tallied up and compared to election results. Should be really interesting to see what the outcome is and how strong website quality correlates with campaign success.

Within the first 24 hours over 5,000 votes have been cast. Congrats to the New Media team for getting this out the door so quickly (with a little help from the Mechanical Turk)! Go vote on the websites in your state…

VoteTheSite.com was written (in under 2 days!) on an early version of the PHP Framework, the Recess! Framework, I’ve had my head down plugging away on the past couple of weeks. Expect blog activity to pick up as the Recess! Framework moves closer to a public release. Sign up to be notified of the release at RecessFramework.com.

Hello Again, Old Friend: Revisiting a PHP Framework

Wed, 08 Oct 2008 13:26:45 -0400

I'm in the process of replumbing the lightweight PHP application framework I wrote with Joel Sutherland over three years ago. Its original design was inspired by the Java Struts Framework. It enabled us to rapidly develop the first version of New Media Campaigns' website management software. Two summers ago we did a major redesign inspired by the DRY nature of Ruby on Rails. Since then Joel and Josh Lockhart have been tweaking the framework by addressing the issues which crop up after their intensive use while rewriting New Media's system.

Energized by what I saw at the Web 2.0 Expo in NY I'm back at it again. My three big goals with this take:

1) Get it in the wild: Open source with an MIT license. This is going to happen before the New Year. Hopefully sooner. We intended to do this with V2. It actually was publicly available in 2006 for a brief period of time but without any real plan for evangelism. I've been using open source software for a long time and its high time to give back.

2) Play nice in the new RESTful world. If I had to bet on a paradigm for interacting with web APIs I would bet the farm on REST. It is perfectly aligned with the grain of the web. Most existing frameworks written without an emphasis on REST have made awkward face lifts to adapt. When revisiting our own framework and considering how to make it properly RESTful this would have held true if not for...

3) Signficantly Improving the Architecture. Revisiting old code is a joy. It's easy to forget how clever you were and how much work you did. Yet it's very disturbing to realize how hacked some of the fundamental design was. As mentioned, the current version was influenced significantly by what Rails plumbing looked like circa-2006. Since then two really important things have changed: 1) the emphasis on tying closer to HTTP protocol, and 2) two additional years of experience with systems design under the belt.

So, here's to take 3! Bits available for download in upcoming months. E-mail me at krisjordan/gmail if interested in being copied on barely functional, pre-release bits in the mean time. Otherwise, stay tuned.

Brainstorming Ideas with Sharpies & Sticky Notes

Wed, 01 Oct 2008 20:17:48 -0400

After getting a comment from Ric Bretschneider of Microsoft's PowerPoint team I came across the book slide:ology on his site Presentations Roundtable. I picked up slide:ology from a bookstore on my way out to New York and it is a great piece of work (and art!) authored by Nancy Duarte of Duarte Design. Duarte Design is the firm behind Al Gore's "An Inconvenient Truth" Keynote slide deck. They are also the designers of John Doerr's TED talk's deck. Duarte Design is truly first class.

A technique that caught my eye was "Innovating with Sticky Notes" (p28). The premise is to use a sharpie and generate ideas on sticky notes. One idea per sticky note (the bonus of using a sharpie is that is about all you can fit). Just unleash as many ideas as possible and get them up on the wall. Structure and flow can then be orchestrated by rearranging the notes.

I decided to give the sticky note method a shot with a talk I'm giving on the Chord Protocol in a graduate class I'm auditing at UNC. I must admit I really liked the technique after trying it out. It's quick and dirty and prevented me from getting lost in details and aesthetics. My focus remained on the big ideas and overall message I wanted to 'teach'. I would even go so far as to say that it's fun. Once I'm done with the talk I'll narrate a slideshow and throw it up on here so you can see the end result.

This sharpie + sticky notes method is only a detail in the grand thesis of Nancy Duarte's slide:ology. Spend some quality time with this book before you prepare your next presentation, you won't regret it.

Aside: if you're going to use this technique on a wall at a coffee shop be prepared for inquisitive looks! :)

10 High Order Bits from the Web 2.0 Expo in NY

Thu, 25 Sep 2008 18:41:09 -0400

10. Your Web App: Give it a REST

David Heinemier Hansson's session about making Ruby on Rails RESTful cast this battle as an epic one between the REST Rebels and the Imperial WS-* Death Star. It's going to be a tough fight but you know who's gonna win.

REST (Representational State Transfer) is the elegant architecture and set of conventions first presented in Roy Fielding's PhD dissertation "Architectural Styles and the Design of Network-based Software Architectures". It is well aligned with the HTTP protocol and much simpler to implement and use than SOAP, XMLRPC, etc.

Implementing RESTful APIs in web applications is getting really easy with leading frameworks like Rails and Cake supporting REST as a first-class citizen. The Atom format is leading the charge as a RESTful format supported by the big players: Google, Microsoft, Yahoo, Twitter, etc.

9. "It’s Not Information Overload, It’s Filter Failure"

Clay Shirky's talk states that since the invention of the printing press humans have always faced information overload. We have been surrounded by more information than we can consume in an entire lifetime for centuries. The problem is not information over load, it's filter failure. We need better filters.

Jay Adelson of Digg believes building better filters is exactly the mission Digg and other players in the collaborative filter space are addressing.

8. Sensor Driven Data: The Web is Getting Orwellian

With Apple putting GPS in iPhones, Google putting GPS in Android, Nikon putting GPS in the Coolpix P6000, and ... you get the point. GPS, motion sensors, video recorders, microphones, and other sensors are increasingly distributed and surrounding us.

Tim O’Reilly believes a BIG revolution is happening Here. Tim is really bullish on sensor driven data. Where 2.0 has its own O'Reilly Conference. This space is heating up fast.

7. Javascript is Bringing Sexy Back to the Browser

John Resig gave a session on processing.js, his visualization engine running atop the HTML5 Canvas. The canvas has really low level functionality, a la OpenGL for 2-D surfaces, but with the right libraries in place it can lead to some truly impressive results. Flickr's Paul Hammond gave perhaps the most compelling story of the use of Javascript and Canvas. After building Flickr Stats, using Canvas for graph visualizations, a team member loaded a page on an iPhone. It just flat out worked.

Unfortunately my friends over at Microsoft are slowing down the progress here with no planned support for the HTML5 Canvas in IE8. Google's excanvas gets around this for IE users by mapping to VML. Unfortunately excanvas currently only works in quirks mode in IE8. Damnit Microsoft, you've brought IE8 a long ways towards being a modern, friendly player on the web, why not support Canvas? Come on Oz, come on.

6. The Open Web is Nearing the Tipping Point

DataPortability co-founders Chris Saad and Daniela Barbosa gave a great session on the basic motivations behind the movement. The future the DataPortability group is trying to create, one which allows us to owning our data, our contacts, our relationships, etc. and be able to move them freely and easily between the on-line systems we use sounds truly empowering. The big players are joining the party: Microsoft, Google, Facebook, Six Apart, Linked In, Yahoo, Digg, Plaxo, MySpace. But Chris says "Who cares about them? This is a grassroots effort!" Joseph Smarr, Chief Architect of Plaxo, gave another interesting session on the major components of the open web and how they fit together. OAuth, OpenID, Open Social, and others were covered. The feeling I walked away with is that we're a lot closer than I thought.

5. Web Scalability thanks to Async & Danga

"You can’t drop something in 40,000 buckets, synchronously, at once", said Digg's Lead Architect, Joe Stump in his session "Scaling Digg and Other Web Applications". He was referencing what happens when Kevin Rose posts a message on Twitter. (Rose actually has nearly 65,000 followers on Twitter) Asynchronous task queuing is how the folks at Digg, Twitter, and Flickr deal with problems that are really hard to do in real time in any scalable fashion.

Just about all of Brad Fitzpatrick's (of LiveJournal and OpenID fame) lightweight systems software, freely available at Danga.com, seems to be used by the biggest Web 2.0 players to achieve scale. That memcached, gearman, perlbal, djabberd, and mogilefs, all came out of Fitzpatrick and Danga is just incredible. No wonder Google gobbled him up from Six Apart.

4. Web 2.0 Traffic: It's Out-of-Band

The knowledge tidbit that stuck out more in my mind than any other was that Twitter gets 10 times the amount of traffic from its API than it does through its website. It makes sense, I'd just never acknowledged it explicitly. Dion Hinchcliffe's workshop painted a similar story for many other Web 2.0 successes. The canonical example is YouTube with the embedded video. The decision to put html snippets plainly visible, right beside of the video, was perhaps their most genius move. Modern web applications and services are making themselves relevant by opening as many channels of distribution possible through feeds, widgets, badges, and programmable APIs.

3. Cal Henderson's PHP Tent Revival

If not for Cal Henderson I may have never have touched PHP again. I'm probably going to come back to this topic in more depth in a future post but Cal's workshop "Scalable Web Architectures: Common Patterns and Approaches" renewed my interest in, relationship with, and respect for PHP. The funny thing is that wasn't even the point of the talk. Cal and Joe Stump of Digg's succinct point that Langauges Don't Scale is right on. Sure PHP isn't as beautiful, trendy, or well designed as Python or Ruby are. However, some of the design decisions made by PHP's Rasmus, specifically the 'shared nothing'ness, make it a great technology for web applications. There's a reason why Facebook, Digg, Flickr, and co. are still on it.

After Cal's workshop I asked him: if you could do it all over again with Flickr would you choose to go with Python or Ruby? Cal's answer: Nope, I'd do it in PHP.

2. Set Your Baby Free

By grooming and nurturing a web app internally for an extended period of time is you lose a lot of value. Jason Fried's notion of "half a product is better than a half-assed product" is so fitting here. Sandy Jen of Meebo echoes similar notions in her talk: Start out with something simple, see if it works, evolve. Bring your customers into the feedback loop as quickly as possible. Joshua Schachter, founder of delicious, spoke of the exact same sentiments in his talk on "Scaling and Building Social Systems".

1. Want to Set the World on Fire? YOU Better Bring the Fire.

If you are not bringing the heat, get out of the kitchen. Passion was the common thread amongst the most inspiring talks I saw at the conference. Between Gary Vaynerchuk, Jason Fried, and Arianna Huffington the message was consistent: be passionate. I'm going to let Gary roll this one out with his amazingly energetic keynote on building personal brand...

Joseph Smarr - Tying it All Together: Implementing the Open Web

Fri, 19 Sep 2008 16:39:22 -0400

[Live from Web 2.0 Expo 9/16 - 9/19 Follow along the other Expo Talks in RSS.]

Joseph Smarr is the Chief Platform Architect at Plaxo.

Lots of open source building blocks for bringing things together. How do all these pieces sit together and what is the landscape going to look like when the dust settles?

The social web today is very broken. On each site you have to re-create an account, re-enter profile info, re-find friends, re-establish relationships. New social apps have limited options: create yet-another-silo and start from scratch or make a widget inside of an existing walled garden.

There's got to be a better way and there is. Help is on the way. It's coming in the form of new building blocks that establish: who I am, who I know, and what's going on. We're going to aim for the medium level of detail on each of the projects which fit into these building blocks.

Who I am: Creating a portable, durable online identity. OpenID is important in this space. OpenID lets you come to a new website and allows you to log-in with an account that exists on another site. You can sign up and sign in with your existing account. You can then link and share your profile data between sites. When you go through Plaxo's sign up you can sign in with any open ID. This takes you over to your identity provider and allows you to verify that you want to share your information with Plaxo. This is good for users and for Plaxo by reducing friction. Yahoo is OpenID, MySpace is on the way, AOL is signed up, some of Google's properties are supported, this has majorly caught on.

Consolidate your online identity with me-links for rel=me (XFN). The social graph API allows you to query Google using REST for the downstream me links. This makes it easy to find out more information about users by what exists on the web. Again great for both the consumer in not having to duplicate info and great for businesses in terms of getting data into your systems.

Who I know: You need to be able to build and maintain relationships. Until recently the only way you could get at this information was to scrape your webmail address books. It's kind of hacky and insecure. The good news is over the last year that this isn't going away, it's useful, and they've made it easy to practice safe portability. Google, Yahoo, and Microsoft have mechanisms for getting at the information without giving a new service your webmail password. OAuth is a means for sharing private data between trusted sites. A bunch of people came together and came up with a standard way of getting at data. OAuth is supported by Google, MySpace, it's a part of DataPortability. OAuth gives a third party site a token which is revocable. It can be scoped access. Friends-list portability allows for continuous discovery across multiple sites.

What's going on? Because the entire web is becoming social you're creating and doing interesting things on a lot of different sites. You can't walk each site to check and see who is doing what. OpenSocial is trying to define a standard language for social networking applications on the web. You can drop in widgets that work on all social networking sites. OpenSocial is going mainstream and has over 500 million users by the end of the year. Everyone is agreeing on standard APIs at the server-to-server level. RSS and Atom is another important piece which is often overlooked. It's an important standard for sharing "here's what going on right now." If you put RSS together with OAuth you can get private update feeds. Jabber XMPP is becoming more important, too, it started as an open standard for instant messaging. One of the things they built in as a result is that it is federated. It's a good set of open tools for different sites sending messages to each other.

This stuff is out there, it's real, and it fits into these standard blocks. What we'll do now is pull everything together.

How does the friends list portability work?

Tell the site your social graph provider: XRDS-Simple (discovery) + OAuth (access)
Site fetches your data to find local friends: Site fetches your data to find your local friends ??? No standard way to do this yet. A project that's going on and in draft spec is still up and coming is PortableContacts.net.
Site lets you connect to people you want: You can periodically look for new matches.

How does contact portability work?

User signs in with an OpenId: Site fetches OpenID URL -> looks for X-XRDS-Location, Site parses XRDS-Simple doc to discover available APIs
Site tries to access contacts API -> gets a 401: WWW-Authenticate response header specifies OAuth, OAuth discovery (via XRDS) provides OAuth endpoints
Site sends user through OAuth

A resource Joseph wrote on OpenID: http://www.plaxo.com/api/openid_recipe

Jeffrey Zeldman and Panel - Content Matters

Fri, 19 Sep 2008 15:09:25 -0400

[Live from Web 2.0 Expo 9/16 - 9/19 Follow along the other Expo Talks in RSS.]

The Panel: Liz Danzico – Bobulate.com - part information architect, part usability analyst, and part editor. Alex Write “Information Architect” – NyTimes.com – Previously a UX designer and journalist in California. Bre Pettis – “Videographer” – BrePettis.com, Etsy.com Kristina Halverson – “Content Strategist” – BrainTraffic.com Jeffrey Zeldman - “Evangelist” – AlistApart.com, HappyCog.com, Zeldman.com

Paul Ford – “Editor” – Harpers Magazine, ftrain.com, themorningnews.com

Liz Danzico Setting Context for Panel Discussion

The people on this panel are interested in changing your minds on the role of the content in design and user experience mocks. The original name of this talk was: copy matters.

You may have heard these two conflicting views: 1. Content drives traffic. Content certainly is a primary reason for users to come to you. 2. Users don’t read on-line (Jakob Nielsen).

We will be talking about whether we are at a cross-roads. Going back a little bit in history to transitions in media, people referred to television as radio with pictures. When movies and television were first made cameras were stationary. Early MTV was just bands on the stage playing. When we make transitions from old to new we borrow metaphors from old technology and apply them to the new. As we look at the web as a publishing medium we’re looking at a different publishing medium. Different responsibilities for makers, editors, etc.

“We aren’t writing, we are speaking in text.” – Erika Hall of Mule Design.

“The internet looks like writing, but it’s actually a conversation.” –Khoi Vinh, NYTimes.com

What kind of content are we talking about in the panel?

Navigation & Orientation content – Daytum.com, Flickr.com with rotating greetings setting the tone in different languages. Navigation on CNN is an example of clear communication on sub-pages. Matched with URL and page hierarchy.

Labels & Action – Vimeo with its labels. Geni.com as a quick review of what labels are.

Help Content – Tick “Just kidding I Remember Now” link next to “Send me my password”.

Non-textual content – Visual content like election maps, photos, info visualization.

Content, content, ! – Marketing communications sites like Business Week with Editorial Content, etc.

With this new publishing model and these new types of content, how are we going to make it work? Ask the experts!

Q) What is the nature of the content work you do? The reason I asked each of you to the panel is because each of you has a different role in content.

Halverson- Typically our clients are dealing with content that helps discuss their products and services. We are trying to help them wrangle, plan for the creation, creating, standards and structures in place to help them govern that content. I got into content strategy because I was handed wire frame to fill content for websites.

Pettis- I got into video and videos about how to make things. For two years had a show called video projects. I’m a video guy who thinks print is a part of the past.

Zeldman- With the magazine I write, with my website I write. I was a journalist and in advertising and copywrite for a long time. When I started websites back in 1995 the whole thing as that it was self publishing. I thought everyone was going to learn HTML and be self –publishing. I thought everyone would find their voice and try to find an audience. With our client services projects we always start with the content, what it is, what’s there, how they’re going to interact with it. We develop content strategy and architecture before we get to design. If you bring design in fairly late in the process you’ve already worn them down. In terms of the magazine it’s a labor of love.

Q: How do you deal with accommodating both organizing content that is purely visual and content that is textual?

Write- NYTimes publishes a lot of content and we increasingly publish a lot of multimedia content: images, slideshows, interactive flash, etc. Issues that come up is around the metadata layer. All of our photos are in a huge database that exist separately from the article database and content management system. We have a good taxonomy to tag articles but we don’t have the same capabilities with issues. From a design point of view we’re constantly trying to figure out how to weave that content into the site.

Pettis – The thing I’m excited about on the internet is that people find a passion and get into it and publish about it. I’m on the blogging team at Etsy. We have really passionate users. What we do with the blog is open it up so that anyone can pitch ideas. We have over 300 authors on the blog. We have a video team of people who want to point cameras at things and record what they’re thinking and doing. It’s a way of sharing passion and excitement.

Q: How are you helping your clients how to become sophisticated publishers of content?

Halverson – That for me is way down the tracks. As an example take a company with 12 different business units serving 122 different markets. Producing content for a lot of audiences. Our process is A) Figuring out where the content is and who is out there B) establish who is publishing content, reviewing content, etc. C) governing consistent brand standards across the content. There’s a complete infrastructure lacking within many organizations between print marketing and interactive marketing. We start by trying to bring these people into the same umbrella. That challenge in some organization is really difficult. They spend all their money on brand and have no idea how to govern and create content.

Zeldman – I would just like to say it’s mostly luck. Like woody allen said ablout love: sometimes you’re lucky sometimes you’re not. With clients we would turn things over and sometimes the client would use it and take of running, sometimes they wouldn’t. We built a content management system and would write guidelines and sometimes clients would follow them and sometimes they wouldn’t. We’re doing something for a food manufacturer and they make a delicious bar which has a cool brand and has medical implications so the challenge for copy is that there are pages that have to address people with lupus and there are other birds where there are birds with funny sayings. What rules do you give the client for when they use each tone, how to transition? We create matrices and recommendations and if we’re lucky the client has the right people and the right talent to keep it going. You hope that everyone is passionate about the project.

Question: How do you approach content from a user generated view point?

Zeldman – I think it’s both. You have to talk about both sides of the equation.

Liz – More and more it’s our responsibility as designers to think about creating very good frameworks that are well thought through, intuitive, and provide intuitive roles which people can participate in. That’s one step: designers create a framework to participate. The second step is for users to actually be involved. The third is an editorial responsibility of the client to monitor content.

Zeldman – If you abandon the responsibility of editorial control you lose a lot of value. If your content is of high quality you’ll get comments of high quality. Generally because the writing is so good at NYTimes there are some really well thought out comments.

Write – When user generated content works well it’s when it’s well channeled. If there’s a cacophony of noise you can’t get anything out of it. At NYTimes you can comment on certain articles but it’s all moderated by a team of people that try to keep the level of discourse civil. The notion of just opening things up and letting people go after it leads to craziness.

Ford – Another example: how many wiki sites are dead on the web right now?

Zeldman – The first site I worked on for a client was Batman Forever in 1995. We had a forum and seeded some content. It went well initially, people used it. But once we were off the job and the movie stopped needing to sell tickets so they stopped keeping track of the forum. Suck.com later did a piece on it. People were making racist comments and trying to have sex with each other.

Ford - You can get content for free but you can’t get editing for free.

Halverson – We’ve had companies who try to fix their content problems by buying a really expensive CMS. They think the magical content will just arrive. You must plan for, create, and govern content.

Zeldman – Even Flickr is about constraints. They encourage a certain kind of user.

Halverson – Good example but an easy example because the site is for fun. It gets more complicated when the matter is more serious.

Q: What kinds of tips can you give to people who are responsible for creating content?

Write - Part of the design process is what are the words that exist on the page? “The vacuous victory of typesetters over authors.” People tend to think of the web as boxes and content blocks.

Ford – If you are the one doing content make it very easy to get tons of feedback. I’m the sole guy doing web copy on Harpers and I hear back from customers constantly. When you are the person doing the copy it’s your job to make it as straightforward as possible.

Pettis – I’m shocked to hear businesses farming out content and passion.

Halverson – I often work with people who are responsible for content on top of their many other jobs. Don’t conceive of and put boxes on your wireframes for content you don’t have time to create and govern. Scale back.

Zeldman – I think that’s the most important piece of advice I’ve heard today. Scale back. If you don’t have the people to do it, you shouldn’t do it. Grow slowly.

Jay Adelson - Organizing Chaos: The Growth of Collaborative Filters

Fri, 19 Sep 2008 14:30:45 -0400

[Live from Web 2.0 Expo 9/16 - 9/19 Follow along the other Expo Talks in RSS.]

Jay Adelson is CEO of Digg, guiding all aspects of the company’s development, growth and management. Under his leadership, Digg has grown to 26 million visitors per month, and is now considered one of the top socially focused Web sites.

Why do collaborative filters matter? How many of you used google? How many of you have used Digg? Any time you take the interests of a group and use that to filter and create relevance for an audience and a group then that is collaborative filtering. Even search is a sense of collaborative filtering, just think about backrub and page rank or clicks on a search result. This has evolved.

So what’s changed? Now you’re on the web 24 hours a day. In 2003 Berkeley said there were about 2.3 million sites added every day. Now there’s about a terabyte a day added to the net. This data is dynamic. Privacy and sense of privacy has also changed. Younger generation doesn’t have the same issues associated with privacy that we have and our parents have. How I use my away message on AIM, “I’m at lunch”, whereas my teenage baby sitter’s will say “I’m feeling down” “I’m full”. We are moving from a seek culture to a connecting culture.

Let’s break social filtering down into three parts:

1) something like a Digg or a Zeitgeist is the same for everyone.

2) social networks where I create a subset of groups with just my friends. I can’t use my friends as a judging factor for what might be interesting to me.

3) The exciting thing, the point I can leave you with today, is the hyper-personalization opportunity. Instead of looking at a social network, look at everyone and pair you with people like you and use that collective wisdom that are more specifically interesting to you. Since your personal data is going to move from website to the next you have to think about how you can take that information and deliver experiences specific to individual users. Collaborative filters are the key to the monetization to Web 2.0 applications

Arianna Huffington in Conversation with Tim O'Reilly

Fri, 19 Sep 2008 13:56:14 -0400

Arianna Huffington is an author and nationally syndicated columnist in the United States. She is the founder of The Huffington Post, an online news/commentary website and aggregated blog.

Tim: How did you get where you are? People look and think you’re at the center of the new media world.

Arianna: The secret is you must have passion. I fell in love with the web when I realized that people without a platform could have one on the web. I love the obsessive/compulsive nature of the web. Even if you don’t own a printing press or the history of a hundred years in business you can reach an audience. We wanted to do three things: we wanted to have an attitude, we wanted to provide a platform for other bloggers (we now have over 2,000), and we wanted to have a community which is central to what we do.

From day 1 we made sure the comments on the blog were pre-moderated. There is no substitute for pre-moderation. We now have 30 humans who pre-moderate around the clock all year.

Tim: Do the moderators actually participate in discussions?

Arianna: For now all they do is delete ad-hominem, violent attacks.

Tim: Describe your process. I remember a movie about Watergate where there was a huge debate as to whether the news should be front page or not. Do you guys have staff meetings where you debate what news goes front page?

Arianna: We’ve got an editorial team. We’ve got a guy who decides the splash headline. We still believe Iraq is the biggest disaster in the history of US Policy. Even when mainstream isn’t following it anymore we are because it is such a catastrophe.

Tim: So you’re kind of like Rupert Murdoch, you don’t necessarily want to be impartial but can promote causes.

Arianna: There is a huge difference between us because the news we run is based on facts. When you say you’re giving both sides of the story and one side of the story isn’t based on fact it’s not newsworthy to present it.

Tim: How do you read the current political situation?

Arianna: You know, it’s really interesting. Last week we had this amazing phenomenon of the media being distracted. The selection of Sarah Palin was a little bit like a soap opera. People obsessed over all these small stories around her. Did she really sell her plane on eBay? Then, suddenly, reality set-in and the house is on fire. We’ve woken up and I’m glad about it. I wrote about it and felt that Sarah Palin was a Trojan Horse. I really feel she is a major danger.

Tim: Let’s talk about financial deregulation. This is a house that’s currently burning. What do you think about what brought us here?

Arianna: What brought us here is the illusion that you can have free markets, unregulated bring about public good. Look at the 85 billion we’ve agreed to put into AIG. America is basically telling the people: if you are big enough you are not going to be allowed to fail. But if you’re an ordinary American and your house is foreclosed then you’re on your own. This isn't what America used to be.

Tim: Coming back around to Huffington Post as a new media phenomenon. You’re in New York, not California, what has that brought to you?

Arianna: We love being in New York. It’s been great for us to be infused in the energy of the city. It’s been a great place for us to recruit young, driven editors. We’ve found great people to work in our technology department. The key is to be surrounded by people who constantly want to invent and reinvent. We are constantly creating new technological tools, bringing in new video, all of it is a part of keeping people engaged.

Tim: Why do you think the conservative blogs have failed and progressive blogs are succeeding?

Arianna: Conservatives do so well on talk radio because they're great at being blowhards. It’s because they can talk without being corrected or checked on facts. They don’t have to speak in truth because no one is pushing back. Progressives do well on the internet because on the internet you have the masses checking your facts and ensuring you’re on top of the truth.

Audience Member: I’ve noticed in America we’re engaging the Islamic world. On the other hand our media organizations lacks people who are Muslim American writers.

Arianna: First of all that is something we’d like to get better at. If anyone wants to write on any subject of any creed just shoot me an e-mail. It doesn’t have to be politics. Half of our traffic isn’t politics. It could be on any subject.

Tim: Do you think the internet will bring greater transparency?

Arianna: As you know, radio was a great tool for fascism. I believe the internet is a great tool for democracy. We see the way Obama is using the internet. Without a doubt without the internet Barack Obama would not be the nominee. It’s increasingly become clear that political leaders need the community to be pushing them to do the right thing. People fascinate themselves with polls but they’re awful. At Huffington Post we’ve got this new feature called Pollstrology.

Tim: Could you speak to any of the trends you’re noticing as you bring groups together at the website?

Arianna: There is something I’m noticing which I believe is going to be the next big thing of the internet: there is a huge need and longing to unplug and recharge. Our subhead is unplug and recharge. If all you care about is success and power, sex and money you are still incredibly better off if you unplug and recharge. Find an oasis. We have 2-5 minute breaks.

Tim: What do you do to recharge?

Arianna: I personally do yoga, I hike, I get enough sleep most of the time. If you have any ideas for how to recharge let us know, send them to me.

[ Follow the Feed for notes on talks from other web leaders & innovators at the Web 2.0 Expo in New York going on this week. ]

Sandy Jen - Scaling Synchronous Web Apps: Lessons Learned from Meebo

Thu, 18 Sep 2008 17:01:58 -0400

[Live from Web 2.0 Expo 9/16 - 9/19 Follow along the other Expo Talks in RSS.]

Sandy is a co-founder of meebo in Mountain View. She majored in Computer Science at Stanford. Sandy is the 'Server Chick'.

Things to keep in mind about scalability: what works for someone else won’t necessarily work for you. You know the most about your stuff. Don’t hire consultants. You built your app. Don’t get married to a technology but don’t be a total flirt. It is a very high cost to rip out your guts and start over again. Remember that this is supposed to be fun. You’re building a product. You always have a customer who will be happy that you’re building this thing for them.

Synchronous web applications are very different from asynchronous web applications. Traditionally “async” implies “more complex”. On the web it’s opposite because the browser is meant to be async. If you’re going to build a synchronous app you’re probably taking something that used to be on the desktop to the web. That’s where all the problems of scaling come into. Meebo is a good example of synchronous on the web, so is Gmail, many games out there.

Doing synchronous on the web is like trying to fill a square hole with a round peg. The hole is that there are a lot of platforms (OS, browsers, etc). When we test a release we test on all the OSes and the browsers and safari and now chrome. Spotty network connections are not going to be 100% stable, still people using dial-up. The limitation of only being able to have 2 open http requests allowed imposes a serious constraint. It’s hard to measure how successful a synchronous app is based on traditional page view metrics. Alexa doesn’t pick up how much traffic Meebo actually gets in relation to other page view based sites.

The peg, the thing you’re shoving into the hole, is the need for instantaneous data transfer. Long polling is challenging because you’re using resources on both the client and the server. This is making the browser do more work. The user experience needs to be seamless and feel fast, light, and feel better than the desktop equivalent.

What is synchronous? What has to be synchronous? What doesn’t have to be synchronous? The more you try to dump into synchronous the more trouble you’re going to have trying to scale it. Sometimes you cheat in order to create the seamless user experience.

Find the right holes for your pegs: don’t underestimate server side architecture! Type of app determines the type of synchronous scaling. Bottlenecks can be anywhere: memory, CPU, bandwidth, storage, disk i/o. Based on the type of app you’re building it will be in different places. You won’t know where all the bottlenecks are until you let it loose. With Meebo it went from one to another to another. We solved the memory problem a while ago and it’s come back since then.

Things that people say are great, synchronous helpers: long polling (COMET) connections without having to poll every 5 seconds. Meebo started with Apache and it wasn’t good for us, so lighttpd was a much better fit. When it comes to compiled vs. interpreted languages it goes either way. We use C but it’s kind of a bitch to hire for because there aren’t many people doing it any more. Databases can be really expensive or really cheap. Start simple and if you need to get more complicated do it when you need to. Memcache is great, we’ll talk about it more later. Load balancers, finally, are just really expensive and you have to buy in pairs.

Simple is better unless you’re rich. First question: what am I using it for? Am I using memcache because everyone else is? We tried it at Meebo but it turns out most of our data wasn’t cacheable. What I gaining? Scalability at the cost of maintainability? Can I use DNS round-robin instead of load balancers? FastCGI vs. web modules vs. PHP? When we first started we didn’t want to reinvent the wheel. We started out really simple with CGI written in PHP. We wound up just writing a module directly into the web server and that’s what we’re still doing today. Start out with something simple, see if it works, evolve. Do I need to save state? Is it persistent? Can I store it in a cookie? Meebo didn’t have user accounts for a year. Launching feedback light is not a bad thing.

There’s a constant tug of war between the front-end and the back-end. Whose bug is it anyway? You have to figure out where the workload makes sense. The browser can be really slow. Most of Meebos users use IE. Say you’re using a web request and you pass a lot of data down to the client to process it can really bog down the user experience. Pick one, release it, and ask if it’s slower or faster than the last release. Your users know more about your product than you do. Listen. Efficiency with data transfer: when we first started out I picked variable names with single letters to save bandwidth. Once we started hiring it was confusing.

Must find a balance between good enough vs. perfect. Perfection is enough simplicity in the system to allow for adaptation. Users don’t care how clever you are, they just want their product to work. Long polling isn’t perfect, browsers have quirks. Sometimes perfect is not good enough (look at Ruby!). Release enough and things will asymptotically approach perfection. Don’t be afraid to try things.

Think ahead but don’t think ahead too much. A great example of this is security. You can spend a long time trying to fix security holes but if your product never ships, who cares? Over designed code is hard to roll back from. Hacky code can work and not be so bad. When you first build you won’t know where you’re going to need to scale so over-thinking the problem is a waste. It’s all about balance.

Nothing simulates real life. Have contingency plans on both front-end and back-end. Don’t build flood gates, build dams: One time we rolled out a feature that took a huge amount of bandwidth and we were able to switch it off. When you roll out features be very transparent with your users and say “hey try this out, let us know what you think”, they’ll get a lot less upset when you have to roll it back.

Be a user of your own product. Don’t be afraid to break your own product. Stay in the loop of your community and stay in touch with the pulse. What is your firefox/ie breakdown? 70% of Meebo’s users use IE. When we use when we use Meebo? IE.

It’s ok to be “Big Brother” in the sense of being aware of what’s going on. Monitor key areas but don’t go overboard on monitoring, you’ll learn to ignore your alerts. Ignoring what systems are telling you in feedback mechanisms are dangerous. Monitoring is being aware of how healthy your system is at any given point. Can I log in? What is our downtime percentage?

Final thoughts are that there are no magic solutions to scalability. It’s important for you to know your system like the back of your hand. Correlate effects to the changes you’ve made in your systems. Do not lose sight of your goal: why are you scaling? Finally, remember, everyone scales differently!

[ Follow the Feed for notes on talks from other web leaders & innovators at the Web 2.0 Expo in New York going on this week. ]

Chris Saad, Daniela Barbosa - Understanding the Basics of Personal Data and DataPortability

Thu, 18 Sep 2008 16:08:51 -0400

[Live from Web 2.0 Expo 9/16 - 9/19 Follow along the other Expo Talks in RSS.]

Chris is Co-Founder and Chairperson at DataPortability Project. Daniela is Chairperson, Steering Committee and Co-founder at the Dataportability Project.

There have been many tech inflection points. Intel gave birth to the standardized PC architecture. Windows became a standard for GUIs, drivers, and all the plumbing which allowed for a whole new class of applications to be built. TCP/IP led to the standardization of the the internet. HTTP/HTML brought us the web. We’re moving further and further up the step and we’re getting to the point where we should be thinking about standardizing on the data.

The data portability video was done very early on. You’ve got data everywhere, accounts everywhere, profiles everywhere, friends everywhere, contact details. Upload your photos, avatars, music, rinse, and repeat. Again and again and again. Network fatigue. Your data locked up in someone else’s hands. DataPortability is all about creating a web where information can freely flow through the network.

Imagine owning and controlling your relationships. Imagine controlling your calendar, images, and other content. Today we all join Flickr, Digg, Twitter, Facebook, etc. Imagine instead that these applications joined you and that you are the data. These applications had to ask for access to the data. Why? So you can sync your friends between Twitter and Del.icio.us and keep them sync’ed. Go to Kodak.com and print your photos from Facebook.

Today we share we comment, we rate, we create. Users sign up, fill out profile, add friends, interact with your stuff, you make money, they share your stuff, you get more traffic. Everything but interacting with your stuff and making money is friction that is repeated at every web site. Why the data lock-in model is actually a myth. Example: Amazon.com has a lot of data on you on their site. Why in the world would they want to release this data? It is because Amazon only has a small slice of the data. They don’t know the searches you do on Google. The data they do have is expiring rapidly. In a DataPortability system you get more data, reduced network fatigue, and more usage of everything.

DataPortability means you can access and synchronize the data you have between multiple services.

In January Robert Scoble, with Plaxo, scraped his contacts from Facebook. This is what we refer to as Scoblegate. It set off a flurry of discussion around the idea of being able to move your data between websites and applications.

DataPortability, as a group, is not writing any code or providing any code as a group, we’re advocating a stack. There are lots of open questions around security, privacy, ownership, business models, and user education. How can we evangelize and educate for the user?

Many of the big vendors are playing: Microsoft, Google, Facebook, Six Apart, Linked In, Yahoo, Digg, Plaxo, MySpace. But who cares about them! “If you’re out there I don’t mean you specifically.” (Haha) The DataPortability Project is an open, grass-roots effort.

We have a governance model where everyone can join and get involved immediately. We are experimenting with a radically transparent leadership model. Everyone will own this because everyone has helped to build it. It’s not a product, nor a service, it’s an idea. That means YOU, yes YOU.

Joe Stump - Scaling Digg and Other Web Applications

Thu, 18 Sep 2008 15:03:02 -0400

[Live from Web 2.0 Expo 9/16 - 9/19 Follow along the other Expo Talks in RSS.]

Joe Stump is currently the Lead Architect for Digg where he spends his time partitioning data, creating internal services, and ensuring the code frameworks are in working order.

Digg by the numbers: 30,000,000 Ron Paul fans. 13,000 requests a second, bunches of servers.

“Web 2.0 sucks (for scaling).” Web 1.0 was easy where we had this landrush of just getting content on-line.

Web 2.0 somebody had a bright idea that we would turn content over to the users. The problem is people like creating a lot of shit. Web 1.0 was easy to scale because I only needed to worry about a could hundred thousand some records. Now we’ve got a lot more to worry about. Another thing I hate is AJAX which makes interacting with websites really easy. It gives users the ability to create shit even faster.

Making your PHP code 300% faster doesn’t matter, it’s not where your bottlenecks are. “PHP Doesn’t Scale” – Cal Henderson. PHP doesn’t scale, Java doesn’t scale, Ruby doesn’t scale – languages don’t scale. When you’re worrying about scale and storing 4 billion kitten photos: how you program it probably doesn’t matter.

What’s scaling? Scaling is specialization. As you get bigger and as you grow the solutions being sold to you by vendors won’t cut it. You have to cut your database into different pieces and make it very specialized and specific to your needs. We’re going to talk about some of the techniques we use at Digg. Scaling is also about severe hair loss. I’m not joking. I’m going bald. It’s tough. It’s not easy. You can’t do it alone.

Often people get confused with scaling out and scaling up. You get to a point where you can’t scale up anymore. You can’t just buy more expensive machines at some point. Everyone is scaling out right now with lots of crappy boxes. We expect to fail.

Your mom lied; don’t share. Decentralize, expect failures and just add boxes. Amazon is one of the best at this.

CAP Theorem says you can only pick two of the following three: strong Consistency, high Availability, Partition tolerance.

What are my options? Denormalize, eventually consistent, parallel, asynchronous, specialize.

Denormalization is necessary in partitioned solutions and it’s becoming a huge problem for Digg. If you’re not using queues and messaging systems you’re going to want to look into gearman and djabberd. You wonder why things are going slow and you realize you’re doing 5 synchronous trips to the database. You’ve got to make these calls async with either http calls or gearman. One thing Digg is big on is running the numbers before you try and fix a problem. Run the numbers to make sure things actually will work. We’ll discuss a case of this.

Memcached, OMG Files! (MogileFS) Digg uses for icons and photos, Gearman is a massively distributed fork, and the new favorite toy: MemcacheDB “Will be the biggest new kid on the block in scaling.” Initial tests on a laptop yielded 15,000 writes a second. The developer behind this took Berkley DB and Memcache and brought them together.

Caching techniques: cache forever and explicitly expire, have a chain of responsibility. We had a generic expiration time on all objects at Digg. The problem is we have a lot of users and a lot of users that are inactive. Chain-of-Responsibility pattern creates a chain: mysql, memcache, apc, PHP globals. You’re first going to hit globals, if it has it you’ll get it straight back, if not go to the next link in the chain, etc. Used at Facebook and Digg. If you’re caching fairly static content you can get away with a file based cache, if it’s something requested a bunch go with memcache, if it’s something like a topic in Digg we use apc.

Partition your data horizontally (rows a-f on one machine) and vertically (some columns on one table, some on another table). Horizontal when you have so much data you need to spread it across a lot of servers. Vertical scaling: Instead of altering tables, add a new table and add new columns to it, this avoids downtime. Abstract your data access so that the partitioned details are hidden from the user.

Green badges at Digg are the bane of Joe’s existence. Similar problem to what Twitter and Digg have. If you take a message from one place and drop it in a bunch of other buckets. Kevin rose has 40,000 followers. You can’t drop something into 40,000 buckets synchronously. 300,000 to 320,000 diggs a day. If the average person has 100 followers that’s 300,000,000 Diggs day. The most active Diggers are the most followed Diggers. The idea of averages skews way out. “Not going to be 300 queries per second, 3,000 queries per second. 7gb of storage per day. 5tb of data across 50 to 60 servers so MySQL wasn’t going to work for us. That’s where memcachedb comes in.” The recommendation engine is a custom graph database from the R&D department and is eventually consistent. An example of problems you run into at real big scale on a social website.

[ Follow the Feed for notes on talks from other web leaders & innovators at the Web 2.0 Expo in New York going on this week. ]

Andrew Turner, Mikel Maron - Trends and Technologies in Where 2.0

Thu, 18 Sep 2008 12:52:12 -0400

Notes from the Web 2.0 Expo - NY talk given by Andrew Turner and Mikel Maron.

As people are going out and gathering information on their own we’re collecting a lot of geo-aware data. This is becoming a really hot area. Nokia and TomTom just made big acquisitions. Every Web 2.0 service is starting to add location. You can start mining this information with tools like geocodr.

How do you start gathering them together? We started a company called geocommons. We’re taking this massive amount of data and trying to pull it all together. It’s an open database of freely available data with creative commons license. You can see where it came from and who posted it. You can search the data.

What about when your communities are supplying a lot of data? In Detroit the city is geocoding walking trails. With Hurricane season there are lots of people geocoding where shelters are. A local NYC company social lite is doing place marking with bars using mobile web. Android has a lot of applications which are innovating on the geo aware capabilities of the phone.

Mapvertising is one way in which people are trying to make money in this space. But it’s hard. You don’t want to do a search for a romantic restaurant near you and get back a Hooters advertisement.

Once people are sharing all of this there is a problem of privacy. Flickr is looking at casual privacy where you can set who is allowed to see where your photos were taken. Fire eagle is a location brokering system. If you trust Yahoo! they can be the trusted holder of your location. You can specify which sites get which granularity (only zip code, for example) of knowledge of your data.

NeoCartography sites like EveryBlock and is trying to focus on the data as opposed to the street. You can look at and understand the area based on data. OpenCycleRoute allows you to re-render a map with the best routes for bikes.

We’re launching GeoCommons Maker in a couple of weeks which allows you to create proper maps.

Burning Man Experiment

You may think it’s a bunch of naked hippies in the desert blowing things up, and it is, but it’s a whole lot more than that. You show up in a desert and within a week you have a miniature city. It provides a canvas for trying out upcoming geocoding technologies.

We collected over 100 gigabytes of data over the week. These are early results. Why is this important for 2.0 Expo? This is a look at what these technologies can enable. Burning Man Earth was a really interesting iteration in Where 2.0.

We took remote sensing data every day. We used pictearth.com, diydrones.ning.com, openaerial.com. We got a small plane with a camera. We were gifted 200 Gallons of Fuel. We took a flight path every morning. Really cool pictures of how the event evolves day after day.

Processing with ERMapper, ESRI, Photoshop, Sweat. You also are recording with a GPS device which gives you where each photo is centered. Takes a lot of sweat to get everything lined up.

Underlying all of the projects we did is the GeoDjango platform. GeoDjango provides facilities for doing mapping. You can do interesting queries on geographic data pretty much for free. We took the PDF view of the city map, rasterized, and did rectification. Used ESRI, WMS, and Tiling software (TileCache). OpenLayers is an opensource mapping library in Javascript, has editing tools which allow you to draw over a rectified map.

In the Future we’re going to turn this into a social application.

OpenStreetMap (OSM) is a free map for the world like Wikipedia for maps. We export from GeoDjango to OSM XML. Import into OSM through its REST AP. Mapnik + mod_tile. The output is a tiled map. Flickr asked if they could take our tiles and use it for people to tag photos. Was really easy to do because we used basic tiles.

We need to start making our map tiles and our geotags time aware because the earth changes. By using OpenStreetMap you can get Garmin maps for free. There’s a freeware product called cGPSMapper. We used Garmin Rinos because they have radio built in so you can see where your friends are. At burning man it was incredibly useful. We also did vehicle tracking by sending packets over ham radio and APRS. Signal picked up by a digipeater which sends the data on. Some which take that positioning data and post it to the internet.

OpenViewProject.org – we were at WhereCamp at Yahoo and hacked Google street view’s data. Google sent a cease and desist. So a friend of mine bought a lot of gear and a tricycle so that he could do it himself.

Gigapans – a gigapan is a gigapixel image. Greater than 500 megapixels. NASA designed a little, sub $200 robot that captures panoramas and their software stitches it together. There’s a site Gigapan that allows you to view these massive photos and zoom in.

Kite Aerial Photography – you can script photos for cannons using their developer kit.

Google Earth & SketchUp models – Andrew Johnstone would take photos of art and texture models made in SketchUp.

Panel Discussion - Building in the Clouds: Scaling Web 2.0

Thu, 18 Sep 2008 12:27:40 -0400

Panel: Jason Hoffman (Joyent), Alistair Croll (Bitcurrent), Alex Barnett (From Bungee Labs to Intuit), Dwight Merriman (10Gen), Jinesh Varia (AWS), Pete Koomen (Google)

Panel session driven by Q&A.

Q) Decision between a component centric cloud and a service centric cloud? In a component centric I need to add instances to my app cluster (i.e. AWS), and in a service centric I write for a specific framework that scales itself (i.e. AppEngine). When does it make sense to focus on each?

A) Hoffman: I think they’ve already converged. It depends on the situation and you do both. The web app tiering has long been dead. You’re already silo’ing your assets. People are going to look at a given functionality in their site and ask what’s the service behind it?

Koomen: With App Engine it’s designed to handle low latency web applications.

Varia: Component clouds are great for flexibility. As the abstractions increase you lose flexibility and you also face lock-in on a technology stack.

Barnett: Scaling for what and why? How much up front consideration do start-ups need to put into becoming scalable? If you’ve only got a set of resources that isn’t infinite how do you face it? The nature and the type of the application will have fundamental implications to the underlying design.

Hoffman: Most web apps don’t have to scale in any reasonable amount of time. Another scaling issue is when you start out bigger and you don’t get enough traffic and have to scale down.

Q) Centralized computing & Distributed computing. Tension going on between centralized and distributed computing. Google has been buying thousands of net scalars and just this morning Amazon announced the cloud delivery service.

A) Merriman: Interesting fact that CDNs are one of the first forms of cloud computing. It’s an easy way to distribute content. Definitely use CDN for static.

Koomen: Scaling is about reducing the constant factor. Has to do with minimizing the amount of work you’re doing in the central server. Whether it’s in the CDN or the client side. It’s about a mentality of reducing what you’re doing on every request.

Hoffman: Amazon was smart about coming out with S3 before coming out with EC2. If you’re dealing with datasets less than a terabyte in size.

Varia: We have been listening a lot. From a scalability perspective many people needed data closer to their customers. Amazon is opening a CDN in 3 continents where the static data will be available from S3 with lower latencies and higher data transfer rates. Customers running RIAs feel it is key to serve content faster.

Q: How much can the edge help?

Hoffman: Outside of serving static content like images the edge doesn’t do anything.

Merriman: I don’t know that I agree with that because if you’re serving to data in Japan.

Hoffman: WAN optimization and network optimization is quite different than edge caching.

Q) How do you measure capacity and performance? What are the metrics you look at?

Kooman: Google cares a lot about CPU and latency. We can scale disk easy.

Hoffman: I think that’s the opposite end. There are things that take up space or move space. Disk space, CPU space, and network space. Then there’s the moving two and from these things. Most people in the real world are not coding against the CPU or CPU bound in a web app. Nobodies writing webapps that saturate the band that comes out of a single server. It takes a long time to fill up a terabyte. What people need is memory and better disk I/O. People still use relational databases. Disk I/O is the main thing.

Barnett: We also worry a lot about the end user experience. We’ve instrumented the AJAX library coming down to track every mouse click and interaction that an application has at a very granular level. You’re able to measure every click in a matter of milliseconds every single click and the latency on web service calls.

Hoffman: There doesn’t currently exist tooling to take end-user experience and feed that all the way back to capacity planning.

Merriman: We had to serve 10 – 20 billion ads per day. There’s a lot of CPU involved in picking which ads to serve. Other issue was just the database. “Have you seen this ad before? How many times?” Lots of data you access in real time and on the back-end on event processing. We looked at CPU a lot and I/O utilization on the database servers.

Varia: At Amazon, metrics is the key. From individual developer, to business, to our whole organization. From a developer we measure in time byte hours which is how much data that person is storing and how it grows. From S3 we measure the number of objects stored (22 billion objects stored) and the number of transactions. We peak at 50,000 transactions per second. We stay ahead of the curve. On the business side we need to understand our segmentation of large, medium, and small businesses.

Barnett: It’s interesting that when we charge for services on a utility model we

Koomen: We’re not going to be able to prevent people from taking out cloud services if they write bad code. So it’s important for us to be able to figure out where the problems exist and bubble that up to the user so they’re not making bad decisions.

Q: How do you guys deal with one rogue app?

Varia: Animoto is a very cool web 2.0 application where you upload your photos and music tracks in a way that it creates a really cool video out of it rendering your photos. They created a Facebook app they went from 25,000 users total, they went to adding 25,000 users every hour. Scaled from 50 servers to 5000 servers in 2 days. They were able to do this because they are built on a cloud platform. They scaled it down during the night time to save on cost. Some of these applications are bursting, no doubt. On an aggregate level the curve is pretty smooth. Amazon takes tremendous pride in figuring out how to add servers and services.

We have certain limits which prevent developers from starting 1000 instances. You are capped at 20 initially. If you want more you have to talk to us. There are security and safety mechanisms in place. If a business has a valid business case we’ll flip the switch.

Hoffman: If you have to spin up new virtual machines to handle traffic bursts you’re going to miss the burst.

Koomen: We deal with bursts like that by dealing with every request agnostically. To address the question from our side on what you do to prevent the users from exploiting a system. We’ve got quotas that measure what individual applications can consume and some knobs to turn.

Clay Shirky - Filter Failures

Thu, 18 Sep 2008 09:55:48 -0400

Clay Shirky is an American writer, consultant and teacher on the social and economic effects of Internet technologies. He teaches New Media as an adjunct professor at New York University's (NYU) graduate Interactive Telecommunications Program (ITP).

How many of you have seen this chart about information? (Chart of exponential data growth.) For 15 years we've been reading the same story about information overload. You can find stories in 93 that are the same as those that showed up in your RSS feed 3 minutes ago. Why is this?

Here's why I think this is: Guttenberg back when he invented the printing press, the cost of printing and the volume of what was being printed in those times was greater than a human could read in their whole life. Since you had to print the books in advance you had to take on all the risk of printing the books. The economic decision was pretty simple: make the editorial team at the publisher figure out what is worth printing.

The internet introduced post-guttenberg economics. The cost of producing anything by anyone has fallen through the floor. There's no economic logic requiring that you filter what's good from bad is now way down stream. What we're dealing with now isn't information overload, we're always dealing with that. Thinking about filter failure is the problem. Everyone has a morning ritual of removing spam from their e-mail. Some combination of a mechanical filter and manual filter.

All of the filter solutions are temporary because you have to assume that the ammount of information coming in is only going to increase. This is a general system design problem. Let me tell you what happened to a friend of mine: a former student, a colleague and a good friend. Last December decided to break off her engagement. She also had to engage in the 21st century practice of changing the status of her relationship. Might as well buy a billboard. She has a lot of friends on Facebook, but she also has a lot of friends on Facebook. She doesn't want all these folks, especially fiance's friends, to find out about this. She goes on to Facebook and thinks she's going to fix this problem. She finds their privacy policy and the interface for managing her privacy. She checks the appropriate check boxes and she's able to go from engaged to single. Two seconds later every single friend in her network get the message. E-mails, IMs, phone is ringing off the hook. Total disasterous privacy meltdown. She did her gradutae thesis on the differences of Friendster, Facebook, and MySpace. She's not an average user. Hard to blame Facebook when they've made so much of an effort. The problem is that managing privacy preferences is an unnatural act. Not something we're used to.

Privacy is a way of managing information flow. What she wanted to do is tell four or five of her friends and allow the information to slowly sweep through the network. That's how it used to work. The big problem we're facing with privacy now is how we're moving from a social system to an engineered, binary system. We live most of our lives not in the bubble of privacy, we have what we called back in the day our personal life. We don't have so much personal life anymore. The inneficiency of information flow wasn't a bug, it was a feature. There was something nice about it being difficult to say things in public. This is a question of filtering. That's a story of outbound information flow. Spam is a story of inbound.

There are some stories that you can't even tell the source. Here' s another story. By the time Chris Avenir was 15 MySpace existed. When he gets to college he enrolls in chemistry class. Because he's 18 he starts the study group on Facebook. He gets 146 of his classmates into the group. Suddenly he's facing 147 charges, one for setting up the group, one for every student he's working with. They claim he's cheating. What has he done? He's crashed two information flows together. To the inside world they say "come here, have conversations" to the outside world they have quality control "students are top notch." What keeps these apart are that the real world is outside of the walls. What he did on Facebook is break down that wall which gives the outside world the ability to see 146 students in a 'study group'. Facebook is not like a fax machine, not like a studygroup meeting. Facebook turns out to be a lot like Facebook. It's different from what's gone before. Only worth spending time on it because of that difference. There is no simple solution to this problem.

This isn't a design problem, it's a mental shift. It's understanding we'll have information overload just the way fish have water. A quote: "when you've faced a problem long enough, maybe it's not a problem, it's a fact." Some of this can be solved with programs: look at Digg, Google, etc. But a lot of it can't. When you're getting deluged with information the question you need to ask yourself is "what filter broke?"

Kris Jordan Blog

Micro PHP's Missing Tenet

"I believe functions are the smallest unit of functionality"

Damnit PHP, why can't you just...

1. Allow functions to be imported like classes

2. Indoctrinate a symbol loading algorithm

How should micro libraries share plain functions until PHP fixes importing and symbol loading?

Finally, will an isaacs please step up? PHP needs a modern package manager.

tl;dr

Footnotes

multimethod.js

What is a multimethod?

Motivating Examples

Multimethod Dispatch Algorithm

A More Detailed Walkthrough

The Basics

Dispatch Function

Deep Equality Matching

API Recap

Installation

How-to Contribute

Fixing WebKit's Accept Header

Contributing to Open Source Software

Refactoring Cache Tables using Memoize

An Exercise for the Reader

Refactor one-off caching via lookup tables with memoize

On Muted Methods, Fluent Interfaces, and Bar Game

What's on my mind...

Recess! Framework Screencast #1

Towards RESTful PHP - 5 Basic Tips

Tip #1: Using PUT and DELETE methods

Tip #2: Send Custom HTTP/1.1 Headers

Tip #3: Send Meaningful HTTP Headers

Tip #4: Don't Use $_SESSION

Tip #5: Test with cURL or rest-client

Tip #6 - Use a RESTful PHP Framework

PHP Class for HTTP Response Status Codes

How Recess Solves Common PHP/MySQL Issues

Dynamic Properties in PHP and StdClass

Simple, Built-in Dynamic Properties

StdClass: Anonymous Objects

How does the quality of political candidates’ websites correlate with campaign success?

Hello Again, Old Friend: Revisiting a PHP Framework

Brainstorming Ideas with Sharpies & Sticky Notes

10 High Order Bits from the Web 2.0 Expo in NY

10. Your Web App: Give it a REST

9. "It’s Not Information Overload, It’s Filter Failure"

8. Sensor Driven Data: The Web is Getting Orwellian

7. Javascript is Bringing Sexy Back to the Browser

6. The Open Web is Nearing the Tipping Point

5. Web Scalability thanks to Async & Danga

4. Web 2.0 Traffic: It's Out-of-Band

3. Cal Henderson's PHP Tent Revival

2. Set Your Baby Free

1. Want to Set the World on Fire? YOU Better Bring the Fire.

Joseph Smarr - Tying it All Together: Implementing the Open Web

Jeffrey Zeldman and Panel - Content Matters

Jay Adelson - Organizing Chaos: The Growth of Collaborative Filters

Arianna Huffington in Conversation with Tim O'Reilly

Sandy Jen - Scaling Synchronous Web Apps: Lessons Learned from Meebo

Chris Saad, Daniela Barbosa - Understanding the Basics of Personal Data and DataPortability

Joe Stump - Scaling Digg and Other Web Applications

Andrew Turner, Mikel Maron - Trends and Technologies in Where 2.0

Panel Discussion - Building in the Clouds: Scaling Web 2.0

Clay Shirky - Filter Failures

Refactor one-off caching via lookup tables with `memoize`