Monsur Hossain

I'm writing a book!

2013-11-14T00:00:00-06:00

My book, CORS in Action, is now available as part of Manning’s Early Access Program! I’ve been spending a lot of busy nights writing, and its crazy to finally see this live! Chapters 1-3 are available now, and more chapters (as well as edits to existing chapters) will be released on a regular basis. CORS in Action will provide a comprehensive overview of how cross-origin resource sharing works, from both a server's and a client's perspective. If you are curious about CORS, have a look here:

http://www.manning.com/hossain/

For a limited time, use code dotd1114tw at checkout out to recieve 50% off!

Favorite music of 2012

2012-12-12T00:00:00-06:00

Here are my favorite songs from 2012. This year I kept a running Rdio playlist of music I liked, so it was easy to pull this together. The tracks aren't in any particular order (well, alphabetical order), but if I had to rank my top 5 it would go:

Younger Us - Japandroids (with my favorite lyric of the year: "Remember that night you were already in bed, said "fuck it" got up to drink with me instead")
Angels - The xx
Thinkin Bout You - Frank Ocean
We Are Never Ever Getting Back Together - Taylor Swift (yes, seriously)
We Are Young - fun. (I love the juxtaposition of this song against the Japandroids song)

There were a lot of great albums this year, but for me, two perched above the rest and stayed on repeat for much of the year:

Japandroids - Celebration Rock - Is there a more awesome or appropriate album title? Kicks ass from start to finish.
The xx - Coexist - This album is like coming up for air after Celebration Rock. Begins and ends with two beautiful love songs.

benchmark.js: how it works

2012-12-11T00:00:00-06:00

I've been playing around with benchmark.js to do some performance testing in JavaScript. Its an easy-to-use library, but because its so easy, it hides a lot of its details behind the scenes. Mathias Bynens and John-David Dalton, the authors of benchmark.js, wrote a great overview of how benchmark.js works. But I wanted a better understanding of what's going on, so I took a deeper look at the code.

Here's an example of a performance test using benchmark.js. This particular test measures the performance of adding a span element to the page:

<html>
<body>

<a href="#" onclick="bench.run({async: true}); return false;">run test</a>

<div id="mydiv"></div>

<script src="https://raw.github.com/bestiejs/benchmark.js/v1.0.0/benchmark.js"></script>
<script>

var bench = new Benchmark('insertNode',

// The function to test
function() {
  mydiv.insertAdjacentHTML('beforeend', '<span></span>');
},

// Additional options for the test
{
  'setup': function() {
    var mydiv = document.getElementById('mydiv');
  },
  'teardown': function() {
    mydiv.innerHTML = '';
  }
});
</script>

</body>
</html>

The code above represents a single benchmark, which is the performance test and any associated code. The fundamental unit of work in a benchmark is the test function. This is the thing we are measuring. Traditional performance tests ask the user to specify how many times to run the test (for example, run this test 1000 times and tell me how long it takes). The magic of benchmark.js is that it automatically calculates the number of iterations. The user is responsible for writing the test, benchmark.js handles the rest.

Check out the weird scoping in this particular test: mydiv is declared in the setup function, but is referenced in the test function and the teardown function. This works because benchmark.js doesn't call those functions directly. Instead, it strips the JS code out of those functions and "compiles" it to this:

function(t1354904061091) {
  var r1354904061091,
      s1354904061091,
      m1354904061091=this,
      f1354904061091=m1354904061091.fn,
      i1354904061091=m1354904061091.count,
      n1354904061091=t1354904061091.ns;

  // From setup()
  var mydiv = document.getElementById('mydiv');

  s1354904061091=n1354904061091.now();
  while(i1354904061091--) {
    // From test function.
    mydiv.insertAdjacentHTML('beforeend', '<span></span>');
  }
  r1354904061091=(n1354904061091.now()-s1354904061091)/1e3;

  // From teardown()
  mydiv.innerHTML = '';

  return {elapsed: r1354904061091, uid: "uid1354904061091"};
}

This is the heart of a benchmark.js test. Don’t let the obfuscated variable names fool you; this code is very simple. It looks like a performance test one would write by hand: call the setup function, start the timer, run the test function a few times, measure the time, and finally call the teardown function. I call this chunk of code a unit. The setup, test, and teardown functions in a unit all share the same scope, and are run in the context of the benchmark itself (i.e. this === the benchmark instance).

Units form the building blocks of a cycle. There is almost a 1-to-1 mapping between a unit and a cycle, but a cycle actually consists of two units. The first unit runs the setup, test and teardown functions one time to see if the code throws any errors (this is a nice feature; we don't want to run the entire test only to discover an error in the teardown function). If there are no errors, the unit runs again, this time with multiple iterations. (I’m not sure why the error check is done once per cycle, it seems like something that could be done once per benchmark).

So how does benchmark.js actually determine how many times to run a test function? Benchmark.js aims to run a test as fast as possible without sacrificing accuracy. It finds this sweet spot by running a few cycles to get a sense of how long the test runs. I call this the analysis phase. It starts by dipping its toes in the water with a few iterations, and then continues to increase the number of iterations until it reaches a percent uncertainty of at most 1% (or a min time or max time is reached, if specified by the user).

A note about the timer used to run the test. The code above measures time by calling the obscure n1354904061091.now(). The actual timer could be any one of the following:

The W3C's high-resolution timer (which is just starting to make its way into browsers and Node.js).
Chrome's Interval microsecond timer (which can be enabled with the --enable-benchmarking flag).
Java's System.nanoTime() timer (exposed through a Java applet).
Wade Simmons' Node.js microtime module.
JavaScript's Date object (low resolution, only used if none of the above are available).

Benchmark.js chooses the timer with the finest resolution available on the platform. It takes the timer's resolution into account when calculating the number of iterations. So while using JavaScript's Date object (with a resolution of 15ms) is not ideal, benchmark.js compensates by running the tests for more iterations.

Once a number of iterations is calculated, benchmark.js enters what I call the sampling phase. This phase runs the tests and stores the results of each cycle in the Benchmark.prototype.stats.sample array. The number of iterations is stored in Benchmark.prototype.count. This is the number of iterations per cycle, not the total number of iterations across all cycles. The count is not necessarily fixed; it may increase if benchmark.js determines it needs more iterations to get an accurate reading.

One point of confusion is the Benchmark.prototype.cycles property. It sounds like it should store the total number of cycles run by the benchmark. But instead it stores the number of cycles run during the analysis phase. If you’d like the number of cycles run during the sampling phase, look at the length of the Benchmark.prototype.stats.sample array.

As the test runs, the Benchmark.prototype.stats object stores the results. The stats are calculated and updated after each cycle, so they will exist even if the benchmark is aborted before finishing. There are a few places where results are stored:

Benchmark.prototype.hz - the calculated number of iterations per second. If you want one number to represent the test results, this is it (it is the number jsperf.com displays after running a test).
Benchmark.prototype.stats - various stats, including mean, margin of error, standard deviation and other goodies.
Benchmark.prototype.times - various timing related stats.

A benchmark also emits various events during the course of a test:

onStart - Called once, before the entire benchmark starts.
onCycle - Called after each cycle completes. Fires during both the analysis and sampling phases.
onComplete - Called once, after the entire benchmark completes.
onError - Called if the JS code has an error.
onAbort - Called if the test is aborted.
onReset - Reset properties (and abort if running).

Finally, benchmarks can also be organized into a suite. A suite is a collection of benchmarks, and is useful for grouping benchmarks. A suite has methods to operate over its benchmarks (such as forEach), as well as an analogous set of events that operate at the suite level (for example, onCycle is called after each benchmark completes).

I hope this gives a good introduction to what happens when runnning a benchmark.js test. To recap, here's an outline of how a test is executed:

For each benchmark in a suite:

Fire event: Suite.onStart()
For each benchmark:

Fire event: Benchmark.onStart()
For each sampled run

Run unit once to check for errors

setup()
testfn()
teardown()

Run unit multiple times and measure results

setup()
for each Benchmark.count

testfn()

teardown()

Fire event: Benchmark.onCycle()

Fire event: Benchmark.onComplete()
Fire event: Suite.onCycle()

Fire event: Suite.onComplete()

enable-cors.org

2012-12-10T00:00:00-06:00

Michael Hausenblas and I have just released a revamped version of the enable-cors.org site. The site includes resources on using CORS, including details on how to configure CORS for various servers. It also has test-cors.org, which helps test CORS support on various servers. The site's source code is hosted on GitHub, so feel free to contribute!

Why I Vote

2012-11-04T00:00:00-05:00

Its around this time of the election year (and over and over throughout the years) that I hear the common refrain of “your vote doesn’t matter” (especially in a decidedly “blue” state like Illinois). Yes, I understand the mathematics of how one vote out of 100 million will not make a dent in the outcome. But I don’t believe that means you shouldn’t vote.

Consider the Jim Crow laws that promoted racial segregation and permeated the South for almost 100 years. One reason for their existence was that the same politicians were elected into office year after year. There was nothing shady about this, they were legitimately elected. The problem was that those who would vote to change the laws had no vote. From Wikipedia:

...the establishment Democrats were passing laws to make voter registration and electoral rules more restrictive, with the result that political participation by most blacks and many poor whites began to decrease. Between 1890 and 1910, ten of the eleven former Confederate states, starting with Mississippi, passed new constitutions or amendments that effectively disfranchised most blacks and tens of thousands of poor whites through a combination of poll taxes, literacy and comprehension tests, and residency and record-keeping requirements. Grandfather clauses temporarily permitted some illiterate whites to vote.

Voter turnout dropped drastically through the South as a result of such measures.

...Those who could not vote were not eligible to serve on juries and could not run for local offices. They effectively disappeared from political life, as they could not influence the state legislatures, and their interests were overlooked.

Many of the arguments justifying voting tend towards a philosophical call to duty. But here is a concrete example from this country’s history of how significant voting is. Regardless of whether a large group of people can’t vote (due to unjust voter registration laws back then) or won’t vote (due to voter apathy now), the impact on the ballot is the same. I’m not saying that not voting leads to racist laws, but only that our rights should not be taken for granted; without a vote, without that voice, we stand to lose a lot.

Thoughts on the CORS preflight cache

2012-09-07T00:00:00-05:00

Cross-Origin Resource Sharing (CORS) opens up some awesome possibilities for accessing APIs from JavaScript. But certain characteristics of REST APIs work against CORS:

JSON APIs return a Content-Type of application/json (while other formats have their own content types).
API resources are addressed with unique URLs.

Measure this against the rules for CORS preflights:

A request for a Content-Type other than application/x-www-form-urlencoded, multipart/form-data, or text/plain triggers a preflight request. This means that every request to a JSON API (even GET or POST requests) triggers a preflight.

In order to reduce the number of preflight requests, CORS has the concept of a preflight cache. However the preflight information is cached for an origin/url pair. That means each unique url has its own preflight cache. An API with the same preflight response across all urls will still receive a preflight request for each unique request.

Finally consider the fact that WebKit browsers cache preflight response for a maximum of 5 minutes (source code). According to a comment in the code, this is to prevent cache poisoning. (Firefox has a limit of 24 hours, I have no idea if IE has a limit). This cache limit can be misleading: even with an Access-Control-Max-Age value of, say, 3600 (seconds), developers will see a new preflight request after only 5 minutes.

Taken together, these issues limit the usefulness of the preflight cache. The most useful case would be an app that hits the same endpoint over and over again (maybe some sort of polling mechanism?). But for a long running app making requests to a variety of urls (a common scenario for most web apps), the cache does little to avoid the preflight performance hit.

There are some hacks that can help improve on this:

The API content type can return one of the accepted “simple” values for Content-Type, such as application/x-www-form-urlencoded, multipart/form-data, or text/plain. This will avoid the preflight request altogether (for GET/POST only, and only if there are no other custom headers)

Variables in the path can be moved to query parameters. This reduces the API’s url space, and fewer urls means a higher preflight cache hit ratio.

At the extreme, an API format that funnels all requests through a single url (such as JSON-RPC) will ensure that the preflight cache applies to all requests.

Of course, all these suggestions sacrifice the “RESTfulness” of the API (if you're into that sort of thing).

Ideally, it would great if preflight information could be cached for an entire domain or a wildcard path. I proposed this on the W3 publicapps mailing list, but there are concerns over how to do this securely. Still, I think finding some solution to this issue would improve performance and make CORS a more palatable option for APIs.

Differences in JavaScript escaping functions

2012-07-30T00:00:00-05:00

My previous post forced me to delve into the differences between the escape, encodeURI, and encodeURIComponent functions. Here's a table that highlights the differences between each (generated, of course, via JavaScript):

ASCII char	escape()	encodeURI()	encodeURIComponent()

(Note that the '*' character is not encoded by any of these functions.)

UTF-8 in JavaScript

2012-07-20T00:00:00-05:00

Working with string encodings in JavaScript can sometimes be frustrating. My latest frustrations came when working with the atob and btoa browser functions. These functions convert between a binary string and a Base64 encoded ASCII string. But they blow up when faced with Unicode:

> btoa('\u0227'); Error: INVALID_CHARACTER_ERR: DOM Exception 5

The MDN docs on btoa point to the following functions from Johan Sundström for working with Unicode:

function encode_utf8( s ) { return unescape( encodeURIComponent( s ) ); } function decode_utf8( s ) { return decodeURIComponent( escape( s ) ); }

Armed with these functions, btoa gives the expected result:

> btoa(encode_utf8('\u0227')); "yKc="

Johan’s functions work, but they feel like a bit of black magic. So I decided to delve into each part of the encode_utf8 function to understand why it works.

encodeURIComponent

The encode_utf8 function starts with encodeURIComponent. encodeURIComponent is defined in the latest ECMAScript spec:

The encodeURIComponent function computes a new version of a URI in which each instance of certain characters is replaced by one, two, three, or four escape sequences representing the UTF-8 encoding of the character.

What struck me right off the bat is that encodeURIComponent is tied to UTF-8. Its not like similar functions other languages where the encoding can be specified as a function argument. This is fine if you want to work with UTF-8, but not if you want to work with other encodings (although there are ways to manage fixed-length encodings using ArrayBuffers).

Here’s an example of using encodeURIComponent on the Unicode character U+0227 (ȧ, LATIN SMALL LETTER A WITH DOT ABOVE):

> encodeURIComponent('\u0227'); "%C8%A7"

The result is a percent-encoded string of each byte of the UTF-8 representation of the character. Looking at the documentation for Unicode character U+0227, we can verify that the UTF-8 representation in hex is indeed 0xC8 0xA7.

The ECMAScript definition is vague; it states that encodeURIComponent replaces "certain characters", without defining what those characters are. But a quick hack shows that any character over 0x7E is encoded, so I think it's safe to say that any non-ASCII character will be encoded by encodeURIComponent. Since percent-encoding only uses the numbers 0-9, letters A-F and the ‘%’ character, the resulting string is guaranteed to be ASCII.

Now we could stop here. All btoa needs is an ASCII string, which encodeURIComponent provides:

> btoa(encodeURIComponent('\u0227')); "JUM4JUE3"

But there are a few drawbacks to this. The first is that the resulting string is not the same bytes as the input string. It is an encoded representation of the original bytes. This could make things annoying or difficult when interoperating with other systems.

The second drawback is that encodeURIComponent produces a larger string. A single Unicode character could take up to 4 bytes in UTF-8, which would produce a URI-encoded string of 12 characters. After Base64 encoding (which makes strings larger), the output string would be much larger than the input. In order to tame the string size, Johan turns to the unescape function.

unescape

Unlike encodeURIComponent, the escape/unescape functions are NOT defined by the latest version of ECMAScript. According to JavaScript: The Definitive Guide (6th Edition, page 855):

Although unescape was standardized in the first verison of ECMAScript, it has been deprecated and removed from the standard by ECMAScript v3. Implementations of ECMAScript are likely to implement this function, but they are not required to.

But seeing as how these functions have been a part of browsers from almost the beginning, I doubt they will disappear anytime soon.

The unescape function unescapes a percent-encoded string. It works with the standard percent-encoding (%XX) as well as the non-standard Unicode (%uXXXX) percent encoding. Here’s the result of unescaping the string from the original example:

> var e = encodeURIComponent('\u0227'); console.log(e); "%C8%A7" > var u = unescape(e); console.log(u); "È§"

"È§" are the ASCII characters for C8 and A7, respectively:

> u.charCodeAt(0).toString(16); "c8" > u.charCodeAt(1).toString(16); "a7"

Calling unescape is very different from calling decodeURIComponent. decodeURIComponent interprets the string with UTF-8, which means it would combine the two encoded characters back into their original UTF-8 representation. unescape merely returns the characters without any interpretation.

The benefits of using unescape is that we now have a representation of the actual UTF-8 bytes, and it is a smaller string:

> e.length 6 > u.length 2

The bytes in the new string are the ASCII-representation of the bytes in the UTF-8 representation of the original string.

Conclusion

Putting it all together, here are the transformations the string goes through in encode_utf8:

Original Unicode string	encodeURIComponent	unescape
ȧ	%C8%A7	È§

Johan’s bit of black magic works because it turns a Unicode string into an encoded UTF-8 string, and then returns the ASCII representation of the encoded bytes. What I love about this method is that it uses the fact that browser's have multiple encoding functions to its advantage. Using only the pair of encodeURIComponent/decodeURIComponent or escape/unescape would be a no-op; the functions would cancel each other out and return the original string. The genius of Johan's method is that it uses the two different encoding functions in tandem to get what we need.

App Reset 2012

2012-07-14T00:00:00-05:00

I just finished a mobile app reset to go along with my password reset. The recent Facebook email fiasco gave me pause; though I wasn't affected, it was a reminder of how much power apps can have, and how important app permissions are.

So I took a good look at the apps I had installed and deleted all but the ones I really wanted. It was easy to delete the apps I no longer use. But taking things a step further, I deleted any app that also has a decent mobile web experience. Most services have at least a simple mobile site, while sites like Twitter or Google+ have sites that rival their apps. There are still a few features missing from the browser, like camera and notification support, but I don't really miss them yet (and I’m trying to cut back on notifications anyway).

I imagine there will come a day when all or most phone features will have a corresponding web API. While that gives the browser a lot of power, it still feels safer to me since:

browser APIs operate on an ask-first permissions model. For example, the browser asks the user before giving location information to the site. Contrast this to the app model, which must ask for all permissions up front, even if the user doesn’t need or want them.
if there is a browser vulnerability, the fix applies across all websites. In an app model, each company is responsible for its app’s own security.

I'm focused on the consumer security aspect here, but there are a lot of other good reasons to develop for the mobile web.

What’s left on my phone are apps that can only operate as apps (such as Kindle, BeyondPod, Sonos and Rdio), and then a bunch of Chrome bookmarks. The bookmarks look like app icons, but they all lead to mobile web sites:

Password Reset 2012

2012-07-10T00:00:00-05:00

I recently finished a huge revamp of my password management system by moving to a one-password-per-site model. Here’s why I decided on this model and how I implemented it.

I was using a base password system to generate passwords I could memorize. The recent breaches at high-profile sites like Zappos, Last.fm and LinkedIn made me rethink this strategy. Remembering passwords doesn’t scale with the growing number of online services we use. One-password-per-site is perhaps the best model (and maybe even a bit overkill), but I was nervous about trusting my passwords to a password manager. But a few realizations made me more comfortable:

I do a horrible job of remembering usernames and passwords anyway. Every time I’m faced with a login form, I sit there staring blankly trying to remember how to get in. I had a handful of passwords and usernames I would cycle through, but trying to remember the right combination was often futile (at worst, I’d get locked out before I hit the right combo). I found myself using the “reset password” link out of convenience. I don’t want these various username/password combinations taking up space in my head. I want to fill those nooks and crannies of my brain with more useful bits of information. Because...
Passwords are just a means to an end. It’s a subtle point, but there’s nothing intrinsic in a password that makes it a necessary part of accessing your data. Passwords are merely the inconvenient price we pay for online convenience. If I lost online access tomorrow, I could still call customer service in order to manage my bank account. I can always request a new password if I absolutely need to access a site but don’t know my password. Forgetting a password is not the end of the world: just get a new one.
Device consolidation. Even when memorizing passwords, the only devices I trust with my passwords are my laptop and the phone. Once I’ve logged into these devices, I seldom need to log back in again (especially thanks to Chrome sync). I avoid logging in from public computers, and can’t imagine a critical situation where I couldn’t wait to log in from one of my devices.

Once I was comfortable with the idea of one-password-per-site, here’s how I implemented it:

All passwords are stored in KeePass. I only need KeePass for a Mac (via KeePassX) and an Android phone (via KeePassDroid), but it is also available on various platforms if necessary. I like KeePass because its free, open-source, and not cloud-based. I’ve heard a lot of great things about LastPass, but I’m still not comfortable with the idea of a third-party managing my passwords. The system I describe here has many of the benefits of a cloud password manager without introducing a third-party player.
Each site gets its own unique password generated by KeePass. I generate a 25 character password with letters, numbers and symbols (unless the site has its own restrictions).
There are certain passwords I still need to memorize, such as the password to the KeePass database itself (of course), passwords to apps I share with my wife (such as Hulu+ and Rdio), passwords to anything work-related and various odds and ends (like my router password).
I also memorize the password to my Google account. This is important because my email is on Gmail, and email is the proxy for identity on most services.
The KeePass database is stored in Google Drive. This allows me to access the database from any computer I trust.
Two-factor auth is enabled on my Google account.
I have a printed copy of my passwords that I keep with my other important documents (such as passports). This is stored somewhere only my wife and I know how to access.
The flow for logging into a site goes like this:
- Open up KeePassX.
- Find the site I want to log into to
- Hit Ctrl-U to navigate to the site
- Hit Ctrl-B to copy the username, paste that into the site
- Hit Ctrl-C to copy the password, paste that into the site
There are probably browser plugins that would make this process easier, but I’m happy with it for now. The flow in Android is similar, except KeePassDroid provides a handy shortcut to copy the username/password from the notification bar.

I’ve been migrating sites to this scheme over the past month, and I’m happy with the results. I don’t waste time staring blankly at a login screen anymore. I just hit a few keys and keep moving forward.

Why I'm writing my own Instapaper

2012-04-25T00:00:00-05:00

I'm a huge fan of "read-later" apps like Instapaper and Pocket. But I'm not a fan of how they present you with little more beyond a list of links. Managing these lists can quickly feel like a burden. When using Instapaper, I find myself spending more time searching for something to read rather than actually reading.

What I want is an app that not only saves links, but also suggests what to read next. My ideal app would have the following features:

Bubble up the sources that are most important to me.
Categorize links by subject (e.g. "technology", "music", etc).
Categorize links by type ("video", "podcast" etc).
Highlight popular content. In the same vein, demote content as it becomes less relevant (For example, a news item about the current Republican race probably won't be relevant a year from now).
Be aware of content length in order to show me shorter links when I have a few minutes to kill, or give me a longer article when I have more time.

In order to play around with some of these ideas, I wrote a proof-of-concept app myself. The app is hosted on App Engine and uses Pinboard as a backend. You can browse the source code, but its not hosted anywhere for others to play with. I'm definitely not looking to write yet another "read-later" app. My hope is that someday tools like Instapaper will incorporate features like this.

Here are some details on how the app works.

Links are added via a bookmarklet. Only the link's url and title are saved. I'm not including the entire contents of the page, which is one of the features of the Instapaper bookmarklet.
Once the link reaches the server, it passes through a series of filters for post-processing and categorization.
Links are first categorized by type. If the link is a video (from YouTube or Vimeo), it is assigned the "to watch" tag. Links from GitHub are categorized as "to hack". Browsing a project on GitHub is a different experience from reading an article; I usually want to spend some time digging into the project's source code and running it locally. Everything else is categorized as "to read". I can also imagine more categories in the future, like books I want to read or movies I want to watch.
High priority sites are tagged with the "c:priority" tag (The "c:" prefix denotes a category tag; I may drop the prefix in the future). This is the first tag I visit when I want to read something, and it ensures that I'm up-to-date with the sites I care about. Other categories include "c:news", for stuff like NY Times articles, and "c:tech" for tech-related sites. Eventually this categorization could be done by some semantic text analysis API like Alchemy.
The source and referrer domains are included as tags, so I can easily browse links from a particular location.
In addition to the bookmarklet, I've implemented post-by-email support. This is useful on mobile platforms where installing a bookmarklet isn't feasible.
All this data is stored in Pinboard. You can see the results in my Pinboard account. I chose Pinboard because of its great tagging support. Tags are the glue that ties the app together. Since this is an experimental app, I like the flexibility that tags offer.

This code is still a work in progress, here are some features I'd like to add in the future:

Improved browsing. Saving links is only half the story; the app also needs an easy and intuitive way to find content. I'm currently using Pinboard's web UI, but I envision building an interface that is context-aware and allows sorting along the many dimensions mentioned above. For example, browsing the "to watch" tag could present video thumbnails rather than a list of links.
Store the content size. I'd like to be able to sort the content by size, so that I can pick an article appropriate for the time I have. The size of a page can easily by retrieved from JavaScript by using document.body.innerHTML. But its not that simple, since a page could be littered with html and ads but not a lot of content. Instapaper faces the same content extraction problem when presenting its "clean" reading interface.
Better stats reporting. The downside of using Pinboard is that I'm constrained to Pinboard's data model. There are a few fields such as "date added" and "date read" that could be useful for reporting stats. Low reading stats, or a lot of links from one category that haven't been read, can signal an opportunity to improve the app.
One of the nice features of Pinboard is that it shows you how many other people have saved the same link. This is kind of a proxy for how popular a link is. I wish Pinboard had the option to sort links by popularity; the API doesn't seem to have this option either.

At the end of the day, I want this app to do the heavy lifting of organizing links for me so I don't have to. This frees up my time to actually do some reading!

oscars.txt

2012-01-24T00:00:00-06:00

Ack, my eyes! The Academy Award nominees were announced today, and their site is just so, so, so painfully bad. It so difficult to browse the nominees, I'm not even going to link to it here.

So I've created the anti-Oscars site; just a simple text file with the nominees:

https://raw.github.com/monsur/oscars.txt/master/oscars.2012.txt

No frills, no ads, no bullshit, just the information you need. I thought about more exotic formats like JSON, but this is all I can do on my lunch hour. If you'd like to do more with it, here's the "source": https://github.com/monsur/oscars.txt

Leaving GoDaddy: why this goes beyond Dec 29th

2011-12-30T00:00:00-06:00

GoDaddy’s recent drama around SOPA was just the swift kick in the pants I needed to actually make the switch (The fact that the company has since changed its tune makes no difference to me). A lot of the press around this has been focused on the Dec 29th "boycott" day, but I think the lessons learned go beyond that. Here are my takeaways from the transfer experience:

There are choices. I thought GoDaddy was the only game in town, and that any other domain registrar out there must be even worse. This is simply not true. There are a bunch of registrars out there that people have used and recommended for years.

The domain transfer process is not hard. I avoided the transfer process because I assumed it was just too much of a pain to bother. But its not that bad at all. The entire process hinges on the EPP code: get the code from your old registrar, give it to the new registrar, and let them handle the rest.

It doesn’t stop here. I don’t expect the 5 domains I’m transferring away from GoDaddy to make much of a difference to them. But the next time I need to register a domain, it will not be with GoDaddy. The next time a friend asks where he should register a domain, I will not recommend GoDaddy. And if I meet someone who wants to switch from GoDaddy but thinks its too hard, I’m prepared to explain how easy it actually is. December 29th is just the beginning.

I am in control. But most of all, this experience has taught me not to feel trapped when it comes to domain registration. Rather than be at the mercy of the domain registrar, I am the one in control. If it doesn’t work out with my new registrar, I have the confidence and know-how to always switch.

Listening to music 2011

2011-12-21T00:00:00-06:00

2011 was the first year where my music listening was driven completely from the cloud. This is a milestone for me; in the past I’ve always had some physical component to the music I owned, whether it be an MP3 or a CD. For someone who’s first music was on a cassette tape, this is the future.

I rely on Rdio for most music, and if it is not available there, I buy/download it and then upload it back to Google Music. Google Music has also been good for capturing free mixes or CDs not available online. I haven’t saved a single MP3 to my computer after adopting this hybrid Rdio/Google Music listening system (And in fact, I don’t think I’ve saved any file locally, and only locally, in a long time).

I’m surprised that I’ve settled on this hybrid listening system. I’m a big fan of subscription services, and never bought into Steve Jobs’ claim that “People want to own their music.” But storage services such as Google Music help fill in the blanks when music isn’t available from a subscription service.

Looking towards the future, I hope to see even this subscription/storage split disappear. Right now, music sources are tied directly to music players. I can only listen to Rdio tracks from Rdio’s site/app, same for Google Music (or Spotify or Mog or Pandora or Amazon Cloud Player). I would love to see music move towards a model where audio sources are decoupled from audio players. So I could pick a few songs from Rdio, and few more from Google Music, and send them to any player (car, phone, home stereo) without using siloed apps. This would help fulfill the dream of all music, everywhere. Screw world peace, that’s my wish for 2012.

Favorites of 2011

2011-12-19T00:00:00-06:00

Here are some of my favorite songs of 2011 (also available as an Rdio playlist). I wouldn't call this a "best-of", just some songs I've enjoyed over the year:

We're Here For a Good Time (Not a Long Time) - The Besnard Lakes
Romance - Wild Flag
Nervous - Joan as Police Woman
Leftovers - PS I Love You
Serve the People - Handsome Furs
Cruel - St. Vincent
Estimate x 3 - Centro-Matic
Mama - White Fang
The Show Goes On - Lupe Fiasco
Skin and Bones - Parts & Labor
Born Alone - Wilco
Midnight City - M83
You Know What I Mean - Cults
Jamie Marie - Girls
All The Same - Real Estate

Using CORS

2011-11-27T00:00:00-06:00

The tutorial I wrote for html5rocks.com on using Cross-Origin Resource Sharing is now live! Check it out!

Another year

2011-10-06T00:00:00-05:00

One of the features I wish Rdio had is to listen to your collection by year. For example, I'd like to listen to all the tracks in my collection from 2011. So I wrote a Python script to generate Rdio playlists by year:

https://github.com/monsur/another-year

Managing cognitive surplus?

2011-09-27T00:00:00-05:00

In Google+, Twitter, Tumblr and Blogger, we have tools for creating content. In Instapaper, we have tools for saving content. But I’m wondering what, if any, tools we do have for managing our cognitive surplus?

Put another way, you have X hours of free time in a day, how do you choose what to do?

You could pick off articles from your Instapaper queue. But it doesn’t have to be limited to online activities. You also wanted to watch that new TV show, go for a walk, or learn to play the piano. What do you do?

Maybe I’m just overthinking things. if you have some free time, just do what you want at the moment. Rapid-fire sites like Twitter encourage us to drink from the firehose and live in the now. There’s a certain calming effect to this letting go. You can’t keep up with everything otherwise you’ll drown, and most things aren’t as important as you first think.

But in order to truly learn and absorb a subject, you have to spend time with it, which is the opposite of the rapid-fire approach. And as the lists of things we want to consume continually grows, we tend to forget the things we once wanted to learn. Just keeping up prevents us from truly mastering anything.

The new Delicious launched today. Various articles about the relaunch touch on the problem of information overload, and I was hopeful that the new Delicious would help manage our cognitive surplus. But browsing the site today, it still feels like a list-making site at heart. This isn’t a bad thing, but I’m not convinced its helping with information overload.

So, I’m curious, how do you manage your cognitive surplus?

GoogleCL and the Google+ API

2011-09-15T00:00:00-05:00

If you are a command-line person, GoogleCL is a quick way to get acquainted with the Google+ API. Try this:

Install GoogleCL using the instructions here.
Run the following command line: google plus activities list me public

You will need to authenticate with your browser the first time through, but after that, things will work without the browser. Then you can do something like running the output through "say":

google plus activities list me public --fields items/object/content --maxResults 1 | say

to have your computer read you your Google+ posts!

This is a silly example, but it highlights the power of the Google APIs Discovery Service. Upon launch, the Google+ API was available in GoogleCL, the Google APIs Explorer, and the .NET, GWT, Java, Objective-C, PHP, Python, Ruby and Go API clients! And as the Google+ API grows, these tools will almost instantly grow right along with it. Pretty neat, huh? You can learn more about how this all works here: http://code.google.com/apis/discovery/

I got a new phone!

2011-09-12T00:00:00-05:00

Finally, after months of waiting and speculation, I got the Droid Bionic on Verizon. There are a lot of great phones rumored to be coming out soon, but in the end I wanted to be on Verizon, I wanted 4GLTE, and I wanted it now!

So whats my verdict?

I love it!

Here are some more details.

The Good

The phone is fast and responsive! Seriously, I have yet to notice any lag when scrolling apps and web pages.
4GLTE - Chicago seems to get excellent 4GLTE coverage. I’ve been running speed test from various locations, and the average speed seems to be around 10Mbps. And I got this gem when driving down Lawerence:
Phone quality sounds great too; I have yet to experience a dropped call or diminished quality. Those three points above address my biggest frustrations with AT&T/Edge.
The screen is bright and sharp. It looks great in direct sunlight. Some users have mentioned seeing a grid in the qHD display. And I can see what they mean; every now and then I’ll catch it in my periphrial vision. It doesn’t bother me, but maybe it can be a nuisance for some.

The Bad

Size. The phone is huge. My hand started cramping up the first day! (I'm better now, thanks for asking). I prefer the smaller form factor like the Nexus or iPhone, but sadly this seems to be the trend with new phones. I'm guessing they want to promote more media usage; the Bionic even has a mini-HDMI port, which I’m sure I’ll never use. I was worried that it would feel bulky in my pocket, but the is thin enough to avoid that.
I lost my Angry Bird levels! Anyone know how to get them back?

Rdio vs. Spotify

2011-08-02T00:00:00-05:00

Ever since Spotify launched in the US, I've been curious about the availabilty of music on Rdio vs. Spotify. So I wrote this app to query the Rdio and Spotify API for results. Enjoy!

http://canilisten.appspot.com/

Chrome Extensions and Cross-Domain Requests

2011-07-07T00:00:00-05:00

Saving this here mostly for my own reference. Chrome Extensions have two "modes" when making cross-domain XHR requests:

If the domain is in the "permissions" section of the manifest.json file - The request doesn't have an "Origin" header, and it always succeeds.
If the domain is not in "permissions" - The request includes an "Origin" header with the value "chrome-extension://..." This indicates that the request is a CORS request, and the response must have a valid Access-Control-Allow-Origin header in order to succeed.

Batting pi

2011-03-29T00:00:00-05:00

A little nugget from yesterday's Baseball Today podcast. Apparently Jim Thome has the single-season batting average closest to pi (or pi/10 to be exact). In 1995, Thome went 142 for 452, for a batting average of .3141592, or 6 decimal digits of pi. The next number in pi is a 6, while in Thome's average is a 9.

How I Tumblr

2011-01-06T00:00:00-06:00

My last few posts have explored the types of lists I have and how to keep them manageable. Lately I've been consolidating the list of Things I Like on Tumblr. Tumblr's flexible interface allows it to store and display various types of information. So far I'm using my Tumblr site to store the following:

Movies I've watched
Books I've read
Music I like
Quotes I like
Links I like
Videos I like
Some combination of all of the above

I like Tumblr for the sense of curation you get from hand-crafting each post. Delicious is great for storing and tagging links, but scrolling through a bland list of links leaves you feeling cold. Tumblr allows each post to have quotes, images, flash, and html descriptions. This flexibility gives each post some room to breathe. Capturing a movie I liked can include an image and a review. I can post Friday I'm in Love from Rdio along with a cool Threadless T-Shirt, just because. And I like how all the items from this amorphous list of Things I Like can be stored in one place. Its like you're curating your own museum of interesting stuff.

I keep these random stuff organized with the following tags:

If a post is about a specific media type, I tag it as such (e.g. "book", "music" "movie")
If I finished consuming the media, I tag it with "done". So books I've read and movies I've watched get tagged with "done". I can never remember what I've read or watched over the course of the year, the "done" tag is a digital memory of that.
If I liked the item, I'll also tag it with "like".

So at the end of this year, I can filter the tags by "movie" and "done" to see all the movies I watched, and then filter the list further by "like" to see the movies I liked. Same for books and music I like. In fact, searching for "music" and "like" will basically be a playlist of my favorite songs from the year!

So how is this different from a blog? As I mention in my previous post, this blog is reserved for longer, original, more thoughtful pieces. Tumblr captures quick snippets of consumption. The difference is volume. Tumblr will receive a much higher volume of posts than Blogger, and I like being able to separate the two, otherwise the Blogger posts get drowned out in the Tumblr noise. This quote from Anil Dash captures a similar sentiment (captured, of course, on my Tumblr):

make sure to blog any idea that’s worth preserving. It’s perfectly fine to tweet about trivialities — I do it all the time! But if you’re tweeting about your work, your passion, or something meaningful to you, you owe it to your ideas to actually preserve them somewhere more persistent.

Consolidating my online identity

2010-12-02T00:00:00-06:00

There are a lot of choices when it comes to social sites, and those choices can lead to confusion. When faced with a new link to share, I wonder: where should this go? Twitter? Facebook? My blog? The answer is never consistant, which leads to fragmentation across those various sites.

My reason for listing my various lists was to consolidate and bring some order to this chaos. I wanted to figure out the minimum set of sites I could participate in while still capturing the stuff I want to share. The results can be found here:

http://monsur.hossa.in

Those are the nine sites I find myself actively participating in (more or less). Each has their own role, which break down as follows:

Blogger - This site you're reading now, meant for longer pieces written by me.
Twitter - For short bursts of thought.
Flickr - My photos, mostly just random stuff taken on the phone.
GitHub - For code projects
Rdio/Last.fm - Captures the music I'm listening to.
Tumblr - Captures anything else I consume and like. This includes random links, photos, videos, quotes, along with books I've read, movies I've watched, and songs I like.
Buzz - Consolidates a lot of the information above so that its all available in one place.
Facebook - I don't really participate on Facebook, but...

There are other sites, such as LinkedIn, which didn't make the cut. I have an account, but I just don't spend enough time on those sites to justify highlighting them.

Its interesting to think that I no longer host any of my own content. There was a time when all this information would live on a web hosting account; now its all in the cloud. Personally I like it that way; let someone else deal with the scaling and maintenance issues. I'm comfortable just maintaining index.html on my web host.

The sites above mostly fall into the categories I laid out in my previous post. Blogger, Twitter, Flickr and GitHub for "Things I Create". Rdio, Last.fm and Tumblr for "Things I Like". The "Things I Want to Consume" category isn't exposed publicly, because who cares to see a list of things I want to do?

Love making lists

2010-10-25T00:00:00-05:00

I love making lists! I'll make a list for just about anything anything; it frees my head to focus on other things. So here's another list, a list of my lists:

My (rarely updated) blog - Blogger
Twitter
My photos - Flickr
Things I want - Amazon Wishlist
Books I want to read - Amazon Wishlist
Music I want to hear - Amazon Wishlist
Movies I want to watch - Netflix Queue
Recipes - Bookmarks stored/tagged in Delicious
Links I like - Delicious, Email
Music I like - Last.fm
Restaurants I want to try - Google Maps
Restaurants I like - Google Maps
Code I wrote - GitHub
Links I want to read - Instapaper
Videos I want to watch - Boxee Queue
Stuff I've seen that I like (pictures, quotes, etc) - Tumblr
Quotes from books - kindle.amazon.com
And of course, the TODO list, which I just keep in a Google Doc

Some of those lists I hardly ever use; others I visit on a daily basis. I don't really think about where all this data is stored, but looking over it all now, the lists can be categorized into three broad categories:

Category	List	Location
Things I create	blog, twitter, photos, source code	Blogger, Twitter, Google Buzz, Picasa/Flickr, GitHub
Things I want to consume	Things I want, books/links to read, music to hear, movies/videos to watch, restaurants to try, recipes to try, todo list	Amazon Wishlist, Instapaper, Netflix Queue, Boxee Queue, Delicious, Google Maps
Things I like	Links, music, restaurants, quotes, random stuff	Delicious, Last.fm, Google Maps, Tumblr, kindle.amazon.com

The first category, "Things I create", is small by nature because its a lot easier to consume than to create. And even when one does create, its limited by our own capabilities. I can write code or snap a cellphone pic, but I'll never write a book or record a hit song (sadly).

The second category, "Things I want to consume" is the biggest and longest of the lists. It's an ever expanding list, growing at a rate far faster than I can ever consume. The lists are fragmented across different sites: one for books, one for movies, one for restaurants, etc. Even when the list is similar, they are captured in different locations. For example, movies vs. online videos (Netflix vs Boxee), or books vs. online articles (Amazon vs. Instapaper).

Lastly, the "Things I like" category is like a mirror of "Things I want to consume". Its represents things that have survived the filter of consumption. If I like a song, its gets "loved" on Last.fm. Highlighted text on my Kindle automatically gets updated on kindle.amazon.com (which I wish was public).

It surprises me how little parity there is between the sites that help you manage these two categories. An album I want to hear starts on an Amazon Wishlist, but ends up on Last.fm if I like it. An article I want to read starts on Instapaper but ends up on Tumblr. The flow feels disjointed, and some information isn't even captured. For example, I'm not quite sure where to say that I like a book or a movie. Sure I can share a favorite movie on Twitter or Facebook, but its just some text, the "movieness" of that item is lost. I think this is why all these social networks can feel overwhelming sometimes; they aren't organized lists, they're out of control blobs!

Though these lists are fragmented across various sites, there is a flow to them.

Create things to consume -> Queue up things to consume -> Highlight things you enjoyed consuming

Someone writes a blog post. That post ends up on someone else's "to read" list, who then reads it and adds it to their "favorite posts" Delicious account. But looking over the fragmented list of sites above, I don't see that flow being captured, especially not across all media types.

I'd love to see a site geared towards list addicts like myself. Some features could include:

The ability to add various media types. Books, music, video, links, restaurants, articles, concerts etc... anything is fair game.
Let me view the entire combined list, or filter on each media type (book, movie, online video, album, etc), or filter or other variables (date added, etc)
Customize views for each media type. Songs/Albums show the album art, addresses appear on a map.
Tell me where I can get it. Links, of course, point to where the actual item lives. Songs/albums have 30 second clips from sites where I can buy them. Books have Amazon links with an extra link if they're available on the Kindle. Movies or TV shows have pointers to where I can stream them on Netflix or Hulu or wherever.
Once I consume an item, let me say whether I liked it.
Don't make me write a review
Items I liked can be filtered in different ways: media type, date I liked it, date the media type was created. This would enable me to easily, say, see all my favorite songs from the past year and expose it as a mix.
And of course, let me share these things, so others can add the things I like to their own list. Expose the lists as a feed (but only if an item is not marked private).

Delicious comes really close to being a "universal list engine". The simple ability to tag items means that you can use Delicious for a list of just about anything (as long as it has a link). But the output format of Delicious is a bit boring. Its tough to pick a movie or a restaurant from text links. I think Delicious would benefit from views tailored to a particular link type. For example, movie links could display the movie poster, or restaurant links could map the restaurant location.

Sites like Netflix or Amazon or Tumblr also come close. But at some level they are all siloed along specific media types rather than across categories. I would love to see a site take a step back and organize along the broad actions of things to consume and things consumed.

A Book Apart

2010-07-05T00:00:00-05:00

My copy of HTML5 for Web Designers arrived on Saturday, and I finished it over the long weekend. It's 85 densely-packed pages that's useful for anyone interested in HTML5 (not just web designers). I'm a big fan of short, nutrient-rich books like this (JavaScript: The Good Parts is another example). They give enough information to start tinkering with a specific technology, but aren't mired in details that can be researched in-depth later. This book is also an example of great print design; its just cool to look at! I'm looking forward to the other titles coming up from A Book Apart.

The back-up plan: Amazon S3

2010-07-02T00:00:00-05:00

My last post was an overview of our home backup system. Part of that system includes storing a subset of files online with Amazon S3. Here are some of the details

The Cost

So you'll be hard pressed to find an online backup solution that doesn't cost something. But the difference with Amazon S3 is that these costs grow unbounded. This can be a scary thought; the more data you have, the more it costs. However, to make this tradeoff for a few reasons:

Trust - Amazon is a trusted company that's been around for ages (in Internet time) and has always put users first.
Flexibility - S3 has a simple programming interface with a wide variety of frontends for interacting with the data. There aren't any limits on what kind of data I can store or where that data comes from (Some online backup services limit you to a single computer, don't allow network attached storage, or have size limits.)
Cost - While the cost does grow unbounded, it grows slowly. 100Gigs of data would be $15/month, and I won't have close to that much data for some time. Storing 30Gigs puts you at about $5/month, which is similar to many of the other online backup services. And hopefully over the years, as the cost of storage decreases, so will the costs of Amazon S3.

The Tools: Jungle Disk

There are a bunch of tools out there for backing up your data to S3; anything from simple shell scripts to high-octane GUI apps. I started by looking at Jungle Disk. I remember evaluating Jungle Disk years ago; it was one of the first S3 clients, with a reliable development schedule. If I recall correctly, I think it was even open source.

Jungle Disk has a large suite of useful features. But it also has a monthly fee ($2/month, first 5 gigs free, $3/month to backup a network drive, which I need). If you're looking for an easy-to-use and powerful solution, Jungle Disk sounds great. But I just can't bring myself to pay another monthly fee on top of the Amazon S3 costs.

The Tools: Other options

I was surprised that most of the popular S3 backup clients were pay apps. My ideal client would be a feature rich open source application or script. Among these, s3sync and s3fs kept coming up.

s3sync is a Ruby script that syncs folders between S3. The app itself hasn't been updated since 2008, but the forums are active. Unfortunately, when I tried s3sync it threw an exception. I didn't bother debugging why. Lazy me.

s3fs uses Fuse to mount an S3 bucket as a folder. However I ran into some errors using MacFuse (and I read that others have too), the app hasn't had any substantial code updates in a while, and it looks like the developer is focused on a pay product.

Now I'm not faulting any of these applications for having a pay component. I'm glad to see these apps are thriving and developers are being rewarded for their hard work. However, I must say I'm surprised there aren't more open source options for S3 backup. S3 has been around for years, the interface is well documented and stable, I guess I expected more.

The Tools: jets3t

I finally settled on jets3t, which is an open source Java command-line app. It was written by James Murty, who literally wrote the book on Amazon S3. Jets3t is well supported, simple to use, and most of all, It Just Works (for me). Jets3t offers a whole suite of tools; I ended up using the command line Synchronize app. After a few test runs between a small set of data and an S3 bucket, I was ready to turn it loose on my personal files.

Here's the Jets3t command I used to run the backup.

bin/synchronize.sh -g -b UP bucketName/backups/readynas /Volumes/personal

It took about a week to back up all 30gigs of my personal folder. Synchronize is remarkably robust; while it encountered S3 errors every now and then, the Synchronize app retried the requests accordingly and never crashed. I was lucky that Amazon was offering free data transfer for the month of June (they have since extended that offer until November).

This command runs every 2 weeks. The initial upload took a while, but each incremental backup after that shouldn't take too long.

Looking forward

This system works well for now, but I keep wondering if it will scale. I use 100Gigs as an imaginary limit. Storing 100Gigs will cost $15/month. I'd be willing to pay that much of peace of mind. I estimate I'm still years away from 100Gigs, but who knows. As we take more photos, and as personal video takes a more prominent role in our media, we may reach it faster than I think.

I also have to consider the cost of restoring 100Gigs of data. At our present bandwidth, that could take weeks. Amazon does offer an Import/Export service (you send them a drive, they put your data on it and send it back); that sounds like a much better option for large amounts of data. Hopefully by the time I reach that much data, bandwidth speeds will have improved, or there will be better options at a lower cost.

The back-up plan

2010-06-26T00:00:00-05:00

Every year that goes by, I think more and more about backup strategies for our home computers. As more of our lives become digitized, it's increasingly important to insure that these memories are preserved. And I'm not talking about backing up a single computer here and there. I mean a comprehensive backup strategy that encompasses all the devices in a household with multiple levels of redundancy.

Two years ago, I would have said "The Cloud" is the answer to everything. But as I quickly learned, it's impractical to backup anything more than a few gigs of data online. Network speeds are not yet capable of transferring hundreds of gigs in any reasonable timeframe.

I recently revisited this question of backups, and here's my current thinking. Hopefully this is something that will scale for years to come.

Overview

Here's an overview of what our backup system looks like:

The key to this system is redundancy. I've tried to build in an adequate level of redundancy so that the data is replicated multiple times across different locations.

Network Attached Storage

I have a ReadyNas NV+ to consolidate all our data. Photos, music, movies, documents, all of it goes on the 2TB storage array. The ReadyNas uses Netgear's proprietary X-RAID technology to offer redundancy should any single drive in the array fail. I split the drive into three partitions:

Personal - Stores personal photos/videos, documents, backups from other computers, and a personal Mercurial source control system (For any code I play around with). This currently comes out to about 30 gigs (mostly photos). This entire partition is backed up to Amazon S3.
Media - Stores mp3s, movies, and any large format media files. The files here are too large to store online.
Backups - Stores Time Machine backups of other laptops.

Backing up Home Computers

At regular intervals, I run a simple rsync script to backup all the home computers to the ReadyNas' "personal" partition. I only backup the user's home folder, which is a considerable space saver (about 10gigs vs. 70gigs for the entire computer).

(Tools such as SuperDuper! will backup an entire computer to a bootable drive, but frankly, this seems overkill to me. Most of the software I use is free or easily replaceable. If a laptop were to fail, the applications would not be missed; I even enjoy a fresh install every few years. The data is what's most important, and that is what gets backed up to the ReadyNas.)

Full computer back ups do have their benefits, which is why I still run Time Machine. I like how Time Machine offers versioned files in an easy-to-use interface. The Time Machine backups go to a special partition on the ReadyNas.

Devices like digital cameras, flip cam, and iPhones are also saved to the ReadyNas' "personal" partition.

Backing up the ReadyNas

Backing up all our devices to the ReadyNas offers decent protection, but since the ReadyNas is always connected to the network, it is vulnerable to accidental file deletion and human error. So the entire ReadyNas is itself backed up to an external USB drive (again using rsync). This doesn't have to happen often, but it should happen regularly. I try to do it once a month, around the same time I do the monthly bills.

(Backup experts say you should store your external backup somewhere offsite, so its protected from fire, theft, etc. I'd like to say that I store my backup offsite, but the truth is, it just doesn't happen. Lugging the drive back and forth between my office is just a pain. Maybe a fireproof safe would be a worthy investment.)

Backing up to Amazon S3

Even with the ReadyNas, even with the external drive, I have a nagging sense of paranoia about my most valuable data. Wedding photos, photos of my newborn son, these are all files that are irreplaceable. These files, along with the other files in the "personal" partition, get backed up to Amazon S3. There is a cost associated with Amazon S3, I estimate less than $5 a month. Its a small price to pay for peace of mind. However this cost will grow, and its something I'll need to keep an eye on and adjust with time.

I don't backup the "media" or "backup" partitions to S3. These partitions are over 100gigs and would prove costly and time-consuming to store on S3.

Conclusion

With this backup strategy in place, any piece of data exists in at least four times: once at the original location, once on the ReadyNas, once in Time Machine (on the ReadyNas) and once on the external USB drive. The most important pieces of data are backed up one more time to Amazon S3. Most of these steps can be automated, the only additional work is plugging and unplugging an external USB drive once a month. Will this scale? I hope to explore that and other details of this backup system in future posts.

My favorite tracks of 2009

2010-01-01T00:00:00-06:00

Here's a mix of some of my favorite songs from 2009. Its not really a top ten list or ranked in anyway, its just a good old-fashioned mix tape.

Zero - Yeah Yeah Yeahs
Something Is Squeezing My Skull - Morrissey
What Are You Willing To Lose - Lucero
The Beat of Our Own Drum - JC Brooks & The Uptown Sound
Could Have Been - Lee Fields
To Save Me - M. Ward
Kick Drum Heart - The Avett Brothers
Little Secrets - Passion Pit
Lisztomania - Phoenix
Come Saturday - The Pains of Being Pure at Heart
Living North - Cymbals Eat Guitars
Too Many Birds - Bill Callahan
French Navy - Camera Obscura
The Neighbors - St. Vincent
Two - The Antlers
Pulled Fences - The Wrens

Album of the Decade: The Meadowlands

2010-01-01T00:00:00-06:00

I don't have the patience to rank my favorite albums of the decade. But The Wrens' Meadowlands is by far my favorite album of the decade, and perhaps of all time. I believe the right music has a way of finding you at the right time. The Pixies' guttural screams were just the right soundtrack for my teenage years. In the same way, The Meadowlands was the soundtrack of my twenties. I entered the decade out of college, lived the startup life, fell in love, got married, and as the new decade begins, I have a kid on the way. No album captures the joys and heartaches of these life-altering transitions better than The Meadowlands.

XmlHttpRequest and status/statusText

2007-12-28T00:00:00-06:00

When an XmlHttpRequest object makes a request to a server, it stores the server’s response in the “status” and “statusText” fields. “Status” holds the status code, while “statusText” holds the reason phrase, as defined in the HTTP/1.1 specs.

Recently I found that the status and statusText returned by the XmlHttpRequest object does’t always match what actually comes from the server. To test this out, I created a page that receives various XmlHttpRequest responses and compares them to the expected values. You can try it out in your own browser here:

For the most part, browsers are compliant, with a few grammatical differences in statusText. Here are some of the major discrepancies:

Firefox 2.0.0.11
408 Request Timeout	Firefox throws a javascript error when accessing “status” and “statusText” for this particular status code.

Internet Explorer 6
204 No Content	status = 1223 statusText = Unknown
301 Moved Permanently 302 Found 303 See Other 307 Temporary Redirect	In all these cases, IE returns: status = 12150 statusText = Unknown

Safari 3.0.4
Although Safari returns the correct status codes for most instances, the statusText is “OK” for ALL status codes! Browsing through the source code for WebKit, it looks like this is a known issue; there is a comment that says “FIXME: it would be nice to have a way to get the real status text eventually“.

Opera 9.25
204 No Content 304 Not Modified 504 Gateway Time-out	Returns status = 0 with no statusText
401 Unauthorized	Returns incorrect status 403
407 Proxy Authentication Required	Returns incorrect status 403
417 Expectation Failed	Returns nothing

I’m sure this isn’t an issue for most users, as all the boilerplate XmlHttpRequest code I’ve seen only checks for status ‘200′ (which all browsers return correctly). However as AJAX applications become more complex, this might become more relevant.