persistent.info

Google Reader and Closure Tools

2009-11-09T08:07:00.001-05:00

Since Google Reader makes heavy use of the recently-open sourced Closure Tools, Louis Gray asked me to give a client's perspective on using them. He wrote up a great post summarizing my thoughts, and if you'd like to see the raw input, I've included it below:

There are three pieces to the Closure Tools, the compiler, the library and the template system. They appeared gradually, roughly in that order. The compiler in its current incarnation dates back to Gmail in 2004 (which is why Paul Buchheit refers to it as the "Gmail JavaScript compiler"), with the library and the template system starting a couple of years later.

Reader development started in early 2005, which meant that we always had the compiler available to us, and so except for early prototypes, we always ran our code through it. Until the last month leading up to the Reader launch in October 2005, the size benefits of the compiler were less important, since we were less focused on download time (and performance in general) and more on getting basic functionality up and running. Instead, the extra checks that the compiler does (e.g. if a function is called with the wrong number of parameters, typos in variable names) made it easier to catch errors much earlier. We have set up our development mode for Reader so that when the browser is refreshed, the JavaScript is recompiled on the server and is used with the page when it is reloaded. This results in a tight development loop that makes it possible to catch JavaScript errors as early as possible.

Since Reader development started before the library and template tools were available, we had homegrown code for doing both. There was actually shared code that did some of the same things as basic library functionality (e.g. a wrapper around getting the size of the window, handling different browser versions and quirks). However, that shared code was of various vintages (copied from project to project) and therefore not very consistent in style or quality. Erik Arvidsson's post talks a bit more about the inception of the Closure library (he's one of the co-creators, along with Dan Pupius).

Reader began using the Closure library and template systems gradually, first for new code and then replacing usages of the old shared library and our homegrown code. It was a gradual process, though I tried to keep it organized by doing an audit of all the usages of old code and their Closure equivalents, so that work could more easily be divided up (this was handled during "fixit" periods, where we focus on code quality more than features).

The benefits of the compiler system are tremendous. The most obvious are the size ones, without it Reader's JavaScript would be 2 megaytes, with it it goes down to 513K, and 184K with gzip (the compiler's output is actually optimized for gzip, since nearly all browsers support it). However, all of the above-mentioned checks, as well as many more that have been added over the past few years (especially type annotations) make it much more manageable to have a large JavaScript codebase that doesn't get out of control as it ages and accumulates features.

The library means that Reader is much less concerned about browser differences, since it tries very hard to hide all those away. Over time, the library has also moved up the UI "stack", going from just basic low level code (e.g. for handling events) to doing UI widgets. This means that it's not a lot of work to do auto-complete widgets, menus, buttons, dialogs, drag-and-drop, etc. in Reader.

One thing to keep in mind is that, as mentioned in the announcement blog post, these tools all started out as 20% projects, and for the most part are still dependent on it. If one project needs a feature from the compiler or the library that doesn't exist, they're encouraged to contribute it, so that other teams can benefit too. To give a specific example, Reader had some home-grown code for locating elements by class name and tag name (a much more rigid and simplified version of the flexible CSS selector-based queries that you can do with jQuery or with the Dojo-based goog.dom.query). As part of the process of "porting" to the Closure library, we realized that though there was an equivalent library function, goog.dom.getElementsByTagNameAndClass, it didn't use some of the more recent browser APIs that could it make it much faster (e.g. getElementsByClassName and the W3C Selector API). Therefore we not only switched Reader's code to use the Closure version, but we also incorporated those new API calls in it. This ended up making all other apps faster; it was very nice to get a message from Dan Pupius saying that the change had shaved off a noticeable amount of time in a common Gmail operation.

You can tell that there's something special about this when you look at the ex-Googlers cheering its release. If it had been some proprietary antiquated system that they had all been forced to use, they wouldn't have been so excited that it was out in the open now :)

If you'd like to know more about Closure, I recommend keeping an eye on Michael Bolin's blog. He already has a few posts about what makes it special, and I'm sure there are more coming.

PuSH Bot: PubSubHubbub to XMPP Gateway

2009-10-12T10:04:00.003-04:00

When XMPP support in Google App Engine was announced, it occurred to me that it would be pretty easy to use it to do a PubSubHubbub-to-XMPP bridge. Other things came up, but I was reminded again of the possibility when a "Oh, you didn't you see my Reader share?" conversation happened in a Partychat room. A bit of searching turned up someone else with the same idea, except it wasn't quite as user-friendly as it could be (i.e. as user friendly as a quasi-command-line interface, which isn't saying much).

After a bit of weekend hacking I've created PuSH Bot, which lets you subscribe to any PubSubHubbub-enabled feed and get notified of updates via XMPP (e.g. to your Google Talk account). It has some niceties like use of feed auto-discovery so you can just specify web page URLs, OPML import for bulk subscribing and throttling of updates. The homepage lets you know how to get started.

The code is available, though there shouldn't be anything too special about it. Feed parsing is done via ROME (as is OPML parsing, though that needed some patching to get working). Feed auto-discovery is handled by Google AJAX Feed API because life is too short for HTML parsing.

Twitter PubSubHubbub Bridge

2009-08-26T10:46:00.001-04:00

During the Twitter DDoS attacks, there was a thread on the Twitter API group about using PubSubHubbub to get low latency notifications from Twitter. This would be an alternative to the streaming API that Twitter already has. The response from a Twitter engineer wasn't all that positive, and it is indeed correct that the streaming API already exists and seems to satisfy most developers' needs.

However, my interest was piqued and I thought it might be a useful exercise to see what Twitter PubSubHubbub support could look like. I therefore decided to write a simple bridge between the streaming API and a PubSubHubbub hub. The basic idea was that there would be a simple streaming client that would in turn publish events to a hub. The basic flow would be:

(created using Kushal's Diagrammr)

I'm using FriendFeed as the PubSubHubbub client, but obviously anything else could substitute for it. The "publisher" is where the bulk of the work happens. It uses the statuses/filter streaming API method* to get notified of when a user of interest has posted, and then it notifies the reference hub that there is an update. It also has a companion Google App Engine app that serves feeds for Twitter updates. This is both because the hub needs a feed to crawl and because the feed needs to have a <link rel="hub"> element, something which Twitter's own feeds don't have. Unfortunately the publisher itself can't run on App Engine since the streaming API requires long-lived HTTP connections, and App Engine will not let requests execute for more than 30 seconds. I considered using the tasks queue API to create a succession of connections, but that seemed too hacky.

In any case, it all seems to work, as this screencast shows:

On the right is the Twitter UI where messages are posted. In the middle is the publisher which receives these messages and relays them to the hub. On the left is FriendFeed which gets updates from the Hub.

Latency isn't great, and as mentioned in the group thread, Twitter could have to deal with the hub being slow. Part of the reason why latency isn't great is because the hub has to crawl the feed to get at the update, even though the publisher already knows exactly what the update is. This could be fixed by running a custom hub (possibly even by Twitter, see the hub can be integrated into the publisher's content management system option), with the flow becoming something like:

In the meantime, here's the source to both the publisher and the app.

* This was called the "follow" method until very recently.

Twitter Streaming API from Python

2009-08-24T23:10:00.000-04:00

I'm playing around with Twitter's streaming API for a (personal) project. tweetstream is a simple wrapper for it that seemed handy. Unfortunately it has a known issue that the HTTP library that it uses (urllib2) uses buffering in the file object that it creates, which means that responses for low volume streams (e.g. when using the follow method) are not delivered immediately. The culprit appears to be this line from urllib2.py (in the AbstractHTTPHandler class's do_open method):

fp = socket._fileobject(r, close=True)

socket._fileobject does have a bufsize parameter, and its default value is 8192. Unfortunately the AbstractHTTPHandler doesn't make it easy to override the file object creation. As is pointed out in the bug report, using httplib directly would allow this to be worked around, but that would mean losing all of the 401 response/HTTP Basic Auth handling that urllib2 has.

Instead, while holding my nose, I chose the following monkey patching solution:

# Wrapper around socket._fileobject that forces the buffer size to be 0
_builtin_socket_fileobject = socket._fileobject
class _NonBufferingFileObject(_builtin_socket_fileobject):
  def __init__(self, sock, mode='rb', bufsize=-1, close=False):
    builtin_socket_fileobject.__init__(
        self, sock, mode=mode, bufsize=0, close=close)

# Wrapper around urllub2.HTTPHandler that monkey-patches socket._fileobject
# to be a _NonBufferingFileObject so that buffering is not use in the response
# file object
class _NonBufferingHTTPHandler(urllib2.HTTPHandler):
  def do_open(self, http_class, req):
    socket._fileobject = _NonBufferingFileObject
    # urllib2.HTTPHandler is a classic class, so we can't use super()
    resp = urllib2.HTTPHandler.do_open(self, http_class, req)
    socket._fileobject = _builtin_socket_fileobject
    return resp

Then in tweetstream's urllib2.build_opener() call an instance of _NonBufferingHTTPHandler can be added as a parameter, and it will replace the built-in HTTPHandler.

Exporting likes from Google Reader

2009-07-16T12:02:00.002-04:00

I started this as another protip comment on this FriendFeed thread about Reader likes but it got kind of long, so here goes:

Reader recently launched liking (and a bunch of other features). One of the nice things about liking is that it's completely public*. It would therefore make sense to be pretty liberal with liking data, and in fact Reader does try to expose liking in our feeds. If you look at my shared items feed you will see a bunch of entries like:

<gr:likingUser>00298835408679692061</gr:likingUser>
<gr:likingUser>11558879684172144796</gr:likingUser>
<gr:likingUser>07538649935038400809</gr:likingUser>
<gr:likingUser>09776139491686191852</gr:likingUser>
<gr:likingUser>02408713980432217881</gr:likingUser>
<gr:likingUser>05429296530037195610</gr:likingUser>

These are the users that have liked. Users are represented by their IDs, which you can use to generate Reader shared page URLs. More interestingly, you can plug these into the Social Graph API to see who these users are.

Liking information isn't just limited to Reader shared item feeds. If you use Reader's view of a feed, for example The Big Picture's, you can see the <gr:likingUser> elements there too. This means that as a publisher you can extract this information and see which of your items Reader users find interesting.

For now liking information that is included inline in the feed is limited to 100 users, mainly for performance reasons. That number may go up (or down) as we see how this feature is used. However, if you'd like to get at all of the liker information for a specific item, you can plug in an item ID into the /reader/api/0/likers API endpoint, and then get at it in either JSON or XML formats.

* I've seen some wondering what the difference between liking, sharing and starring is. To some degree that's up to each user, but one nice thing about liking is that it has less baggage associated with it. We learned that if we try to redefine existing behaviors (like sharing) users get upset.

HTML Color OneBox

2009-03-03T20:07:00.001-05:00

Work is the sort of place that cares about specific colors, so HTML color hex triplets come up in conversation quite often. Neil suggested that this should be a OneBox in search results. It occurred to me that this could done via the Subscribed Link feature that we offer for search results. It turned out that subscribed links can use gadgets, which meant that an inline preview of colors was even possible. Regular expression matching also meant that I didn't have to list out every color by hand. This page has more information on the OneBox, or you can subscribe directly.

Once you have installed this, you can search for things like #fafafa or #ccc and get an immediate preview (in fact, the # can be omitted).

Porting Twitter Digest to Google App Engine

2008-05-11T20:26:00.002-04:00

I've been meaning to play around with Google App Engine for a while, and as a quick project, I decided to port Twitter Digest to it (not as exciting as Kushal's Millidunst Calculator). This looked to be pretty straightforward: the original version was already in Python, and wasn't very complicated (just a single CGI script). It did indeed end up pretty easy; the whole process took a couple of hours.

The first step was to port the script from CGI-style invocation to the App Engine webapp framework. Then I looked into what it would take to get Python Twitter (the library I used for fetching data from Twitter) running. Switching it from urllib2 to urlfetch was pretty painless (though I don't use the posting parts of the API, so I didn't check if those work too). The other part of the library that I was relying on was its caching mechanism (since the digests are daily, there's no point in querying Twitter more often). DeWitt (the library's author) had thoughtfully put the caching functionality into a separate class, so it was easy to replace it with another one that implemented the same interface but was backed by App Engine's datastore.

The result (complete with App Gallery entry) is not that exciting, in the sense that it functions identically to the original. The only issue that I've run into so far is that when there are several cache misses, the URL fetches can take long enough that the request hits App Engine's deadline. However, since the successful fetches are cached, repeating the request will eventually succeed (so if consuming the digest via a feed, this shouldn't be a big deal). Ideally the urlfetch functionality would also support asynchronous fetches, since it would be easy to adapt the code to fetch all user timelines in parallel.

Update on 11/23/2008: Since I've gotten some requests for the modifications to twitter-digest that I made to get it to run on App Engine, here's a patch.

Intern on the Google Reader team

2008-04-24T08:38:00.000-04:00

Having interns has worked out well for the Reader team. Following my blog post, we were very pleased to get Nitin Shantharam and Jason Hall to help us out with Reader development. Their stints on the team resulted in a a bunch of features, and Jason is now back at Google working full-time (Nitin wasn't a slacker, he's just still in school).

We're looking for another intern or two this year. Internships generally last a couple of months to twelve weeks, are for full-time students, and would be in Google's Mountain View, California office. You can work on either Reader's backend (a C++ system for crawling millions of feeds, handling lots of items being read, shared, starred or tagged per second) or frontend (Java servers and JavaScript/AJAX-y craziness) depending on your interests and experience.

If you or anyone you know is interested in this internship, contact me at mihaip at google dot com. This page also has more general information about interning at Google.

persistent.coffee

2008-04-03T14:46:00.000-04:00

I've been trying my hand at latte art. Though I have a very long way to go, I've been documenting my efforts, with a hope of learning from my mistakes. Blogger's mobile support makes it pretty easy to collect pictures, and I've finally gotten around to making a decent template for the "blog."

coffee.persistent.info is the result. Technically, this isn't a Blogger template, since I just have some static HTML as the content. Instead, it uses the JSON output that Blogger's GData API supports. Rendering the page in JavaScript allows for more flexibility. I wanted to make pictures that I liked take up 4 slots (a layout inspired by TwitterPoster). This imposed additional constraints (in order to prevent overlap between sequential large pictures). The display is generally reverse-chronological starting from the top left, but images are occasionally shuffled around to prevent such overlaps. There is also a bit of interactivity, the pictures are clickable to display larger versions. To help with all this, I've been experimenting with jQuery (also on Mail Trends), and am liking it quite a bit.

Mail Trends

2008-03-25T10:32:00.011-04:00

I get a lot of email (especially at work). I'm trying a Inbox Zero-like approach in order to keep up with it. Though that's helping me to stay on top of things, I had the nagging feeling that I was probably on too many mailing lists, and that some of them were probably not worth it from a signal-to-noise ratio perspective.

Ideally something like the Reader Trends or Search History Trends page would exist for Gmail. I thought I could perhaps build it myself, but the absence of an official Gmail API deterred me. However, it occurred to me that the recently added IMAP support could act as an API of sorts. It should be easy to get just the message headers and slice and dice them to extract the stats that I was interested in.

Thus was born Mail Trends, an IMAP-based email analysis project. It can generate a bunch of tables, graphs and distributions based on time of day, senders, recipients, mailing lists, etc. To get a feel for what it can output, see the results of running it on a piece of the Enron Email Dataset. To run it over your own email, see the getting started page. As a caveat, the program currently loads everything into memory, so my run on 200,000 messages resulted in 1.6 gigabytes being used. You may want to use the --max_messages= flag to limit the dataset, at list for initial runs.

The project is still in its early stages, so patches and suggestions are definitely welcome (my email address is at the footer). You can also subscribe to the feed of check-ins to see changes as they are made. The plan wiki page has a very brief outline of what I'm planning on working on next.

Two Safari 3.1 Tips

2008-03-22T14:23:00.001-04:00

Safari 3.1 is out, and I've upgraded my Mac to it. Besides some issues with arrow keys in Reader (we're on it), it's working out well. Here are two hidden prefs that you may find useful:

defaults write com.apple.Safari IncludeInternalDebugMenu -bool true

The Develop menu that 3.1 includes is nice, but it seems to supplant the old "Debug" menu (i.e. the preference key that used to toggle it - IncludeDebugMenu - now toggles the "Develop" menu). The old menu had functionality that isn't present in the official one, most notably the "Caches" window that displayed the number of live JavaScript objects and made tracking down memory leaks much easier. If you'd like to bring back the old menu, you can use the new IncludeInternalDebugMenu key shown above

defaults write com.apple.Safari TargetedClicksCreateTabs -bool true

First spotted on Twitter, this forces new windows to open in tabs, one feature that I missed from my Firefox days.

Decommissioning Mscape Software

2008-02-18T21:50:00.000-05:00

Before I became enamored with web development, I used to be a Mac software developer under the Mscape Software moniker. My first public release (clip2cicn, a helper tool for making the icon resources necessary for Kaleidoscope schemes) was almost 10 years ago, on June 26, 1998. My flagship product was Iconographer, an icon editing tool.

I haven't had time for Mscape Software for about 4 years (the last Iconographer release was in July of 2003, and I last touched the code in early 2004). The site was still up, and I kept receiving registrations (Iconographer was a $15 shareware product). As time went on, I began to feel more and more guilty about not providing any support for (paying) customers. To this end, I have finally gotten around to dismantling Mscape Software, replacing the site with a placeholder with download links for all products. I've also put up a registration code for Iconographer so that the (annoying) registration reminder can be shut off.

I've gotten some requests to open-source Iconographer, but I'm not sure I'll have time even for that. I'm not 100% sure I can build the product given software on hand (it was built with CodeWarrior). Even then, the codebase shows its age (it was my first large-ish project and still uses many Classic constructs like a WaitNextEvent event loop).

plusplusbot: Karma for anything and anyone

2007-11-16T10:14:00.000-05:00

The recent release of Foamee reminded me that I had never blogged here about plusplusbot, a little toy of a site that I started to work on this past summer. The idea behind plusplusbot is that you can express your (dis)pleasure with something or someone over Twitter, and the site will keep track of the target's score over time. All of this is done over Twitter, in a barnacle-like fashion. The thinking being that Twitter handles message delivery over a variety of mediums (Jabber, SMS, Web) and provides a ready-made social network to piggyback on (you can view the activity of you and those you follow, for example, here's mine). This concept should be familiar to those that have enountered "karmabots" in chatrooms.

The targets can be anything. Companies, food, people, even Twitter itself. For whatever reason, the site has caught on most with Japanese Twitter users, so the homepage is often incomprehensible.

Technically, the site is not too exciting. Like Twitter Digest, it also uses python-twitter to talk to Twitter and templet for simple templates. The site itself is not dynamic at all, instead a script running on one of my machines polls Twitter every two minutes (since it has to fetch both friend updates and direct messages, fetching both every minute would go over Twitter's 70 requests/hour API rate limit). If it determines that a new plusplus or minusminus has been sent, it re-generates those pages (the user's and the target's) and uploads them. The simple design means that the site can be hosted nearly anywhere.

Macros for the new version of Gmail

2007-11-05T21:09:00.001-05:00

As those of you on the new version of Gmail might have noticed, nearly all Greasemonkey scripts that used to work on the old version no longer do. Even though it looks pretty similar, the new Gmail is entirely different from a JavaScript, HTML and CSS perspective, so this isn't surprising. Some of the scripts are no longer necessary. For example, saved searches aren't really needed, since searches now get their own URLs and can be bookmarked.

However, my Macros script is still needed; while the new version of Gmail does have additional keyboard shortcuts, it's still not possible to do everything from the keyboard. I've therefore ported it for the new version of Gmail, to install it, click below.

Install Gmail Macros

More specifically, the following keyboard shorcuts have been ported over:

g: Go to label
l: Apply label
b: Remove label
e: Archive (regardless of view, unlike "y")
d: Discard (mark as read and archive)

There is also a new keyboard shortcut, which the old script didn't have:

f: "Focus" the current view (only show unread, starred or inbox messages)

You may remember the script supporting other keyboard shorcuts. Since the new version of Gmail supports additional shortcuts, those have obsoleted. The new ones are:

shift + i: Mark as read
shift + u: Mark as unread
shift + 3: Move to trash (not actually new, but not many people seem to know this one)
shift + 8 followed by a, n, r, u, s, or t: Select all, none, read, unread, starred, or unstarred

For those of you adventurous enough to look at the script source, you'll notice that it uses a gmonkey object that is present on the window, which in turn gets you a gmail object with methods like getNavPaneElement() and getActiveViewType(). What this means is that the version of Gmail, in addition to being faster, also has semi-official support for Greasemonkey scripts. I'm pretty sure docs for this API will be out soon, but in the meantime, feel free to look at the script and use a tool like Firebug to investigate the properties of the gmonkey and gmail objects and play around.

Update on 11/06/2007: And here are the semi-official docs.

Feed Proxy: View Raw RSS/Atom Feed Data in Firefox 2.0

2007-10-01T20:24:00.000-04:00

Firefox 2.0 has a nice RSS/Atom feed preview feature, which pretty-prints the feed and allows you easily subscribe to it in a feed reader. While this is nice from a user's standpoint, it can get in the way when trying to debug or otherwise see a raw feed as a developer. There is no option to turn this mode off, which seems to be making some people unhappy.

To help with this use-case, I made Feed Proxy, a simple service that gives you back the raw XML output (by adding a 512 byte dummy header that makes Firefox not invoke its feed preview code). Its page has more details on how to invoke it and what its caveats are.

Valleywag, brought to you by....Google?

2007-09-26T18:11:00.000-04:00

Gawker Media's use of virtual hosting for its sites was being discussed in PartyChat, so to verify this, I ran host valleywag.com to compare it with host fleshbot.com. Imagine my surprise when I saw this output:

$ host valleywag.com
valleywag.com has address 69.60.7.199
valleywag.com mail is handled by 1 aspmx.l.google.com.
valleywag.com mail is handled by 5 alt1.aspmx.l.google.com.
valleywag.com mail is handled by 5 alt2.aspmx.l.google.com.
valleywag.com mail is handled by 10 aspmx2.googlemail.com.

If I am interpreting this correctly, Valleywag's email is being handled by Gmail/Google Apps, which is slightly ironic.

Twitter Digest

2007-08-31T10:30:00.000-04:00

There are quite a few people on Twitter that I'd like to follow but can't, since they update way too often. For example, I'd like to know how Craig Hockenberry's MobileTwitterrific work is going. John Gruber often posts advance tidbits from Daring Fireball. However, since Twitter doesn't seem to allow different notification levels for contacts, I'd rather not get overwhelmed with their updates on a regular basis.

Twitter does have an API though, and a pretty complete one at that. Additionally, python-twitter makes it pretty easy to code against it. The result is that it was pretty easy to make Twitter Digest. You can give it a bunch of usernames and it will update a page once a day with all of those people's tweets. Feeds can also be generated, so that you can subscribe to your digest in your favorite feed reader. To give an example, here's my digest (and its feed.)

Since python-twitter already has the the concept of caching API replies, this was especially easy to develop. Simply by juggling with cache timeout values, API replies are kept around for up to a day, thus I don't have to persist anything in my code. All digests are refreshed at GMT day boundaries because supporting timezones seemed like too much work.

Digests can be thematic too. Here's one for the Twitter team. I'm sure there are other uses too. I'm pretty happy with how this turned out, and the only thing missing is better reply support (if a message is in reply to another, then (optionally) show that message inline too). Right now the API is missing support for this (even though the regular UI surfaces this information), but I'm sure something can be hacked together.

Update on 9/3/2007: Adding support for "in reply to" messages turned out to be easy enough (look for the first message from the other username that's before the one that's replying, but only up to an hour earlier, in case it's not actually connected).

iPhone Game Development: A Tale of Failure

2007-07-22T15:29:00.001-04:00

Having acquired an iPhone, I've been playing around with it to see just how much can be done with the "SDK" that is the MobileSafari browser. I thought it would be fun to see if I could make a simple game for it, since that's different enough from the kind of UI coding I do at work.

Most of the games for the iPhone that I've encountered thus far were of the puzzle/card variety, and there wasn't anything too exciting/button-mashing/mindless. It seemed like this was mainly caused by the limitations of the events that the browser exposes. With no real mousemove events, and with mousedown events actually firing when the cursor was lifted (i.e. at the same time as mouseup), it was hard to get anything to happen on the screen quickly in response to a user action. One avenue that I tried was to have the screen covered in narrow vertical <div>'s, and to listen for click events on each of them. Unfortunately, while this worked when each was clicked separately (piano key-style), it didn't work when the user dragged his finger from one to another, since this would switch the browser into panning mode, even if the page/viewport was set to be only 320 pixels wide (i.e. no scrolling could actually occur).

Since the viewport panning does not seem to expose any events, I thought that maybe there could be other ways of catching this user action. When scrolling quickly, the iPhone renders newly-visible areas with a checkerboard pattern before actually displaying their contents. I wondered if this meant that images contained in these areas would not be loaded until they were scrolled into view and rendered. If so, it would be possible to use a load event listener as a notification that the user had scrolled to a particular part of the screen. Unfortunately this didn't seem to be the case, images are loaded as soon as the document is displayed, even if they are off the screen.

I then remembered that the iPhone supports an additional gesture, a two-fingered scroll that can be used to scroll text-areas, elements with overflow: auto and possibly other nodes. It would appear that scroll events trigger for this scrolling motion, meaning that it's possible to detect and get continuous updates for this. Though having to use two fingers seemed a bit clumsy, this seemed pretty promising insofar as it allowed for quick responses to user input.

I had lofty goals for my iPhone game (I thought a vertical scrolling shooter was a good fit for the dragging input method and the vertical screen layout), but I decided that I should do something simple for my first attempt. I thought I would recreate aa racing game that I hard first played on my TI-82 calculator, where a car that can only be moved left or right moves at a constant vertical speed through a winding, narrowing road, with the objecting being to avoid hitting the sides. I thought this would map well to a horizontal scrolling motion that I could get with a scroll event handler as mentioned above.

After writing a simple random road generator and the animation code, I was about to add collision detection. Out of curiosity, I tried to move the "car" left/or right as road moved by, even without collision detection, to see if the speed was too high. To my surprise, the road stopped moving as soon as I put my two fingers on the screen. It appeared that intervals (set up with window.setInterval) do not fire when two-finger scrolling mode is engaged. I thought that maybe I could refresh the animation from the scroll event handler, but that only seems to fire when there is actual motion, thus smooth animation can't be ensured.

My failed attempt is visible here. Before pressing the "Start" button, use the two-fingered scrolling motion left and right to confirm the the "car" (the blue square) moves as expected. Then press "Start" and obverse the reasonably smooth animation. Unfortunately it all stops when placing two fingers on the screen again.

This is rather disappointing, since there doesn't seem to be any fundamental technical reason why this should be the case. We're not talking about Mac OS 9 here (there, due to the way UI events were processed, normal event loop execution would be suspended while a menu was held open or a scrollbar thumb was dragged). It's quite clear, given what native iPhone apps can do, that the platform has quite a lot of CPU and graphics leeway. It's therefore unfortunate just how constrained the MobileSafari browser is, since just a few additional hooks would allow much richer interactions with the user.

Bad Headlines

2007-05-04T09:43:00.000-04:00

Headlines seen over the past few weeks on the start page that Verizon forces on its users, and what they refer to in parentheses:

"Daily Life is a Struggle" (life in Baghdad)
"All Sides Deny Deals" (Iran/UK sailors hostage situation)
"Panel Seeks Documents" (Alberto Gonzales' federal prosecutors firings)
"Cheruiyot Wins Marathon" (Robert Kipkoech Cheruiyot won the Boston Marathon)
"Gates: Clock is ticking" (Defense Secretary Robert Gates on the situation in Iraq)
"Donkey Becomes Witness" (Dallas man is sued for noise disturbance caused by a donkey)
"Dems Predict a Win" (House Democratic leaders predict that they can win a vote on a bill calling for a withdrawal from Iraq)
"Candidates to Debate" (The Republican presidential candidate debate)

Someone forgot to read up on microcontent headlines.

Tiled Windows Are the Way of the Future

2007-04-11T11:40:00.000-04:00

Sorry Bill Atkinson. I know you worked really hard to make regions and thus overlapping windows work. But since I now have 30 inch displays at both home and work, I've been tiling my windows and using Desktop Manager (and soon Spaces?) to switch between workspaces. Not having to deal with z-ordering makes mental context switches faster.

Communicating through screenshots

2007-04-06T13:32:00.000-04:00

My posts to the Reader blog often have screenshots, and I'm pretty picky about what goes in them. It's nice to see that others have noticed this too. My most recent post has a link to Dinosaur Comics, which the author seems to appreciate. Similarly, I intentionally left a reference to Valleywag in my trends post, which they noticed.

In some fashion, Apple does this too. A patent of theirs had a BBEdit reference, which amused the author of the software.

Two Web Development Tips

2007-03-25T20:32:00.001-04:00

Memory Leaks

Fighting memory leaks in fancy AJAXy apps is a way of life for most UI developers. Besides Internet Explorer-specific issues, leaks can also occur due to inadvertent remaining references to objects you don't actually need (frequent culprits are global event registries and history/state management stacks). Safari/WebKit turn out to have some useful built-in hooks for tracking these down. By enabling Safari's debug menu and choosing the "Show Caches Window" command from there, you can see JavaScript object counts, force a garbage collection and see the types of objects that still have references pointing to them (note that garbage collection has been improved in WebKit nightly builds so you'll probably want to use that).

Drip can provide some similar information for IE, and Leak Monitor finds some Firefox-specific leak types, but it would be good for browser/add-on creators to provide even more debugging information to authors.

Browser Bugs

WebKit and Firefox/Mozilla both have publicly visible bug tracking systems. As they work on their respective new versions, regressions will occur that will break web sites, possibly including yours. Often, this is a genuine bug in the browser code, and hopefully it'll be fixed before the final release. However, it can sometimes be due to incorrect assumptions made by your code which no longer hold true when browser code is tightened up. Both WebKit's and Mozilla's Bugzilla instances support generating RSS feeds from search results. By subscribing to feeds for bugs that mention your product name, you can stay on top of such bug reports. For example, here's a shared page (with feed) for the bugs that mention Reader on either site.

Determining Twitter's Growth

2007-03-16T11:01:00.000-04:00

Opinion on Twitter is divided. What seems to be undisputed is that right now it's growing very quickly. I was curious just how "quickly" quickly was, preferably going beyond just anecdotal "my network doubled in size in the last 5 seconds" kind of observations. It seems like Twitter assigns globally unique, incrementing IDs to all messages it receives. By looking at the values of these IDs over time, it's possible to see how many status messages Twitter is keeping track of. I've generated a logarithmic graph of this.

I'm not sure why there was an inflection point in early November. It's also possible that this is affected by technical changes on Twitter's side. Still possibly interesting. Also, Joshua's post on autoincrement considered harmful is related and an interesting read.

Update: As it was pointed out to me in the comments, Andy Baio had the same idea except he executed it more throughly.

Intern on the Google Reader team

2007-03-09T15:22:00.000-05:00

The Reader team is hoping to have a student intern or two this coming summer. We're fast moving and always have more ideas than manpower, so an internship can be quite rewarding as far as the "working on real stuff" factor goes. For example, our intern last year, Brad, worked on the subscribe and feed search functionalities of the new Reader that launched last September. You can intern in Google's New York or Mountain View offices, working on either Reader's frontend/UI or backend.

If you or anyone you know is interested in this internship, contact me at mihai at persistent dot info. This page also has more general information about interning at Google.

Blogger Migration Part II: Getting Data Into Blogger

2007-02-25T16:55:00.000-05:00

Blogger doesn't have any built-in entry importing facilities. My plan for dealing with this was to use their API to re-post all of the entries I had exported from Movable Type. A quick test showed that such entries could be back-dated, which was my main concern.

Using some sample code that I found as a starting point, it was pretty easy to write a simple importing script for entries. Since I needed to parse the Atom responses from the API (e.g. to get at entry IDs), the Universal Feed Parser came in handy. Since I had used the Convert Line Breaks option on quite a few posts, I had to HTML-ify some post bodies before sending them to Blogger (I've turned off Blogger's similar setting. In addition to being only at the blog level, instead of a per-post setting, I've decided that the closer to regular HTML my posts are, the easier it will be to migrate again in the future).

The Blogger API allows for posting of comments too, but unfortunately without the ability to impersonate others (even when anonymous comments are enabled). The solution was to impersonate the regular (non-API) comment posting form, which does allow for authors to be specified (but no backdating, which is why all imported comments are dated February 10th). This was made slightly trickier since a security token is required (to avoid XSRF attacks), so the posting page had to be scraped first to extract it.

Finally, since Blogger proved to be occasionally flaky when doing many API operations in a row, I had to add some simple checkpointing so that if the process failed I could restart it and it would continue from where it left off. Once I did all that, importing 350+ entries and 600+ comments took around an hour, but worked flawlessly.

I've uploaded an archive of my code that I used for the importing. It's not the cleanest it could be, but others may find it useful too. Additionally, in the time that has passed since I began my importing, it looks like a similar script has appeared that does a similar import operation from WordPress, which may be worth a look too.