persistent.info

Porting Twitter Digest to Google App Engine

2008-05-11T20:26:00.001-04:00

I've been meaning to play around with Google App Engine for a while, and as a quick project, I decided to port Twitter Digest to it (not as exciting as Kushal's Millidunst Calculator). This looked to be pretty straightforward: the original version was already in Python, and wasn't very complicated (just a single CGI script). It did indeed end up pretty easy; the whole process took a couple of hours.

The first step was to port the script from CGI-style invocation to the App Engine webapp framework. Then I looked into what it would take to get Python Twitter (the library I used for fetching data from Twitter) running. Switching it from urllib2 to urlfetch was pretty painless (though I don't use the posting parts of the API, so I didn't check if those work too). The other part of the library that I was relying on was its caching mechanism (since the digests are daily, there's no point in querying Twitter more often). DeWitt (the library's author) had thoughtfully put the caching functionality into a separate class, so it was easy to replace it with another one that implemented the same interface but was backed by App Engine's datastore.

The result (complete with App Gallery entry) is not that exciting, in the sense that it functions identically to the original. The only issue that I've run into so far is that when there are several cache misses, the URL fetches can take long enough that the request hits App Engine's deadline. However, since the successful fetches are cached, repeating the request will eventually succeed (so if consuming the digest via a feed, this shouldn't be a big deal). Ideally the urlfetch functionality would also support asynchronous fetches, since it would be easy to adapt the code to fetch all user timelines in parallel.

Intern on the Google Reader team

2008-04-24T08:38:00.000-04:00

Having interns has worked out well for the Reader team. Following my blog post, we were very pleased to get Nitin Shantharam and Jason Hall to help us out with Reader development. Their stints on the team resulted in a a bunch of features, and Jason is now back at Google working full-time (Nitin wasn't a slacker, he's just still in school).

We're looking for another intern or two this year. Internships generally last a couple of months to twelve weeks, are for full-time students, and would be in Google's Mountain View, California office. You can work on either Reader's backend (a C++ system for crawling millions of feeds, handling lots of items being read, shared, starred or tagged per second) or frontend (Java servers and JavaScript/AJAX-y craziness) depending on your interests and experience.

If you or anyone you know is interested in this internship, contact me at mihaip at google dot com. This page also has more general information about interning at Google.

persistent.coffee

2008-04-03T14:46:00.000-04:00

I've been trying my hand at latte art. Though I have a very long way to go, I've been documenting my efforts, with a hope of learning from my mistakes. Blogger's mobile support makes it pretty easy to collect pictures, and I've finally gotten around to making a decent template for the "blog."

coffee.persistent.info is the result. Technically, this isn't a Blogger template, since I just have some static HTML as the content. Instead, it uses the JSON output that Blogger's GData API supports. Rendering the page in JavaScript allows for more flexibility. I wanted to make pictures that I liked take up 4 slots (a layout inspired by TwitterPoster). This imposed additional constraints (in order to prevent overlap between sequential large pictures). The display is generally reverse-chronological starting from the top left, but images are occasionally shuffled around to prevent such overlaps. There is also a bit of interactivity, the pictures are clickable to display larger versions. To help with all this, I've been experimenting with jQuery (also on Mail Trends), and am liking it quite a bit.

Mail Trends

2008-03-25T10:32:00.011-04:00

I get a lot of email (especially at work). I'm trying a Inbox Zero-like approach in order to keep up with it. Though that's helping me to stay on top of things, I had the nagging feeling that I was probably on too many mailing lists, and that some of them were probably not worth it from a signal-to-noise ratio perspective.

Ideally something like the Reader Trends or Search History Trends page would exist for Gmail. I thought I could perhaps build it myself, but the absence of an official Gmail API deterred me. However, it occurred to me that the recently added IMAP support could act as an API of sorts. It should be easy to get just the message headers and slice and dice them to extract the stats that I was interested in.

Thus was born Mail Trends, an IMAP-based email analysis project. It can generate a bunch of tables, graphs and distributions based on time of day, senders, recipients, mailing lists, etc. To get a feel for what it can output, see the results of running it on a piece of the Enron Email Dataset. To run it over your own email, see the getting started page. As a caveat, the program currently loads everything into memory, so my run on 200,000 messages resulted in 1.6 gigabytes being used. You may want to use the --max_messages= flag to limit the dataset, at list for initial runs.

The project is still in its early stages, so patches and suggestions are definitely welcome (my email address is at the footer). You can also subscribe to the feed of check-ins to see changes as they are made. The plan wiki page has a very brief outline of what I'm planning on working on next.

Two Safari 3.1 Tips

2008-03-22T14:23:00.001-04:00

Safari 3.1 is out, and I've upgraded my Mac to it. Besides some issues with arrow keys in Reader (we're on it), it's working out well. Here are two hidden prefs that you may find useful:

defaults write com.apple.Safari IncludeInternalDebugMenu -bool true

The Develop menu that 3.1 includes is nice, but it seems to supplant the old "Debug" menu (i.e. the preference key that used to toggle it - IncludeDebugMenu - now toggles the "Develop" menu). The old menu had functionality that isn't present in the official one, most notably the "Caches" window that displayed the number of live JavaScript objects and made tracking down memory leaks much easier. If you'd like to bring back the old menu, you can use the new IncludeInternalDebugMenu key shown above

defaults write com.apple.Safari TargetedClicksCreateTabs -bool true

First spotted on Twitter, this forces new windows to open in tabs, one feature that I missed from my Firefox days.

Decommissioning Mscape Software

2008-02-18T21:50:00.000-05:00

Before I became enamored with web development, I used to be a Mac software developer under the Mscape Software moniker. My first public release (clip2cicn, a helper tool for making the icon resources necessary for Kaleidoscope schemes) was almost 10 years ago, on June 26, 1998. My flagship product was Iconographer, an icon editing tool.

I haven't had time for Mscape Software for about 4 years (the last Iconographer release was in July of 2003, and I last touched the code in early 2004). The site was still up, and I kept receiving registrations (Iconographer was a $15 shareware product). As time went on, I began to feel more and more guilty about not providing any support for (paying) customers. To this end, I have finally gotten around to dismantling Mscape Software, replacing the site with a placeholder with download links for all products. I've also put up a registration code for Iconographer so that the (annoying) registration reminder can be shut off.

I've gotten some requests to open-source Iconographer, but I'm not sure I'll have time even for that. I'm not 100% sure I can build the product given software on hand (it was built with CodeWarrior). Even then, the codebase shows its age (it was my first large-ish project and still uses many Classic constructs like a WaitNextEvent event loop).

plusplusbot: Karma for anything and anyone

2007-11-16T10:14:00.000-05:00

The recent release of Foamee reminded me that I had never blogged here about plusplusbot, a little toy of a site that I started to work on this past summer. The idea behind plusplusbot is that you can express your (dis)pleasure with something or someone over Twitter, and the site will keep track of the target's score over time. All of this is done over Twitter, in a barnacle-like fashion. The thinking being that Twitter handles message delivery over a variety of mediums (Jabber, SMS, Web) and provides a ready-made social network to piggyback on (you can view the activity of you and those you follow, for example, here's mine). This concept should be familiar to those that have enountered "karmabots" in chatrooms.

The targets can be anything. Companies, food, people, even Twitter itself. For whatever reason, the site has caught on most with Japanese Twitter users, so the homepage is often incomprehensible.

Technically, the site is not too exciting. Like Twitter Digest, it also uses python-twitter to talk to Twitter and templet for simple templates. The site itself is not dynamic at all, instead a script running on one of my machines polls Twitter every two minutes (since it has to fetch both friend updates and direct messages, fetching both every minute would go over Twitter's 70 requests/hour API rate limit). If it determines that a new plusplus or minusminus has been sent, it re-generates those pages (the user's and the target's) and uploads them. The simple design means that the site can be hosted nearly anywhere.

Macros for the new version of Gmail

2007-11-05T21:09:00.001-05:00

As those of you on the new version of Gmail might have noticed, nearly all Greasemonkey scripts that used to work on the old version no longer do. Even though it looks pretty similar, the new Gmail is entirely different from a JavaScript, HTML and CSS perspective, so this isn't surprising. Some of the scripts are no longer necessary. For example, saved searches aren't really needed, since searches now get their own URLs and can be bookmarked.

However, my Macros script is still needed; while the new version of Gmail does have additional keyboard shortcuts, it's still not possible to do everything from the keyboard. I've therefore ported it for the new version of Gmail, to install it, click below.

Install Gmail Macros

More specifically, the following keyboard shorcuts have been ported over:

g: Go to label
l: Apply label
b: Remove label
e: Archive (regardless of view, unlike "y")
d: Discard (mark as read and archive)

There is also a new keyboard shortcut, which the old script didn't have:

f: "Focus" the current view (only show unread, starred or inbox messages)

You may remember the script supporting other keyboard shorcuts. Since the new version of Gmail supports additional shortcuts, those have obsoleted. The new ones are:

shift + i: Mark as read
shift + u: Mark as unread
shift + 3: Move to trash (not actually new, but not many people seem to know this one)
shift + 8 followed by a, n, r, u, s, or t: Select all, none, read, unread, starred, or unstarred

For those of you adventurous enough to look at the script source, you'll notice that it uses a gmonkey object that is present on the window, which in turn gets you a gmail object with methods like getNavPaneElement() and getActiveViewType(). What this means is that the version of Gmail, in addition to being faster, also has semi-official support for Greasemonkey scripts. I'm pretty sure docs for this API will be out soon, but in the meantime, feel free to look at the script and use a tool like Firebug to investigate the properties of the gmonkey and gmail objects and play around.

Update on 11/06/2007: And here are the semi-official docs.

Feed Proxy: View Raw RSS/Atom Feed Data in Firefox 2.0

2007-10-01T20:24:00.000-04:00

Firefox 2.0 has a nice RSS/Atom feed preview feature, which pretty-prints the feed and allows you easily subscribe to it in a feed reader. While this is nice from a user's standpoint, it can get in the way when trying to debug or otherwise see a raw feed as a developer. There is no option to turn this mode off, which seems to be making some people unhappy.

To help with this use-case, I made Feed Proxy, a simple service that gives you back the raw XML output (by adding a 512 byte dummy header that makes Firefox not invoke its feed preview code). Its page has more details on how to invoke it and what its caveats are.

Valleywag, brought to you by....Google?

2007-09-26T18:11:00.000-04:00

Gawker Media's use of virtual hosting for its sites was being discussed in PartyChat, so to verify this, I ran host valleywag.com to compare it with host fleshbot.com. Imagine my surprise when I saw this output:

$ host valleywag.com
valleywag.com has address 69.60.7.199
valleywag.com mail is handled by 1 aspmx.l.google.com.
valleywag.com mail is handled by 5 alt1.aspmx.l.google.com.
valleywag.com mail is handled by 5 alt2.aspmx.l.google.com.
valleywag.com mail is handled by 10 aspmx2.googlemail.com.

If I am interpreting this correctly, Valleywag's email is being handled by Gmail/Google Apps, which is slightly ironic.

Twitter Digest

2007-08-31T10:30:00.000-04:00

There are quite a few people on Twitter that I'd like to follow but can't, since they update way too often. For example, I'd like to know how Craig Hockenberry's MobileTwitterrific work is going. John Gruber often posts advance tidbits from Daring Fireball. However, since Twitter doesn't seem to allow different notification levels for contacts, I'd rather not get overwhelmed with their updates on a regular basis.

Twitter does have an API though, and a pretty complete one at that. Additionally, python-twitter makes it pretty easy to code against it. The result is that it was pretty easy to make Twitter Digest. You can give it a bunch of usernames and it will update a page once a day with all of those people's tweets. Feeds can also be generated, so that you can subscribe to your digest in your favorite feed reader. To give an example, here's my digest (and its feed.)

Since python-twitter already has the the concept of caching API replies, this was especially easy to develop. Simply by juggling with cache timeout values, API replies are kept around for up to a day, thus I don't have to persist anything in my code. All digests are refreshed at GMT day boundaries because supporting timezones seemed like too much work.

Digests can be thematic too. Here's one for the Twitter team. I'm sure there are other uses too. I'm pretty happy with how this turned out, and the only thing missing is better reply support (if a message is in reply to another, then (optionally) show that message inline too). Right now the API is missing support for this (even though the regular UI surfaces this information), but I'm sure something can be hacked together.

Update on 9/3/2007: Adding support for "in reply to" messages turned out to be easy enough (look for the first message from the other username that's before the one that's replying, but only up to an hour earlier, in case it's not actually connected).

iPhone Game Development: A Tale of Failure

2007-07-22T15:29:00.001-04:00

Having acquired an iPhone, I've been playing around with it to see just how much can be done with the "SDK" that is the MobileSafari browser. I thought it would be fun to see if I could make a simple game for it, since that's different enough from the kind of UI coding I do at work.

Most of the games for the iPhone that I've encountered thus far were of the puzzle/card variety, and there wasn't anything too exciting/button-mashing/mindless. It seemed like this was mainly caused by the limitations of the events that the browser exposes. With no real mousemove events, and with mousedown events actually firing when the cursor was lifted (i.e. at the same time as mouseup), it was hard to get anything to happen on the screen quickly in response to a user action. One avenue that I tried was to have the screen covered in narrow vertical <div>'s, and to listen for click events on each of them. Unfortunately, while this worked when each was clicked separately (piano key-style), it didn't work when the user dragged his finger from one to another, since this would switch the browser into panning mode, even if the page/viewport was set to be only 320 pixels wide (i.e. no scrolling could actually occur).

Since the viewport panning does not seem to expose any events, I thought that maybe there could be other ways of catching this user action. When scrolling quickly, the iPhone renders newly-visible areas with a checkerboard pattern before actually displaying their contents. I wondered if this meant that images contained in these areas would not be loaded until they were scrolled into view and rendered. If so, it would be possible to use a load event listener as a notification that the user had scrolled to a particular part of the screen. Unfortunately this didn't seem to be the case, images are loaded as soon as the document is displayed, even if they are off the screen.

I then remembered that the iPhone supports an additional gesture, a two-fingered scroll that can be used to scroll text-areas, elements with overflow: auto and possibly other nodes. It would appear that scroll events trigger for this scrolling motion, meaning that it's possible to detect and get continuous updates for this. Though having to use two fingers seemed a bit clumsy, this seemed pretty promising insofar as it allowed for quick responses to user input.

I had lofty goals for my iPhone game (I thought a vertical scrolling shooter was a good fit for the dragging input method and the vertical screen layout), but I decided that I should do something simple for my first attempt. I thought I would recreate aa racing game that I hard first played on my TI-82 calculator, where a car that can only be moved left or right moves at a constant vertical speed through a winding, narrowing road, with the objecting being to avoid hitting the sides. I thought this would map well to a horizontal scrolling motion that I could get with a scroll event handler as mentioned above.

After writing a simple random road generator and the animation code, I was about to add collision detection. Out of curiosity, I tried to move the "car" left/or right as road moved by, even without collision detection, to see if the speed was too high. To my surprise, the road stopped moving as soon as I put my two fingers on the screen. It appeared that intervals (set up with window.setInterval) do not fire when two-finger scrolling mode is engaged. I thought that maybe I could refresh the animation from the scroll event handler, but that only seems to fire when there is actual motion, thus smooth animation can't be ensured.

My failed attempt is visible here. Before pressing the "Start" button, use the two-fingered scrolling motion left and right to confirm the the "car" (the blue square) moves as expected. Then press "Start" and obverse the reasonably smooth animation. Unfortunately it all stops when placing two fingers on the screen again.

This is rather disappointing, since there doesn't seem to be any fundamental technical reason why this should be the case. We're not talking about Mac OS 9 here (there, due to the way UI events were processed, normal event loop execution would be suspended while a menu was held open or a scrollbar thumb was dragged). It's quite clear, given what native iPhone apps can do, that the platform has quite a lot of CPU and graphics leeway. It's therefore unfortunate just how constrained the MobileSafari browser is, since just a few additional hooks would allow much richer interactions with the user.

Bad Headlines

2007-05-04T09:43:00.000-04:00

Headlines seen over the past few weeks on the start page that Verizon forces on its users, and what they refer to in parentheses:

"Daily Life is a Struggle" (life in Baghdad)
"All Sides Deny Deals" (Iran/UK sailors hostage situation)
"Panel Seeks Documents" (Alberto Gonzales' federal prosecutors firings)
"Cheruiyot Wins Marathon" (Robert Kipkoech Cheruiyot won the Boston Marathon)
"Gates: Clock is ticking" (Defense Secretary Robert Gates on the situation in Iraq)
"Donkey Becomes Witness" (Dallas man is sued for noise disturbance caused by a donkey)
"Dems Predict a Win" (House Democratic leaders predict that they can win a vote on a bill calling for a withdrawal from Iraq)
"Candidates to Debate" (The Republican presidential candidate debate)

Someone forgot to read up on microcontent headlines.

Tiled Windows Are the Way of the Future

2007-04-11T11:40:00.000-04:00

Sorry Bill Atkinson. I know you worked really hard to make regions and thus overlapping windows work. But since I now have 30 inch displays at both home and work, I've been tiling my windows and using Desktop Manager (and soon Spaces?) to switch between workspaces. Not having to deal with z-ordering makes mental context switches faster.

Communicating through screenshots

2007-04-06T13:32:00.000-04:00

My posts to the Reader blog often have screenshots, and I'm pretty picky about what goes in them. It's nice to see that others have noticed this too. My most recent post has a link to Dinosaur Comics, which the author seems to appreciate. Similarly, I intentionally left a reference to Valleywag in my trends post, which they noticed.

In some fashion, Apple does this too. A patent of theirs had a BBEdit reference, which amused the author of the software.

Two Web Development Tips

2007-03-25T20:32:00.000-04:00

Memory Leaks

Fighting memory leaks in fancy AJAXy apps is a way of life for most UI developers. Besides Internet Explorer-specific issues, leaks can also occur due to inadvertent remaining references to objects you don't actually need (frequent culprits are global event registries and history/state management stacks). Safari/WebKit turn out to have some useful built-in hooks for tracking these down. By enabling Safari's debug menu and choosing the "Show Caches Window" command from there, you can see JavaScript object counts, force a garbage collection and see the types of objects that still have references pointing to them (note that the garbage collection has been improve in WebKit nightly builds so you'll probably want to use that).

Drip can provide some similar information for IE, and Leak Monitor finds some Firefox-specific leak types, but it would be good for browser/add-on creators to provide even more debugging information to authors.

Browser Bugs

WebKit and Firefox/Mozilla both have publicly visible bug tracking systems. As they work on their respective new versions, regressions will occur that will break web sites, possibly including yours. Often, this is a genuine bug in the browser code, and hopefully it'll be fixed before the final release. However, it can sometimes be due to incorrect assumptions made by your code which no longer hold true when browser code is tightened up. Both WebKit's and Mozilla's Bugzilla instances support generating RSS feeds from search results. By subscribing to feeds for bugs that mention your product name, you can stay on top of such bug reports. For example, here's a shared page (with feed) for the bugs that mention Reader on either site.

Determining Twitter's Growth

2007-03-16T11:01:00.000-04:00

Opinion on Twitter is divided. What seems to be undisputed is that right now it's growing very quickly. I was curious just how "quickly" quickly was, preferably going beyond just anecdotal "my network doubled in size in the last 5 seconds" kind of observations. It seems like Twitter assigns globally unique, incrementing IDs to all messages it receives. By looking at the values of these IDs over time, it's possible to see how many status messages Twitter is keeping track of. I've generated a logarithmic graph of this.

I'm not sure why there was an inflection point in early November. It's also possible that this is affected by technical changes on Twitter's side. Still possibly interesting. Also, Joshua's post on autoincrement considered harmful is related and an interesting read.

Update: As it was pointed out to me in the comments, Andy Baio had the same idea except he executed it more throughly.

Intern on the Google Reader team

2007-03-09T15:22:00.000-05:00

The Reader team is hoping to have a student intern or two this coming summer. We're fast moving and always have more ideas than manpower, so an internship can be quite rewarding as far as the "working on real stuff" factor goes. For example, our intern last year, Brad, worked on the subscribe and feed search functionalities of the new Reader that launched last September. You can intern in Google's New York or Mountain View offices, working on either Reader's frontend/UI or backend.

If you or anyone you know is interested in this internship, contact me at mihai at persistent dot info. This page also has more general information about interning at Google.

Blogger Migration Part II: Getting Data Into Blogger

2007-02-25T16:55:00.000-05:00

Blogger doesn't have any built-in entry importing facilities. My plan for dealing with this was to use their API to re-post all of the entries I had exported from Movable Type. A quick test showed that such entries could be back-dated, which was my main concern.

Using some sample code that I found as a starting point, it was pretty easy to write a simple importing script for entries. Since I needed to parse the Atom responses from the API (e.g. to get at entry IDs), the Universal Feed Parser came in handy. Since I had used the Convert Line Breaks option on quite a few posts, I had to HTML-ify some post bodies before sending them to Blogger (I've turned off Blogger's similar setting. In addition to being only at the blog level, instead of a per-post setting, I've decided that the closer to regular HTML my posts are, the easier it will be to migrate again in the future).

The Blogger API allows for posting of comments too, but unfortunately without the ability to impersonate others (even when anonymous comments are enabled). The solution was to impersonate the regular (non-API) comment posting form, which does allow for authors to be specified (but no backdating, which is why all imported comments are dated February 10th). This was made slightly trickier since a security token is required (to avoid XSRF attacks), so the posting page had to be scraped first to extract it.

Finally, since Blogger proved to be occasionally flaky when doing many API operations in a row, I had to add some simple checkpointing so that if the process failed I could restart it and it would continue from where it left off. Once I did all that, importing 350+ entries and 600+ comments took around an hour, but worked flawlessly.

I've uploaded an archive of my code that I used for the importing. It's not the cleanest it could be, but others may find it useful too. Additionally, in the time that has passed since I began my importing, it looks like a similar script has appeared that does a similar import operation from WordPress, which may be worth a look too.

Understanding feed reader marketshare numbers

2007-02-18T17:48:00.001-05:00

Update on 2/25: FeedBurner has published a post discussing this same issue but providing numbers for their whole userbase, which makes it even more interesting.

Ever since the Reader team announced that we were making public subscriber counts (thanks Justin), bloggers have been excitedly posting about the bumps they're seeing in their subscriber stats. I'm obviously very happy that Reader is getting all this attention, and that we turn out to be quite popular when compared to other feed readers. However, these statistics need a bit of interpretation. Most people post charts of their subscriber counts, like this one for this blog:

For web-based readers where feeds are fetched on behalf of multiple users, the subscriber number is based on what the site reports. To the best of my knowledge, with the exception of My Yahoo!, these number are total subscribers, even if an account is inactive. Unless the site is aggressive about cleaning up inactive accounts, these numbers are only upper bounds on the number of actual readers that you have.

A more interesting number to look at is how many viewers each item gets from each feed reader. FeedBurner provides this as part of their TotalStats package. By embedding a small tracking image in your burned posts and looking at referrers, it's possible to see these item-specific views. Here are how many views and clicks my post from yesterday got in various feed readers:

From this it would appear that Reader has an even bigger lead over Bloglines (though given the biases in this blog's readership, I'm not reading too much into this). There are other factors involved here too. The user bases for feed readers are not identical, if an item appeals more to one population than another, that may skew things. Additionally, some readers (especially homepage-style ones like My Yahoo!, Google Personalized Homepage and Netvibes) don't have to display the item body and allow users to jump straight to the post page. These would show up in the "Clicks" column but not in the "Views" one.

What becomes apparent is that none of these statistics provide a complete picture of your readership, but that when used together they can still give you broad trends and help you tailor your content to your audience.

Blogger Migration Part I: Getting Data Out of Movable Type

2007-02-17T10:57:00.000-05:00

The first step in migrating away from Movable Type was to get all of my entries and comments in a structured format that could be parsed and uploaded to Blogger*. MT doesn't hold data hostage, there is a documented import/export format. Six Apart considers the format "lossy", in that it doesn't save a complete snapshot of your blog. I decided that what it did contain was good enough, though it turn out that what it lacked (entry IDs and permalinks) did make things slightly more difficult. A search on code.google.com for Python code to parse the format turned up Transfusion which does just that (searching for one of the magic strings in the format, CONVERT BREAKS specifically, was the easiest way to track this down).

As I was skimming through the exported entries, I saw that they weren't quite HTML. I had used MTMacro to create various shorthand tags for linking to entries, reference to images, etc. Similarly, I used MTCodeBeautifier to pretty-print code samples. None of these were getting evaluated when exporting, and even if they had been, I probably would have wanted to tweak their output anyway (e.g. to change URLs). Generally, it seemed like the time I had spent customizing my Movable Type installation with cruft-free URLs, plug-ins, etc. would be directly proportional to the time I would have to spend migrating away from it.

One of the more prevalent macros I had used was one of the form <entryLink id="NNN">foo</entryLink> so that I could link to my past entries. Unfortunately, since entry IDs were not included entry IDs issue, there was no easy way to turn these into actual links, since the exported information did not contain entry IDs or URLs. In the end, I ended up converting these by hand.

That's it for the exporting part, part II will contain the Blogger import process and part III the template/design reasoning.

* the other migration option that I was considering was WordPress. However, the idea of having to do SQL queries to serve traffic didn't seem that appealing given my current provider's slow SQL performance. WordPress.com would have been a hosted option, but if I was going to relinquish control of the installation, it might as well be to a Google product.

Switched to Blogger

2007-02-10T23:58:00.000-05:00

Partly because I was fed up with Movable Type's rebuild times, but also for dog food reasons, I've moved my blog over to Blogger (custom domains was the final missing piece). Redirects should be in place and no links should break, but feed readers will most likely see a bunch of new entries (I didn't see an option in FeedBurner to suppress duplicates). Please leave a comment if you see anything amiss.

I'll be writing up more details this coming week about the work that was necessary to migrate.

More Efficient FeedBurner Visitor Tracking

2007-01-29T09:47:00.000-05:00

I was quite happy to see that FeedBurner had integrated site visitor tracking in addition to their already great feed usage statistics. Actually using it with my blog was just a matter of cutting and pasting some code into my template. Here's the snippet that I was offered for Movable Type:

<script src="http://feeds.feedburner.com/~s/PersistentInfo?i=<$MTEntryPermalink$>" type="text/javascript" charset="utf-8"></script>

This seems reasonable, but it bothered me that the tracking URL varied for each entry. This meant that for my front page, 7 different (but nearly identical) pieces of JavaScript would have to be fetched, slowing down the rendering of my site. Others have complained too. I think a better way of implementing this would have been to have something like:

<script src="http://tracking.feedburner.com/tracking.js" type=text/javascript" charset="utf-8"></script> <script type="text/javascript">FBTrack("PersistentInfo", "<$MTEntryPermalink$>");</script>

That would have meant that the same tracking code can be used across all sites thus most users would have it cached (I think that's what Google Analytics relies on). Unfortunately, that's not something that I can change on my own.

However, when running into a related problem (a FeedFlare was stuck on) it was pointed out to me that the i=... part of the script URL can be removed if all that's desired is the visitor tracking and not per-entry features like FeedFlares. This means that at least across my entire site there is only one tracking script URL that visitors have to load. It turns out FeedBurner documents this, but only in the "Others..." category (the bottom "If you want to use StandardStats only" section).

Moving my Gmail Greasemonkey scripts to code.google.com

2006-12-30T18:44:00.000-05:00

I wanted to experiment with code.google.com's project hosting feature, and I thought an easy way to get started would be with my Gmail-related Greasemonkey scripts. This is their new home. I'm not sure how far I'll go with the "project" part (e.g. whether I would ever give anyone else commit privileges, though patches are welcome), but this at least lets me familiarize myself with Subversion.

I've also decided to actually serve all the scripts from the repository, so that there isn't an extra publishing step when making changes (thus far I don't feel the need for stable versions). There is also a feed for all changes, so this should make it easy to stay up to date even if I don't explicitly blog about them. The feed is generated with an XSLT based on Norman Walsh's transform.

I also took this opportunity to make a few improvements to the scripts. The macros one can now also select saved searches when using the "g" keyboard shortcut (contributed by another user). The saved searches script now namespaces the saved data by the current user's email address. This should enable it to work better when there are multiple Gmail users per computer or Google Apps for Your Domain is being used.

Greasemonkey Script Updates

2006-11-08T10:14:00.000-05:00

There have been some reports that my Gmail Macros Greasemonkey script no longer works. I started to look into fixing this, and figured that while I was at it, I should incorporate the improvements that others have made to the script back into the original one. Both posts about it have a growing number of comments, including patched versions with various tweaks. Additionally, there is a Google Group dedicated to Gmail power users that seems to have made even more changes. The script has been updated and now contains the following additional commands:

b: Remove label
z: Mark unread
o: Expand/collapse all messages in a conversation
shift-x + a/n/r/u/s/t: Select all/none/read/unread/starred/unstarred
h: Show help (reference for built-in and the script's keyboard shortcuts)

Thanks go to Karl, Anand and Michael (?) for these improvements. If I have missed anyone, please let me know.

On a related note, my Google Reader/Gmail integration script has spawned two variants, one by Rob Radez and another by Winston. They went for slightly deeper integrations, if you like the original you may like these too.