Taylor Hughes — Blog

HTTPS: Insist on secure authentication

taylor@taylor-hughes.com (Taylor Hughes) — Tue, 15 Dec 2009 03:43:51 +0000

While checking out some other to-do list options about four months ago, I noticed something disturbing: Ta-da List, a popular to-do list app and top Google result for “to-do list”, doesn’t use SSL for its login or sign-up forms.

This was shocking to me, since using HTTPS for authentication has always been a no-brainer. Since then, however, I’ve been flabbergasted to find two new services that also launched without HTTPS support for sign-up and authentication: Brizzly, a twitter client, and TeuxDeux, another to-do list app. Brizzly is awesome — really, I love it — but I haven’t used it since I discovered this issue a month or so ago. (Update: Brizzly has added HTTPS for authentication; I am free to use it again.)

I contacted support on all three sites — which I consider to be high-profile, professional efforts — and all three essentially said they’d get around to adding SSL for authentication at some point in the future.

Frankly, I don’t understand how they can consider plaintext password transmission acceptable for their users, now or ever.

Perhaps the problem is misunderstood. Maybe web developers aren’t concerned with unsecured wireless networks or the off chance that somebody is doing deep packet inspection in between their users’ web browsers and their datacenters.

But I think they should be concerned — especially about unsecured wireless networks — and not risk exposure of their users’ sensitive data.

Vulnerable information: Users’ passwords are valuable

My first assumption going into this is that passwords are valuable information and users trust us as web developers to keep them safe.

Passwords are valuable because people reuse passwords all the time. I work at a network security company, and even I reuse the same set of passwords over and over again for a variety of services I use — it’s just unavoidable when you have to sign up for a new web site virtually every day.

Thus, the password I use at some random site is probably the same password I use for any number of other sites — and most people probably use the same password for their bank accounts, too. Maybe even the same username.

This isn’t how things have to be — you can store your passwords in the Apple Keychain or use a program like 1Password to remember unique, complicated passwords for you for each site you use. But, for the typical user, this is not the case.

Note: I am not as worried about sessions being hijacked because session hijacking has a limited scope. For example, with a hijacked twitter session, someone could post malicious tweets, but he can’t log into my bank’s web site. Passwords are private data that people trust is kept secret.

Anatomy of an insecure login

Here’s how a typical insecure login happens over HTTP:

You type your username and password in a form and hit “Login”
The browser “form-encodes” the data you typed into the form so it looks sort of like this: username=taylor&password=mypassword
A web request is created including the form-encoded data
The request data leaves your computer and travels over your local wired or wireless network
The request data leaves your home or office, traveling through your ISP’s network
The data leaves your ISP and travels to its destination through a variety of core routers and switches, which are generally regarded as secure
Your data arrives at the datacenter where the servers live and is routed to the proper destination
The server gets your request, checks your credentials and issues you a session cookie, which is something you’ll identify yourself with in subsequent requests
The response from the server — including your new session cookie — flows back through the same channels and into your browser.

The parts highlighted in red are places where your password, transmitted in plaintext, could be intercepted. Let’s look at those spots a little more in detail.

Wireless networks: Unsecured wireless as a bullhorn

At home, you probably use WPA or (gasp) WEP security to keep your neighbors from accessing your wireless router. But those technologies also encrypt your network’s traffic, so the hacker sitting outside with a laptop and a Pringles can can’t easily get a look at the juicy gossip you’re reading on Perez Hilton.

Most public access points — coffee shops, airports, etc. — use unsecured wireless networks. These unsecured wireless networks rely on HTTPS and other secure transmission media to protect your sensitive data while it flies through the air. Thus, on an unsecured wireless network, your insecure HTTP requests and AIM conversations and torrent downloads are blasting out over the airwaves for anybody to read.

That’s basically like announcing your passwords over a bullhorn for the entire coffee shop to hear.

You can try this experiment on your home computer: Download Wireshark, start collecting packets from your own network interface, and login to Ta-da List. You’ll see an HTTP packet with a form-encoded version of your login information contained in a HTTP POST request. It looks sort of like this, except the password isn’t blurred out.

It’s right there, in plaintext.

This is easily the most dangerous point for unsecured HTTP transmissions, and the point that should most worry both users who are logging in using an insecure form and the web developers who created them. You should assume that every packet you transmit on an unsecured wireless network is being read by someone else.

Your corporate LAN or ISP, and the datacenter your data ends up in

Your network traffic, somewhere down the line, is probably being siphoned off and monitored by some assortment of devices. The core Internet is not so much a worry, but your local ISP, your corporate LAN or your university’s vast network are probably being tapped at various points and inspected.

Heavy-duty switches nowadays come equipped with ports specifically designed for this: SPAN ports copy the data off for various analyses, or you can buy a network tap to do effectively the same thing. This is how intrusion detection systems work: You plug an IDS into a port that essentially copies off all the traffic on your network, then the IDS performs some packet inspection to figure out if anything bad is going on. You can do this at immense speed, applying “filters” or “signatures” to gigabits of traffic every second without breaking a sweat. IDS technology is extremely popular, making up a huge portion of the cybersecurity performed at nearly every major organization across the globe.

The point is not that security groups are out to get you, it’s that deep packet inspection at line speeds is not difficult.

With the system I work on every day, for example, I could write a filter to extract Brizzly usernames and passwords in just a few minutes. You could even do it with a simple packet capture device filtered by destination IP: You just need access to the network anywhere in between the source and destination.

The sheer volume of the traffic on most big networks is the only thing working to your advantage in this case — it’s nearly impossible to look through everything in detail, so it’s unusual that your password would be stolen over a wired connection. But I would rather not take my chances.

Technology to the rescue: HTTPS

Luckily, it doesn’t matter who is listening if you encrypt HTTP requests containing passwords using HTTPS. SSL uses a public-key handshake to guarantee two things:

My password is being transmitted to the correct recipient (Ta-da List, etc.), and
My password information is encrypted and unreadable by anyone else who doesn’t own a supercomputer.

Further, it’s easy to configure: You buy an SSL certificate and install it, flip on SSL, write a rewrite directive in your HTTP server, and you’re set — nobody can read your users’ passwords except you. No app code changes necessary.

HTTPS is not going to solve every security problem — how’s your password storage, by the way? — but it’s an easy step in the right direction. Web developers should use HTTPS for sign-up and authentication, no questions asked.

Disclaimer, or qualifier, depending on how you look at it: I work at a network security company that builds a device to analyze networks and figure out who the bad guys are.

The four dimensions of developer happiness

taylor@taylor-hughes.com (Taylor Hughes) — Tue, 23 Jun 2009 01:38:41 +0000

I’ve spent a lot of time thinking about what I like about programming for a living, what I like about my job in particular and what could make it all even better — and I’ve boiled it down to four dimensions that need to be satisfied by a workplace to make it truly rock. I’ll call them “Taylor’s Dimensions of Developer Happiness.”

If you’re firing on all four cylinders, congratulations — you’ve found a great place to work.

The code: writing good software

As software engineers, the product of our toil is essentially a pile of text files — but the quality and composition of them is vital. Even more important is the collaborative, thoughtful process that goes into creating a lasting set of reusable, maintainable components.

Gauging this category might mean answering questions from the Joel Test, but there’s even more. How about some of these:

Do you believe in testing? Does someone on your team freak out if a code path is untestable (or untested)?
Do you do code reviews? Have you been talked into refactoring a nasty bit of code recently?
Do you have deep discussions about the merit of various source control systems?

All of these things individually are great — but the more that resonates with you, the more likely it is that there’s experience behind the process you’re following to build good code.

The coworkers: working with people you like

I am a strong believer that the people you spend eight hours a day with should be people you’d gladly spend the rest of the evening with at a bar or restaurant. If you can’t get along with them over drinks, you sure as hell won’t be able to build a quality product with them.

That isn’t to say you should be best friends with everyone you work with — just that you enjoy spending the majority of your daily life with them and that you trust in their ability to make decisions that impact you on a daily basis.

This is mostly something that goes without saying, but it’s easy to glaze over or forget the importance of the team’s social makeup when you’re interviewing or distracted by a job offer.

The mission: building a great product

I saw a talk recently by one of the co-founders of Alice.com talking about what has led to the success of his previous start-ups. He said something to this effect: Start-up employees are on a mission to change the world; they need to know the mission and be revved up about the product to make it all come together. In fact, he said the start-up employee is one of the only advantages start-ups have.

Whether or not you work for a start-up, however, the mission is extremely important: You need to be commuting to work every morning for a reason.

You should be able to delve into an exciting summary of the work you’re doing and talk my ear off for about a half hour without stopping. It should keep you up at night. The end result of your work is extremely important: You need to be proud and excited about what you’re doing.

The challenge: learning as you go

The last dimension to developer happiness: The environment you work in should push you with new challenges on a constant basis. This means learning new things and venturing into unknown territory every day.

Lots of different things can be challenging, and in a start-up environment especially, challenges will crop up out of nowhere all the time: technological nightmares, interpersonal conflicts and everythingelseinal challenges abound.

But I think it’s important to put yourself in an environment that breeds those challenges, and to avoid situations you know you can handle.

The… end

And that’s it. This is mostly the product of my working in a small start-up and evaluating the options out there in the rest of the world — so, take it with a grain of salt. I am fairly certain, however, that if you can cover those four bases, the job is a great fit.

Going Django-less on App Engine

taylor@taylor-hughes.com (Taylor Hughes) — Tue, 16 Jun 2009 03:09:25 +0000

I just finished migrating DNZO to run on the webapp framework, after running on Django 1.0 via app-engine-patch for the past several months. I learned a valuable lesson in the process of switching: The webapp framework is really nice and, perhaps more importantly, way faster than using a zipimported version of Django.

It makes sense that using a zipped up copy of Django would be slow: You’re making the webserver inflate a copy of a framework you’re using to serve requests before it can serve them. The most important thing to note here, however, is that App Engine aggressively kills off application instances, especially during the day (when overall system load is presumably higher) — so you don’t get the opportunity to reuse those inflated copies of Django very often.

As a result, I ended up being hit with the initial load penalty frequently: Almost every request during the day took upwards of two full seconds. And that totally sucks.

So, I started out the transition by building a new section of DNZO that uses the webapp framework. Obviously you lose a lot of goodies provided by Django, but the webapp framework provides a basic class structure, some URL mapping and resolution, and it’s generally really nice and easy to extend. It provides nothing I don’t want, and the stuff I do want that it doesn’t have (like straightforward cookie support — really?) is pretty easy to whip up. Before long, I had a basic structure that worked well and I was ready to migrate the rest of DNZO to the same framework.

A few subversion commits later and we have the new version of DNZO: Now with page loads that are about 5 to 10 times faster than when using the zipimport’d version of Django. Fresh page loads usually take between 300 and 500 milliseconds, down from two to three full seconds: the difference between feeling slow and feeling pretty snappy.

Movin’ on up, to the client side

taylor@taylor-hughes.com (Taylor Hughes) — Mon, 30 Mar 2009 01:12:08 +0000

I launched a new feature for DNZO last week: fast, JavaScript-based task sorting. Instead of forcing a new page load, tasks are sorted near-instantly on the client side. In order to enable bookmarking and the back button, I used the PrototypeXtensions.js library’s history manager — after a bit of hackery to get it working as I expected.

This move to JavaScript for sorting mirrors a common theme I’ve experienced in developing DNZO: I’ve slowly moved a lot of things over to the client side to increase responsiveness, minimize server-side requests and keep the requests that do need to happen light.

Original plan: JavaScript as frosting

When I started building DNZO, I initially designed the JavaScript side of things as sort of a frosting on top of the server side logic.

For example, clicking “edit” on a task fired an Ajax.Request to the edit link’s href (e.g. /t/1234), whose server-side counterpart supplied the markup for editing that task. It was really nice, because I had less state to manage and didn’t have to know anything about the structure of the tasks table. Similarly, script.aculo.us’s Ajax.Autocompleter initially loaded autocomplete suggestions from the server side by requesting something like /projects/?q=DN…, and the script just displayed the suggestions once it got them. The App Engine datastore did most of the work for me, and since the suggestions were updated whenever tasks were saved, I didn’t have to worry about suggestions becoming out of date.

The frosting technique provided three primary advantages:

Less client-side logic. When you write your app this way, the client side is pretty dumb. My edit link just replaced something with an Ajax call and the autocompleter just asked somebody else for suggestions. Totally straightforward and less prone to cross-browser quirks.
Cleaner initial markup. In the case of the edit button, I didn’t have to include each task’s inline-editing markup (e.g. input fields, submit button, etc.), so the initial page size was smaller. There were also fewer hidden elements sitting around and nothing output as JSON for client-side consumption — there was no need for it.
Very DRY. Using Ajax.Autocompleter for autocomplete meant the JavaScript side of things knew nothing of how to generate suggestions or how to keep a list of things to suggest. Moving this all to the client side with InstantAutocompleter meant I had to duplicate some of that functionality: I had to both update the list of things to suggest on the client side and permanently store that list of suggestions on the server side.

Unfortunately, the typical workflow for a user (read: me) ended up looking like this:

Try to do something (e.g. click “edit”, type in part of a project name, etc.).
Wait for an ajaxy spinner to stop spinning. Tap foot during a brief pause until the display finally changes.
Complete the action.

Hitting the “edit” link and then waiting about a second for the request to complete was frustrating. Waiting for a list of suggestions to come back from an Ajax call reduced its utility to a point where I just didn’t use the autocomplete at all. And, similarly, waiting for an entire page load just to reverse your sort order just sucked.

Revised plan: JavaScript provides core functionality

So, I have a new strategy: Make things happen instantly in your web app if you can, even if you have to repeat yourself in JavaScript and even if you have to work a little harder on the client side. Things need to happen immediately, or the app will feel slow.

I think the main issue is that I originally thought of DNZO as a web site rather than a web application. I didn’t realize that I valued the app’s responsiveness over its code zen factor — especially when the web app purports to help people get things, er, done-zo.

Load Google Analytics asynchronously

taylor@taylor-hughes.com (Taylor Hughes) — Mon, 30 Mar 2009 01:06:48 +0000

So at some point last week I realized that Google Analytics was slowing down DNZO‘s page loads. I had just been using the default Google Analytics loader code, which spits out its <script> blocks inline. That behavior causes the browser to load the tracking stuff before continuing with anything else, and that’s not cool.

After some searching I found a blog post at Ajaxian that talks about the issue. I didn’t immediately find a Prototype-based solution for Analytics in particular, however.

So I wrote a bit of Prototypey JS to write the elements out once the page is loaded, and I’ve noticed a dramatic improvement in page load time.

Here’s the code; it’s pretty straightforward:

<script type="text/javascript">
function _checkGAT() {
  if (typeof _gat == 'undefined') { 
    setTimeout(_checkGAT, 200); 
  } else {
    try {
      var pageTracker = _gat._getTracker('UA-123456-7');
      pageTracker._trackPageview();
    } catch(err) {}
  }
}
Event.observe(window, 'load', function(e) {
  $$('head').first().appendChild(new Element('script', { 
      src: ("https:" == document.location.protocol) ? 
            "https://ssl." : "http://www." +
            "google-analytics.com/ga.js"
     }));
  _checkGAT();
});
</script>

Hopefully someone finds this useful. ;)

Getting things DNZO

taylor@taylor-hughes.com (Taylor Hughes) — Fri, 6 Mar 2009 05:08:59 +0000

Some time last summer, I decided I needed a to-do list. I had too much to keep track of, and things were starting to get messy. But Ta-da List is too simple, Remember the Milk is too complicated, and desktop to-do list apps like TaskPaper and OmniFocus don’t sync to multiple computers, which is a deal breaker for me.

So, a Google App Engine hackathon followed by approximately six months of a few hours here and there and voilà: DNZO (“done-zo”) is born. Sign up with your Google Account today!

Features

Quickly add new tasks with a responsive interface focused on making it easy to get started. Once you’ve added some tasks, quickly edit, complete, delete or re-order them at will.
Sort or filter your tasks based on projects, contexts and due dates. Click a project, context or due date to see all the matching tasks — and sort on any columns you like. Don’t use one column or another? Hide it.
Add multiple lists to better organize disparate tasks. Keep “Groceries” here and “Work Stuff” there, or put them together in the same list under different projects — we’re flexible.
Track what you’ve done with the archived tasks interface. See every task you’ve completed today, this week, last week or last month, and give yourself a pat on the back when you see everything you’ve accomplished.

Give DNZO a try — I look forward to any feedback, either by comment or by e-mail. Happy to-doing!

Introducing InstantAutocompleter

taylor@taylor-hughes.com (Taylor Hughes) — Thu, 12 Feb 2009 04:31:21 +0000

I spent some free time over a weekend recently writing my own Prototype-based JavaScript autocompleter. Holding out hope that someone else could find it useful, I’ve created a Google Code project for it — check it out!

InstantAutocompleter handles matching within a single keystroke event and doesn’t wait around to appear, so your choices come up immediately. Further, the first choice may or may not be selected by default, and you can also adjust matching patterns and token separators as regular expressions for autocompletion of items in a list.

The obvious choice for this sort of thing in the Prototype-and-script.aculo.us realm is Autocompleter.Local, a control that ships with script.aculo.us. But I found it clunky and occasionally intrusive: it takes a bit to show up, and it can get in the way if it falls behind and provides a match after you’re already done. InstantAutocompleter is meant as a replacement for Autocompleter.Local and should be even easier to use by both end-users and programmers alike.

(Some of the event-handling code takes vague cues from Autocompleter.Base, so thanks to the developers who put that package together.)

Give InstantAutocompleter a shot over at the InstantAutocompleter demo page, or visit the project page to find out more.

To App Engine or not to App Engine

taylor@taylor-hughes.com (Taylor Hughes) — Wed, 7 Jan 2009 05:07:30 +0000

The web development landscape has gotten a lot more interesting over the past several years. Web frameworks have been popping up like dandelions, new and interesting languages have become popular and even entire hosted web platforms have emerged.

With the advent of platforms like Google App Engine (and hey, AppJet, I’m looking at you), starting your next project is not just about deciding on a language anymore. Instead, it’s about weighing trade-offs between entire platforms: from the language and available frameworks to the way the app is deployed and scaled using them.

In deciding what to use for my next project, I’m weighing the benefits and costs of using App Engine — and it’s no straightforward task.

The hard parts

Datastore design — What’s that line about premature optimization, again? Well, with Google App Engine, you’re basically confronted with the task of designing a scalable data layout from the very early stages of developing your application. Datastore queries are restricted in ways such that you simply can’t not think about the effect they’ll have as you scale to millions of users — which is obviously not a pressing concern for most endeavors.
Application monitoring — Performing queries that would be simple in a standard RDBMS are not straightforward in App Engine, so figuring out even how many users you have or how many records of a certain type exist in your datastore is not easy. (To do it, you’ll need to actually keep track of those counts somewhere as the data is coming in or perform some very heavy calculations on demand.) Furthermore, there’s no interface to the actual hardware or software running your application besides the sparse App Engine administration console — so more interesting statistics about performance and deeper analysis into your traffic are hard to come by.
Limited framework access — Django ships with App Engine and can be easily included in any project (an old version of it, but you can upgrade it yourself manually if you like). But as you’ll quickly discover, almost none of the amazing features of Django are enabled for App Engine. A lot of the easy administration interfaces provided by the framework are impossible to use with the aforementioned datastore restrictions, and as such, you’re on your own.
Quotas — Quotas are something I haven’t actually run into with App Engine, but they’re a scary upper bound when you’re building what’s sure to be the Next Big Thing. Some of them would easily be reached with a few hundred users: An app that sends out e-mail notifications frequently could easily hit a paltry 2000 e-mails a day, for example, with no way yet to buy more quota — though we hear it’s coming soon.

The reward

Wonderfully easy development setup — The App Engine development server is wonderful. As I noted in my previous post, there are some inconsistencies with the production environment, but you really can’t beat the way it all works. The OS X client is really spectacular in particular. While designing and coding an App Engine app isn’t always the easiest task, the development server and mock datastore are just great.
Wonderfully easy deployment — Deploying new versions of an app to App Engine is great. With some tips from a Google IO talk called Building Quality Apps on App Engine, I setup a deployment script to safely deploy new versions of the code by incrementing the application’s version number to match my Subversion revision number, which is further utilized in the app for smart caching of minified JavaScript and CSS — and it’s a breeze. I can deploy a new version, check it out to make sure it works as planned and turn it on as the default version quickly and easily. If something blows up, fix and repeat or revert. Fantastic.
Built-in scaling — It’s a disadvantage because you have to think about it early on, but it’s an advantage because once you do, you theoretically don’t have to again. The hardware and software automatically grow your app across Google’s infrastructure, and as long as you’ve structured your data properly and had efficiency in mind you should be set as traffic grows. You never have to think about moving to a distributed database infrastructure after the fact or about how to go from a single server or set of servers to a server farm. Twitter on App Engine probably would have just… worked.
No network administrator needed — In a similar vein, system configuration, maintenance, backup and the like are all taken care of with App Engine for free. You never have to worry about keeping your version of PHP up to date or migrating to a new version of PostgreSQL or upgrading your hard drives.

Non-conclusion

There are a lot of costs and a lot of benefits to using App Engine. If you’re planning on building a big app with a lot of users, it could be worth it to put the extra time in to build it out with Google — but it’s not an easy decision by any means.

When not to use App Engine

taylor@taylor-hughes.com (Taylor Hughes) — Thu, 4 Sep 2008 23:47:44 +0000

A recent post by Brad Feld reminded me to write this.

When Google App Engine was released in April, I was instantly intrigued by what amounts to a potentially huge new platform for building web apps. With App Engine, you can write a web app with access to a BigTable datastore, hosted for free on Google hardware, with the ability to scale to millions of users. For now, your application must be written in Python, but you can leverage most of Django to make the web application development process less hostile.

This sounds like the perfect medium to build just about anything: possibly a little more work than throwing together a Rails app, but no database or server maintenance, no hosting costs, and Google-scale scalability for free. But alas: Google App Engine is not ideal for quick, small-scale projects — for a variety of reasons.

When I set out to build my new blog using App Engine, I figured I would have a few tasks ahead of me:

Learn enough Python and Django to get me going.
Design new templates (“views” in Railspeak) that incorporate what I want in the new blog.
Build the models (provided by the App Engine API) and controllers (“views” in Djangospeak) to serve up the content.
Import my existing data: 140-some blog entries and about 1200 comments stored in a MySQL database on my old host.

At first glance, #2 doesn’t seem like anything new, #1 and #3 sound like a moderate challenge, and #4 should be easy given #1 and #3, besides transportation and translation of the existing data itself.

As it turned out, however, existing data import and template rendering were the two biggest issues I had in building my new blog. Unfortunately for me, Google App Engine is built for quick, efficient requests that can scale to millions of users, not clunky requests that require lots of rendering.

App Engine requires performance tuning

Toss together a fairly complicated blog with threaded comments and pretty text formatting, and you’ve got some serious processing that typically needs to be done on every request: Certain data needs to be retrieved from the datastore, templates need to be parsed and included (recursively in the case of threaded comments), text of all kinds needs to be translated and formatted in many ways with a handful of external libraries, and finally all of it needs to be cobbled together and sent back to the user.

For the typical small-to-midsize project, nobody cares that all this needs to happen on every request because that sort of optimization takes time and care.

As it stands, however, App Engine considers every request for any page of my blog a “high CPU” request — usually 2-3 times above the “high CPU threshold.” Some unknown-but-small quantity of “high CPU” requests is tolerated by GAE, but I get a warning log message on just about every page load.

Here’s the worst part: I didn’t hear anything about my requests being “high CPU” requests until I deployed the application and started importing the existing data — after developing in the local App Engine development environment for something like a month and a half.

App Engine apps can’t have background processes

What this meant for me is that I couldn’t simply hack together a crappy, one-time-use script to import my existing blog data. Instead, I had to write a request handler that could import part of my existing data, and then I had to make a ton of quick requests in rapid succession to get all the data loaded. If I tried to import it all at once, I (a) ran out of memory or (b) ran out of execution time, both of which did not happen in the development environment.

What’s more, the regular updates of my Flickr photo stream and Last.fm statistics also have to happen by way of actual Web requests, instead of having the ability to set up some sort of cron for background maintenance tasks.

And that sucks.

The App Engine API is still fresh

As I mentioned under the previous two headings, one of the most annoying parts about the process of getting my App Engine-powered blog up and running was discovering, at the last minute, new development constraints upon deployment. These surprises highlighted the fact that, while it is pretty awesome, the App Engine development server does not mirror the production environment in some really important ways:

The development server does not have request timeouts or request memory limits.
The development server does not know what makes a “high CPU” request.
The development datastore is extraordinarily slow: so slow that I spent quite a bit of time optimizing my application to use more memory and reduce database queries — which ended up being a bad move because of memory constraints in production.

What’s more, I ran into a few very frustrating bugs in the deployment process:

An issue with appcfg.py reported a completely opaque error message when attempting to update a malformed YAML configuration file. This issue did not affect the development server, which happily parsed and utilized the YAML locally.
An issue with administrative user account management meant that a new version of my blog needed to be uploaded before I could successfully login to the application as an administrator.

The App Engine documentation is sparse

There, I said it.

The important stuff is covered by App Engine API documentation, but only the most basic use cases for the APIs are elaborated on. There just isn’t a whole lot of meat in the docs.

I found myself constantly asking, “Well, what if I was doing X instead of Y?”, where Y was the most basic example and X was slightly more contrived: “But what if I’m building a tree of nested comments and I want them all to load in one datastore call?” Especially for something that ended up being so picky about optimization, you’d think there would be a focus on that sort of thing in the docs.

As an added blow, my experience searching for just about any problem I was having yielded few-to-no results: Looking for help using Django? Sure. Using App Engine? Sure. Using Django and App Engine? Hmmm… Using Django and App Engine and only experiencing the error when you try to deploy and can’t log in? Good luck.

Finally

In closing, App Engine isn’t for low-volume projects unless you’re a glutton for punishment. It’s just not a platform designed or optimized for rapid development — it’s a platform for lightweight multiuser apps that you intend to scale to thousands or millions of users.

And for that, it sounds killer.

Two years with Last.fm

taylor@taylor-hughes.com (Taylor Hughes) — Thu, 14 Aug 2008 17:34:28 +0000

I joined Last.fm in July, 2006, and have been “scrobbling” at least some portion of what I listen to to the service ever since. What this means is that every time I play a song in iTunes, Last.fm records the play and generates statistics for my listening history over time.

For awhile now I’ve been meaning to take a look at the progression of music I’ve found since I joined the service. I ended up writing a quick Ruby script to download my weekly top artists from the service since I joined and calculate the first time I (theoretically) ever listened to them.

I’ve distilled the raw results down to what I think is an accurate portrayal of my new artist discovery over the past two years.

I say “distilled” because the original results had a lot of music I had already listened to before signing up for Last.fm — Last.fm just didn’t know about it beforehand. (The stats also do not include plays from my iPod or car, or, before 2007, the plays from my home computer; so these are slightly inaccurate.)

So, here’s the list. It’s formatted as such: Artist Name → the number of songs I’ve listened to by the artist, according to Last.fm, over the past two years.

The journey begins:

2006

July-August

Belle and Sebastian → 277 plays
Sufjan Stevens → 1584 plays
The New Pornographers → 128 plays
OK Go → 302 plays
The Roots → 78 plays
Editors → 162 plays

Belle and Sebastian, OK Go and the New Pornographers were all recommendations from fellow Heralders at the time; Sufjan Stevens and the Roots were some sort of personal experiment (I think); and the Editors were a recommendation from John Zeratsky.

September-December

Nada Surf → 142 plays
Lady Sovereign → 60 plays
of Montreal → 422 plays
Ray LaMontagne → 153 plays
Junior Boys → 262 plays
The Sea and Cake → 118 plays

Nada Surf was a recommendation from Ryan Swan; Lady Sovereign, Junior Boys and the Sea and Cake were all Jeff Greene’s; and of Montreal and Ray LaMontagne were inspired by Molly’s visit to my apartment around November.

2007

March-April

Headlights → 134 plays
Mika → 13 plays recorded, more in reality
Elton John → 59 plays
Grizzly Bear → 233 plays
Blonde Redhead → 163 plays

Elton John was a tip from my Finance professor; Headlights came from Jeff G.; Mika and Blonde Redhead were the result of two different newspaper reviews; and Grizzly Bear came from Nate and Elyssa.

July-August

Weekly → 52 plays
Air → 42 plays
The Be Good Tanyas → 93 plays

I’d been meaning to give Air a try for some time; Weekly was a John Z. recommendation; and the Be Good Tonyas came from Jeff G.

September-December

The Beatles → 1155 plays
Feist → 351 plays
Steely Dan → 364 plays
Menomena → 246 plays
Interpol → 513 plays
Kings of Convenience → 127 plays
The Knife → 615 plays
Chromeo → 140 plays

I’d also been meaning to give the Beatles a spin, and it really worked out (as you can see by their enormous play count). Feist and Menomena came from John Z.; Steely Dan from Nate; Interpol from a binge drinking session at the Irish Pub; and Kings of Convenience, the Knife (awesome!) and Chromeo came from my sister.

2008

January-March

Aphrodite → 73 plays
Lily Allen → 74 plays
José González → 203 plays
Vampire Weekend → 43 plays
The Rapture → 57 plays
The Bird and the Bee → 165 plays
Bon Iver → 228 plays
The Helio Sequence → 593 plays
Joni Mitchell → 250 plays
James → 143 plays

Aphrodite and Lily Allen were recommendations from Molly; José González was a result of listening to his cover of “Heartbeats”, originally by the Knife; Vampire Weekend came from Matt Dolbey; The Rapture and the Bird and the Bee came from Joselyn; Bon Iver and the Helio Sequence came from two online reviews at Killer Park Job; Joni Mitchell and James were recommendations from Joel.

April-Present

Yael Naim → 180 plays
The Good, The Bad and the Queen → 11 plays
Gotye → 116 plays
The Hood Internet → 188 plays
Paper Route → 57 plays
Neil Young → 136 plays
Viva Voce → 114 plays
Grand Archives → 42 plays
Annuals → 122 plays

Yael Naim came from the television and Apple, Inc.; the Good, the Bad and the Queen and Paper Route were from Joselyn; Gotye and the Hood Internet were from John Z.; Neil Young was a result of random coincidence; and, finally, Viva Voce, Grand Archives and Annuals were all recommendations from Last.fm itself.

Whew.

So, there you have it.

Two years of finding new music to enjoy.

What’s next?