Anatomy of a Webapp

Screenshots!

2007-05-17T23:58:00.001-07:00

Hey,

I realize I have disappeared from the blog for a long time, but I swear I have been hard at work on the web app itself. After getting the basic functionality done (adding links, searching, etc.), I have spent way too much time on the look and feel of the interface.

On the one hand, I've learned a lot about ajax and dynamic web design. On the other I might suffer a stroke if I find one more bug in Firefox or IE.

At this point I will switch over to working on the rating system. For R2 to work, users have to be able to weed out bad links, and approve of useful ones. We also need to add a way for users to flag spam.

-Ben

Firefox Extension Progress

2007-05-16T23:47:00.000-07:00

So I've begun the Firefox extension. Writing an extension in XUL and JS for Firefox is likely one of the least pleasant programming experiences I've had. My main problem with it is the lack of decent tools for debugging. As far as I can tell, unless you have a glaring syntax error somewhere in your code, you never hear about any problems from the browser. Rather, your JS thread just exceptions out silently and stops execution.

Despite these difficulties, I think we've beaten down the hard part of the plugin. You can now pull up a sidebar with a keyboard shortcut and it will notice what track what page your eyeballs are on. All that remains now is to yank data with the API and show it in a snazzy way. As proof that the extension is somewhat functional, enjoy this screenshot.

If anyone has any experience in building extensions like this please drop me a line and give me some hints :) I'm spending more time looking at example code than I am writing my own, but more information will help. Also, Ben should post sometime soon with a screenshot of how he's prettied up the web interface.

And back to our regular schedule

2007-05-05T22:21:00.000-07:00

Sorry for that long interlude. I'll chalk most of it up to The Coachella Valley Music Festival which took up five days of our lives, and I've definitely spent the week recovering as well. In case you were wondering the show rocked. I'm not sure if I'll ever get over how amazing Ratatat was.

Anyway, back to R2. Things are actually on schedule! There is an interface, though it isn't exactly pretty. I just spent an hour and a half refactoring our django code so it is quite pretty and will be nicer to work with. I learned from this post that you can separate your views into separate files for ease. I did the same with our URLs and I'm contemplating doing the same to our models. We now keep primary, API, and Ajax code in totally separate files.

Speaking of Ajax, I also rewrote all of the JavaScript in the site using the Prototype library. That library is awesome. JavaScript is downright incomplete without it. I'm pretty new to real JS coding so if you have any tips for me leave a comment. I can generally get by with it but I have no eye for front-end work in general.

We've also had two meetings with our TA, both of which went very well. We were way ahead of schedule for the first week so things worked out in our second meeting even though we spent 5 of the 7 interleaving days in SoCal. The group that meets before us is developing an XBox 360 game, so when we get there they are demo'ing on an XBox while I just have my MacBook. It's a little intimidating.

We have big plans for this week, and I'll probably spend the rest of the night and some of Sunday on the project. Here's what we hope to have done this week:

API 100% ready to ship.
Some fancy Ajax to enhance the UI (we have type-inference, we need auto-title-getting and some others)
Implement the user system
Get the browser plugin to hello world state

It should be pretty doable, but I think I have a few midterms this week so we'll see.

Oh, and of course, enjoy some screenshots of the testing UI at 1140 lines of python, 279 lines of template/html, and a few dozen lines of CSS and JS.

Development Roadmap

2007-04-21T16:57:00.000-07:00

Part of the project proposal was to write a roadmap for development, and here it is.

April 20, 2007 - database backend, simple ui
April 27, 2007 - define api, improve adding ui
May 4, 2007 - implement api, web adding ui done
May 11, 2007 - begin browsing ui, plugin
May 18, 2007 - finish simple browsing, add through plugin
May 27, 2007 - begin graphical browsing, have plugin lookup data
June 3, 2007 - finish browsing, polish site, start ui for plugin
June 10, 2007 - polish plugin, publish site, add seed relations, data

I feel I could get this done a lot faster if I weren't taking 5 other classes, but oh well. It seems pretty reasonable.

Still alive

2007-04-16T11:29:00.000-07:00

Hey folks, just wanted to let you know that we're still alive and working on R2. Other school-related things kept me busy this weekend. Tomorrow I give a presentation on R2 to the rest of the Software Project class. I'll let you know how it goes.

If anyone is perchance going to be at the Netvibes party in SF tonight shoot me an email. I'll be there.

Announcing Relates2

2007-04-12T20:10:00.000-07:00

The project shall be called Relates2! We call it R2 for short.

I turned in the first draft of my project proposal yesterday. I'll post a PDF of the final here after I meet with a tutor and revise it and whatnot. The whole thing is 13 pages long and definitely not worth reading because proposals are by nature repetitive, but here's the introduction and the feature set. I'll get into the gritty technical details this weekend when I actually start coding (finally).

--
Summary
Relates2 is a web-based application where users can declare a relationship between any two web addresses (URIs). URIs can be associated by any relationship in R2's database, which can be amended and added to by users under guidance by administrators. Once the relationship between two sites is known, linked content can either be embedded throughout the web through our API or can be viewed using our Firefox plugin. In the end users enjoy a richer browsing experience that is augmented by relevant largely user-generated content.

Feature Specification
R2 is made up of several interlocking but conceptually distinct components: the backend, the web interface, the API interface, and the browser plugin.

Database/Backend
This part of the product is invisible to the user and thus we are free to implement it any way we see fit. All it really has to do is be reliable and store the data required to run the rest of the site: every relation we recognize, every type of resource we recognize, every pair of related URIs, users, rules, and various statistics.

Web Interface
This is the first user-facing part of the product. Through it, users can:

Link two sites. Upon entering the URIs of the two sites, R2 tries to infer the types of the URIs and suggests them to the user. The interface also suggests a relationship between the two resources if it knows their types. The user is free to override either. Once entered, the link is stored in the database and available for querying.

View related content. The user can enter a URI and then view a list of all content related to that URI. The related content interface will at the very least allow the user to sort by type of relation and popularity of the link. When content links correspond to media the media will be previewed inline. The web interface will also allow a user to agree or disagree with a link. This interface is the part of the project most open to interpretation. Though it is not of the highest priority we believe that there is a lot of innovation that could go into this part of the site, including graphical and interactive methods of viewing relationships.

View new content. The user can view a list of all the recently linked resources. Sorting can be done at least by popularity and relationship.

API
The API allows developers to do anything that can be done through the web interface, only through XML and without any presentation necessary. There will also be API calls to return pre-formatted information for easy embedding on the web. The API will feature useful presentation-minded features that prepare content for embedding, i.e. a method such as getThumbnail that knows how to extract a thumbnail from common sources such as Flickr and YouTube. To use the API developers will need to have an API key in much the same way that the Flickr and Facebook APIs require.

Browser Plugin
The last component of R2 will be a browser plugin targetted at Firefox users. It will normally run as an icon in the corner of the browser window, out of the user's view until the user wishes to use it. When the user views a site that has related content known to R2, the icon changes colors to signify that it has information for the user. If interested, the user can click the icon and a sidebar will appear. Related content will be listed as it is in the web interface, with media shown inline where possible and various sorting options available. The user can quickly add other content through the plugin as well.

TechCrunch covers web annotations

2007-04-12T12:06:00.000-07:00

Too bad TechCrunch didn't decide to write this article about web annotations a month and a half later. Otherwise, I would hope that we were on it because I think Artoo is in the same product space but does web annotations much better.

Annotating the web is definitely not a new idea, as the article points out. However, I think we have a better idea of what kinds of annotations users actually want to utilize now than we did before. We also know that users have something valuable to contribute to a website. I think that the solutions covered by TechCrunch don't quite have the right understanding still.

If you squint your eyes del.icio.us is nothing more than a web annotation tool. It lets users decorate a site with a few tags to describe it, although no one cares what these tags are once they get to a website. It's a tiny way to remix the web.

These tools go far beyond the functionality of del.icio.us, of course. The annotation is supposed to enhance the browsing experience once a user has reached a page. This is exactly the goal of Artoo.

I really think that the latest generation of annotation tools misses the point of what users want yet again. 'Web graffiti' is too harsh of a term to describe them, but not much too harsh. They focus on text and actual markups on the physical page. They expect that whoever has something worth saying about the page also runs across the page and has the plugin installed. They also expect that users viewing annotations on a page trust the block of text or inline image left by some other user. This is the crux of their shortcomings: they believe that annotations can stand on their own.

I fundamentally disagree that an annotation can stand on its own. Artoo abstracts the notion of an annotation by adding a level of indirection. Rather than letting a user's contribution atop a site stand alone, Artoo gives the annotation context and legitimacy. A web resource (anyone have a better name for this? I mean a video, a picture, a web page, just any piece of content.) can stand on its own, for it is used to doing that. Yelp.com reviews exist perfectly well far away from the things they review because Yelp is an entity on its own with its own sense of legitimacy. A random inline annotation on a webpage has no such luck; it is untrustworthy by default because this is the wild, wild web.

The simple addition of indirection makes Artoo innovative far beyond what anyone will call web graffiti. I wish TrailFire, Stickis, ShiftSpace, Fleck, and Diigo all the best, but their products are inherently limited when compared to Artoo.

We figured out a name

2007-04-11T11:05:00.000-07:00

Thanks to those who suggested names for our project. Last night whilst stuck in a diner for 2 hours my girlfriend and I figured out a name that not only makes some measure of sense but also came with an available domain name

However, the name is top secret for a bit. For now we will be referring to the project as Artoo.

About our PRODUCT

2007-04-09T01:13:00.000-07:00

__PRODUCT__ is what I'm calling our project in our proposal (due on Thursday, so soon!) since I cannot for the life of me come up with an interesting name that is available in domain form.

But I can tell you what it is we intend to build. Maybe readers can help us come up with a name. Critiques of the idea and features are also welcome, of course.

We want to connect pieces of content, largely the user-generated variety, to other pieces of content with some kind of human-understandable meaning. We want to see video someone shot at a protest when we read the coverage by the LA Times. We believe that when you look at the site for the Hilton San Francisco you should also be able to easily see its reviews on Yelp and photos on Flickr. The web is a limitless resource, but it is so huge that it can be unmanageable. All we have to guide us through it is the noble hyperlink, but sometimes we need something more.

So __PRODUCT__ is a webapp that allows users to associate two URIs together under some chosen relation. The URIs are typed so we can make sure that http://somephoto DEPICTS http://somethingelse, if that makes any sense. With our help users can always add more types and relations. The extensibility of the system is key.

Users will glue together two websites through the web interface to __PRODUCT__. Likely people who create the user-generated content that lives on the web will want to connect their work to related pieces of work. Either their photo depicts a place, their video shows the reality of a news story, or their blog post replies to another blog post. So much of what we create on the web is connected in really interesting ways, but these connections are either unknown or cost at least the time of a Google search to find.

Once __PRODUCT__ has information about these connections, so much that is interesting can be done with it. It can be browsed in a real graphical web on our website. You can formulate interesting, logical queries based on real relationships rather than the best guess of a complicated spidering algorithm. Through our API related content can be embedded throughout the web. Just imagine if your Flickr photos of a developing news story showed up right next to the text of a story on the New York Times' site. And through our Firefox extension your entire browsing experience could be enhanced by having related content available whenever a sidebar is open - Flickr and YouTube thumbnails, Yelp ratings, and any other kind of media that can be published, all a click away and in your browser.

It's entirely open-ended, we are just providing the tool that creates an opportunity. I think some of the most novel uses of this different form of tagging are way different than what I have thought of (so far I just have connecting old media with new and propagating tiny pieces of content). With its open API and the cleverness of a community I think that the sky is the limit for __PRODUCT__.

About Ben

2007-04-08T21:19:00.000-07:00

Hi,

I'm Ben. I'm a senior, graduating at the end of this quarter. I am originally from Minnesota, but I have transplanted to California. Like Jason, I program mostly in C/C++ and Python, though I have experience in Java, Javascript, etc. After school, I'm looking forward to taking some time off to travel in New Zealand. Afterwords I will settle down and get a real job in the Bay Area.

Personally, my interests are all over the place. I like poetry, national parks, religions (I'm a religious studies minor), politics, and dogs. My music tastes are also eclectic, but I reserve a special place in my heart for Daft Punk.

About Jason

2007-04-08T18:15:00.000-07:00

Hi again, this is Jason. I'll make Ben get on and post something later. I wanted to offer a bit about myself to kick off the blog.

So I'm actually a junior right now, but I'm graduating in the middle of next year so I'm doing my senior project this quarter. I am originally from Texas but now I live in the Bay Area. I started programming sometime in high school and I've been pretty intent on becoming a programmer ever since. I worked as a dev intern at meebo, which is awesome, last summer and I will be there again this summer. I code in C/C++ and Python primarily, but I've done a good deal of Java and LISP and can get by in pretty much any popular language if necessary. After I get out of this school I plan to take some time off and launch a company of my own.

Aside from the geek credentials, though, I am a person. I like post-punk and post-rock music, which sounds really pretentious but actually isn't that much so. I live in a co-op house called Kairos, as does Ben. I manage the kitchen, where I like to wash dishes and cook extravagant food. I TA an assortment of classes, this quarter being CS107 Programming Paradigms. I really like hobo jokes and every other Wednesday night I can be found holding a glass of the cheapest wine that doesn't come in a bag or a jug known to man.

Welcome to Anatomy of a Webapp

2007-04-08T16:29:00.000-07:00

Hi there. I am Jason, one of the writers of this blog and a developer of a yet-to-be-named webapp for the CS194 Software Project class at Stanford University. Ben, the other developer, and I will be writing about our experience building a webapp from the ground up.

This will be our 'senior project,' a project that every CS major at Stanford has to do to graduate. The class is 10 weeks long and by the end, on June 12, our project will be complete. The class actually stops meeting after this week and we don't meet again until the end of the quarter to demo what we have accomplished. Our only deliverables are a 12-15 page project proposal, due this Thursday, and the finished product. We will be meeting weekly with a TA for the class to discuss our progress and seek advice, but every bit of planning and every line of code is written by the two of us.

Expect posts to come every few days. We'll explain just what our project is about soon, as well as a bit about ourselves. And once we ramp up coding we'll tell you about features as they're implemented, show off screenshots, complain about school, and generally try to show as much of the experience of building a software project as possible.