ecmanaut

Death and IE

2011-10-08T09:42:00.000-07:00

This post is a brief celebration of succession on the web. Not Mosaic begat Netscape begat Mozilla begat Firefox succession, but Firefox 4 begat Firefox 5 begat Firefox 6 begat Firefox 7 and Chrome N begate Chrome N+1 type succession. Google got it right with Chrome last decade, and not long thereafter, Mozilla got it too this year.

Quoting that post's quote of Steve Jobs' 2005 Stanford commencement address (worth watching, if you haven't),

Death is very likely the single best invention of Life. It is Life’s change agent. It clears out the old to make way for the new. Right now the new is you, but someday not too long from now, you will gradually become the old and be cleared away.

This very strongly applies to web browsers, and as it turns out, the best thing a browser version can do (besides getting things right in the first place) is dying even more quickly than it came. I wonder if this enlightened insight might have sprung with Mozilla, if only Firefox could have kept its inaugural name, Phoenix, which it now fully embodies, but at least now, this gift has finally been given to web developers:

You need not bother writing applications (or perfecting layouts) to high-fidelity for old browsers, for their time is short and their better ancestors replace them quickly.

Assuming your deployment domain is web pages. As browser add-ons go, if you still maintain one of the old style XPI design your work burden has shot through the roof instead, unless you have near zero UI footprint, happen to only need and use APIs that time proves to remain stable and host it on addons.mozilla.org, where they bump its maxVersion every few weeks for you when it still seems to work. Failing either, you have to manually update and test it seven times per year. In Chrome, this has premeditatedly been addressed by not promiscuously offering any APIs it can't support in the long run, at the cost of limiting how much add-ons can do. Firefox, in this regard, remains the only browser where add-on authors can innovate 100% of the browser, and in this regard it has filled an important need in the browser eco-system (and still does). It is both its greatest feature and its greatest burden. Hopefully this will be mitigated to some extent with Jetpack and the new add-on builder.

Returning to the paramount topic of graceful, benevolent, rapid-evolution-supportive death, Microsoft's IE does not yet get it. Like a third world nation befallen by sudden prosperity, it has doubled its reproduction rate while keeping its mortality rate constant. As noted in mentioned posts, this does not bring just better browsers faster, it brings over-population, and makes web development unsustainable. Or, as Paul Irish put it in the first post: it pollutes the browser market.

In this respect, the conditional comments and many "backwards compatibility simulation" modes IE bring you are not mainly tools helping web developers make sites work on Internet Explorer, but a huge extra burden of work forced onto web developers, to cope with IE's broken release process. This is Microsoft's job, not ours, and we should be outraged with them for forcing it on us.

It is not the rotting corpses of IE6, IE7, IE8 or IE9 that all need to die to give space to IE10, which still smells fresh, it is a broken release process that needs to get addressed and brought up to date with best browser practices of the decade. It is the reinvention of sudden death, not just the gift of new life, that must come to Redmond, too. I applaud the IE team for making it their business to get up to speed with the web's evolution, but it's less the whats than the hows that are important to get right now. SVG is great, but sort out the release process before taking on, say, webGL. It can wait. Fixing the world's hard problems like over-population is harder than running after the latest ooo-shiny! - but the alternative is systemic collapse.

For a browser, it is better to live a great but short life and go out with a boom, than it is to burden its extended family with a never-ending old age in an insufferable early-set rigor mortis. However you feel about Steve Jobs he lived and died in this way, never holding back, never growing stale of mind nor action, and the world was better off for it.

Running an old rails 2.3.8 with rvm

2011-09-29T12:36:00.000-07:00

I was helping set up a local (legacy) rails 2.3.8 server on a macbook today, autonomous from the system ruby installation. This was a bit messy, as modern versions of rubygems conflict with the old rails 2.3.8 environment to the tune of:

Uninitialized constant ActiveSupport::Dependencies::Mutex (NameError)

...when you try to start the server. Here's the recipe I came up with:

Install rvm.

# Install ruby 1.8.7 and downgrade its rubygems to 1.5.3:
rvm install 1.8.7 && \
  rvm use 1.8.7 && \
  rvm gem install -v 1.4.2 rubygems-update && \
  rvm gem update --system 1.4.2 && \
  update_rubygems && \
  echo 'Okay.'

# Install all the gems you need, for instance:
rvm gem install -v 2.3.8 rails && \
  rvm gem install -v 3.0 haml && \
  rvm gem install -v 2.1 authlogic && \
  rvm gem install -v 1.0 dalli && \
  rvm gem install -v 0.2 omniauth && \
  rvm gem install -v 2.7.0 erubis && \
  rvm gem install -v 1.3.3 sqlite3 && \
  echo 'Gems installed!'

If needed, run rake db:setup in your rails tree to set up its databases.
Done! rails/script/server -p your_port is ready for action.

Optimized SVGs at gist.github.com

2011-07-05T23:46:00.000-07:00

Lately, I've been having a lot of fun hand optimizing SVG files for size, a bit like Sam Ruby does for his blog (it is highly instructive peeking at his collection, as I think I have mentioned before).

For me, SVG has something of the magical flair I first found in HTML in the nineties, back when it was the new bleeding edge New Thing, but I argue that it's even more fun than HTML was. The W3C SVG specs are not prohibitively difficult to read, and of course you have much greater graphical freedom than structured documents can afford you (duh!).

Like Sam, I try for something presentable in a kilobyte or less (uncompressed, though modern good browsers are as happy to render SVG:s delivered with content-encoding: gzip, of course, as long as they are otherwise correct and delivered with an image/svg or image/svg+xml content-type), and to never enforce a fix width in the svg tag itself – so they just beautifully grow to any size you want them to be, with no loss of quality.

Which is the other main beauty of SVG, besides being fly-weight, standardized, widely supported and still growing broader support in all the main-stream browsers. Over the last few years, the SVG front has been progressing happily and now is very practically useful already, for at least those of us that care most about Chrome, Firefox and Opera (I get the perception that Opera's often rather exceptional lead on the SVG support front is largely or even solely the work of Erik Dahlström, but I might exaggerate a bit).

Anyway, this weekend, I had fun turning the Laughing Man (from Ghost in the Shell: Stand Alone Complex) that @elmex vectorized at some point, into a 1000 byte version of my own, also featuring the gentle rotating text seen in the anime series (YouTube), via declarative animation (so there is no javascript involved here).

Edit: I initially missed an excellent opportunity here to plug Jeff Schiller's scour, which is an ideal first step when you start from an SVG source file. Be sure to run with -p something-large, as its defaults are being lossy about precision, which cuts needed decimals from some input files. With -p 99 you'll be on the safe side. Experiment with low single-digit numbers if you like (the current default – 5 – is often good), but make sure things are still looking right, or you may just ruin your file, rather than optimizing it for tiny foot-print. Broken images don't get extra credit for also being small!

The result is in this gist (if you install this user script, you can swap between *.svg text/plain source code image/svg rendition) or to the left, if your browser renders inline SVG:s properly in HTML content (I recommend the gist over view source, as Blogger inserts linebreak HTML tags unless I strip out all newlines first).

What surprised me, when I made this user script, is how far standards support has come in modern browsers: in order to inject the <svg> tag I create for the rendered version, I had to feed the source through DOMParser (as setting .innerHTML is lossy from treating the SVG content as text/html, not text/xml), and the few lines doing that, just magically worked in all of Chrome, Firefox 4 and Opera 11 (de-jQuery:fied, to make more sense outside of the script's context) with no special extra effort on my part:

// turn the raw SVG source string into an XML document:
svg = (new DOMParser).parseFromString(svg, 'text/xml');

// import it into an SVGSVGElement in this document:
svg = document.importNode(svg.documentElement, true);

// and insert that element somewhere in the document:
document.body.appendChild(svg);

To me, that's a rather clear sign SVG is ready for prime time now.

While github reports having no plans on serving *.svg gists with an image content-type (they don't want people using gists for image hosting, I guess, even though it's sad you can't easily preview without saving to disk or using my hack above) I still think the light-weight gist community oriented sharing is good for this kind of thing. Others happily forked the octocat SVG I similarly format converted a while ago from the github About page, and milligramme made this much spacier version.

I gather all my SVG play in a svg-cleanups repository on github, if anyone wants to get inspired the fork or follow way, and occasionally tweet about it. If you find this kind of exercise as much fun, I love hearing about it; here, on Twitter, github, or elsewhere. I believe it's good teaching and learning for the web as a whole, too. Any logos, trademarks and the like above are property of their respective owners.

Draw your own Github SVGs, step by step

2011-06-04T04:28:00.000-07:00

I SVG:ified and played a little further with the logo material from the recently published Github about page, and then tonight I figured it would be fun to visualize the elegant process by which a raw SVG image is built up, piece by piece, by rather basic building blocks. With just a little bit of javascript magic to help you, here is how you piece together your own github schwag from scratch (works like a charm in Chrome, Firefox 4, and presumably any other modern browser that can handle inline SVG images

head over to the blog post itself

If all you see is a button and the description of each step, that just means your browser doesn't natively handle inline SVG, which of course is a bit of a shame. Another reason I was curious to try this is for seeing how inline SVG:s fare in feed readers. (Google Reader seems to fail.)

Nicer source code for the images than in the page (Blogger insisted on filling all the whitespace with html junk) in these gists: github-logo.svg, octocat.svg. gist.github.com currently doesn't serve .svg files as image/svg, so you'll have to save them locally first to see them rendered.

Source code for the step-by-step drawing is in this gist; MIT licensed, if you want to fork away, adopt, adapt or whatnot. Have fun! I did. :-)

Add-on hosting: Mozilla vs Google vs Opera

2011-05-29T23:18:00.000-07:00

I habitually develop browser user scripts to stream-line things for myself (and others) on the web – and a half random scattering of them tends to end up proper add-ons, when I think the benefit of them being easy to find, install and reuse by others merits the added work for myself of packaging them and submitting them to an add-ons gallery.

This happens rarely enough for me to forget some details of the process (yet often enough to be annoying), hence this post to document salient parts of it, applaud parts of where hosts worked things out really well, and note where there are holes to patch up. (I don't cover Safari since I have not made any Safari add-ons.)

(screenshot taken here)

Firefox add-ons: addons.mozilla.org, a k a AMO

Your add-ons are listed here: addons.mozilla.org/en-US/developers/addons
Add-on URL: addons.mozilla.org/en-US/firefox/addon/your-configurable-addon-slug
Public data: current version, last update time, compatible browser versions, website and, optionally: all add-on detail metrics, if the developer wants to share them (excellent!)
Public metrics: total download count, average rating, number of ratings
Detail metrics: TONS: mainly installation rate and active installed user base over time, broken down by all sorts of interesting properties or in aggregate, graphed and downloadable in csv format. Notable omission: add-on page traffic stats. Public example: Greasemonkey stats
Developer page linked from the public add-on page when you're logged in: NO
Release process: Manual review process that can take a really long time, as AMO often is under-staffed, and hasn't successfully incentivized developer participation in the process to an extent as to make it not so.

Summary: great stats and an ambition to make information public and transparent.

(screenshot taken here)

Chrome add-ons: chrome.google.com/webstore, or the Chrome Web Store
Previously lived at chrome.google.com/extensions, the Chrome extensions gallery

Your add-ons are listed here: chrome.google.com/webstore/developer/dashboard
Add-on URL: chrome.google.com/webstore/detail/your-addon-signature[/optional-add-on-slug]
Public data: current version, last update time, if installed: a checkmark and the sign "Installed", instead of the "Install" button (excellent!)
Public metrics: total download count, average rating, number of ratings
Detail metrics: If you jump through the relatively tiny hoop of creating and tying a new Google Analytics account to your add-on, you get detailed add-on page traffic stats, over in Google Analytics. Notabe omission: all metrics about your installed user base :-/
Developer page linked from the public add-on page when you're logged in: NO
Release process: Automated review process, making yourself the only release blocker, in practice. This is bloody awesome!

In summary: A really delightful release process that doesn't waste your time. The metrics part is really disappointingly missing the most interesting data; I hope that will improve with time. There are other essential missing docs too: to find out that you can self host your add-on AND publish it in the chrome web store, with the same add-on id, you need excellent Google fu, or this secret knowledge:

first (before your initial web store submission) build your add-on locally
store the .pem file in some good location
rename it key.pem
copy that .pem file into the add-on directory
zip up that directory (so the top level of the zip file only has one directory)
NOW (and for all later versions, IIRC) upload this zip file to the Chrome Web Store
…and it will use your add-on signature, instead of that of Google's secret .pem

(slightly de-horizontalized screenshot taken here)

Opera add-ons: addons.opera.com/addons/extensions

Your add-ons are listed here: addons.opera.com/developer (this page is near impossible to find by navigation, and was the reason I created this blog post :-)
Add-on public URL: addons.opera.com/addons/extensions/details/addon-slug[/version (redirects visitors here)]
Add-on developer URL: addons.opera.com/developer/extensions/details/addon-slug/version (no redirect help, but links to all other versions)]
Public data: current version, last update time, add-on size (excellent!)
Public metrics: total download count, average rating, number of ratings
Detail metrics: NONE. Notabe omission: all data about your installed user base, AND add-on page traffic :-(
Developer page linked from the public add-on page when you're logged in: NO
Release process: Manual review process, rewarding the behaviour of publishing as little data about your add-on as possible, since each bit of data is a plausible blocker. (Example: it is a bad thing to link to your development site on github, unless it is a repository that only has Opera-addon-centric stuff in it, and not, say, covers the source code used to build add-ons all browsers it caters.)

Summary: young; still a far cry from easy to find your way around. I accidentally managed to log in via my email address (after having reset the password), which had gotten the same password as the account nickname I had published from before, but was otherwise treated as another account, with its own (empty) list of add-ons. So I first found no way to upgrade my existing add-on and just uploaded a new one. When I found the mistake, there was no way to abort the review process I had started (but I could edit it to add some reviewer notes for the probably equally confused reviewer). What probably looks like a really well integrated set of connected Opera sites to an Opera employee is a huge maze of web pages, none related to publishing your add-on, to an add-on developer.

All of these hosting sites have developer log-in of some sort, yet fail to link the developer's admin view from the public add-on page. For Opera, this hurts really badly, as there is so much Opera noise around, which does not try to help you publish your add-on; for the others, it's a smaller nuisance (it's also entirely possible that I learned where to find the "Developer Hub" and "Developer Dashboard" links long ago enough to now find them near intuitive).

Update: I just found Opera's dashboard link! Your logged-in name is the link that takes you to your dashboard of add-ons. And it only took knowing the wanted url, opening a web console and typing [].slice.call(document.links).filter(function(a){ return a.href == "https://addons.opera.com/developer/"; }) in it. What do you know? :-)

Github Improved 1.7

2011-05-28T02:47:00.000-07:00

After a whole year in development as a user script (initial mention, formerly known as "Github: unfold commit history"), a brief test-drive as an Opera extension at the 2010 add-on con, I finished up the missing bits and pieces to make today's Github Improved! chrome extension available in the Chrome Gallery / Web Store (1.7 now also available in the Opera extension gallery).

New since last release is the little tag delta (Δ) links that let you instantly see what happened between, say, Greasemonkey 0.9.5 and the previous release tag (it recognizes anything where the only thing differing between tags are their numeric parts, which also means it's not going to handle fancy 1.6rc1 type tag names littered with semantic non-numeric parts):

And, as prior users may find even more importantly, that it doesn't hog any shifted keyboard shortcuts, which for some reason had the side effect of making arrow up pull in a pageful of diffs.

Full documentation of all features, and a few screenshots, is available on the Chrome Web Store page. I also take every opportunity to mention that it really shines best together with the AutoPatchWork extension, or something equivalent which unfolds the commit history pagination as you scroll off page. Enjoy!

Google Closure Library bookmarklet

2011-05-21T20:33:00.000-07:00

Any self-respecting javascript library should have a debug bookmarklet that lets you load it into a page, so you can tinker with it (the library, or the page, for that matter) without any messy overhead like downloading the most recent version, building it with the specific flags and sub-components you want, saving the current page, adding library setup code to it, and reloading that page before you can do squat. I found Google Closure Library a bit lacking in that regard, so here's my take on it:

It has two modes of operation; if all you want to do is load the goog object into the current page (or overwrite the one you already have with one that has a different set of goog.require()s), just customize which those requires should be, and you're set; it creates an iframe, in which it loads the library (goog.require only works during page parsing time, as it uses document.write() to load its dependencies), and then overwrites the top window's goog identifier.

The second mode is good for writing your own bookmarklets making use of some Closure Library tools; provide your own function, and it will instead get called with the freshly loaded goog object, once it's ready. At the moment, I have only played with this in Google Chrome, but feel free to fork away on github, if you tinker in additional improvements for other browsers.

Finally, here's a bookmarklet version to save or play with, which will prompt you for just which closure dependencies you want to load: closure – drag it to your bookmarks toolbar or equivalent if you want to keep it around. Enjoy!

A "No News is Good News" Tsunami Feed

2011-05-21T12:15:00.000-07:00

I have subscribed to the NOAA/NWS/West Coast and Alaska Tsunami Warning Center's feed of ahead-of-time tsunami warnings for the U.S. West Coast, Alaska, and British Columbia coastal regions (example information statement) for some time. Most, like this one, are "wolf-cry" statements stemming from some seismic activity somewhere that won't generate any tsunami, marked with the all-important body phrase "The magnitude is such that a tsunami WILL NOT be generated" - meaning I don't really care about them.

Today I made a Yahoo pipe, filtering away those from the feed. Here is the resulting feed, containing only positive tsunami warning statements. Behind the scenes, it's created by the simple YQL statement

select * from rss where url="http://wcatwc.arh.noaa.gov/rss/tsunamirss.xml" and description  like "%tsunami <strong>WILL NOT</strong> be%"

(there's some newline or similar after "be" in the original feed, and it doesn't seem like YQL does any whitespace normalization, but this matches well enough to filter the stuff).

The resulting feed is my enrapturing contribution to today's supposed end of the world. You may note that the feed is currently (and hopefully still, by the time you read this :-) empty - which is, of course, good news. Enjoy!

On a related hacker's note, it would be really handy to extend Google Reader with a "filter this feed" feature that created these more on the fly, without mucking about for an hour in the Yahoo Pipes interface, and then changed the old subscription to the new feed.

Chrome + NaCl + libmodplug + ... = tracker modules on the web

2011-05-16T00:59:00.000-07:00

Some notes I took while looking into how to build a libmikmod_x86_32.nexe and libmikmod_x86_64.nexe to get Chrome tracker module playing support from javascript (the work seems to be done already in naclports) via NativeClient, for all the MOD, S3M, XM, IT, 669, AMF, AMS, DBM, DMF, DSM, FAR, MDL, MED, MTM, OKT, PTM, STM, ULT, UMX, MT2 and PSM formats:

In a Chrome 11 profile, go to about:flags, and click Enable under "Native Client", and then the "Relaunch Now" button at the bottom (alternatively: start the session with the --enable-nacl command-line flag)
Download the NaCl SDK (for Chrome 11, in my case; eventually the ABI will supposedly freeze and cover a wider range of versions)
Download depot_tools, extract it and export NACL_SDK_ROOT=$PWD in that directory (technical overview here)
Check out naclports
Probably optional: comment out all the RunInstallScript lines in naclports/src/packages/nacl-install-all-bitsize.sh except the one you're interested in (in my case: RunInstallScript libmodplug-0.8.7 nacl-libmodplug-0.8.7.sh)
Run nacl-install-all-bitsize.sh 64 (or 32 for a 32-bit build), which, in my case, built $NACL_SDK_ROOT/toolchain/mac_x86/nacl64/usr/lib/libmodplug.a
About here my research petered out, as I found this smaller hack for XM-only playback by some forthcoming Japanese fellow (git repository here; English translation of same page c/o Google Translate), which has basic functionality
If you got curious about my original venue and proceeded further, I'd love to hear about it, especially if you managed to build the final nexe:s; do post a comment!

Github tag and branch labels

2011-05-04T02:03:00.000-07:00

I just made a little update to my github improved! user script; now it shows you branch and tag labels in the commits view, like this:

There's been some more mystery meat features slipping in there too somewhat unannounced; if you click a committer icon a little filter panel opens on top that lets you see how many commits in the view were by whom, and if you click one of those, hide those commits. I got the idea when I was playing with Autopatchwork at some point, unpaginating a whole repository's worth of commits and wanted to slice and dice the view a bit, get aggregate stats and the like. This is what Greasemonkey's early history (from 0.8 and back) looks like, in terms of authors (not committers) involved, for example:

Autopatchwork could use some more coverage on the English-speaking web, by the way, because of its neat way of aggregating user data from contributors. Instead of a dedicated backend server someone maintains for the script to work, it's using a wiki-like JSON database for public domain content: Wedata.net; define your json schema and let anyone that wants to fill in, edit and co-maintain the data (here: unpagination url regexps and node xpaths). The editing process looks a bit like this. I am especially glad to have found it, as I've been wanting a service like that for a long time but not really wanted to host it, and having a public cloud sync point for mini-applications like this to update their localStorage copies of the data from is a neat trick.

Anyway, happy githubbing! As with so many little features before it, I already can't fathom going back to a github without this feature (and I've only been using it since yesternight). It helps a lot seeing what went into which release at a glance, without doing all sorts of manual work. It would be nice for it to mark branch-off points too, but it would require some kind of merge-base type api end-point, for digging up where a branch's closest common ancestor is to all other branches. Maybe time for devising a neat response format and crafting a little api feature request.

gravatar.rb

2011-04-03T02:22:00.000-07:00

I made myself a little command-line tool gravatar.rb today, which eats email addresses and outputs "gravatar-url email@address" lines, for all email addresses that has a customized gravatar created.

It's fun to run git log|gravatar for projects you've got checked out; here's yui3:

(People showing up more than once either have multiple email addresses registered with gravatar that they commit via or borrow someone else's picture. :-)

Installing perl modules on Snow Leopard and Homebrew

2011-03-29T00:46:00.001-07:00

Preamble: this post is a great read if you really want to have your Perl stuff in /usr/local, instead of in /Library/Perl, the MacOS X way. Otherwise, maybe not so much! (The error I had stumbled on, which made me think cpan didn't just work was, apparently, that I at some point had used sudo cpan, and gotten permissions on things out of whack. If cpan was a little bit more helpful, chances are I would have figured it out sooner -- but let's not toss a blog post that was fun to write and still has a few useful nuggets, despite being built on an otherwise shoddy fundament. :-)

First up: this isn't any more Homebrew specific than that I use the perl Apple ships with Snow Leopard (5.10 at this time of writing, for 10.6.7 - I'm on a Macbook Air myself), with all its built-in DTrace and MacOS integration goodness. Read that fine page for authoritative data; I neither am not pretend to be a perl savant.

Using the system perl is how Homebrew prefers it – unlike some other packaging systems I shall not mention here, as a service to other fine people trying to google for "perl mac snow leopard -this -that". Should you for any reason wish to mention those in comments on this post, you may call them "Funk" and "Warts" (or "MacWarts"). I may delete your comment if you don't. (Nothing personal; just sayin'. And you might as well not, anyway, I am unlikely to be able to help you.)

Anyway, so I want my perl modules installed in /usr/local, and I want them in my @INC when I run /usr/bin/perl and it's a shame that the system perl only looks in any of these, right?

/Library/Perl/Updates/5.10.0/darwin-thread-multi-2level
/Library/Perl/Updates/5.10.0
/System/Library/Perl/5.10.0/darwin-thread-multi-2level
/System/Library/Perl/5.10.0
/Library/Perl/5.10.0/darwin-thread-multi-2level
/Library/Perl/5.10.0
/Network/Library/Perl/5.10.0/darwin-thread-multi-2level
/Network/Library/Perl/5.10.0
/Network/Library/Perl
/System/Library/Perl/Extras/5.10.0/darwin-thread-multi-2level
/System/Library/Perl/Extras/5.10.0

Wrong! (I told you: read this fine manual written by knowledgeable people!) Still, a neat thing they describe is that the MacOS X perl installation will happily read /Library/Perl/5.10.0/AppendToPath and /Library/Perl/5.10.0/PrependToPath and append / prepend any directories they list (one line per path, normal unix style) to @INC. Or, of course, any other version than 5.10.0 – perl -v will tell you what you've got installed. So, as root, I appended /usr/local/lib/perl5/site_perl/ to my AppendToPath.

Next up, assuming you've already made a mess trying to install stuff with cpan (I had), you may want to do what I did, which is rm -rf ~/.cpan (which permanently nukes it; should you want to be able to undo that, you of course instead move it aside somewhere – you're solely responsible for your actions here, as always). Then run cpan. It'll ask you tons of questions you might not know the answer to any better than I did if you say no, or give slightly okay defaults if you say yes. Some frustrating experimentation later, I came down with this recipe, which worked for me; first yes it, and then when you get the prompt, type o conf. It'll probably look a bit like this, if the defaults remain about the same:

cpan[1]> o conf
$CPAN::Config options from '/Users/jhs/.cpan/CPAN/MyConfig.pm':
    commit             [Commit changes to disk]
    defaults           [Reload defaults from disk]
    help               [Short help about 'o conf' usage]
    init               [Interactive setting of all options]

    applypatch         []
    auto_commit        [0]
    build_cache        [100]
    build_dir          [/Users/jhs/.cpan/build]
    build_dir_reuse    [1]
    build_requires_install_policy [ask/yes]
    bzip2              [/usr/bin/bzip2]
    cache_metadata     [1]
    check_sigs         [0]
    colorize_debug     undef
    colorize_output    undef
    colorize_print     undef
    colorize_warn      undef
    commandnumber_in_prompt [1]
    commands_quote     undef
    cpan_home          [/Users/jhs/.cpan]
    curl               [/usr/bin/curl]
    dontload_hash      undef
    dontload_list      undef
    ftp                [/usr/bin/ftp]
    ftp_passive        [1]
    ftp_proxy          []
    getcwd             [cwd]
    gpg                []
    gzip               [/usr/bin/gzip]
    histfile           [/Users/jhs/.cpan/histfile]
    histsize           [100]
    http_proxy         []
    inactivity_timeout [0]
    index_expire       [1]
    inhibit_startup_message [0]
    keep_source_where  [/Users/jhs/.cpan/sources]
    load_module_verbosity [v]
    lynx               []
    make               [/usr/bin/make]
    make_arg           []
    make_install_arg   []
    make_install_make_command [/usr/bin/make]
    makepl_arg         []
    mbuild_arg         []
    mbuild_install_arg []
    mbuild_install_build_command [./Build]
    mbuildpl_arg       []
    ncftp              []
    ncftpget           []
    no_proxy           []
    pager              [/usr/bin/less]
    password           undef
    patch              [/usr/bin/patch]
    prefer_installer   [MB]
    prefs_dir          [/Users/jhs/.cpan/prefs]
    prerequisites_policy [ask]
    proxy_pass         undef
    proxy_user         undef
    randomize_urllist  undef
    scan_cache         [atstart]
    shell              [/bin/zsh]
    show_unparsable_versions [0]
    show_upload_date   [0]
    show_zero_versions [0]
    tar                [/usr/bin/tar]
    tar_verbosity      [v]
    term_is_latin      [1]
    term_ornaments     [1]
    test_report        [0]
    unzip              [/usr/bin/unzip]
    urllist           
    use_sqlite         [0]
    username           undef
    wait_list          undef
    wget               [/usr/local/bin/wget]
    yaml_load_code     [0]
    yaml_module        [YAML]


cpan[2]>

Then paste this little snippet, which sets most things right (it's a good decade to use a UTF-8 terminal):

o conf term_is_latin 0
o conf check_sigs 1
o conf make_arg -j3
o conf make_install_arg -j3
o conf makepl_arg PREFIX=/usr/local
o conf mbuildpl_arg --install_base /usr/local
o conf commit

This will have perl use two cores when compiling modules, and use Module::Signature if you install it, which seemed like a nice enough idea (at least if you have gpg installed; you may want to skip the check_sigs part, or install it first, if not -- via brew install gpgme for instance). And, most importantly, it'll put stuff in /usr/local at installation time, so perl will find it at invocation time. (Some day, that might just work right out of the box, in MacOS too.)

And that's it. Try it with cpan Module::Signature for instance, if you enabled that and you'll eventually end up with a fatter /usr/local/lib/perl5/site_perl. You are encouraged to drop other helpful tips here, especially if you know a thing or two about perl or cpan that I totally should have mentioned. Happy hacking! :-)

Musings on DNA, code and otherwise

2011-01-14T00:50:00.000-08:00

From college days, I recall microbiology as one of the more fascinating fields I never opted to pursue, but still developed a latent interest in. The evolved, highly complex and cross-connected execution context of biology, at the deepest levels where it meets (or indeed is) the realities of chemistry and physics speaks to my geek genes. There are so many levels of exciting, beautiful and interesting stories, events and developments woven together and buried in genome! Not to mention it is open source!

You want to grok something? UTSL! If you don't yet understand it, it's because you are not knowledgeable enough. Go learn! Experiment. Think hard. Figure it out. And then share what you learned, so others can reach longer, make it do more, be more efficient, go faster, taste better, or what have you. I think choosing mad computer science over mad genetics was more of a cultural thing gut draw towards openness; away from the lock-ins of lawyers of looming heavy corporations protecting the intellectual off-spring stemming from the money invested into powering their geeks' forays into the unknown, and toward the almost open eco system of what today amounts to the web and places like github, where people promiscuously and chaotically spin off of each others' work making stuff better together, for fun and profit.

But back to genetic code, as inspired by a read of an article on MS, schizophrenia and bipolar disorder as related to endogenous retroviruses (meaning stuff we all carry as common heritage, though not activated in most of us). One of the things we were taught in my classes was about the huge amount of introns (DNA between genes that is seemingly not active, as it isn't observed to ever get transcripted into proteins -- which is essentially how microbiological code execution happens) observed in all living organisms (about 98% of human DNA is non-coding DNA). Of course the depth level of the insights we absorbed were rather meagre, and although omnipresent easy-access Wikipedia was not yet around, it felt fairly obvious that the rest, at the very least, would end up having all sorts of other functions or effects under conditions and circumstances that would relate to it somehow. If you have ever researched (or exploited) a buffer overflow in some program, you probably know that wherever the program pointer accidentally ends up, whatever junk or data happens to be in that memory region is suddenly, for all intents and purposes (or despite them :-), is now code! Maybe it will do something interesting. Or nefarious. Probably nefarious, really.

Biology, chemistry and physics, of course, just add a gazillion more dimensions to "functions and effects of things", while also imposing a bus-load of peculiar constraints, and introducing still more environmental conditions from the rather chaotic virtual machine of physics, chemistry and biology (being three different zoom levels at which things happen to shoot your code in the foot) combined.

If you thought debugging code is hard, it's actually a delightful walk in the park, for the most part -- free of all sorts of concerns like the ethics of feeding useful test arrays into your debugger, that your test subjects can't keep dying all the time until you figure it out, and so on, and that we're probably not debugging software that co-evolved on a planetary scale, over a couple of billion years, but was probably caused by, at the very most, a couple of decades worth of ingenuity from probably less than thousands (or at most millions) of humans, all more or less acting with actual intents, often worthy of words like "design" and "intent". We're kind of lucky, that way! And oftentimes, the whole code base is even your very own damn fault! And you could even know full well what every line of your code is supposed to do, even though you might not yet have discovered all the things it should probably also do to work under the conditions it'll be subject to when unforgiving users run it. (Even yourself!)

I wonder how far into the future we have to go to see iLife speaking to the creative class doing justice to its name, providing us with the tools to be creative with the workings of micro- (or, for that matter, even macro-)organisms, in the same manner it's letting today's happy amateurs go wild with arts like graphics, music and video. Lots of people have rather rigid ideas about what amounts to art, science, work, life style, communication and so many other arbitrary things you can do for whichever reasons propel you, based, among other things, on rules, rulers or methods used in each field, motivations behind them, et cetera. Try breaking a few; you end up in far more interesting places. And make and bring friends, along the way!

Reading Chrome, Firefox and Opera extensions in Emacs

2011-01-09T01:06:00.001-08:00

If you're a fan of emacs' archive-mode that lets you load zip files and the like as it lets you load regular directories, and know that Firefox xpi, Chrome crx and Opera oex extensions are really just thinly wrapped zip files under the hood, you might expect Emacs to grok them, just the same. I did, and was a little surprised when it didn't plain work. (So maybe I'm running an old Emacs build?)

Anyway, I looked into the matter and did a little poking about to see what setup would do the trick. In the case of Firefox and Opera, it's really easy, as the xpi and oex "formats" only amount to changing the filename extension and making sure you have the right files in there -- so all you have to do is make sure that ("\\.xpi\\'" . archive-mode) exists in your auto-mode-alist, and ("\\.xpi\\'" . no-conversion) in your auto-coding-alist (there is a little snippet for your .emacs at the end of this post), and the corresponding oex variant.

In the Chrome case (which prepends a Chrome header with a format version, an RSA public key and signature from the extension author), besides the corresponding crx change for same variables, I also had to patch the lisp/arc-mode.el file lightly:

arc-mode.el.patch at pastebin

If you try loading a crx and find it already works, someone upstream already applied the same or a better patch than mine to your installation, but otherwise, read on. To find where it is on your system, type C-h archive-mode <return> and follow the archive-mode is a compiled Lisp function in `arc-mode.el'. link, apply the patch and M-x byte-compile-file the result again.

The .emacs snippet that makes it all kick in automagically:

.emacs snippet at pastebin

Note that for the Chrome extension case, you won't be able to edit and save any of the contents in place, as you can with plain zip (xpi, oex) files; the magic to remove/re-add public keys and signatures will be somebody else's late Saturday hack to complete. :) Enjoy!

Vintage Firefox

2010-12-26T16:13:00.000-08:00

When I occasionally need to check how far back a feature was introduced in Firefox, it usually turns out I only have the last few versions installed, and takes a while before I find where mozilla.org hit them, so here goes: a complete archive of the latest (mac -- tweak the urls or browse yourself for Linux, Windows or otherwise) release of every major version of Firefox from 0.8 to date (at current time of writing, 3.6.13 is the most recent Firefox released) off of ftp://ftp.mozilla.org/pub/mozilla.org/firefox/releases/, where mozilla.org hosts them, and the Firebug releases you need to do any useful web development work with them:

Firefox 4.0 beta 8 (Gecko 2 / Firebug 1.7X.0a7)
Note: as neither of is released yet, consider both links outdated when you read this (the Firebug xpi especially is in development, as hinted by the X)
Firefox 3.6.13 (Gecko 1.9.2 / Firebug 1.6.0 or 1.5.4)
Firefox 3.5.16 (Gecko 1.9.1 / Firebug 1.5.4)
Firefox 3.0.19 (Gecko 1.9 / Firebug 1.4.5)
Firefox 2.0.0.20 (Gecko 1.8.1 / Firebug 1.3.4b2, 1.2.0 or Firebug 1.0.5)
Firefox 1.5.0.12 (Gecko 1.8 / Firebug 1.05)
Firefox 1.0.8 (Gecko 1.7)
Firefox 0.9.3 (Gecko 0.9)
Firefox 0.8 (Gecko 0.8)

(Corresponding Gecko and Firebug versions, the second and third most messy to find Firefox entity, from the Gecko page on MDC and the install.rdf of the corresponding Firebug xpi:s archived at getfirebug.com.)

Never ever download and install Firefox from any other site! There is a huge, shady "pay per install" market of people bundling other people's software with malware (bot nets, typically); read all about it from Symantec (pdf) or Kevin Stevens (also pdf, the latter from Black Hat DC 2010). This goes for all executable programs you download and install on your machine; if you have no reason to trust the source, get it from a source you do trust.

Opera 11 extensions @ Add-on-Con

2010-12-12T00:40:00.000-08:00

I attended Add-on-Con this year again (I think I missed last year's), and figured I should sum up some of the interesting and/or useful stuff I picked up on while there. What I found most gratifying was the upcoming Opera (11) add-on API, which takes on the same approach as the Chrome and Safari herd, with the clean, HTML5 / postMessage / content script based design we have grown to expect, with optional background and popup pages. (Mozilla kind of wants to get there too, but does not expect to have anything remotely near it for at least another year, and I don't think it's even on any road maps yet.)

In Opera's current version, they don't go quite as far as does Chrome with its process separation, so an extension's popup can actually read from and write to localStorage from the same origin as its background page (which lets you simplify the message passing a little if you're not aiming for easy porting). Your content script runs in its own global object, but it has a window reference to the current window object, as the page sees it, and in the interest of not automatically leaking your script's guts into the page's global scope, it is not in the inheritance chain, so you actually have to use long references like window.document and window.localStorage to access them. (I tried assigning this.__proto__ = window;, which in theory should import all window properties in my content script's scope, and while that seemed to allow the former, it still failed on the latter, so I guess I hit unsupported stuff or beta bugs.)

To try it out, I made my github improved user script an Opera 11 extension. Run rake to build the zipped-up oex from the content script if you check out or fork the codebase, which puts the content script in includes/ (all scripts in includes/ with a // ==UserScript== block with Greasemonkey-style @include / @exclude rules run, not at the DOMContentLoaded event, but at page start, so you have to set your own DOMContentLoaded listener in a browser forked section if you want to execute under the same page conditions under Opera as in Firefox or a Chrome content script set to load at page ready).

Touting standards, Opera decided that a browser extension is essentially a Widget, and reused the W3C Widget standard they helped coin a while back, making its manifest sit in config.xml in the oex (a zip file) root directory (and emphasized that it also means you have to have an index.html next to it – the background page – but that it can be empty, if you like). For instance:

<?xml version="1.0" encoding="utf-8"?>
<widget xmlns="http://www.w3.org/ns/widgets" version="1.0.0-0">
  <name>Github improved</name>
  <icon src="icon.png"/>
  <author href="http://ecmanaut.blogspot.com/">Johan Sundström</author>
  <description xml:lang="en">Adds github changeset unfolding and other site improvements.</description>
</widget>

This also means that you can use Widget storage and config-based preference init, if you like. It indeed even seems to be the recommended way, as that gets you unlimited storage (actually: the same total cap Opera sets for itself globally), whereas your background page's localStorage gets subjected to the same arbitrary per-domain storage size cap that web sites are.

As for extension distribution, they of course have their own extension gallery you may want to submit your things to, and like all human-driven approval processes, they have a slightly anal process to get through first. Essentially, the less you can state about your add-on, the fewer opportunities you'll have to fail their test criteria; I was advised by the presenters not to add an add-on homepage, for example.

Currently, every submission attempt of yours in the approval process requires you to update your version number, so you can't aim for releasing a final 1.0 and treat all the attempts on the way there as release candidate 1 through N; my test add-on went through three revisions before coming out, and ended up named 1.0.0-0 in the end. It also was required to list change notes, that are reflected on the download page even when for an add-on that never existed before. Badly implemented good intentions also featured in the insistent numerous-stage wizard walking you through many steps every time (including screen shots, icons, et c), but there is at least the option to copy data from a previous version.

I think I may make Opera extensions of some of the extension-worthy hacks I do, but having tried the Chrome extension gallery's really smooth process, showing how it should be done, I don't think I'll have the patience to jump through Opera's hoops very often unless they improve notably.

To me, the bottom line is that Opera's add-on system doesn't make it prohibitively difficult to support Opera too, if you are already making an add-on for Chrome that does not rely on lots of deeper-integration Chrome API:s. It still surprises me that while they have the best technical user javascript implementation (and first too) a browser has ever had, they gave it the worst UI (a config switch listing a directory into which you yourself drop user scripts manually, after having modded them to add a DOMContentLoaded event listener to start the main part of the script) ever. Their js hooks still thorougly beat both Greasemonkey and Chrome, yet the lack of installation when visiting a *.user.js file locks that feature set in for only the really expert of browser users.

Comfy browser performance testing

2010-09-19T18:21:00.000-07:00

I have been poking around with some performance tests and profiling lately, first with Steve Souders' Browserscope, which gets the job done, at some convenience cost of setting up a publicly hosted web page and tying it to a Google account (that you might have to first create), and, for loading performance, different ways of organizing script tags, images, iframes, style sheets and so on, Cuzillion, by same author.

Today, I found and toyed around a bit with jsperf.com by Mathias Bynens, which does much of the boring setup job for you, when it comes to micro-benchmarking some snippet of javascript versus another, assuming the thing you want to test is synchronous, and doesn't involve complex preconditions. (I wonder what makes Array copying using a saved Array.prototype.slice significantly slower than the other more evenly performance-matched variants?) As should always be stated when mentioning benchmarks and optimization, this kind of micro-benchmarking is less relevant than finding your code's hot spots and picking better algorithms where relevant. That said, it's still academic fun to play with this kind of thing.

For this kind of small test, jsperf.com does all the Browserscope setup work for you, and lets others improve (fork) your test after the fact, adding versions you didn't think of yourself that do the same thing. I have a vague plan of trying to make it run some performance tests for a little code snippet I wrought up recently that does a deep copy of a nested (JSON:able) structure from a parent window where there may be crud on Object.prototype (the code should run in an iframe free of such), to benchmark vs a mere JSON.parse(JSON.stringify(taintedObject), which would also clean away the crud:

var _slice = [].slice, _toString = Object.prototype.toString;
function array(arrayish) {
  return _slice.call(arrayish, 0);
}

function isArray(obj) {
  return '[object Array]' === _toString.call(obj);
}

// Take a nested structure from a hostile window (presumably full of crap in its
// Object.prototype, et cetera) and return a cleaned-up version with this window
// object's (pure) Array and Object constructors.
function deepClone(obj) {
  if ('object' !== typeof obj) return obj;
  if (null === obj) return null;
  if (isArray(obj)) return array(obj).map(deepClone);
  var clean = {};
  for (var key in obj)
    if (obj.hasOwnProperty(key))
      clean[key] = deepClone(obj[key]);
  return clean;
}

What surprised me about the above code, when testing it on the kinds of payloads it would typically handle (data somewhat heavy on largeish strings), was that it actually was a bit faster than using the browser native JSON codecs, whereas, for larger inputs, native code always won out. Sometimes you just have to test stuff, to find out what wins.

JSONView with copy & paste support

2010-09-13T08:22:00.000-07:00

Last weekend I picked up Ben Hollis' JSONView extension to tinker it to keep JSON viewed through it copy-and-paste:able, both as a whole and in part. It was rather fun work and within some few hours I had a patch that did the trick, and a few other ones to boot:

I sent Ben a patch, but not having heard anything back, I figured I might as well share the results here too, as it's all MIT licensed goodness (as is my patched version, of course). You can either download an installable xpi build, or fork my repository from github.

My edits are all on a branch named after my github user -- a practice I can recommend, when mirroring some svn upstream repository (with git-svn, for instance); it simplifies the process of tracking vendor updates to a mere git svn rebase on master (for you -- git-svn, sadly, is not able to push its metadata about the non-git upstream to github) and lets other people reuse your pristine vendor branch for their own purposes.

Enjoy your easier-to-handle JSON! I guess next step would be to integrate these improvements into the Chrome fork Jamie Wilkinson is maintaining. Any takers?

Friendlier github commits pages

2010-08-23T03:55:00.000-07:00

I figured I should announce the news of a much improved github commits page I have been tinkering with for a while, that lets you unfold full details about commits (individually, or in batch) without leaving the page. A screenshot of what the Greasemonkey master branch commits might look like with a few unfolded:

Usage is intuitive; just click anywhere in a commit that isn't already a link, and the commit will load in place (unless already loaded) and unfold its whole changeset, or refold itself again, when clicked a second time. Similarly, individual files changed can be folded and unfolded much the same way, though only by clicking their headers (I often want to copy and paste stuff from diffs, so no click magic there). To unfold or refold every commit on the page, either hit "f", or click the "(un)fold all commits" link at the top or bottom right of the page.

This user script (install link, userscript.org page with docs and complete change log) is incidentally compatible with extensions like AutoPagerize too, in case you prefer scrolling for content to finding and clicking "Older" (h) and "Newer" (l). And if you just want to try it out without installing anything, try this bookmarklet on any github commits page.

And it's all done by a hundred lines of jQuery, about a dozen lines of css and another dozen lines of Greasemonkey / Chrome portability code that breaks out of their respective sandbox environments to run what the script needs to run in the context of the web page itself.

That latter piece is useful too, by the way, modeled on a trick Anthony Lieuallen came up with for the Greasemonkey wiki at some point. This is what it looks like, including full docs:

The above-referred jQuery is already loaded by github's page template, so I don't need to bother grabbing or shipping one of my own. If you are hacking a more hostile site, and run Greasemonkey you might want to add a // @require http://ajax.googleapis.com/ajax/libs/jquery/1.4.2/jquery.min.js line to your script header instead (I haven't tested, but if you try to run jQuery in the mozilla javascript sandbox anything beyond really basic stuff fails due to the common pitfalls that jQuery, sanely, does not much attempt to evade).

All above code is freely poachable (read: MIT license) at github, with as much or little attribution as you like. I would of course be especially happy to see the github people adopt this for their default page templates, as it's really useful stuff. Share and enjoy!

Showing inline HTML comments of Paul Graham's

2010-07-28T03:00:00.001-07:00

I browsed a few of the older of Paul Graham's Essays tonight, dug up my Paul Graham click-to-inline footnotes user-script, which wasn't installed in this Google Chrome profile (install link here), and peeked at the source of a few of the essays which it still doesn't grok.

In doing so, I happened upon some HTML comments -- the next level of shaved-off cutting-floor material left around for our prying eyes, of you will -- many of which were interesting, much like his foot-notes. So I hacked up a new version of the script, which inlines those too, showing them as little <a>, <b>, <c>, onward -- expanding to something looking like an HTML comment when clicked.

Share and enjoy!

To test drive the new feature, you might want to re-read Why Smart People Have Bad Ideas, A Unified Theory of VC Suckage and The Age of the Essay. (And before anyone mentions it: no, I didn't actually get to making it augment his old-style [non-]markup for footnotes. Maybe next time. ;-)

Optimizing SVGs

2010-07-14T22:59:00.000-07:00

The other night I came across this cool Tree of Life page, featuring some pdfs and images of the family relationship of all life on Earth. Great stuff. Among them, this simplified rendition divided into about a hundred sub-families, and their relations:

You see our really ancient common heritage starting at 0 radians, progressing through evolution towards the really highly evolved creatures at two pi radians; birds, crocodiles, turtles and (you are here!) mammals (but in reverse order; sorry -- us mammals are not the last cry in evolution in all ways conceivable :-).

I liked it, but it felt wrong that it was trapped in a pdf; this kind of thing should really be a Scalable Vector Graphics image (SVG, henceforth) with cut-and-pastable text, and both readable and hackable right in the page source, for people like you and me that like to poke around in things.

So I made an exercise turning it into a somewhat nice SVG, to see both how small I could make it, without much effort, and where browsers are at in terms of rendering an inline SVG, these days. I haven't actually tested yet, so it'll be a fun surprise for me too, upon publishing this post. And if your browser doesn't render it, you still had the rasterized version above, or the source pdf (35008 bytes long).

Oh, and for the curious, there's a public git repository of all the changes on github, one step at a time, from the first version (where it's helpful to have a friend that has Adobe Illustrator, for instance, to do an initial machine translation of the pdf to a workable yet messy SVG). For reference, this page does not embed the minimized end result, which weighed in at 14852 bytes (or 6038, gzipped to an svgz).

(I consider those cheating, as the line data itself has been compressed somewhat beyond the point where it's still hackable.)

If you want to play around with this kind of thing, and get familiar with hand-editing SVG files, I can whole-heartedly recommend Sam Ruby's great library of sub-kilobyte hand-made SVG:s. While I can't find a statement to attest to it at the moment, I believe they are all freely MIT licensed (I think I asked him in person at SVG Open 2009), encouraging you to learn from and play with them. It is a great resource if you want to start playing with this yourself and want to pick up on some of the tricks of the trade, since they, on average, contain pretty much 100% signal, 0% noise.

Oh, and the SVG specification when you are curious about something specific. If you want to learn a minimal subset only that can do almost everything, look at the <path d="turtle graphics here"/> attribute.

And here is the outcome of my own craftsmanship, for the browsers that get it:

Unfortunately Blogger intersperses it with <br> tags if I leave the new-lines in, so see github for a cleaner version. No luck with my current set of browsers, with at least this doctype and HTML version. It does degrade to showing the text content of all the families, though, which a PDF wouldn't.

Google styleguides for JSON APIs

2010-07-11T18:49:00.000-07:00

I just eyed through Google's dos and don'ts style guide for when exporting JSON APIs. Overall it's pretty good, ranging from the very basics of "abide by the JSON specification" (though stated at depth, presumably for the JSON illiterate, with all implications of what data types are available, what syntax is legal and the like) to how to do naming, grouping of properties, how to introduce and deprecate identifiers, what to leave out, how to represent what kinds of data, and so on.

It of course doesn't guarantee that the outcome will be good APIs (the JSON exported by Google Spreadsheets, at least in the incarnations I peeked at some years ago, was an absolutely horrible auto-translation from XML that even mangled parts of the data due to its imperfect representation of a data grid, for instance), but it prevents does protect against many needless pitfalls.

Time

Not all of its tips are great, though. The rest of this post is a rant about time, and how it's more complicated than you think (unless you have ever run across this). I specifically want to warn about its recommendation on Time Duration Property Values (my emphasis that it's talking about amounts of time rather than timestamps, which ISO 8601 is great for), which it suggests to be encoded ISO 8601 style. Example (comments are of course not part of the output):

{
  // three years, six months, four days, twelve hours,
  // thirty minutes, and five seconds
  "duration": "P3Y6M4DT12H30M5S"
}

Don't do this!

That is a daft idea and/or example. Think about it for a moment. If you truly want to convey the length of a duration for a period of three years, six months, four days, twelve hours, thirty minutes and five seconds, the total number of seconds of that should be computable with perfect precision, right? To this application, after all (whatever it is) -- the number of days, hours, minutes -- and even those last five seconds -- are significant, so we should get them right.

Here be dragons. Human readouts of time durations like the one above don't convey that information. If you talk about durations, you have to pick one, well-defined unit of time (or -- less usefully -- a number of units that translate to each other by well-defined rules, requiring no additional data inputs) of time, and stick to that one.

I'd recommend picking either days, seconds, milliseconds or nanoseconds as your (one!) unit of choice, dependent on what kind of duration you represent and what kinds of likely uses it has. Declare the unit (so the property name suggests the unit, if you're kind) and stick to integers.

Because the number of seconds in three years, six months, four days, twelve hours, thirty minutes and five seconds depends on when you start counting, and/or what you mean by "year", "month", "day" and "hour" (earth-centric time is complicated -- some minutes even have more than 60 seconds).

Typically, it's in reference to some specific reference time, from which to increment the year by 3, the month number by six, the date by four days, and finally add another 12h, 30m and 5s. But we didn't get a reference time; we just have a duration. It's like a vector denoting a coordinate in a coordinate system. You can't tell what it points at, without knowing where it points from.

And humans happily think up some well-defined case like counting from midnight this January 1st, finds that it works out to becoming 2013-06-04 12:30:05 after adding, and maybe even computes it to 108041406 seconds total and believes it's a well-defined amount of time. It just isn't; those six months only just turned out to be 182 days because 2013 isn't a leap year. If we had started counting from 2009 and ended up on 2012-06-04 12:30:05, they would have wound up 183. And if we had started counting from March 1st instead of January 1st, they would have been 185. No matter how you see that, we've suddenly got all this seemingly second-precision duration -- with a fuzz margin of plus or minus a day and a half -- a range of more than 250,000 seconds, which compares most unfavourably to advertised second-precision.

And if you decide that your years are a special 365 * 24 * 60 * 60 seconds long, then ten years from now won't be July 11th, but July 8:th. Humans might disagree.

So if you have want to represent a duration of time, and you want the API consumer to know how long that duration is, pick a unit and pass an integer. And if, say, your first API version had a timestamp of the start of something and you want API v2 consumers to be able to tell how long after that it completed, pick a unit and pass an integer. And if you have good reason to believe that the consumer (maybe a human) is interested in the end time, pass start and stop timestamps. Programs consuming your JSON will have access to date functions that can compute the amount of time between them, or what date and time it will be after a fix number of given time units from a reference time.

Thank you for taking your time to indulge in thinking and talking about time over JSON APIs.

List hardlinks on HFS volumes

2010-07-09T22:22:00.000-07:00

This one goes out to all of you mac users out there.

I recently made a little hack hardlinks that takes a path (or inode) of a hardlinked file and lists the paths of all of its clones (including its own), provided it lives on an HFS volume. Usage is simple:

sudo hardlinks path

or

sudo hardlinks -c inode

You need to have the hfsdebug-lite binary Amit Singh provides installed for it to work, and if you're on a modern mac, you need to install the Rosetta tools from your MacOS X install disk to get hfsdebug to run. (That sudo is needed for hfsdebug to access the raw device of the volume -- after that, hfsdebug drops privileges to the nobody user.)

Source code:

A peek at Dropbox

2010-07-06T19:58:00.000-07:00

On a friend's suggestion, I had a peek at Dropbox for syncing directories between multiple computers, sharing files with people without posting email attachments, and the like. It's got many rather useful properties, but seems a little immature in the unix world; it is unaware of file modes and symlinks (so symlinks will show up as real files or as nothing at all, if orphaned), making it less suitable for syncing git checkouts across multiple machines, as I was hoping to. As noted in the linked thread, though, the upcoming 0.8 release will get aware of file modes, which is a good start.

What it does seem really good for, though, is auto-syncing preference files, data sets of stuff you want comfy access to wherever you are, breaking through NATs to sync stuff to home machines behind firewalls, and the like. One of the neater ideas I came across in the otherwise mostly uninteresting comments on this tips and tricks post was to set up a home machine to poll for torrent files dropped into some Dropbox-synced directory and start downloading them (to another directory, presumably), instead of doing the same via ssh.

The free plan covers 2GB data stored (plus keeping 30 days of backup history), and if you sign up through a referral link (here's mine), both signee and referrer get a quarter-gig extra quota.

Their iPhone application delivers browsability of the files you sync (your own and those others share with you -- and it should be noted that "sharing" currently implements read/write access, only, so you'll want to keep backups and/or trust sharees as you trust yourself) and lets you micromanage pictures into your on-phone picture album one by one, lets you play music and video, but not add them to your on-phone music library.

Similarly, it lets you can micromanage a picture at a time back from the device photo library to cloud storage (and connected computers), or make (now read-only), copy and email urls to any of the files in your dropbox, instead of mailing them as large attachments.

I am not surprised that it doesn't much address the main pain points of the major data interop inconvenience that is the iPhone (App store terms probably don't allow them to), but I was more than a little surprised that it doesn't measure up to what a normal rsync does yet for typical machine-to-machine file transfers yet. It does a good job as a dual-direction sync feature for basic data between yourself and non-technical friends and relatives, though.

Stop gif animations in Chrome with escape

2010-07-02T00:02:00.000-07:00

It occurred to me that one of the basic browser features still missing in Chrome, to turn off gif animations as you hit the escape key, ought to be implementable as a tiny user script through canvas:

document.addEventListener('keydown', freeze_gifs_on_escape, true);

function freeze_gifs_on_escape(e) {
  if (e.keyCode == 27 && !e.shiftKey && !e.ctrlKey && !e.altKey && !e.metaKey) {
    [].slice.apply(document.images).filter(is_gif_image).map(freeze_gif);
  }
}

function is_gif_image(i) {
  return /^(?!data:).*\.gif/i.test(i.src);
}

function freeze_gif(i) {
  var c = document.createElement('canvas');
  var w = c.width = i.width;
  var h = c.height = i.height;
  c.getContext('2d').drawImage(i, 0, 0, w, h);
  try {
    i.src = c.toDataURL("image/gif"); // if possible, retain all css aspects
  } catch(e) { // cross-domain -- mimic original with all its tag attributes
    for (var j = 0, a; a = i.attributes[j]; j++)
      c.setAttribute(a.name, a.value);
    i.parentNode.replaceChild(c, i);
  }
}

It mostly works, though for gif images loaded from another domain, we're unfortunately still out of luck. I hope Chrome will soon offer an extension flag for doing privileged canvas operations, such as drawImage, for an image loaded from another domain, like here.

That privilege could even involve a manual extension review process in the Chrome extension gallery, for all I care; it is jarring that we can't fix user experience bugs like this due to the enforced security model.

Edit: As suggested in the tip below, we don't really need to .toDataURL the image, although that gives the best results on pages that apply css styling to img tags that we won't inherit to the canvas tag. The script has been updated to work everywhere; direct install link here.