ecmanaut

GroupOn

2013-02-21T05:27:00.000-08:00

My startup got acquired by GroupOn (where I shall be used for Good, not Evil :-).

I started this Monday. So far, it is a little disorienting to work in what is best described as a huge confederation of startups. It should hopefully get better once chipping away at stuff to chip away at making stuff better.

Beliefs as Era Indicators

2013-01-18T00:32:00.001-08:00

The future is already here — it's just not very evenly distributed.

A common Gibson quote, but have you reflected on how many levels it applies to? It is not just about technology, for instance – which might be a typical first understanding of the message. Let me portray how it applies to beliefs, and how it may be an even more useful picture to carry in your mind.

If you were to travel back in time to the renaissance, Terminator style, not a thread on your body, you would be from the future. But almost nothing about you would set you apart much from anyone else of the era – your strange dialect and manneurisms, relative lack of calluses from not having done much as manual labour goes – but nothing that would place you in another time, rather than from some other place – except what is in your head.

All sorts of things you believe (and more importantly don't believe) set you apart as a member of a different age. Some living in those days share some of your beliefs about how the solar system is arranged, for instance (the future was unevenly distributed back then, too, after all), but most don't, and would find you a strange eccentric full of funny made-up ideas, uneducated about how things are. You would probably find that it makes it easier to live peacefully by not insisting on convincing others about what you know and hold as true, and find people laughing at or dismissing you much less, the less you do, and the more you just blend in with the beliefs of the era. Getting on with life just takes precedence over distributing your ideas across the population of the era.

It is nice to live peacefully; most people probably prefer it, both then and now. And much like what you would behave like, to live in the past, most people we ever meet in life behave, today, rather than taking every opportunity to challenge other people's beliefs you know or have reason to believe different from yours. And consequentially, we live in a patchwork of partially overlapping belief worlds, medieval to far future, every moment of every day of our lives – much more than we would see in technology, say.

Discovering, exposing yourself to new beliefs (especially the high-impact core beliefs that affect your perception of yourself and the world), and challenging and upgrading those you currently hold, is the quickest way from now to the future. It always has been, whatever time you live in. Since the future is not evenly distributed, your future is one different from everybody else's, so you can zoom right past all your peers without anyone even knowing or noticing. Many of the people you meet daily in life live in places you will never set foot in, both having solutions and problems of times long gone, times yet to come, or futures you will never reach, as seen from your own time line.

While one might assume that most of our beliefs about ourselves and the world are derived from facts, they are generally not. More typically, they are based on other beliefs instilled by social upbringing and education, our perceptions or best guesses about experiences we have had and made – and very rarely subjected to any rigorous challenges. If I ask you how many senses you have, chances are very large you might come up with the number five without even for the briefest moment considering to make an inventory of your biosensory inputs – because someone taught you that, at some point. You would not challenge me if I remarked that you have a very good sense of balance stemming from an ingenious set of tubes in your inner ear, because this belief is a piece of trivia that is about language – a (today) dated agreement about what constitutes a sense, in the English language's traditional classification of nouns – not because you believe that your body lacks this sensory equipment. But unchallenged, it might never have occurred to you that this number five is a piece of random inherited baloney that we keep passing on to coming generations. Many – or hopefully most – of our beliefs are replaceable like this, when we realize they are incorrect, incomplete, irrelevant, just unhelpful, or worse, downright deceptive.

When people talk about life-long learning, I think they mostly talk about new skill sets, and rarely new beliefs. I used to think of it that way, before pondering what things I have learned over the last decade of the greatest worth to me as a human, in my everyday life. While some of that is certainly life skills like love languages and how-to type knowledge, most of it are things like a revised understanding of nutrition, that sugar is a drug, high-fructose corn syrup (the main ingredient of soft drinks and typical candy) is technically a kind of poison and how they affect me and my perceptions of the world when ingested, how clothes confine and isolate people both from themselves and each other, notions about how to assess and change what I believe and why, the importance of being wrong, in order to discover what is right – and many, many more. Picking up new skill sets, in terms of life impact, rank way down on the scale, by comparison.

Love and respect your teachers, and listen with indulgent curiosity to people expressing opinions you don't believe in. Because, apart from your imagination, that is your best source for new useful beliefs. And if you are more extrovert-wired, it may be the richer source of the two. And don't just listen for the substance of it, ask why and try to understand, as best you can, the motivations behind them, when it is better than "because someone told them". All learning worth having should come with explanations, and conversely, suspect all knowledge you hold that you can't derive from something. When you can't derive why something is true, you will rarely realize when that knowledge expires because of changes in the world, new discoveries and technologies with consequences to the old knowledge.

I do not rank myself a great teacher, in part from having little patience to pick gentle language or stick mostly to neutral ground, which are great skills I appreciate in people I do rank as such. My own style of communication is largely weaned among slightly autist-leaning programmers that charge at things passionately and pick and choose selectively from what is flung around, narrowing in on beliefs via what proves useful, while rarely or never interchanging belief with identity. The best are often wrong, and often change their mind, as soon as they discover they were wrong, and in such a culture, charging on strongly and having people point out your errors when spotted narrows in on better ideas far quicker than stepping lightly and covering your trail in qualifiers and disclaimers. While strongly conflict averse myself, I find merit in this efficient way of prestige-less shortcuts to arrive at better truths, and will note that it immediately breaks down out of context, if applied in contexts where skins are less thick, people more married to their beliefs and less selective about their take-aways.

If I note as one of my relative-future beliefs that I observe to be ill distributed around me (surveying my nearest grocery store's dairy section is more than enough data needed for that) that "light" products, for the most part, substitute healthy fat (that play a part in some pleasant sought-after taste or texture) with unhealthy processed sweeteners trying to replicate the same, to keep up with beliefs about fat that shot to the sky after some spectacularly bad science in the early 1980s, and causing more problems than they solve, I may be dispensing useful data, but not particularly good teaching. If I link to Robert Lustig talking about the same, I am somewhat better, but still not very good: I have not particularly rigged you to be interested in the subject, and chances are slim that his explanations seek you out at a time you are interested in them.

Especially as my motives for raising the example is mostly to self-indulgently lament living in a future, steeped in a contemporary medieval world not yet based on said future. When, one or a few generations from now, everybody believing in today's common notion about dieting and counting non-qualified carbs, and everybody passing on such beliefs to future generations – once they have all died out, my future will likely slowly get more evenly distributed, but until then, I can only nurse, co-create and choose local clusters of future in a sea of past. The more you live in the future, the more everything that surrounds you is all locked in the past.

While my tone may deceivingly marry "future" with value judgments as "good" and "past" as "bad", my intent is more in line with "more useful" and "less useful", to myself and other members of that world. You are more adapted to your surrounding era by holding its beliefs, however unaware of what might be incremental or radical improvements to its status quo by believing otherwise. I am fortunate to live in a time, with a mentality and income where, upon discovering that I don't see clearly as far as others, and diagnosing the cause as astigmatism, I could patch my lenses via some LASIK surgery in 2012, instead of changing my behaviour and wearing external optical patchwork every day of the rest of my life. Many either don't have that option, or worse, have, but never find out, even if they would have chosen it and been successful attempting it.

The core point I have made above is that beliefs, more than most things, define who we are, what we do and what comes out of it, how we feel about ourselves, what paths we choose, what we aspire to, worry about, avoid, and why; whom we share what with and when, and that we are at choice about them to the extent we choose, and never more. They can be pliable or rigid, set in stone or actively eager to get replaced, iterated, and superseded. And they exist in a multidimensional soup of other beliefs, most unreflected upon by those holding on to them. Some are benign, many illicitly planted, some detected, many not. Lots of beliefs are comforting but disastrous, like denying climate change, choosing a set seeming to incur less responsibility. Some are chosen to have an easier time coping – and sometimes believing falsehoods can help you live with a higher quality of life than you would have with a greater awareness, if knowing makes you brood or get plagued by guilt.

Life is a set of choices, but choices are downstream from beliefs. Upstreams from beliefs are whatever processes of multiplication and elimination you instill. Make them good ones. Live long, and prosper! :)

iOS app search engine definitions

2013-01-01T21:13:00.003-08:00

To have chrome search for iOS apps from the url bar by typing "iapp app-name", add a search engine definition (Command-, type "search", click "Manage search engines", scroll to the bottom of the list and add a new entry) like this one:

iOS apps     iapp     https://www.google.com/search?q=inurl%3Ahttps%3A%2F%2Fitunes.apple.com%2Fus%2Fapp%2F%20%s

(after tweaking "us" to whatever country code your apps come from)

It would be neat if the iOS app store had its own search page somewhere on the web so we didn't have to rely on Google's rather poor rendition of the search results, but I haven't found one.

The more json minded of you might also add an accompanying "iappjs" version, for the heck of it:

iOS apps json     iappjs     https://itunes.apple.com/search?country=us&media=software&limit=200&term=%s

Surprise is data

2012-12-31T09:43:00.002-08:00

One of the best ways to leverage subconscious data about myself, my experiences, assumptions and otherwise, is to notice and write down what I get surprised about – especially when traveling, or in changing circumstances in any other capacity. I am fairly certain this idea comes from one of Paul Graham's essays (though I couldn't find which – if you know, do say, and I'll link it; it's a great read).

I was visiting family in Sweden over Christmas, and what surprised me the most during those ten days, was how wonderfully quiet it is in over there. In a way this is old news to me, as the reverse was by far the biggest surprise about moving to the US three years ago – everything everywhere pollutes the audio spectrum with ambient noise. It is not just the traffic, but air-conditioning, refrigerators, dish-washers, washing machines, dryers, computers, and all the senseless amounts of intentional noise that trains make, music playing everywhere in places of commerce, and so on. Being surrounded by it for so long, I have apparently become so steeped in the noise that my body equates it with an omni-present tax on human well-being. As quality of life goes, I much prefer paying Sweden's tax rate over this sensory tax.

It's a difference in engineering culture: in Sweden, low (or ideally no) noise emissions is an important and expected fitness criterion, for everything – and has been for at least two decades, probably more. Noise levels are one of the data points listed (in dB(A)), when buying a new fridge or dish-washer, the governmental institution Arbetsmiljöverket (loosely translated "the work agency") offers a free iPhone / Android app for measuring ambient noise, and both in the large (architecture, work place environment, city planning, et cetera) and small (tools, implements), fighting noise is not just a matter of engineering pride or regulations, but also pretty much taken for granted.

In the US, it often feels like the noise aspect is a built-in proof that something is working properly, rather than the opposite. In the eighties, you could learn to hear that your C64 basic program was operating correctly via resonance frequencies from the CPU that leaked out through the loudspeaker (due to improperly grounded circuits, I presume), and any car mechanic worth their salt can tell a lot about a car by the sounds it makes.

I am surprised so little has changed here since those days.

View rendered source bookmarklet

2012-11-25T18:16:00.002-08:00

I haven't made a bookmarklet from scratch in a long while, but after looking at this codepen I wrote a while ago to demonstrate copying one javascript document into another, it occurred to me that it could easily become a modern "view rendered source" bookmarklet, that might even work on iOS devices and the like, where a view source feature is sorely missing.

Here is the result: view rendered source

On clicking it, you get a "view source" iframe with the source of the current document, in whichever state it was when you clicked the button, and when you click (or tap) that it goes away again.

I have yet to try it on an iDevice myself, but I have high hopes. To my surprise, manually entered bookmarklets like this one refuse to run in a modern Safari, though smaller ones that load their js payload over the wire instead seem to work, so I might make a gist of the code too, if that is what it takes.

Enjoy!

Absolute url from a relative url and base_url

2012-10-18T17:28:00.000-07:00

Today I needed a javascript url resolver to get absolute urls for urls mentioned in css files - resolved against the url of the stylesheet, not the current page. Fortunately, your browser already implements this natively, it's just not exposed in DOM APIs, so it needs a little DOM cleverness to coax out the functionality:

function resolveURL(url, base_url) { var doc = document , old_base = doc.getElementsByTagName('base')[0] , old_href = old_base && old_base.href , doc_head = doc.head || doc.getElementsByTagName('head')[0] , our_base = old_base || doc_head.appendChild(doc.createElement('base')) , resolver = doc.createElement('a') , resolved_url ; our_base.href = base_url; resolver.href = url; resolved_url = resolver.href; // browser magic at work here if (old_base) old_base.href = old_href; else doc_head.removeChild(our_base); return resolved_url; }

You can play around with it a little here, to see that your browser supports it, too. You should even be able to use a relative URL as the base_url parameter, which should get resolved against the page url -- which here is the jsfiddle url shown, as that demo is running in an embedded iframe, rather than on this blog itself:

It of course won't work in node.js, but hopefully it'll be useful to something you or others are doing, too. Use as you like; it's all public domain / MIT licensed goodness, whichever you fancy.

Facebook and the Human Mind

2012-07-06T08:11:00.000-07:00

I am seeing more of Facebook these days, from an indirect angle, lending a point of view on the phenomenon that lets me observe the behaviour it is driving in its subjects at large, rather than participating, and observing the behaviours I exhibit. The latter is both significantly more challenging, due to perception bias, and a microscopically smaller data point, to a sea of data backdrop.

Facebook has created a niche where its inputs are humans (expressing human behaviour) and where its outputs are massive amounts of highly measurable data about said humans, and all the ways they affect, relate to and interconnect with each other, and other entities of the world. Both that data, and the behaviours that end up amassing the data, are highly profitable -- and exponentially more so, the larger its body of subjects. But let's look closer and deeper at the human behaviours it promotes, where it gets interesting:

I think I see the fundamental component of life itself, as it applies to behaviour: any behaviour which increases the likelihood of recreating itself, increases the rate of repeating itself, and increases the accuracy of the reproduction of itself, is (from a natural selection point of view,) more fit. Whenever this mechanism randomly shows up in nature (which only needs to happen once in a universe, as the seed mutates, and down the line reproduces the incredible wealth of complexity some unironically like to call "creation"), we call it Life. This behaviour, then, is a form of life that lives, reproduces and mutates in humans, a second order life form Richard Dawkins and others call memes. And Facebook operates as an incubation chamber for this life form and its hosts, a catalyst of the set of behaviours that drive the reaction. Facebook is an echo-chamber for memes -- creating, spreading and remixing memetic life, provisioning it with an ever increasing body of fertile human minds to live (and evolve) in.

Let me establish some vocabulary to help clarify this post: a Facebook operator, as I use the word in this post, refers neither to any Facebook employee or direct partner, but to anyone using Facebook's behaviour measurement tools monitoring what transpires in the system, what type of stimulus (under what circumstances) is how successful at driving engagement. I believe these tools are available to anyone creating or owning a Facebook Page, but otherwise for the most part invisible to everyday users of the Facebook medium itself. For reuse of already established Facebookian language, where I write "activity", you may think "meme", if you so prefer, or a Facebook Page post of some sort, be it a video, picture, link, or just a piece of text, as this is their chosen granularity of measurement / reporting.

What a Facebook operator sees is essentially how successful they are at producing and catalyzing user behaviour that spreads, as expressed by per-activity virality and reach factors, measuring reproduction rate (fecundity), number of subjects (how many humans an activity got exposed to), and on aggregate what shape those flows took; if it funneled through a few people that caused large multiplication factors, or through a large number of people that each produced a tiny multiplication factor, say.

Philosophically, I find the emerging picture telling a story of our culture's present state, and future direction, as the phenomenon seems strongly self-reinforcing. The story I see emerging behind and throughout the data is to drive humans to addictive-compulsory behaviour in general, and the dopamine cycle of attention-deficit disorder in particular. Dopamine is released in response to an action as a biochemical reward. Facebook, in a substantial way, operates like an enormous (human-focused) Skinner box, set on tightening the dopamine hit cycle, optimizing that loop to tighten it as much as possible, and delivering as successful stimulus as possible -- where the measure of success is to what extent each activity creates a dopamine blip in the body of people they get exposed to and to what extent they then choose to pass it on to their peers.

This phenomenon is not isolated to Facebook in particular, they just happen to be at its leading edge. I am sure Google Plus, Twitter and others in this field very much want to do the same, but just don't yet have (or expose) the same refined measurement tools to its operators.

We should not be surprised that Facebook, a company embracing "The Hacker Way" at its core, is first to deliver on this at scale; more than any profession I know, hackers embrace and understand the mechanics of tight cycles, and optimizing your development cycle (by adopting whatever tools, processes and mechanisms you can create, imagine, or buy for money, that shortens or streamlines it), and how massively it pays off in rate of progress. In Facebook's case, I see them doing the same to the smallest unit of activity in their Skinner box, and outsourcing the optimization to any external, willing, self-interested entity which helps them turn a profit for free. (Google is attempting to do the same to the web at large via Google Analytics, for that matter, but have a substantially lower degree of control, and higher granularity of measure.)

I am less interested in (and would kindly discourage any comments to that effect) all sorts of judgmental aspects, condemning Facebook (or any other entity) or similar for its role in this. Facebook, Twitter, Google Plus and others are, for better or worse, a replacement for the mainstream media journalism of the past, and in all the above I believe I have outlined the same mechanisms it has been rehashing over the decades it has existed, only at a much higher pace, due to the now far tighter and more highly instrumented cycle. The tabloid genes that thrived in their old, leakier Skinner box, is thriving and evolving in and to this shiny new Skinner box and shows up in all its most pre-digested forms, following laws of least-common-denominatorialist journalism, largest-emotional-impact journalism and similar -- all far more attuned to impact-on-subject than any other measurement of fitness or higher order utility such as whether the content serves its recipient in some way, informing them, making them happier, or more successful, as opposed to just more entertained and engaged with the medium.

I believe it's important to realize that it's somewhat useless to moralize over what transpires in the system or what people "ought" to be doing instead with their time, or what operators ought to be broadcasting into other people's spans of attention, as what we see emerge here is just a mirror image of what behaviours the human mind is conditioned for and dutifully reproduces, as evolution taught her, and which served her well in the few hundred thousand years to date, in the kind of circumstances she lived under throughout most of that period.

More usefully, I think, is to think of how we choose to expose ourselves to these kinds of tools, knowing that we are humans with very similar biological underlying programming, and to assess what values we derive from their use, how much and in what formats, under which circumstances and in which company (online, here) we subject ourselves to these media. Especially, you might benefit from thinking of it as you would a recreational drug, as its impact on you at a biochemical level is similar, as implemented by your biological hardware and social software.

And, much like our judgment is impaired by drugs and we do things that we at the time don't see the consequences of, down the line, our use of these online social systems, to varying (but substantial) degrees hides the future impact of our actions from us, as the visibility of our actions is multiplied million-fold from what they used to be in the pre-internet days of humanity, from which we still draw most of our genes and behaviours. Especially as this activity data trickles to places your typical drunken stupor was safely insulated from just a decade ago, and as power centers wielding all sorts of control over our lives, have started taking an interest in this data haven, and what bits of trivia about us they may draw upon to influence decisions concerning us. This is entirely invisible today, where we used to have at least the semblance of control in the age when the people we saw around us, were a decent hint to the reach of what transpired at any point in time.

Mind your step, be wary of what values you derive from what you invest your time and attention in, and maybe occasionally stop for a moment to think about how you prefer to engage your mind and connect with yourself, other humans and the world of thoughts and ideas. You get to choose all of the above, as long as you actually do choose, as opposed to following your unthinking hard-wired programming.

Github TV ad

2012-06-24T18:28:00.000-07:00

This is a transcript of the Github TV ad. If you haven't seen it, think "Apple commercial", Jonathan Ive's dreamy narrative, British accent, the whole jive, and you'll have it about right. Okay, you're set; cue soft music:

Here at Github, we like to fork the best ideas in the valley and spin them a little different. You might not have heard about our philantropic branch (because we don't like to brag), but it's actually the secret behind some of our most groundbreaking new innovations.

Github Biolabs is how Tom Preston-Werner, in late 2007 and before we were even founded, re-imagined Google's "20% time". We started Github as an incubator, here at the San Francisco asylum for mad scientists, and some of our most brilliant employees began their career doing 20% time with us, as a precursor to their first parole leave. None has left us yet.

[Cut to a person lying in the grass under a huge tree wearing a light visor, skimming through and tweaking its genome via Minority Report style gestures. Chromosomes fly by in the air above, zooming in and out, as she unfolds, marks, cuts, re-folds, pans, pulls, merges, rebases, pushes and splices sequences into the tree. She is wicked fast.]

At Github Biolabs, we have four saps on tap, an expansive green-house, and a beautiful arboretum. It is here that our scientists, using recombinant DNA splicing techniques, in this social open source setting, literally Create New Apples.

[Cut back to narrator, taking a bite out of one. Fade to white. Fade to logo:]

Death and IE

2011-10-08T09:42:00.000-07:00

This post is a brief celebration of succession on the web. Not Mosaic begat Netscape begat Mozilla begat Firefox succession, but Firefox 4 begat Firefox 5 begat Firefox 6 begat Firefox 7 and Chrome N begate Chrome N+1 type succession. Google got it right with Chrome last decade, and not long thereafter, Mozilla got it too this year.

Quoting that post's quote of Steve Jobs' 2005 Stanford commencement address (worth watching, if you haven't),

Death is very likely the single best invention of Life. It is Life’s change agent. It clears out the old to make way for the new. Right now the new is you, but someday not too long from now, you will gradually become the old and be cleared away.

This very strongly applies to web browsers, and as it turns out, the best thing a browser version can do (besides getting things right in the first place) is dying even more quickly than it came. I wonder if this enlightened insight might have sprung with Mozilla, if only Firefox could have kept its inaugural name, Phoenix, which it now fully embodies, but at least now, this gift has finally been given to web developers:

You need not bother writing applications (or perfecting layouts) to high-fidelity for old browsers, for their time is short and their better ancestors replace them quickly.

Assuming your deployment domain is web pages. As browser add-ons go, if you still maintain one of the old style XPI design your work burden has shot through the roof instead, unless you have near zero UI footprint, happen to only need and use APIs that time proves to remain stable and host it on addons.mozilla.org, where they bump its maxVersion every few weeks for you when it still seems to work. Failing either, you have to manually update and test it seven times per year. In Chrome, this has premeditatedly been addressed by not promiscuously offering any APIs it can't support in the long run, at the cost of limiting how much add-ons can do. Firefox, in this regard, remains the only browser where add-on authors can innovate 100% of the browser, and in this regard it has filled an important need in the browser eco-system (and still does). It is both its greatest feature and its greatest burden. Hopefully this will be mitigated to some extent with Jetpack and the new add-on builder.

Returning to the paramount topic of graceful, benevolent, rapid-evolution-supportive death, Microsoft's IE does not yet get it. Like a third world nation befallen by sudden prosperity, it has doubled its reproduction rate while keeping its mortality rate constant. As noted in mentioned posts, this does not bring just better browsers faster, it brings over-population, and makes web development unsustainable. Or, as Paul Irish put it in the first post: it pollutes the browser market.

In this respect, the conditional comments and many "backwards compatibility simulation" modes IE bring you are not mainly tools helping web developers make sites work on Internet Explorer, but a huge extra burden of work forced onto web developers, to cope with IE's broken release process. This is Microsoft's job, not ours, and we should be outraged with them for forcing it on us.

It is not the rotting corpses of IE6, IE7, IE8 or IE9 that all need to die to give space to IE10, which still smells fresh, it is a broken release process that needs to get addressed and brought up to date with best browser practices of the decade. It is the reinvention of sudden death, not just the gift of new life, that must come to Redmond, too. I applaud the IE team for making it their business to get up to speed with the web's evolution, but it's less the whats than the hows that are important to get right now. SVG is great, but sort out the release process before taking on, say, webGL. It can wait. Fixing the world's hard problems like over-population is harder than running after the latest ooo-shiny! - but the alternative is systemic collapse.

For a browser, it is better to live a great but short life and go out with a boom, than it is to burden its extended family with a never-ending old age in an insufferable early-set rigor mortis. However you feel about Steve Jobs he lived and died in this way, never holding back, never growing stale of mind nor action, and the world was better off for it.

Running an old rails 2.3.8 with rvm

2011-09-29T12:36:00.000-07:00

I was helping set up a local (legacy) rails 2.3.8 server on a macbook today, autonomous from the system ruby installation. This was a bit messy, as modern versions of rubygems conflict with the old rails 2.3.8 environment to the tune of:

Uninitialized constant ActiveSupport::Dependencies::Mutex (NameError)

...when you try to start the server. Here's the recipe I came up with:

Install rvm.

# Install ruby 1.8.7 and downgrade its rubygems to 1.5.3:
rvm install 1.8.7 && \
  rvm use 1.8.7 && \
  rvm gem install -v 1.4.2 rubygems-update && \
  rvm gem update --system 1.4.2 && \
  update_rubygems && \
  echo 'Okay.'

# Install all the gems you need, for instance:
rvm gem install -v 2.3.8 rails && \
  rvm gem install -v 3.0 haml && \
  rvm gem install -v 2.1 authlogic && \
  rvm gem install -v 1.0 dalli && \
  rvm gem install -v 0.2 omniauth && \
  rvm gem install -v 2.7.0 erubis && \
  rvm gem install -v 1.3.3 sqlite3 && \
  echo 'Gems installed!'

If needed, run rake db:setup in your rails tree to set up its databases.
Done! rails/script/server -p your_port is ready for action.

Optimized SVGs at gist.github.com

2011-07-05T23:46:00.000-07:00

Lately, I've been having a lot of fun hand optimizing SVG files for size, a bit like Sam Ruby does for his blog (it is highly instructive peeking at his collection, as I think I have mentioned before).

For me, SVG has something of the magical flair I first found in HTML in the nineties, back when it was the new bleeding edge New Thing, but I argue that it's even more fun than HTML was. The W3C SVG specs are not prohibitively difficult to read, and of course you have much greater graphical freedom than structured documents can afford you (duh!).

Like Sam, I try for something presentable in a kilobyte or less (uncompressed, though modern good browsers are as happy to render SVG:s delivered with content-encoding: gzip, of course, as long as they are otherwise correct and delivered with an image/svg or image/svg+xml content-type), and to never enforce a fix width in the svg tag itself – so they just beautifully grow to any size you want them to be, with no loss of quality.

Which is the other main beauty of SVG, besides being fly-weight, standardized, widely supported and still growing broader support in all the main-stream browsers. Over the last few years, the SVG front has been progressing happily and now is very practically useful already, for at least those of us that care most about Chrome, Firefox and Opera (I get the perception that Opera's often rather exceptional lead on the SVG support front is largely or even solely the work of Erik Dahlström, but I might exaggerate a bit).

Anyway, this weekend, I had fun turning the Laughing Man (from Ghost in the Shell: Stand Alone Complex) that @elmex vectorized at some point, into a 1000 byte version of my own, also featuring the gentle rotating text seen in the anime series (YouTube), via declarative animation (so there is no javascript involved here).

Edit: I initially missed an excellent opportunity here to plug Jeff Schiller's scour, which is an ideal first step when you start from an SVG source file. Be sure to run with -p something-large, as its defaults are being lossy about precision, which cuts needed decimals from some input files. With -p 99 you'll be on the safe side. Experiment with low single-digit numbers if you like (the current default – 5 – is often good), but make sure things are still looking right, or you may just ruin your file, rather than optimizing it for tiny foot-print. Broken images don't get extra credit for also being small!

The result is in this gist (if you install this user script, you can swap between *.svg text/plain source code image/svg rendition) or to the left, if your browser renders inline SVG:s properly in HTML content (I recommend the gist over view source, as Blogger inserts linebreak HTML tags unless I strip out all newlines first).

What surprised me, when I made this user script, is how far standards support has come in modern browsers: in order to inject the <svg> tag I create for the rendered version, I had to feed the source through DOMParser (as setting .innerHTML is lossy from treating the SVG content as text/html, not text/xml), and the few lines doing that, just magically worked in all of Chrome, Firefox 4 and Opera 11 (de-jQuery:fied, to make more sense outside of the script's context) with no special extra effort on my part:

// turn the raw SVG source string into an XML document:
svg = (new DOMParser).parseFromString(svg, 'text/xml');

// import it into an SVGSVGElement in this document:
svg = document.importNode(svg.documentElement, true);

// and insert that element somewhere in the document:
document.body.appendChild(svg);

To me, that's a rather clear sign SVG is ready for prime time now.

While github reports having no plans on serving *.svg gists with an image content-type (they don't want people using gists for image hosting, I guess, even though it's sad you can't easily preview without saving to disk or using my hack above) I still think the light-weight gist community oriented sharing is good for this kind of thing. Others happily forked the octocat SVG I similarly format converted a while ago from the github About page, and milligramme made this much spacier version.

I gather all my SVG play in a svg-cleanups repository on github, if anyone wants to get inspired the fork or follow way, and occasionally tweet about it. If you find this kind of exercise as much fun, I love hearing about it; here, on Twitter, github, or elsewhere. I believe it's good teaching and learning for the web as a whole, too. Any logos, trademarks and the like above are property of their respective owners.

Draw your own Github SVGs, step by step

2011-06-04T04:28:00.000-07:00

I SVG:ified and played a little further with the logo material from the recently published Github about page, and then tonight I figured it would be fun to visualize the elegant process by which a raw SVG image is built up, piece by piece, by rather basic building blocks. With just a little bit of javascript magic to help you, here is how you piece together your own github schwag from scratch (works like a charm in Chrome, Firefox 4, and presumably any other modern browser that can handle inline SVG images

head over to the blog post itself

If all you see is a button and the description of each step, that just means your browser doesn't natively handle inline SVG, which of course is a bit of a shame. Another reason I was curious to try this is for seeing how inline SVG:s fare in feed readers. (Google Reader seems to fail.)

Nicer source code for the images than in the page (Blogger insisted on filling all the whitespace with html junk) in these gists: github-logo.svg, octocat.svg. gist.github.com currently doesn't serve .svg files as image/svg, so you'll have to save them locally first to see them rendered.

Source code for the step-by-step drawing is in this gist; MIT licensed, if you want to fork away, adopt, adapt or whatnot. Have fun! I did. :-)

Add-on hosting: Mozilla vs Google vs Opera

2011-05-29T23:18:00.000-07:00

I habitually develop browser user scripts to stream-line things for myself (and others) on the web – and a half random scattering of them tends to end up proper add-ons, when I think the benefit of them being easy to find, install and reuse by others merits the added work for myself of packaging them and submitting them to an add-ons gallery.

This happens rarely enough for me to forget some details of the process (yet often enough to be annoying), hence this post to document salient parts of it, applaud parts of where hosts worked things out really well, and note where there are holes to patch up. (I don't cover Safari since I have not made any Safari add-ons.)

(screenshot taken here)

Firefox add-ons: addons.mozilla.org, a k a AMO

Your add-ons are listed here: addons.mozilla.org/en-US/developers/addons
Add-on URL: addons.mozilla.org/en-US/firefox/addon/your-configurable-addon-slug
Public data: current version, last update time, compatible browser versions, website and, optionally: all add-on detail metrics, if the developer wants to share them (excellent!)
Public metrics: total download count, average rating, number of ratings
Detail metrics: TONS: mainly installation rate and active installed user base over time, broken down by all sorts of interesting properties or in aggregate, graphed and downloadable in csv format. Notable omission: add-on page traffic stats. Public example: Greasemonkey stats
Developer page linked from the public add-on page when you're logged in: NO
Release process: Manual review process that can take a really long time, as AMO often is under-staffed, and hasn't successfully incentivized developer participation in the process to an extent as to make it not so.

Summary: great stats and an ambition to make information public and transparent.

(screenshot taken here)

Chrome add-ons: chrome.google.com/webstore, or the Chrome Web Store
Previously lived at chrome.google.com/extensions, the Chrome extensions gallery

Your add-ons are listed here: chrome.google.com/webstore/developer/dashboard
Add-on URL: chrome.google.com/webstore/detail/your-addon-signature[/optional-add-on-slug]
Public data: current version, last update time, if installed: a checkmark and the sign "Installed", instead of the "Install" button (excellent!)
Public metrics: total download count, average rating, number of ratings
Detail metrics: If you jump through the relatively tiny hoop of creating and tying a new Google Analytics account to your add-on, you get detailed add-on page traffic stats, over in Google Analytics. Notabe omission: all metrics about your installed user base :-/
Developer page linked from the public add-on page when you're logged in: NO
Release process: Automated review process, making yourself the only release blocker, in practice. This is bloody awesome!

In summary: A really delightful release process that doesn't waste your time. The metrics part is really disappointingly missing the most interesting data; I hope that will improve with time. There are other essential missing docs too: to find out that you can self host your add-on AND publish it in the chrome web store, with the same add-on id, you need excellent Google fu, or this secret knowledge:

first (before your initial web store submission) build your add-on locally
store the .pem file in some good location
rename it key.pem
copy that .pem file into the add-on directory
zip up that directory (so the top level of the zip file only has one directory)
NOW (and for all later versions, IIRC) upload this zip file to the Chrome Web Store
…and it will use your add-on signature, instead of that of Google's secret .pem

(slightly de-horizontalized screenshot taken here)

Opera add-ons: addons.opera.com/addons/extensions

Your add-ons are listed here: addons.opera.com/developer (this page is near impossible to find by navigation, and was the reason I created this blog post :-)
Add-on public URL: addons.opera.com/addons/extensions/details/addon-slug[/version (redirects visitors here)]
Add-on developer URL: addons.opera.com/developer/extensions/details/addon-slug/version (no redirect help, but links to all other versions)]
Public data: current version, last update time, add-on size (excellent!)
Public metrics: total download count, average rating, number of ratings
Detail metrics: NONE. Notabe omission: all data about your installed user base, AND add-on page traffic :-(
Developer page linked from the public add-on page when you're logged in: NO
Release process: Manual review process, rewarding the behaviour of publishing as little data about your add-on as possible, since each bit of data is a plausible blocker. (Example: it is a bad thing to link to your development site on github, unless it is a repository that only has Opera-addon-centric stuff in it, and not, say, covers the source code used to build add-ons all browsers it caters.)

Summary: young; still a far cry from easy to find your way around. I accidentally managed to log in via my email address (after having reset the password), which had gotten the same password as the account nickname I had published from before, but was otherwise treated as another account, with its own (empty) list of add-ons. So I first found no way to upgrade my existing add-on and just uploaded a new one. When I found the mistake, there was no way to abort the review process I had started (but I could edit it to add some reviewer notes for the probably equally confused reviewer). What probably looks like a really well integrated set of connected Opera sites to an Opera employee is a huge maze of web pages, none related to publishing your add-on, to an add-on developer.

All of these hosting sites have developer log-in of some sort, yet fail to link the developer's admin view from the public add-on page. For Opera, this hurts really badly, as there is so much Opera noise around, which does not try to help you publish your add-on; for the others, it's a smaller nuisance (it's also entirely possible that I learned where to find the "Developer Hub" and "Developer Dashboard" links long ago enough to now find them near intuitive).

Update: I just found Opera's dashboard link! Your logged-in name is the link that takes you to your dashboard of add-ons. And it only took knowing the wanted url, opening a web console and typing [].slice.call(document.links).filter(function(a){ return a.href == "https://addons.opera.com/developer/"; }) in it. What do you know? :-)

Github Improved 1.7

2011-05-28T02:47:00.000-07:00

After a whole year in development as a user script (initial mention, formerly known as "Github: unfold commit history"), a brief test-drive as an Opera extension at the 2010 add-on con, I finished up the missing bits and pieces to make today's Github Improved! chrome extension available in the Chrome Gallery / Web Store (1.7 now also available in the Opera extension gallery).

New since last release is the little tag delta (Δ) links that let you instantly see what happened between, say, Greasemonkey 0.9.5 and the previous release tag (it recognizes anything where the only thing differing between tags are their numeric parts, which also means it's not going to handle fancy 1.6rc1 type tag names littered with semantic non-numeric parts):

And, as prior users may find even more importantly, that it doesn't hog any shifted keyboard shortcuts, which for some reason had the side effect of making arrow up pull in a pageful of diffs.

Full documentation of all features, and a few screenshots, is available on the Chrome Web Store page. I also take every opportunity to mention that it really shines best together with the AutoPatchWork extension, or something equivalent which unfolds the commit history pagination as you scroll off page. Enjoy!

Google Closure Library bookmarklet

2011-05-21T20:33:00.000-07:00

Any self-respecting javascript library should have a debug bookmarklet that lets you load it into a page, so you can tinker with it (the library, or the page, for that matter) without any messy overhead like downloading the most recent version, building it with the specific flags and sub-components you want, saving the current page, adding library setup code to it, and reloading that page before you can do squat. I found Google Closure Library a bit lacking in that regard, so here's my take on it:

It has two modes of operation; if all you want to do is load the goog object into the current page (or overwrite the one you already have with one that has a different set of goog.require()s), just customize which those requires should be, and you're set; it creates an iframe, in which it loads the library (goog.require only works during page parsing time, as it uses document.write() to load its dependencies), and then overwrites the top window's goog identifier.

The second mode is good for writing your own bookmarklets making use of some Closure Library tools; provide your own function, and it will instead get called with the freshly loaded goog object, once it's ready. At the moment, I have only played with this in Google Chrome, but feel free to fork away on github, if you tinker in additional improvements for other browsers.

Finally, here's a bookmarklet version to save or play with, which will prompt you for just which closure dependencies you want to load: closure – drag it to your bookmarks toolbar or equivalent if you want to keep it around. Enjoy!

A "No News is Good News" Tsunami Feed

2011-05-21T12:15:00.000-07:00

I have subscribed to the NOAA/NWS/West Coast and Alaska Tsunami Warning Center's feed of ahead-of-time tsunami warnings for the U.S. West Coast, Alaska, and British Columbia coastal regions (example information statement) for some time. Most, like this one, are "wolf-cry" statements stemming from some seismic activity somewhere that won't generate any tsunami, marked with the all-important body phrase "The magnitude is such that a tsunami WILL NOT be generated" - meaning I don't really care about them.

Today I made a Yahoo pipe, filtering away those from the feed. Here is the resulting feed, containing only positive tsunami warning statements. Behind the scenes, it's created by the simple YQL statement

select * from rss where url="http://wcatwc.arh.noaa.gov/rss/tsunamirss.xml" and description  like "%tsunami <strong>WILL NOT</strong> be%"

(there's some newline or similar after "be" in the original feed, and it doesn't seem like YQL does any whitespace normalization, but this matches well enough to filter the stuff).

The resulting feed is my enrapturing contribution to today's supposed end of the world. You may note that the feed is currently (and hopefully still, by the time you read this :-) empty - which is, of course, good news. Enjoy!

On a related hacker's note, it would be really handy to extend Google Reader with a "filter this feed" feature that created these more on the fly, without mucking about for an hour in the Yahoo Pipes interface, and then changed the old subscription to the new feed.

Chrome + NaCl + libmodplug + ... = tracker modules on the web

2011-05-16T00:59:00.000-07:00

Some notes I took while looking into how to build a libmikmod_x86_32.nexe and libmikmod_x86_64.nexe to get Chrome tracker module playing support from javascript (the work seems to be done already in naclports) via NativeClient, for all the MOD, S3M, XM, IT, 669, AMF, AMS, DBM, DMF, DSM, FAR, MDL, MED, MTM, OKT, PTM, STM, ULT, UMX, MT2 and PSM formats:

In a Chrome 11 profile, go to about:flags, and click Enable under "Native Client", and then the "Relaunch Now" button at the bottom (alternatively: start the session with the --enable-nacl command-line flag)
Download the NaCl SDK (for Chrome 11, in my case; eventually the ABI will supposedly freeze and cover a wider range of versions)
Download depot_tools, extract it and export NACL_SDK_ROOT=$PWD in that directory (technical overview here)
Check out naclports
Probably optional: comment out all the RunInstallScript lines in naclports/src/packages/nacl-install-all-bitsize.sh except the one you're interested in (in my case: RunInstallScript libmodplug-0.8.7 nacl-libmodplug-0.8.7.sh)
Run nacl-install-all-bitsize.sh 64 (or 32 for a 32-bit build), which, in my case, built $NACL_SDK_ROOT/toolchain/mac_x86/nacl64/usr/lib/libmodplug.a
About here my research petered out, as I found this smaller hack for XM-only playback by some forthcoming Japanese fellow (git repository here; English translation of same page c/o Google Translate), which has basic functionality
If you got curious about my original venue and proceeded further, I'd love to hear about it, especially if you managed to build the final nexe:s; do post a comment!

Github tag and branch labels

2011-05-04T02:03:00.000-07:00

I just made a little update to my github improved! user script; now it shows you branch and tag labels in the commits view, like this:

There's been some more mystery meat features slipping in there too somewhat unannounced; if you click a committer icon a little filter panel opens on top that lets you see how many commits in the view were by whom, and if you click one of those, hide those commits. I got the idea when I was playing with Autopatchwork at some point, unpaginating a whole repository's worth of commits and wanted to slice and dice the view a bit, get aggregate stats and the like. This is what Greasemonkey's early history (from 0.8 and back) looks like, in terms of authors (not committers) involved, for example:

Autopatchwork could use some more coverage on the English-speaking web, by the way, because of its neat way of aggregating user data from contributors. Instead of a dedicated backend server someone maintains for the script to work, it's using a wiki-like JSON database for public domain content: Wedata.net; define your json schema and let anyone that wants to fill in, edit and co-maintain the data (here: unpagination url regexps and node xpaths). The editing process looks a bit like this. I am especially glad to have found it, as I've been wanting a service like that for a long time but not really wanted to host it, and having a public cloud sync point for mini-applications like this to update their localStorage copies of the data from is a neat trick.

Anyway, happy githubbing! As with so many little features before it, I already can't fathom going back to a github without this feature (and I've only been using it since yesternight). It helps a lot seeing what went into which release at a glance, without doing all sorts of manual work. It would be nice for it to mark branch-off points too, but it would require some kind of merge-base type api end-point, for digging up where a branch's closest common ancestor is to all other branches. Maybe time for devising a neat response format and crafting a little api feature request.

gravatar.rb

2011-04-03T02:22:00.000-07:00

I made myself a little command-line tool gravatar.rb today, which eats email addresses and outputs "gravatar-url email@address" lines, for all email addresses that has a customized gravatar created.

It's fun to run git log|gravatar for projects you've got checked out; here's yui3:

(People showing up more than once either have multiple email addresses registered with gravatar that they commit via or borrow someone else's picture. :-)

Installing perl modules on Snow Leopard and Homebrew

2011-03-29T00:46:00.001-07:00

Preamble: this post is a great read if you really want to have your Perl stuff in /usr/local, instead of in /Library/Perl, the MacOS X way. Otherwise, maybe not so much! (The error I had stumbled on, which made me think cpan didn't just work was, apparently, that I at some point had used sudo cpan, and gotten permissions on things out of whack. If cpan was a little bit more helpful, chances are I would have figured it out sooner -- but let's not toss a blog post that was fun to write and still has a few useful nuggets, despite being built on an otherwise shoddy fundament. :-)

First up: this isn't any more Homebrew specific than that I use the perl Apple ships with Snow Leopard (5.10 at this time of writing, for 10.6.7 - I'm on a Macbook Air myself), with all its built-in DTrace and MacOS integration goodness. Read that fine page for authoritative data; I neither am not pretend to be a perl savant.

Using the system perl is how Homebrew prefers it – unlike some other packaging systems I shall not mention here, as a service to other fine people trying to google for "perl mac snow leopard -this -that". Should you for any reason wish to mention those in comments on this post, you may call them "Funk" and "Warts" (or "MacWarts"). I may delete your comment if you don't. (Nothing personal; just sayin'. And you might as well not, anyway, I am unlikely to be able to help you.)

Anyway, so I want my perl modules installed in /usr/local, and I want them in my @INC when I run /usr/bin/perl and it's a shame that the system perl only looks in any of these, right?

/Library/Perl/Updates/5.10.0/darwin-thread-multi-2level
/Library/Perl/Updates/5.10.0
/System/Library/Perl/5.10.0/darwin-thread-multi-2level
/System/Library/Perl/5.10.0
/Library/Perl/5.10.0/darwin-thread-multi-2level
/Library/Perl/5.10.0
/Network/Library/Perl/5.10.0/darwin-thread-multi-2level
/Network/Library/Perl/5.10.0
/Network/Library/Perl
/System/Library/Perl/Extras/5.10.0/darwin-thread-multi-2level
/System/Library/Perl/Extras/5.10.0

Wrong! (I told you: read this fine manual written by knowledgeable people!) Still, a neat thing they describe is that the MacOS X perl installation will happily read /Library/Perl/5.10.0/AppendToPath and /Library/Perl/5.10.0/PrependToPath and append / prepend any directories they list (one line per path, normal unix style) to @INC. Or, of course, any other version than 5.10.0 – perl -v will tell you what you've got installed. So, as root, I appended /usr/local/lib/perl5/site_perl/ to my AppendToPath.

Next up, assuming you've already made a mess trying to install stuff with cpan (I had), you may want to do what I did, which is rm -rf ~/.cpan (which permanently nukes it; should you want to be able to undo that, you of course instead move it aside somewhere – you're solely responsible for your actions here, as always). Then run cpan. It'll ask you tons of questions you might not know the answer to any better than I did if you say no, or give slightly okay defaults if you say yes. Some frustrating experimentation later, I came down with this recipe, which worked for me; first yes it, and then when you get the prompt, type o conf. It'll probably look a bit like this, if the defaults remain about the same:

cpan[1]> o conf
$CPAN::Config options from '/Users/jhs/.cpan/CPAN/MyConfig.pm':
    commit             [Commit changes to disk]
    defaults           [Reload defaults from disk]
    help               [Short help about 'o conf' usage]
    init               [Interactive setting of all options]

    applypatch         []
    auto_commit        [0]
    build_cache        [100]
    build_dir          [/Users/jhs/.cpan/build]
    build_dir_reuse    [1]
    build_requires_install_policy [ask/yes]
    bzip2              [/usr/bin/bzip2]
    cache_metadata     [1]
    check_sigs         [0]
    colorize_debug     undef
    colorize_output    undef
    colorize_print     undef
    colorize_warn      undef
    commandnumber_in_prompt [1]
    commands_quote     undef
    cpan_home          [/Users/jhs/.cpan]
    curl               [/usr/bin/curl]
    dontload_hash      undef
    dontload_list      undef
    ftp                [/usr/bin/ftp]
    ftp_passive        [1]
    ftp_proxy          []
    getcwd             [cwd]
    gpg                []
    gzip               [/usr/bin/gzip]
    histfile           [/Users/jhs/.cpan/histfile]
    histsize           [100]
    http_proxy         []
    inactivity_timeout [0]
    index_expire       [1]
    inhibit_startup_message [0]
    keep_source_where  [/Users/jhs/.cpan/sources]
    load_module_verbosity [v]
    lynx               []
    make               [/usr/bin/make]
    make_arg           []
    make_install_arg   []
    make_install_make_command [/usr/bin/make]
    makepl_arg         []
    mbuild_arg         []
    mbuild_install_arg []
    mbuild_install_build_command [./Build]
    mbuildpl_arg       []
    ncftp              []
    ncftpget           []
    no_proxy           []
    pager              [/usr/bin/less]
    password           undef
    patch              [/usr/bin/patch]
    prefer_installer   [MB]
    prefs_dir          [/Users/jhs/.cpan/prefs]
    prerequisites_policy [ask]
    proxy_pass         undef
    proxy_user         undef
    randomize_urllist  undef
    scan_cache         [atstart]
    shell              [/bin/zsh]
    show_unparsable_versions [0]
    show_upload_date   [0]
    show_zero_versions [0]
    tar                [/usr/bin/tar]
    tar_verbosity      [v]
    term_is_latin      [1]
    term_ornaments     [1]
    test_report        [0]
    unzip              [/usr/bin/unzip]
    urllist           
    use_sqlite         [0]
    username           undef
    wait_list          undef
    wget               [/usr/local/bin/wget]
    yaml_load_code     [0]
    yaml_module        [YAML]


cpan[2]>

Then paste this little snippet, which sets most things right (it's a good decade to use a UTF-8 terminal):

o conf term_is_latin 0
o conf check_sigs 1
o conf make_arg -j3
o conf make_install_arg -j3
o conf makepl_arg PREFIX=/usr/local
o conf mbuildpl_arg --install_base /usr/local
o conf commit

This will have perl use two cores when compiling modules, and use Module::Signature if you install it, which seemed like a nice enough idea (at least if you have gpg installed; you may want to skip the check_sigs part, or install it first, if not -- via brew install gpgme for instance). And, most importantly, it'll put stuff in /usr/local at installation time, so perl will find it at invocation time. (Some day, that might just work right out of the box, in MacOS too.)

And that's it. Try it with cpan Module::Signature for instance, if you enabled that and you'll eventually end up with a fatter /usr/local/lib/perl5/site_perl. You are encouraged to drop other helpful tips here, especially if you know a thing or two about perl or cpan that I totally should have mentioned. Happy hacking! :-)

Musings on DNA, code and otherwise

2011-01-14T00:50:00.000-08:00

From college days, I recall microbiology as one of the more fascinating fields I never opted to pursue, but still developed a latent interest in. The evolved, highly complex and cross-connected execution context of biology, at the deepest levels where it meets (or indeed is) the realities of chemistry and physics speaks to my geek genes. There are so many levels of exciting, beautiful and interesting stories, events and developments woven together and buried in genome! Not to mention it is open source!

You want to grok something? UTSL! If you don't yet understand it, it's because you are not knowledgeable enough. Go learn! Experiment. Think hard. Figure it out. And then share what you learned, so others can reach longer, make it do more, be more efficient, go faster, taste better, or what have you. I think choosing mad computer science over mad genetics was more of a cultural thing gut draw towards openness; away from the lock-ins of lawyers of looming heavy corporations protecting the intellectual off-spring stemming from the money invested into powering their geeks' forays into the unknown, and toward the almost open eco system of what today amounts to the web and places like github, where people promiscuously and chaotically spin off of each others' work making stuff better together, for fun and profit.

But back to genetic code, as inspired by a read of an article on MS, schizophrenia and bipolar disorder as related to endogenous retroviruses (meaning stuff we all carry as common heritage, though not activated in most of us). One of the things we were taught in my classes was about the huge amount of introns (DNA between genes that is seemingly not active, as it isn't observed to ever get transcripted into proteins -- which is essentially how microbiological code execution happens) observed in all living organisms (about 98% of human DNA is non-coding DNA). Of course the depth level of the insights we absorbed were rather meagre, and although omnipresent easy-access Wikipedia was not yet around, it felt fairly obvious that the rest, at the very least, would end up having all sorts of other functions or effects under conditions and circumstances that would relate to it somehow. If you have ever researched (or exploited) a buffer overflow in some program, you probably know that wherever the program pointer accidentally ends up, whatever junk or data happens to be in that memory region is suddenly, for all intents and purposes (or despite them :-), is now code! Maybe it will do something interesting. Or nefarious. Probably nefarious, really.

Biology, chemistry and physics, of course, just add a gazillion more dimensions to "functions and effects of things", while also imposing a bus-load of peculiar constraints, and introducing still more environmental conditions from the rather chaotic virtual machine of physics, chemistry and biology (being three different zoom levels at which things happen to shoot your code in the foot) combined.

If you thought debugging code is hard, it's actually a delightful walk in the park, for the most part -- free of all sorts of concerns like the ethics of feeding useful test arrays into your debugger, that your test subjects can't keep dying all the time until you figure it out, and so on, and that we're probably not debugging software that co-evolved on a planetary scale, over a couple of billion years, but was probably caused by, at the very most, a couple of decades worth of ingenuity from probably less than thousands (or at most millions) of humans, all more or less acting with actual intents, often worthy of words like "design" and "intent". We're kind of lucky, that way! And oftentimes, the whole code base is even your very own damn fault! And you could even know full well what every line of your code is supposed to do, even though you might not yet have discovered all the things it should probably also do to work under the conditions it'll be subject to when unforgiving users run it. (Even yourself!)

I wonder how far into the future we have to go to see iLife speaking to the creative class doing justice to its name, providing us with the tools to be creative with the workings of micro- (or, for that matter, even macro-)organisms, in the same manner it's letting today's happy amateurs go wild with arts like graphics, music and video. Lots of people have rather rigid ideas about what amounts to art, science, work, life style, communication and so many other arbitrary things you can do for whichever reasons propel you, based, among other things, on rules, rulers or methods used in each field, motivations behind them, et cetera. Try breaking a few; you end up in far more interesting places. And make and bring friends, along the way!

Reading Chrome, Firefox and Opera extensions in Emacs

2011-01-09T01:06:00.001-08:00

If you're a fan of emacs' archive-mode that lets you load zip files and the like as it lets you load regular directories, and know that Firefox xpi, Chrome crx and Opera oex extensions are really just thinly wrapped zip files under the hood, you might expect Emacs to grok them, just the same. I did, and was a little surprised when it didn't plain work. (So maybe I'm running an old Emacs build?)

Anyway, I looked into the matter and did a little poking about to see what setup would do the trick. In the case of Firefox and Opera, it's really easy, as the xpi and oex "formats" only amount to changing the filename extension and making sure you have the right files in there -- so all you have to do is make sure that ("\\.xpi\\'" . archive-mode) exists in your auto-mode-alist, and ("\\.xpi\\'" . no-conversion) in your auto-coding-alist (there is a little snippet for your .emacs at the end of this post), and the corresponding oex variant.

In the Chrome case (which prepends a Chrome header with a format version, an RSA public key and signature from the extension author), besides the corresponding crx change for same variables, I also had to patch the lisp/arc-mode.el file lightly:

arc-mode.el.patch at pastebin

If you try loading a crx and find it already works, someone upstream already applied the same or a better patch than mine to your installation, but otherwise, read on. To find where it is on your system, type C-h archive-mode <return> and follow the archive-mode is a compiled Lisp function in `arc-mode.el'. link, apply the patch and M-x byte-compile-file the result again.

The .emacs snippet that makes it all kick in automagically:

.emacs snippet at pastebin

Note that for the Chrome extension case, you won't be able to edit and save any of the contents in place, as you can with plain zip (xpi, oex) files; the magic to remove/re-add public keys and signatures will be somebody else's late Saturday hack to complete. :) Enjoy!

Vintage Firefox

2010-12-26T16:13:00.000-08:00

When I occasionally need to check how far back a feature was introduced in Firefox, it usually turns out I only have the last few versions installed, and takes a while before I find where mozilla.org hit them, so here goes: a complete archive of the latest (mac -- tweak the urls or browse yourself for Linux, Windows or otherwise) release of every major version of Firefox from 0.8 to date (at current time of writing, 3.6.13 is the most recent Firefox released) off of ftp://ftp.mozilla.org/pub/mozilla.org/firefox/releases/, where mozilla.org hosts them, and the Firebug releases you need to do any useful web development work with them:

Firefox 4.0 beta 8 (Gecko 2 / Firebug 1.7X.0a7)
Note: as neither of is released yet, consider both links outdated when you read this (the Firebug xpi especially is in development, as hinted by the X)
Firefox 3.6.13 (Gecko 1.9.2 / Firebug 1.6.0 or 1.5.4)
Firefox 3.5.16 (Gecko 1.9.1 / Firebug 1.5.4)
Firefox 3.0.19 (Gecko 1.9 / Firebug 1.4.5)
Firefox 2.0.0.20 (Gecko 1.8.1 / Firebug 1.3.4b2, 1.2.0 or Firebug 1.0.5)
Firefox 1.5.0.12 (Gecko 1.8 / Firebug 1.05)
Firefox 1.0.8 (Gecko 1.7)
Firefox 0.9.3 (Gecko 0.9)
Firefox 0.8 (Gecko 0.8)

(Corresponding Gecko and Firebug versions, the second and third most messy to find Firefox entity, from the Gecko page on MDC and the install.rdf of the corresponding Firebug xpi:s archived at getfirebug.com.)

Never ever download and install Firefox from any other site! There is a huge, shady "pay per install" market of people bundling other people's software with malware (bot nets, typically); read all about it from Symantec (pdf) or Kevin Stevens (also pdf, the latter from Black Hat DC 2010). This goes for all executable programs you download and install on your machine; if you have no reason to trust the source, get it from a source you do trust.

Opera 11 extensions @ Add-on-Con

2010-12-12T00:40:00.000-08:00

I attended Add-on-Con this year again (I think I missed last year's), and figured I should sum up some of the interesting and/or useful stuff I picked up on while there. What I found most gratifying was the upcoming Opera (11) add-on API, which takes on the same approach as the Chrome and Safari herd, with the clean, HTML5 / postMessage / content script based design we have grown to expect, with optional background and popup pages. (Mozilla kind of wants to get there too, but does not expect to have anything remotely near it for at least another year, and I don't think it's even on any road maps yet.)

In Opera's current version, they don't go quite as far as does Chrome with its process separation, so an extension's popup can actually read from and write to localStorage from the same origin as its background page (which lets you simplify the message passing a little if you're not aiming for easy porting). Your content script runs in its own global object, but it has a window reference to the current window object, as the page sees it, and in the interest of not automatically leaking your script's guts into the page's global scope, it is not in the inheritance chain, so you actually have to use long references like window.document and window.localStorage to access them. (I tried assigning this.__proto__ = window;, which in theory should import all window properties in my content script's scope, and while that seemed to allow the former, it still failed on the latter, so I guess I hit unsupported stuff or beta bugs.)

To try it out, I made my github improved user script an Opera 11 extension. Run rake to build the zipped-up oex from the content script if you check out or fork the codebase, which puts the content script in includes/ (all scripts in includes/ with a // ==UserScript== block with Greasemonkey-style @include / @exclude rules run, not at the DOMContentLoaded event, but at page start, so you have to set your own DOMContentLoaded listener in a browser forked section if you want to execute under the same page conditions under Opera as in Firefox or a Chrome content script set to load at page ready).

Touting standards, Opera decided that a browser extension is essentially a Widget, and reused the W3C Widget standard they helped coin a while back, making its manifest sit in config.xml in the oex (a zip file) root directory (and emphasized that it also means you have to have an index.html next to it – the background page – but that it can be empty, if you like). For instance:

<?xml version="1.0" encoding="utf-8"?>
<widget xmlns="http://www.w3.org/ns/widgets" version="1.0.0-0">
  <name>Github improved</name>
  <icon src="icon.png"/>
  <author href="http://ecmanaut.blogspot.com/">Johan Sundström</author>
  <description xml:lang="en">Adds github changeset unfolding and other site improvements.</description>
</widget>

This also means that you can use Widget storage and config-based preference init, if you like. It indeed even seems to be the recommended way, as that gets you unlimited storage (actually: the same total cap Opera sets for itself globally), whereas your background page's localStorage gets subjected to the same arbitrary per-domain storage size cap that web sites are.

As for extension distribution, they of course have their own extension gallery you may want to submit your things to, and like all human-driven approval processes, they have a slightly anal process to get through first. Essentially, the less you can state about your add-on, the fewer opportunities you'll have to fail their test criteria; I was advised by the presenters not to add an add-on homepage, for example.

Currently, every submission attempt of yours in the approval process requires you to update your version number, so you can't aim for releasing a final 1.0 and treat all the attempts on the way there as release candidate 1 through N; my test add-on went through three revisions before coming out, and ended up named 1.0.0-0 in the end. It also was required to list change notes, that are reflected on the download page even when for an add-on that never existed before. Badly implemented good intentions also featured in the insistent numerous-stage wizard walking you through many steps every time (including screen shots, icons, et c), but there is at least the option to copy data from a previous version.

I think I may make Opera extensions of some of the extension-worthy hacks I do, but having tried the Chrome extension gallery's really smooth process, showing how it should be done, I don't think I'll have the patience to jump through Opera's hoops very often unless they improve notably.

To me, the bottom line is that Opera's add-on system doesn't make it prohibitively difficult to support Opera too, if you are already making an add-on for Chrome that does not rely on lots of deeper-integration Chrome API:s. It still surprises me that while they have the best technical user javascript implementation (and first too) a browser has ever had, they gave it the worst UI (a config switch listing a directory into which you yourself drop user scripts manually, after having modded them to add a DOMContentLoaded event listener to start the main part of the script) ever. Their js hooks still thorougly beat both Greasemonkey and Chrome, yet the lack of installation when visiting a *.user.js file locks that feature set in for only the really expert of browser users.

Comfy browser performance testing

2010-09-19T18:21:00.000-07:00

I have been poking around with some performance tests and profiling lately, first with Steve Souders' Browserscope, which gets the job done, at some convenience cost of setting up a publicly hosted web page and tying it to a Google account (that you might have to first create), and, for loading performance, different ways of organizing script tags, images, iframes, style sheets and so on, Cuzillion, by same author.

Today, I found and toyed around a bit with jsperf.com by Mathias Bynens, which does much of the boring setup job for you, when it comes to micro-benchmarking some snippet of javascript versus another, assuming the thing you want to test is synchronous, and doesn't involve complex preconditions. (I wonder what makes Array copying using a saved Array.prototype.slice significantly slower than the other more evenly performance-matched variants?) As should always be stated when mentioning benchmarks and optimization, this kind of micro-benchmarking is less relevant than finding your code's hot spots and picking better algorithms where relevant. That said, it's still academic fun to play with this kind of thing.

For this kind of small test, jsperf.com does all the Browserscope setup work for you, and lets others improve (fork) your test after the fact, adding versions you didn't think of yourself that do the same thing. I have a vague plan of trying to make it run some performance tests for a little code snippet I wrought up recently that does a deep copy of a nested (JSON:able) structure from a parent window where there may be crud on Object.prototype (the code should run in an iframe free of such), to benchmark vs a mere JSON.parse(JSON.stringify(taintedObject), which would also clean away the crud:

var _slice = [].slice, _toString = Object.prototype.toString;
function array(arrayish) {
  return _slice.call(arrayish, 0);
}

function isArray(obj) {
  return '[object Array]' === _toString.call(obj);
}

// Take a nested structure from a hostile window (presumably full of crap in its
// Object.prototype, et cetera) and return a cleaned-up version with this window
// object's (pure) Array and Object constructors.
function deepClone(obj) {
  if ('object' !== typeof obj) return obj;
  if (null === obj) return null;
  if (isArray(obj)) return array(obj).map(deepClone);
  var clean = {};
  for (var key in obj)
    if (obj.hasOwnProperty(key))
      clean[key] = deepClone(obj[key]);
  return clean;
}

What surprised me about the above code, when testing it on the kinds of payloads it would typically handle (data somewhat heavy on largeish strings), was that it actually was a bit faster than using the browser native JSON codecs, whereas, for larger inputs, native code always won out. Sometimes you just have to test stuff, to find out what wins.