Adequately Good

Replacing `setTimeout` Globally

2011-04-07T00:00:00-07:00

Sometimes, you might want to overwrite built-in global methods like setTimeout and setInterval. If you try, you might find that it’s much harder than you think to accomplish this in every browser, particularly if you ever want to find the originals again. After a lot of painful experimentation, I think I have a definitive solution that works in all browsers with minimal side-effects.

Failed Approaches

I started out with the simple approach:

setTimeout = function() {};
// or
window.setTimeout = function() {};

This seems to work in most browsers. However, Internet Explorer 8 and below are not appreciative of this technique. In the first case, IE will actually throw an exception saying "Object doesn't support this action". The second option works, but it will only affect the value of window.setTimeout, leaving plain old global setTimeout alone. This is workable, but not ideal. Time to look for another solution.

Another Attempt

After speaking with @jcummins about this, we thought about using the var keyword, and seeing if that would help. So that brings me to my next approach:

var setTimeout = function() {};

The good news is that this works across the board! Both setTimeout and window.setTimeout now reference my new function, in all browsers, and you can safely do any assignments needed, no exceptions thrown. But now the question is, where did the original setTimeout go? It’s not an easy thing to find.

You might try something like the following:

var temp = setTimeout,
    setTimeout = function() {};

Now, you might expect temp to contain the original setTimeout, but it unfortunately will come up undefined. This is due to JavaScript hoisting. I even tried var temp = window.setTimeout first, but the property on window was immediately hoisted on top.

So, I resolved to find another way to get at the original value. After some digging, I discovered a reference to the original setTimeout on the window’s prototype, which you can access at window.constructor.prototype.setTimeout. Alright! Things are looking good. Unfortunately, things quickly went downhill.

1. In most browsers, window is constructed by a function named "DOMWindow". That is, window.constructor.name === "DOMWindow". However, in Safari this is not the case, the constructor property instead references Object. We don’t have a way to access DOMWindow directly, so we can’t get to the prototype. Luckily, we can use the ECMAScript 5 __proto__ property, and find setTimeout at window.__proto__.setTimeout. So my catch-all became (window.__proto__ || window.constructor.prototype).setTimeout. 2. It turns out Opera has the same problem as Safari, and we can’t access the correct prototype at window.constructor.prototype. However, it also doesn’t seem able to access it at window.__proto__, leaving us low on options. More on that in a bit. 3. IE7 and below don’t have a constructor or a __proto__ property on window, and there doesn’t seem to be any other way to get direct access to the window’s prototype.

At this point, I declared searching for the original copies of setTimeout outside of the global scope a lost cause, and went back to the drawing board. You could, in theory, instantiate a new <iframe> and copy setTimeout from it, but I didn’t want to introduce that much overhead.

A Solution

At this point, I figured the best solution would be to circumvent JavaScript’s hoisting rules. As we know, hoisting occurs immediately after entering an execution context. So, to dodge it, we’d have to introduce a second execution context. You could do this easily in HTML:

<script>
  var temp = setTimeout;
</script>
<script>
  var setTimeout = function() {};
</script>

However, I was looking for a pure JS solution. So, after some soul-searching, I decided I’d pull out eval.

var temp = setTimeout;
eval("var setTimeout;");
setTimeout = function() {};

Done and done. The most flexible thing to do is just to quickly fix the inconsistency where IE doesn’t allow you to overwrite setTimeout directly, and then proceed to do as you need. Here’s some quick sample code, though you could easily adapt this better into your own project:

var __originals = {
  st: setTimeout,
  si: setInterval,
  ct: clearTimeout,
  ci: clearInterval
};

eval("var setTimeout, setInterval, clearTimeout, clearInterval;");

setTimeout = __originals.st;
setInterval = __originals.si;
clearTimeout = __originals.ct;
clearInterval = __originals.ci;

__originals = undefined;

This snippet will smooth out that inconsistency in Internet Explorer, and allow you to proceed with whatever overrides or replacements you need on those methods. No need to use var again in the future, so you can avoid the hoisting pains. I’ve tested this in IE6, 7, 8, and 9, as well as Chrome, Safari, Firefox 3/4, and Opera, all on Mac and Windows, and it’s rock-solid.

Update! A Better Solution

After further investigation, helped by many including @angusTweets and @kangax, I have fully uncovered the problem in IE, with an extremely simple and safe solution.

It turns out that the problem is not always present. For instance, open up a brand new, blank window in IE7 or IE8, and try the following in the JS console:

window.setTimeout = 1;
setTimeout; // 1
setTimeout = 2; // 2

But then, try this instead (again, in a fresh window):

setTimeout;
window.setTimeout = 1;
setTimeout; // {...}
window.setTimeout; // 1
setTimeout = 1; // Error

So what’s going on here? Well, I’ve come up with a solid theory.

Initially, the property setTimeout exists on the prototype of window, not on window itself. So, when you ask for window.setTimeout, it actually traverses one step on the prototype chain to resolve the reference. Similarly, when you ask for setTimeout, it traverses down the scope chain, then to window, then down the prototype chain to resolve the reference.

I suspect that IE has a built-in optimization where it automatically caches the resolution of implied globals that are discovered all the way down on the prototype of the global object. It would have good reason to do so, as these are commonly requested references, and traversing that chain is costly. However, it must be setting this reference as read-only, since it’s just a caching optimization. This has the unfortunate side-effect of causing an exception to be thrown when attempting to assign to the reference by using it as an lvalue. The only way to kill this new reference is by using var in the global scope, but this puts us in the hoisting problem. What to do?

Assuming this theory is correct, there’s actually a simple solution. To prevent this “optimization”, we simply need to move the reference one step up the chain, directly onto window. Luckily, this is quite easy:

window.setTimeout = window.setTimeout;

That’s it! Assuming you’ve done this before any reference to setTimeout on its own, this will move the reference and completely avoid this problem. You may now safely assign to setTimeout without using var. This works because window.setTimeout, when used as an lvalue (on the left side of the assignment), does not walk the prototype chain, but on the right side, it does. So this will always pull a property out of the prototype chain and put it right on the object. Incidentally, IE exhibits the same problem with every other property on the window prototype, and the same solution will fix them. You can easily find the other properties, but a few of them are alert, attachEvent, scrollTo, blur, and many others.

This solution has been tested in all browsers, has no side effects, and is wonderfully simple. It’s always nice when things work out that way.

Thoughts on the Hashbang

2011-02-11T00:00:00-08:00

There’s been a lot of discussion this week about the “hashbang”, that ugly little bit you find in the middle of URLs like this one: http://twitter.com/#!/ded/status/18308450276. I wanted to provide a rebuttal to the arguments that the hashbang is bad for the Web, based on a lot of discussions we’ve been having inside Twitter since the #newtwitter project began last summer, and have continued right up until today.

It’s Not About the Hashbang

The hashbang is in the unfortunate position of being the messenger of a big change that’s been slowly occurring on the Web in the past few years, and will only continue to pick up steam: many Web domains are now serving desktop-class applications via HTTP, instead of traditional Web sites. For instance, twitter.com is no longer a collection of Web pages that represent a Web site, but is simply an application that you happen to launch by pointing a browser at http://twitter.com*. This has many wide-reaching implications, and the hashbang is merely a side-effect. In this way, the hashbang is an easy-to-hate straw-man, whereas the real debate to be had is about this shift towards applications.

Why Applications on the Web?

Services such as Twitter, Flickr, Facebook, and others have every reason to build rich applications. An application allows an experience that is faster, more seamless, and much better for users than traditional Web sites. However, we do not have to choose to host our apps on the Web, and could easily use other options, such as building a native OS application, or using another cross-platform platform such as Adobe AIR. We all choose Web applications however, because the Web is ubiquitous, well-understood, and mostly superior to other application delivery options.

We get URLs in our Web applications as a legacy of Web sites, and this feature is unique to this class of applications. The power of URLs bring benefits like bookmarks, sharing of application states for free, and search engine indexing. Unfortunately, supporting a standard URL structure for this type of application is not possible in current browsers, without using the URL hash. So, we use the URL hash to take advantage of this sort of thing, but not because we believe our application to be a traditional Web site. We do lose indexability and some other benefits of HTTP we may otherwise have, but these are not losses to our application, since we did not have to use a URL at all. Google has thankfully provided an implementation for crawling applications, based on the hashbang, which brings search engine indexing to the URL hash, and is easy enough to implement.

The Tradeoff

We’ve made a tradeoff, however, in making twitter.com into an application using the hash, which is that it now cannot be both an app and a site at the same time. This capability might not sound compelling, but it is. For instance, browsers without JavaScript cannot access content that has been shared using the URL hash, since that is a marker to the JS app, not to a Web page. In a perfect world, we would have been able to use the original URL structure with the hashes for our application, and served both the site and the app simultaneously. This would be the absolute best thing for users of all kinds, and this debate would not be happening.

There is hope on the horizon, in the form of the HTML5 History API. This new browser feature will allow developers to change the entire URL path and query string without incurring a page refresh. By using this, we could drop the hash, and get all the benefits of traditional Web sites with all the benefits of a desktop-class application. Support for this has been in Chrome and Safari for some time (albeit with a bug we found (now fixed)), and will arrive in Firefox 4. Internet Explorer currently has no plans to implement this in IE9, which is a shame. Had it not been for the bug we found, we would have shipped #newtwitter with HTML5 History on day one (and in fact, the integration is already built).

The Hashbang is a Good Thing

It’s not perfect. It’s not pretty (despite what Google says). But the hashbang is useful. Users benefit from the trend towards full-fledged applications, with increased speed and functionality. URLs are not necessary in this context, but we provide them (via the hash) because they bring tangible benefits for applications. The hashbang also brings a tangible benefit. Someday, we will hopefully be rid of the thing, when HTML5 History (or something like it) is standard in all browsers. The Web application is not going anywhere, and the hash is a fact of life.

* Technically, twitter.com does still host a Web site. For instance, twitter.com/jobs is a normal Web page, as is twitter.com/bcherry when viewed logged-out. We have, however, begun serving an application to logged-in visitors.

Saner HTML5 History Management

2010-07-25T00:00:00-07:00

Hashchange

This event is quite simple. Whenever the window.location.hash property changes, by following a link, setting the property, editing the URL bar, or using back/forward to move through browser history, the “hashchange” event is fired from the window. Using it is really easy:

window.onhashchange = function() {
	alert("hash changed!");
};
window.location.hash = Math.random(); // alerts "hash changed!"

This feature is implemented in recent versions of all major browsers. In older browsers like Internet Explorer 6 and 7, you can easily provide it by polling the hash property on an interval, and manually firing an event when it changes. This is easy to build into a jQuery plugin, which Ben Alman did in the robust jquery.hashchange.js plugin.

History Management

This feature is a bit more complex. Browsers that support it add a window.history object, with the following properties:

window.history.back() and window.history.forward(), which provide programmatic interfaces to browser back and forward functions.
window.history.pushState(stateObj, title, url). This method pushes a new entry into the browser history, which then becomes the browser’s current state. You can provide any JSON-stringifiable object to send with it, and the browser will provide that object again when you navigate to that point (more on that in a bit). More importantly, if you provide a URL, the browser will change the URL displayed in the address bar, without reloading the page. The new URL must be on the same domain, but you can change the rest of it, which is the window.location.pathname and window.location.hash. Changing the URL in this way will not trigger a “hashchange” event, though.
window.history.replaceState(stateObj, title, url). This is just like window.history.pushState, except that the current browser state is removed from the history, so you cannot hit “back” to return to it.
window.onpopstate. This event is fired whenever a state object is removed from the browser history, which occurs on browser “back” or “forward”. State objects are persisted on the user’s hard disk between sessions, which is a nice feature. The object passed into a call to pushState or replaceState is provided as the state property on the event object in the “popstate” event.

This feature is implemented in the latest versions of WebKit, which includes Safari and Chrome. Additionally, the Firefox 4 betas include support for this.

What’s the Use Case?

The new history management stuff is very promising, because it allows a web application to live across many physical URLs, but be run in a single instance. This is important for certain kinds of applications, where using hashes is not universally suitable.

For instance, at Twitter, we currently update your URL hash as you navigate around the application, to make bookmarkable pages like http://twitter.com/#replies. However, we force a full page load for certain pages, most notably profile pages (e.g. http://twitter.com/bcherry) and permalink pages (e.g. http://twitter.com/bcherry/status/18966802499). This is so that those URLs can be copied from the address bar and posted on the web.

We want to make sure that users without JavaScript and search engine bots crawling links to our site will get the correct page from the server (since the browser does not send a hash along to the server). This would not be possible if those URLs used hashes. Unfortunately, this means the application is slower, because a full page load is needed going into and out of those locations.

This is where HTML5 History Management could be useful.

So What’s the Problem?

Unfortunately, the existing implementation of history management is not useful, and not in the spirit of the web.

Our web applications should be built to respond to a URL. Both the client and server versions of an application should understand a shared URL structure, and know how to present the same page to the browser that reflects that URL.

Allowing developers to store extra state information in the browser history is missing the point. The only thing stored in history should be a URL, and the browser can associate a title with it if it chooses.

This is RESTful design, mirrored on client and server. Modern browsers can support changing the URL without reloading the page from the server, and older ones can continue to hit the server every time.

In this way, we can build applications that degrade correctly in older browsers, and when viewed by bots, while providing a faster experience for users with modern browsers.

Enter “pathchange”

Both “hashchange” and pushState/”popstate” should be replaced with “pathchange”, which is an event that fires when the URL changes in any way. This event does not provide any information, the application should inspect the current URL to discover the state it should enter. Relative links within a page should not force page reloads, they should instead just trigger the “pathchange” event.

It turns out that it’s possible to implement this event in modern browsers now, based on the features they already have. Here’s how:

Listen to “hashchange”, and trigger “pathchange” when it occurs
Poll the hash in browsers without “hashchange” support, and trigger “hashchange”, which triggers “pathchange”
With history support, listen to “popstate”, and trigger “pathchange” when it occurs
With history support, intercept all relative links when they are clicked, and prevent normal navigation. Call window.history.pushState(null, null, href) instead, and trigger a “pathchange”.
Provide a helper function to make navigation to new URLs using window.history.pushState, when supported, easy.

I’ve implemented all of this as a jQuery plugin that is quite easy to use:

$(function() {
	$.pathchange.init(); // setup event listeners, etc.
	$(window).pathchange(function() {
		respondToUrl();
	}).trigger("pathchange");

	$.pathchange.changeTo("/foo");
});

I’ve also created a demo page that presents A Saner HTML5 History App that uses jquery.pathchange.js under the hood. Check it out in various browsers to see the HTML5 magic at work, and be sure to use your browser “back” and “forward” buttons, and reload the page a few times.

That’s my take on the HTML5 history features. It’s unfortunate that what the browsers are implementing is not what we really need, but it’s encouraging that they do provide enough to implement what we do really need. Let me know in the comments if you agree, disagree, or have questions about my approach.

___Note Number One: It's also worth pointing out that I discovered a [serious bug](https://bugs.webkit.org/show_bug.cgi?id=42940) in WebKit's implementation of history management while working on this today. In short, the "popstate" event is often lost when the network is occupied, which makes little sense. [Here's a demo page with a reproducible case](http://www.bcherry.net/playground/pushstate) that I threw together. It fires off a request to download an image which takes 1s on every "popstate", which means hitting "back" more than once every second leads to lost history entries and an application that gets out of sync with the URL. You could work around this by polling the URL in addition to listening to "popstate", but it's not a good workaround. Until this is fixed, you'll have to be wary of this if you ship this feature to your users, and it probably is not suitable for very complex AJAX apps. Firefox 4 does not have the same problem.______Note Number 2: This article was originally published around 4am PST on July 26th. The author published a revision around 8pm PST the following day, to make it a little less incohorent and a little more useful. Luckily, the author uses Git to prepare Markdown-formatted articles, so you can [view the diff](http://github.com/bcherry/adequatelygood/commit/eb688c7809e8d5f61f9ed12442d3a578d46fab97) if you'd like to find out what changed.___

Writing Testable JavaScript

2010-07-08T00:00:00-07:00

The engineering culture at Twitter requires tests. Lots of tests. I haven’t had formal experience with JavaScript testing before Twitter, so I’ve been learning a lot as I go. In particular, a number of patterns I used to use, write about, and encourage have turned out to be bad for writing testable code. So I thought it would be worthwhile to share a few of the most important principles I’ve developed for writing testable JavaScript. The examples I provide are based on QUnit, but should be just as applicable to any JavaScript testing framework.

Avoid Singletons

One of my most popular posts was about using JavaScript Module Pattern to create powerful singletons in your application. This approach can be simple and useful, but it creates problems for testing, for one simple reason: singletons suffer state pollution between tests. Rather than creating your singletons as modules, you should compose them as constructable objects, and assign a single, default instance at the global level in your application init.

For example, consider the following singleton module (contrived example, of course):

var dataStore = (function() {
	var data = [];
	return {
		push: function (item) {
			data.push(item);
		},
		pop: function() {
			return data.pop();
		},
		length: function() {
			return data.length;
		}
	};
}());

With this module, we may wish to test the foo.bar method. Here’s a simple QUnit test suite:

module("dataStore");
test("pop", function() {
	dataStore.push("foo");
	dataStore.push("bar")
	equal(dataStore.pop(), "bar", "popping returns the most-recently pushed item");
});

test("length", function() {
	dataStore.push("foo");
	equal(dataStore.length(), 1, "adding 1 item makes the length 1");
});

When running this test suite, the assertion in the length test will fail, but it’s not clear from looking at it why it should. The problem is that state has been left in dataStore from the previous test. Merely re-ordering these tests will cause the length test to pass, which is a clear red flag that something is wrong. We could fix this with setup or teardown that reverts the state of dataStore, but that means that we need to constantly maintain our test boilerplate as we make implementation changes in the dataStore module. A better approach is the following:

function newDataStore() {
	var data = [];
	return {
		push: function (item) {
			data.push(item);
		},
		pop: function() {
			return data.pop();
		},
		length: function() {
			return data.length;
		}
	};
}

var dataStore = newDataStore();

Now, your test suite will look like this:

module("dataStore");
test("pop", function() {
	var dataStore = newDataStore();
	dataStore.push("foo");
	dataStore.push("bar")
	equal(dataStore.pop(), "bar", "popping returns the most-recently pushed item");
});

test("length", function() {
	var dataStore = newDataStore();
	dataStore.push("foo");
	equal(dataStore.length(), 1, "adding 1 item makes the length 1");
});

This allows our global dataStore to behave exactly as it did before, while allowing our tests to avoid polluting each other. Each test owns its own instance of a DataStore object, which will be garbage collected when the test completes.

Avoid Closure-based Privacy

Another pattern I used to promote is real private members in JavaScript. The advantage is that you can keep globally-accessible namespaces free of unnecessary references to private implementation details. However, overuse of this pattern can lead to untestable code. This is because your test suite cannot access, and thus cannot test, private functions hidden in closures. Consider the following:

function Templater() {
	function supplant(str, params) {
		for (var prop in params) {
			str.split("{" + prop +"}").join(params[prop]);
		}
		return str;
	}

	var templates = {};

	this.defineTemplate = function(name, template) {
		templates[name] = template;
	};

	this.render = function(name, params) {
		if (typeof templates[name] !== "string") {
			throw "Template " + name + " not found!";
		}

		return supplant(templates[name], params);
	};
}

The crucial method for our Templater object is supplant, but we cannot access it from outside the closure of the constructor. Thus, a testing suite like QUnit cannot hope to verify that it works as intended. In addition, we cannot verify that our defineTemplate method does anything without trying a .render() call on the template and watching for an exception. We could simply add a getTemplate() method, but then we’d be adding methods to the public interface solely to allow testing, which is not a good approach. While the issues here are probably just fine in this simple example, building complex objects with important private methods will lead to relying on untestable code, which is a red flag. Here’s a testable version of the above:

function Templater() {
	this._templates = {};
}

Templater.prototype = {
	_supplant: function(str, params) {
		for (var prop in params) {
			str.split("{" + prop +"}").join(params[prop]);
		}
		return str;
	},
	render: function(name, params) {
		if (typeof this._templates[name] !== "string") {
			throw "Template " + name + " not found!";
		}

		return this._supplant(this._templates[name], params);
	},
	defineTemplate: function(name, template) {
		this._templates[name] = template;
	}
};

And here’s a QUnit test suite for it:

module("Templater");
test("_supplant", function() {
	var templater = new Templater();
	equal(templater._supplant("{foo}", {foo: "bar"}), "bar"))
	equal(templater._supplant("foo {bar}", {bar: "baz"}), "foo baz"));
});

test("defineTemplate", function() {
	var templater = new Templater();
	templater.defineTemplate("foo", "{foo}");
	equal(template._templates.foo, "{foo}");
});

test("render", function() {
	var templater = new Templater();
	templater.defineTemplate("hello", "hello {world}!");
	equal(templater.render("hello", {world: "internet"}), "hello internet!");
});

Notice that our test for render is really just a test that defineTemplate and supplant integrate correctly with each other. We’ve already tested those methods in isolation, which will allow us to easily discover which components are really breaking when tests of the render method fail.

Write Tight Functions

Tight functions are important in any language, but JavaScript presents its own reasons to do so. Much of what you do with JavaScript is done against global singletons provided by the environment, and which your test suite relies on. For instance, testing a URL re-writer will be difficult if all of your methods try to assign window.location. Instead, you should break your system into its logical components that decide what to do, then write short functions that actually do it. You can test the logical functions with various inputs and outputs, and leave the final function that modifies window.location untested. Provided you’ve composed your system correctly, this should be safe.

Here’s an example URL rewriter that is not testable:

function redirectTo(url) {
	if (url.charAt(0) === "#") {
		window.location.hash = url;
	} else if (url.charAt(0) === "/") {
		window.location.pathname = url;
	} else {
		window.location.href = url;
	}
}

The logic in this example is relatively simple, but we can imagine a more complex redirecter. As complexity grows, we will not be able to test this method without causing the window to redirect, thus leaving our test suite entirely.

Here’s a testable version:

function _getRedirectPart(url) {
	if (url.charAt(0) === "#") {
		return "hash";
	} else if (url.charAt(0) === "/") {
		return "pathname";
	} else {
		return "href";
	}
}

function redirectTo(url) {
	window.location[_getRedirectPart(url)] = url;
}

And now we can write a simple test suite for _getRedirectPart:

test("_getRedirectPart", function() {
	equal(_getRedirectPart("#foo"), "hash");
	equal(_getRedirectPart("/foo"), "pathname");
	equal(_getRedirectPart("http://foo.com"), "href");
});

Now the meat of redirectTo has been tested, and we don’t have to worry about accidentally redirecting out of our test suite.

__Note__: There is an alternative solution, which is to create a `performRedirect` function that does the location change, but stub that out in your test suite. This is a common practice for many, but I've been trying to avoid method stubbing. I find basic QUnit to work well in all situations I've found so far, and would prefer to not have to remember to stub out a method like that for my tests, but your case may differ.

Write Lots of Tests

This is a no-brainer, but it’s important to remember. Many programmers write too few tests because writing tests is hard, or lots of work. I suffer from this problem all the time, so I threw together a little helper for QUnit that makes writing lots of tests a lot easier. It’s a function called testCases which you call within a test block, passing a function, calling context, and array of inputs/outputs to try and compare. You can quickly build up a robust suite for your input/output functions for rigorous testing.

function testCases(fn, context, tests) {
	for (var i = 0; i < tests.length; i++) {
		same(fn.apply(context, tests[i][0]), tests[i][1],
			tests[i][2] || JSON.stringify(tests[i]));
	}
}

And here’s a simple example use:

test("foo", function() {
	testCases(foo, null, [
		[["bar", "baz"], "barbaz"],
		[["bar", "bar"], "barbar", "a passing test"]
	]);
});

Conclusions

There is plenty more to write about testable JavaScript, and I’m sure there are many good books, but I hope this was a good overview of practical cases I encounter on a daily basis. I’m by no means a testing expert, so please let me know if I’ve made mistakes or given bad advice.

Spying Constructors in JavaScript

2010-05-12T00:00:00-07:00

When writing unit-tests for code, a common technique is spying, where you set expectations on a method’s invocation, run some code, and verify that the method was invoked as expected. This is pretty straightforward. Here’s a simple example using JsMockito:

function foo(a) { return a; }
foo = spy(foo);
foo(1);
verify(foo)(1); // verified!
verify(foo)(2); // never run

Here, we’re spying on the foo method, and checking that it was invoked at least once with the parameter 1, and once with the parameter 2. As it turns out, this spy method does not work well with JavaScript constructors, in JsMockito, Jasmine, or many other testing frameworks. The basic problem is that the prototype is not transferred appropriately, so code like this will fail:

function Foo(a) {
	this.a = a;
}
Foo.prototype = {
	bar: function () {
		console.log(this.a);
	}
};

var f = new Foo(1);
f.bar(); // 1

Foo = spy(Foo);
var g = new Foo(1);
g.bar(); // error

verify(Foo)(1); // not reached

It turns out it’s really easy to write a constructor-safe spying function, and it doesn’t even take very many lines of code.

function spy(F) {
	function G() {
		var args = Array.prototype.slice.call(arguments);
		G.calls.push(args);
		F.apply(this, args);
	}

	G.prototype = F.prototype;
	G.calls = [];

	return G;
}

This spy function works just like the one in JsMockito, but it doesn’t fail with constructors. For completeness, here’s an implementation of a simple verify function:

function verify(F) {
	return function () {
		var args = Array.prototype.slice.call(arguments),
			i,
			j,
			call,
			count = 0,
			matched;

		for (i = 0; i < F.calls.length; i += 1) {
			call = F.calls[i];
			matched = true;
			for (j = 0; j < args.length; j += 1) {
				if (args[j] !== call[j]) {
					matched = false;
					break;
				}
			}
			if (matched) {
				count += 1;
			}
		}

		return count > 0;
	};
}

It would be easy to extend this verify implementation to allow more types of verify like .once() or .never(), working off the count variable.

And that’s it! Here’s an example of code that will work with this spy implementation:

function Foo(name, id) {
	this.name = name;
	this.id = id;
}

Foo.prototype = {
	log: function () {
		console.log("Foo %o:%o", this.id, this.name);
	}
};

var f = new Foo("test", 1);
f.log();

Foo = spy(Foo);

var f2 = new Foo("spied", 2);
f2.log();

console.log("verify Foo(\"spied\", 2): %o", verify(Foo)("spied", 2));
console.log("verify Foo(\"something\", 2): %o", verify(Foo)("something", 2));

var baz = {
	spam: function (a) {
		console.log("calling baz.spam(%o), this.other=%o", a, this.other);
	},
	other: 10
};

baz.spam = spy(baz.spam);

baz.spam(1);
console.log("verify baz.spam(1)", verify(baz.spam)(1));
console.log("verify baz.spam(2)", verify(baz.spam)(2));

The other neat thing is that, so long as you’re not trapping stale references to the original constructor function before it got spied, JavaScript’s instanceof operator should work just fine:

function F() {}
F = spy(F)
new F() instanceof F; // true

You can find the complete code (and a bit more) for this excercise at www.bcherry.net/playground/spying-constructors. I hope this was informative. I think I’ll probably end up either contributing a patch to JsMockito with this, or building my own bare-bones set of mocking/spying functions for use with QUnit.

___P.S. It's been some time since I've updated, but I'm hoping this will be the first of many new, interesting JavaScript posts inspired by the work I'm doing at Twitter with @bs, @hoverbird, @ded, and @dsa.___

Debugging Closures and Modules

2010-04-13T00:00:00-07:00

The most common complaint with using closures to keep private variables in JavaScript is that it makes debugging harder. This complaint definitely holds water, and the loss of easy debugging and inspection using the Module Pattern is a serious concern. This is where one of JavaScript’s non-standard but well-supported features comes in: the debugger statement. In this article, I’ll introduce this statement, and show how I use it to solve this deficiency in closure-based code.

Intro to `debugger`

The debugger statement is not standardized, but is provided by all implementations of JavaScript. It essentially acts like a programmatic breakpoint. When popular JavaScript debuggers like Firebug or the IE Developer Tools execute a debugger; statement, execution stops and the script debugger opens up, just like any breakpoint. Here’s a simple example:

function foo() {
	var a = 1;
	debugger; // execution stops until manually resumed
	a = 2;
}
foo();

In the above, we’ll be able to inspect the local variables when our debugger pauses execution, and we’ll see that a is still 1. After resuming execution, the next line will execute and a will be assigned the value 2.

Debugging Modules

The great thing is that execution stops at the debugger statement, complete with the local call stack from the point of execution. This provides a really nice ability to pair with the module pattern, or any other closure-based privacy pattern. Let’s take a simple module, but add extra capabilities to it by exposing a public .debug() method.

var foo = (function () {
	var privateVar = 10;
	return {
		publicFunc: function () {/*...*/},
		debug: function () {
			debugger;
		}
	};
}());

Now we can call foo.debug() from the Firebug console, and the closure of the anonymous constructor is opened up for inspection. By looking at the call stack in our debugger, we can inspect the scope chain for the private state, which will be in local variables. Checking the value of privateVar will be quite easy.

Notice that it’s very important for the .debug() property to be created and assigned during the normal construction. Assigning it after-the-fact will not provide the same functionality, because the local call stack will not contain the anonymous constructor. This is an unfortunate limitation, but there isn’t a way around it.

Safety and Configuration

You might not want to make it so easy to stop program execution by triggering the .debug() function on one of your modules, or you might want to disable this functionality. One approach is to ship a version of your code without this property at all, but that’s difficult to accomplish in some applications. An easier method would be to do some extra checks in the .debug() method, before firing the debugger statement.

return {
	debug: function () {
		if (DEBUG_MODE) {
			debugger;
		}
	}
};

Or, perhaps check for a specific debugger first:

return {
	debug: function () {
		if (DEBUG_MODE && "firebug" in console) {
			debugger;
		}
	}
};

___Note___: I'm assuming you've already configured a global flag named `DEBUG_MODE` in your application.

Methods like this will allow you to provide finer-grained control over when and where your debugger statement runs. If you’re building in a lot of logic, it makes sense to write a global helper function for this, but you’ll have to be careful to preserve the right call stack:

function DEBUG() {
	if (DEBUG_MODE && "firebug" in console) {
		debugger;
	}
}

var foo = (function () {
	return {
		debug: function () {
			DEBUG();
		}
	};
}());

Here we’ll keep our callstack in tact, except the one we care about will be two deep instead of one deep.

Problems With Minifiers

I tried running code with such a .debug() method through the YUI Compressor, and got a disappointing result:

[ERROR] 4:11:identifier is a reserved word

It seems that the YUI Compressor doesn’t know about the debugger statement, which is understandable since it’s non-standard. However, I didn’t want to let this defeat my attempts, so a co-worker and I came up with a workaround. Instead of using a raw debugger statement, we put it behind eval, like this:

function debug() {
	eval('debugger;');
}

Now YUI no longer complains, and the code works just great. However, there are a few caveats here:

Using eval is generally considered dangerous and bad practice. Be sure you understand the implications, and are willing to use this workaround anyways. I think this is a reasonable decision to make, for the intended purpose, but ensure you’ve at least put some thought into it.
JSLint will complain at you. This isn’t such a big deal since it would be complaining about the raw debugger statement anyways.

Conclusions

So that’s my technique for effectively debugging inside closures, especially when using the Module Pattern. I’m not saying this is a great technique for all uses or users, but it works well for me, and nicely solves the most common complaint developers have about the Module Pattern. I’d love to hear alternatives or reasons why my technique is dangerous and should be avoided, so leave a comment!

Object-to-Primitive Conversions in JavaScript

2010-03-30T00:00:00-07:00

Like most object-oriented programming languages, JavaScript provides built-in ways to convert between objects and primitive values, by way of the special toString and valueOf methods. This article will cover the basics of these methods, but then dive into the details of how this stuff really works, bad stuff, performance, and browser support.

Types and Primitives

To understand this article, you’ll need to understand the difference between primitive and non-primitive values in JavaScript. There are 5 primitive types, which are associated with the various primitive values.

Null: The value null.
Undefined: The value undefined.
Number: All numbers, such as 0 and 3.14. Also NaN, and Infinity.
Boolean: The values true and false.
String: All strings, such as "foo" and "".

All other values are non-primitive, including arrays, functions, and plain old objects. For completeness, here are the results of the typeof operator, applied to these values:

typeof null; // "object"
typeof undefined; // "undefined"
typeof 0; // "number" (`typeof NaN` is also "number")
typeof true; // "boolean"
typeof "foo"; // "string"
typeof {}; // "object"
typeof function () {}; // "function"
typeof []; // "object"

__Note__: `typeof null` should _not_ be `"object"`. This is a mistake from the first versions of JavaScript, but it's really too late to fix. A more sensible type would have been `"null"`, but this is what we're stuck with.

If you’ve got that down, then we’re ready to move on to the basics of toString and valueOf. If you’re already familiar with the basics, feel free to skip ahead to “How it Works”.

Basic Usage

We’ll be using a simple example population object that holds a country name and a population. Lets code that up.

function population(country, pop) {
	return {
		country: country,
		pop: pop
	};
}

var america_pop = population("USA", 350e6);
var mexico_pop = population("Mexico", 200e6);
var canada_pop = population("Canada", 200e6);

alert(america_pop); // [object Object]

var north_america_pop = america_pop + mexico_pop + canada_pop;

alert(north_america_pop); // [object Object][object Object][object Object]

This works, but the calls to alert are not very useful. What we’d really like is for the first alert to show '[Population "USA" 350000000]' and the second to show "750000000". So, let’s code that up next.

toString

All objects inherit the method toString from Object.prototype, which returns "[object Object]". However, we can easily override this by providing toString as a method of our object, or its prototype. In this example, we’ll attach it directly to each instance, but feel free to use the prototype instead.

function population(country, pop) {
	return {
		country: country,
		pop: pop,
		
		toString: function () {
			return "[Population " + 
				"\"" + country + "\" " +
				pop +
			"]";
		}
	}
}

var america_pop = population("USA", 350e6);
alert(america_pop); // [Population "USA" 350000000]

__Note__: I'm using __closure__ on the `country` parameter, rather than using `this.country`. This only works due to how the constructor is set up. If you placed `toString` on the prototype, you would need to use `this.country`.

valueOf

All JavaScript objects also inherit the method valueOf from Object.prototype. By default, this method simply returns the object itself, but is generally overridden to convert an object to a Number, or another primitive value, so it can be used by operators like +. We can do the same thing as above to complete our basic example.

function population(country, pop) {
	return {
		country: country,
		pop: pop,
		
		toString: function () {
			return "[Population " + 
				"\"" + country + "\" " +
				pop +
			"]";
		},
		
		valueOf: function () {
			return pop;
		}
	};
}

var america_pop = population("USA", 350e6);
var mexico_pop = population("Mexico", 200e6);
var canada_pop = population("Canada", 200e6);

alert(america_pop); // [Population "USA" 350000000

var north_america_pop = america_pop + mexico_pop + canada_pop;

alert(north_america_pop); // 750000000

Here we’ve defined the valueOf function of our population object to return the population, which should be a Number.

How It Works

As with most things in JavaScript, the process by which toString gets called is not as simple as you’d think. Let’s explore what happens when alert(america_pop) is called.

alert calls GetValue on the reference. This returns the object it points at.
alert calls ToString on the value (this is not the same as the object’s toString)
ToString calls ToPrimitive on the value, passing the hint String.
ToPrimitive calls the object’s internal [[DefaultValue]] method with the hint String.
[[DefaultValue]] calls the toString property of the object, with the object as this.
The result of toString is a primitive value, so it is returned, all the way up the chain to the ToString method.
Since the result is of type String, ToString returns all the way to alert.
alert displays the value.

While this is a lot, it’s pretty straightforward. However, he key mechanism that needs more explaining is the ToPrimitive function. This function is used to take an arbitrary value and get a corresponding primitive value instead. If the input is already a primitive value then the value will be returned without conversion. However, if the value is non-primitive, then it will call the internal [[DefaultValue]] method to find a default value for the object.

[[DefaultValue]] is an internal property of every object. It’s a method that takes an optional hint, which should be either Number or String. If a hint is not provided, it will default to Number unless the object is a Date, in which case it defaults to String (this is silly). After this has been figured out, it will call toString and valueOf, in order, to find a primitive value. This is where the hint comes into play. If the hint is Number, then valueOf will be tried first, but if it’s String then toString will be tried first. Here’s the ensuing process:

If the first method exists, and is callable, call it and get the result, otherwise skip to 3.
If the result of 1 is a primitive, return it.
If the second method exists, and is callable, call it and get the result, otherwise skip to 5.
If the result of 3 is a primitive, return it.
Throw a TypeError exception.

The value that is returned by [[DefaultValue]] is guaranteed to be primitive. If it was not, a TypeError would have been thrown. This also implies that toString and valueOf should return primitives on order to be useful in this context.

Confusion About the + Operator

Here’s an example with a (possibly) unexpected result:

var foo = {
	toString: function () {
		return "foo";
	},
	valueOf: function () {
		return 5;
	}
};

alert(foo + "bar"); // 5bar
alert([foo, "bar"].join("")); // foobar

In this context, we’re using the + operator to do string concatenation. But, foo was not converted to a string using toString, it was turned into a number using valueOf, then used for string concatenation. This probably isn’t what we want, but it is how it works. It’s a side-effect of the overloading of the + operator for arithmetic and string concatenation. The +operator has a well-defined process:

Evaluate the left-hand side, and get the value.
Evaluate the right-hand side, and get the value.
Call ToPrimitive on both the left-hand and right-hand sides (without a hint)
If either primitive value is a String, then skip to 7.
Call ToNumber on both values.
Return the sum of the values.
Call ToString on both values.
Return the concatenation of both values.

Since no hint is passed to the ToPrimitive calls, the hint will be defaulted to Number (unless it’s a Date, which defaults to String). This means that our valueOf function will be called, instead of toString. It’s not until after the primitive values are retrieved that the interpreter decides whether it is going to do string concatenation or arithmetic. That’s why our example above returns "5bar" instead of "foobar".

Bad Stuff

There is one really bad feature of all this, which is that ToPrimitive does not enforce any type-checking on the return values, other than that they are primitive. This means you can write code like this:

var foo = {
	toString: function () {
		return 5;
	},
	valueOf: function () {
		return "foo";
	}
};
alert(foo.toString() + 1); // 6 (bad!)
alert(foo + 1); // "foo1" (no good!)
alert(+foo); // NaN (the worst!)

The valueOf method can be forgiven for not type-checking, because it is more generic. You’d expect it to be able to return any suitable primitive value. However, the toString method has no such excuse. This is simply a bad feature. You can, of course, mitigate by using String(foo) instead of foo.toString(), which will call toString and then convert that result to a string. But you should not have to do this, or worry about this. Please do not make objects with toString methods that do not return strings.

How About Performance?

After understanding the complexity that goes into these implicit conversion, I got curious about how that affects performance. So I decided to test the time it takes to perform an [].join(obj) over 1,000,000 iterations in the major browsers. I did one test with the object being implicitly cast to a string, and one where I called the toString method manually (i.e. [].join(obj.toString())). As expected, the explicit call was faster in most cases.

Firefox 3.6.2: 874ms vs. 320ms - almost 3x faster.
Chrome 5: 94ms vs. 47ms - 2x faster.
Opera 10.50: 155ms vs 182ms - a little slower.
Safari 4: 409ms vs 280ms - almost 2x faster.
Internet Explorer 8: 2856ms vs 2786ms - about the same.
Internet Explorer 9 (preview): 645ms vs 633ms - about the same.

___Note 1___: The Firefox, Chrome, Opera, and Safari tests were all run on a Macbook Pro running OS X 10.5. The IE tests were run on a desktop running Windows 7. [Run the tests yourself here.](http://www.bcherry.net/playground/defaultvalues)___Note 2___: I chose to use the `[].join` method because doing so was most likely to avoid any dead-code elimination optimizations in modern browsers. I've had trouble with this before, in Firefox. I _did_ try testing with the `String()` constructor, with similar results in most browsers. Opera was an exception where using the explicit `toString` was close to __5x faster__. In Firefox, the explicit cast was a bit faster, but both cases were about 100x faster than the `[].join` method (and other browsers), which means the code-path was probably being removed by the dead code eliminator.

The takeaway from this performance test is that it’s always best to call your object’s type-conversion methods directly, rather than relying on the interpreter to do the complex series of method calls and comparisons needed to do it automatically. The Opera 10.50 result is very strange, but it’s not particularly slower, so I wouldn’t worry about it. The gains made in other browsers more than make up for the outlier Opera result.

How About Browser Support?

Like many things in the ECMAScript specification, these processes are complex, and I doubted that all browsers would implement them exactly as specified. So, in that test suite from earlier, I added compliance checks. I was quite surprised to see that all major browsers, including versions of Internet Explorer going back to at least IE 5.5, implement these mechanisms correctly. This is even the case with the awkward handling when developers do things like make toString return a number instead of a string. All browsers handle the code according to the specification. This is great news.

But the specification unhelpfully introduced ambiguity in one particular area: the absence of a hint for the ToPrimitive function. Here’s the exact wording:

All native ECMAScript objects except Date objects handle the absence of a hint as if the hint Number were given; Date objects handle the absence of a hint as if the hint String were given. Host objects may handle the absence of a hint in some other manner.

That the standard explicitly allows browsers to deviate here worried me. Included in that test suite was a check that, in the absence of a hint, Date objects will default to String and Boolean objects will default to Number. All browsers passed this check as well, which means that browser support for all of this functionality seems to be consistent and correct.

Conclusions

I hope this was useful in understanding how these mechanisms work in JavaScript. There are three important things to take away from this article:

Implement toString and valueOf on your commonly-reused objects. They can help you write clearer, more concise code, and make debugging easier too.
All browsers implement object-to-primitive conversion according to the specification, so you can safely consult it for more detail.
When performance is important, always try to call your type-conversion methods directly, instead of relying on JavaScript’s implicit calls.

You can find the test suite used for this article here if you’re interested in trying to replicate my results. Please let me know if you find contradictory results to what I posted here.

Thanks for reading! If you have questions or feedback then leave a comment below or contact me directly.

JavaScript-Style Objects in Python

2010-03-26T00:00:00-07:00

One of JavaScript’s most convenient features is the object syntax. Objects are so easy to work with. You’ve got a lot of ways to make them, although the object literal syntax in particular kicks ass. You can access and assign them with either the dot operator or dictionary syntax. Missing properties just return undefined. This is just a really fantastic object model.

So, today, I sat down to write my first serious Python code in a while, and found myself trying use a dictionary like a JavaScript object. This was a total failure, obviously. Python simply does not have that. Luckily, it’s an incredibly malleable language, if you’re willing to do the legwork. So, in about 30 min, I implemented JavaScript-style objects.

To make a JavaScript-style object, you use js.JsObject().

import js
foo = js.JsObject()

You can assign properties with either dictionary syntax or the dot operator.

foo.bar = 1
foo['baz'] = 2

Reading properties works the same, and properties that don’t exist simply return None.

foo['bar'] # 1
foo.baz # 2
foo.spam # None
foo['spam'] # None

It just prints out as a dictionary when you convert it to a string, as is done by the print operator. It also does the same when viewed in the interpreter.

print foo # {'baz': 2, 'bar': 1}

The properties are iterable just as in JavaScript. This is different than normal Python dictionaries, where you iterate on a tuple of (key, value). Here we just get the keys.

for prop in foo:
	print prop, foo[prop] # bar 1, baz 2

You can also delete properties, using either the dot operator or dictionary syntax. It won’t raise if the property doesn’t exist, either.

del foo.bar
del foo['spam']

You can also easily check if properties exist using the in operator.

'baz' in foo # True
'bar' in foo # False

And object comparison works as in JavaScript. Two will never be the same, even if they have the same properties and values.

js.JsObject() == js.JsObject() # False
js.JsObject() is js.JsObject() # False

And, finally, the constructor function is more flexible than I let on initially. You can pass in keyword arguments, which will end up as properties. You can also pass a dictionary to initialize it with. Or, you can pass both!

foo = js.JsObject(a=1, b=2)
print foo # {'a': 1, 'b': 2}

foo = js.JsObject({'a': 1, 'b': 2})
print foo # {'a': 1, 'b': 2}

foo = js.JsObject({'a': 1}, b=2)
print foo # {'a': 1, 'b': 2}

So yeah, that’s pretty awesome. I thought about handling prototypal inheritance too, but I just don’t see the need. I almost never use prototypes in JavaScript, I’m positive I would never use them in Python. I’m also not crazy about the creation syntax, but I couldn’t find a way to re-purpose the dictionary literal {}, and didn’t want to override built-in names like dict or object. I also wanted to keep with Python conventions and UpperCamelCase the class name, so js.JsObject() is what I settled on.

You’re probably curious about the implementation, so here’s the full source code. I’m probably missing some useful things, but this is what I have so far. It’s quite simple; for the most part it just overrides everything to some form of dictionary method on self.___dict__. You can also find js.py on GitHub.

class JsObject(object):
	def __init__(self, *args, **kwargs):
		for arg in args:
			self.__dict__.update(arg)

		self.__dict__.update(kwargs)

	def __getitem__(self, name):
		return self.__dict__.get(name, None)

	def __setitem__(self, name, val):
		return self.__dict__.__setitem__(name, val)

	def __delitem__(self, name):
		if self.__dict__.has_key(name):
			del self.__dict__[name]

	def __getattr__(self, name):
		return self.__getitem__(name)

	def __setattr__(self, name, val):
		return self.__setitem__(name, val)

	def __delattr__(self, name):
		return self.__delitem__(name)

	def __iter__(self):
		return self.__dict__.__iter__()

	def __repr__(self):
		return self.__dict__.__repr__()

	def __str__(self):
		return self.__dict__.__str__()

So, there you go. I’d love to hear what you think, leave a comment!

JavaScript: Better and Faster

2010-03-21T00:00:00-07:00

“JavaScript: Better and Faster” is the name of a presentation I gave at Slide last week. It was generally well-received by the Slide crew so I decided to put a (slightly edited) copy of the accompanying slideshow online (runs in your browser). It’s not particularly dense, and doesn’t get into too much depth on any one topic, but I think it gives a great practical overview of the JavaScript landscape, including what things to look out for and neat techniques to write better code. The original intent was to cover the ins and outs of writing good, efficient JavaScript, targeted at people who write JavaScript every day, but have a strong background in Python.

I’d like to give a huge thanks to Scott Chacon, whose Showoff software made compiling this slideshow the most enjoyable thing imaginable. It took a little work to port the resultant slideshow to Google AppEngine, but I got it up and running. I’ll probably work on a more generalized Python/AppEngine adaptation of Showoff for my future use, since that’s my web platform and I want to keep using Showoff in the future. This experience also got me really turned on to Markdown, which I wasted no time in integrating into this blog.

This is the first presentation I’ve given in my professional career, and would much appreciate feedback on the slides.

JavaScript Module Pattern: In-Depth

2010-03-12T00:00:00-08:00

The module pattern is a common JavaScript coding pattern. It’s generally well understood, but there are a number of advanced uses that have not gotten a lot of attention. In this article, I’ll review the basics and cover some truly remarkable advanced topics, including one which I think is original.

The Basics

We’ll start out with a simple overview of the module pattern, which has been well-known since Eric Miraglia (of YUI) first blogged about it three years ago. If you’re already familiar with the module pattern, feel free to skip ahead to “Advanced Patterns”.

Anonymous Closures

This is the fundamental construct that makes it all possible, and really is the single best feature of JavaScript. We’ll simply create an anonymous function, and execute it immediately. All of the code that runs inside the function lives in a closure, which provides privacy and state throughout the lifetime of our application.

(function () {
	// ... all vars and functions are in this scope only
	// still maintains access to all globals
}());

Notice the () around the anonymous function. This is required by the language, since statements that begin with the token function are always considered to be function declarations. Including () creates a function expression instead.

Global Import

JavaScript has a feature known as implied globals. Whenever a name is used, the interpreter walks the scope chain backwards looking for a var statement for that name. If none is found, that variable is assumed to be global. If it’s used in an assignment, the global is created if it doesn’t already exist. This means that using or creating global variables in an anonymous closure is easy. Unfortunately, this leads to hard-to-manage code, as it’s not obvious (to humans) which variables are global in a given file.

Luckily, our anonymous function provides an easy alternative. By passing globals as parameters to our anonymous function, we import them into our code, which is both clearer and faster than implied globals. Here’s an example:

(function ($, YAHOO) {
	// now have access to globals jQuery (as $) and YAHOO in this code
}(jQuery, YAHOO));

Module Export

Sometimes you don’t just want to use globals, but you want to declare them. We can easily do this by exporting them, using the anonymous function’s return value. Doing so will complete the basic module pattern, so here’s a complete example:

var MODULE = (function () {
	var my = {},
		privateVariable = 1;

	function privateMethod() {
		// ...
	}

	my.moduleProperty = 1;
	my.moduleMethod = function () {
		// ...
	};

	return my;
}());

Notice that we’ve declared a global module named MODULE, with two public properties: a method named MODULE.moduleMethod and a variable named MODULE.moduleProperty. In addition, it maintains private internal state using the closure of the anonymous function. Also, we can easily import needed globals, using the pattern we learned above.

Advanced Patterns

While the above is enough for many uses, we can take this pattern farther and create some very powerful, extensible constructs. Lets work through them one-by-one, continuing with our module named MODULE.

Augmentation

One limitation of the module pattern so far is that the entire module must be in one file. Anyone who has worked in a large code-base understands the value of splitting among multiple files. Luckily, we have a nice solution to augment modules. First, we import the module, then we add properties, then we export it. Here’s an example, augmenting our MODULE from above:

var MODULE = (function (my) {
	my.anotherMethod = function () {
		// added method...
	};

	return my;
}(MODULE));

We use the var keyword again for consistency, even though it’s not necessary. After this code has run, our module will have gained a new public method named MODULE.anotherMethod. This augmentation file will also maintain its own private internal state and imports.

Loose Augmentation

While our example above requires our initial module creation to be first, and the augmentation to happen second, that isn’t always necessary. One of the best things a JavaScript application can do for performance is to load scripts asynchronously. We can create flexible multi-part modules that can load themselves in any order with loose augmentation. Each file should have the following structure:

var MODULE = (function (my) {
	// add capabilities...

	return my;
}(MODULE || {}));

In this pattern, the var statement is always necessary. Note that the import will create the module if it does not already exist. This means you can use a tool like LABjs and load all of your module files in parallel, without needing to block.

Tight Augmentation

While loose augmentation is great, it does place some limitations on your module. Most importantly, you cannot override module properties safely. You also cannot use module properties from other files during initialization (but you can at run-time after intialization). Tight augmentation implies a set loading order, but allows overrides. Here is a simple example (augmenting our original MODULE):

var MODULE = (function (my) {
	var old_moduleMethod = my.moduleMethod;

	my.moduleMethod = function () {
		// method override, has access to old through old_moduleMethod...
	};

	return my;
}(MODULE));

Here we’ve overridden MODULE.moduleMethod, but maintain a reference to the original method, if needed.

Cloning and Inheritance

var MODULE_TWO = (function (old) {
	var my = {},
		key;

	for (key in old) {
		if (old.hasOwnProperty(key)) {
			my[key] = old[key];
		}
	}

	var super_moduleMethod = old.moduleMethod;
	my.moduleMethod = function () {
		// override method on the clone, access to super through super_moduleMethod
	};

	return my;
}(MODULE));

This pattern is perhaps the least flexible option. It does allow some neat compositions, but that comes at the expense of flexibility. As I’ve written it, properties which are objects or functions will not be duplicated, they will exist as one object with two references. Changing one will change the other. This could be fixed for objects with a recursive cloning process, but probably cannot be fixed for functions, except perhaps with eval. Nevertheless, I’ve included it for completeness.

Cross-File Private State

One severe limitation of splitting a module across multiple files is that each file maintains its own private state, and does not get access to the private state of the other files. This can be fixed. Here is an example of a loosely augmented module that will maintain private state across all augmentations:

var MODULE = (function (my) {
	var _private = my._private = my._private || {},
		_seal = my._seal = my._seal || function () {
			delete my._private;
			delete my._seal;
			delete my._unseal;
		},
		_unseal = my._unseal = my._unseal || function () {
			my._private = _private;
			my._seal = _seal;
			my._unseal = _unseal;
		};

	// permanent access to _private, _seal, and _unseal

	return my;
}(MODULE || {}));

Any file can set properties on their local variable _private, and it will be immediately available to the others. Once this module has loaded completely, the application should call MODULE._seal(), which will prevent external access to the internal _private. If this module were to be augmented again, further in the application’s lifetime, one of the internal methods, in any file, can call _unseal() before loading the new file, and call _seal() again after it has been executed. This pattern occurred to me today while I was at work, I have not seen this elsewhere. I think this is a very useful pattern, and would have been worth writing about all on its own.

Sub-modules

Our final advanced pattern is actually the simplest. There are many good cases for creating sub-modules. It is just like creating regular modules:

MODULE.sub = (function () {
	var my = {};
	// ...

	return my;
}());

While this may have been obvious, I thought it worth including. Sub-modules have all the advanced capabilities of normal modules, including augmentation and private state.

Conclusions

Most of the advanced patterns can be combined with each other to create more useful patterns. If I had to advocate a route to take in designing a complex application, I’d combine loose augmentation, private state, and sub-modules.

I haven’t touched on performance here at all, but I’d like to put in one quick note: The module pattern is good for performance. It minifies really well, which makes downloading the code faster. Using loose augmentation allows easy non-blocking parallel downloads, which also speeds up download speeds. Initialization time is probably a bit slower than other methods, but worth the trade-off. Run-time performance should suffer no penalties so long as globals are imported correctly, and will probably gain speed in sub-modules by shortening the reference chain with local variables.

To close, here’s an example of a sub-module that loads itself dynamically to its parent (creating it if it does not exist). I’ve left out private state for brevity, but including it would be simple. This code pattern allows an entire complex heirarchical code-base to be loaded completely in parallel with itself, sub-modules and all.

var UTIL = (function (parent, $) {
	var my = parent.ajax = parent.ajax || {};

	my.get = function (url, params, callback) {
		// ok, so I'm cheating a bit :)
		return $.getJSON(url, params, callback);
	};

	// etc...

	return parent;
}(UTIL || {}, jQuery));

I hope this has been useful, and please leave a comment to share your thoughts. Now, go forth and write better, more modular JavaScript!

This post was featured on Ajaxian.com, and there is a little bit more discussion going on there as well, which is worth reading in addition to the comments below.

Performance of === vs. ==

2010-03-08T00:00:00-08:00

One particular weirdness and unpleasantry in JavaScript is the set of equality operators. Like virtually every language, JavaScript has the standard ==, !=, <, >, <=, and >= operators. However, == and != are NOT the operators most would think they are. These operators do type coercion, which is why [0] == 0 and "\n0\t " == 0 both evaluate to true. This is considered, by sane people, to be a bad thing. Luckily, JavaScript does provide a normal set of equality operators, which do what you expect: === and !==. It sucks that we need these at all, and === is a pain to type, but at least [0] !== 0.

So, that’s all well and good, but when making the decision to use === instead of == it’s important to understand if there are any performance implications. Reasonable people, like myself, would expect the strict equality operators to be faster than their type-coercing counterparts, because the type coercion must take time. But, being a scientist, I had to set up an experiment to test this hypothesis.

For Science!

My experiment times the execution time of 24 different tests, in the browser you view it in. These tests represent every permutation of the following factors:

Use: Whether the result is computed and thrown away, or assigned into a local variable.
Comparison: Comparing integers vs. integers, strings vs. strings, and integers vs. strings.
Operands: Whether the operands are actually equal or unequal (with forced type coercion in the case of ===).
Operator: Using either == or ===.

Note that I am specifically not testing the relative performance when comparing against values like null, undefined, or false. This is because those are falsy values which have even worse type coercion characteristics. Some integers and strings can, of course, also be falsy, such as 0 or "", but these are normal values which occur during arithmetic or string comparison, so I’ve tested with them.

The tests were run over two million iterations, except in Internet Explorer, where it produced a long-running script error, so I cut it to 500,000 iterations. Here are the browser configurations I tested:

Mozilla Firefox 3.6 (Mac)
Google Chrome 5 (Mac dev channel)
Internet Explorer 8 (iterations reduced by 4x)
Safari 4 (Mac)
Opera 10.5 beta (Mac)
Mozilla Firefox 3.6 (Mac) with Firebug open

Results

My results are available as a Google spreadsheet here. It turns out that there is little practical performance difference between == and ===. While the strict operator is marginally faster (roughly 10%) in most browsers when combined with explicit type conversion, such as a === +b, the only real performance gains will come from avoiding type conversion entirely. Converting a string to an integer for comparison with another integer is significantly slower (up to 10x) than simple comparing two integers. You should never allow integers to be stored as strings internally, as the type conversion will incur a performance penalty.

While that was the basic takeaway from the numbers, I did find one interesting outlier when testing with Firefox. In Firefox, the comparison a === +b is about 20x slower than the equivalent a == b when a is an integer and b is a string integer. This result seems suspicious to me, and nothing similar occurred in any other browser. Oddly, when the Firebug script debugger is turned on, this result changes, and a === +b becomes about 10% faster than the other. I’m not sure what to make of this result, but it does serve as a reminder that integers should always be stored in numbers, not in strings.

Conclusion

Not much surprise in these results, other than the Firefox result. But, it did help me avoid a nagging worry that I’m silently slowing down my code whenever I use === instead of == in my JavaScript.

Again, find the test here and my results here. I hope you found this information interesting. Let me know in the comments if you see different results than I got.

Minimum Timer Intervals in JavaScript

2010-02-26T00:00:00-08:00

I was talking with a co-worker today about the behavior of setTimeout and setInterval when given a small interval, like 0 or 1. The expectation would be that the timer will fire in 0ms, or 1ms. However, as with other things in JavaScript, the reality is a bit different. It turns out that browsers actually have a minimum timer interval which they can’t work any faster than. John Resig wrote about timer performance a few years back, and found this behavior. He’s also covering it in more detail in his new book.

But, I wasn’t happy with data a few years old, so I decided to just go and write my own simple test suite, How Fast is setTimeout in Your Browser?. This page simply runs setTimeout with an interval of 0, 1000 times, and averages the real timeout experienced in each. Go ahead and check it out in your browser of choice.

The Results

Well, it turns out that things aren’t so bad. Most browsers are in the 10-15ms range for their bottom limit, having improved in recent versions. Notable exceptions are Internet Explorer, which has the same bottom of around 16ms in all versions since IE6, and Google Chrome, which, at least since version 4, has a bottom limit closer to 5ms It’s important to keep this limitation in mind when using setTimeout or setInterval. In particular, if you’re looking for consistent timer intervals across browsers, you have to use something >15ms. But, don’t forget that JavaScript is single-threaded, and the timer won’t execute while other code is executing. This means that in the following code sample, you can guarantee that the timer will not run until the loop has completed. You cannot, however, guarantee precisely when that will happen, nor that it will be the next piece of code to run following the loop.

setTimeout(function () { alert("timer"); }, 0);
for (var i = 0; i < 1000; i += 1) {
	// something
}

So it should be safe to use timers with an interval of 0ms when your only expectation is that the timer will fire as soon as it can, but not until after the current code path has completed. Relying on timers to respect the interval you give them is foolish, since, as I’ve shown, they have a lower-bound, and since they wait even after firing, before executing, for other code to return.

The Source Code

This test is really simple. Here’s the complete JavaScript source code:

var target = document.getElementById("target"),
	results = 0,
	iterations = 1000,
	i = 0;

function go() {
	var fn = function () {
			results += new Date().getTime() - d;
			i += 1;
			if (i < iterations) {
				go();
			} else {
				finish();
			}
		},
		d = new Date().getTime();
	setTimeout(fn, 0);
}

function finish() {
	target.innerHTML = "Average timer delay was <span class=\"num\">" + results/iterations + 
		"</span>ms, over <span class=\"num\">" + iterations + "</span> iterations.";
}

go();

And that’s all there is to it. I hope you found this useful or interesting. Timers are very fickle, but also incredibly useful, so it’s worth taking the time to understand what it is they do, and how they do it.

This Blog - Now With HTML5

2010-02-19T00:00:00-08:00

Take a look at this web site. Notice anything out of the ordinary. Hopefully you don’t, none of the layout or styling has changed noticeably recently. However, try a “View Source”. Now do you see something different? This page is now rendered with the following doctype:

<!DOCTYPE html>

That’s right, this web site is now HTML5! In fact it even validates as HTML5 successfully! If you dig past the doctype, you’ll find all sorts of HTML5 semantic elements, including <nav>, <article>, and <time>, among others. Making the transition was reasonably painless with the help of Mark Pilgrim’s excellent article on making the switch. For the most part, I just had to change a few elements and rewrite my CSS. There were a few headaches because Blueprint CSS expects your grid elements to be <div>s, but the following CSS added the right capabilities to my semantic elements to be compatible with Blueprint:

header, footer, nav, article {
	float: left;
	margin-right: 10px;
}

Of course, IE doesn’t understand the new HTML5 elements. Luckily, there’s an easy solution. I chose to write it myself, since it’s so simple, but there’s a publicly available complete solution you can use as well. Here’s mine, for reference (in the <head>):

<!-- IE HTML5 Compatibility -->
<!--[if IE]>
	<script type="text/javascript">
		(function () {
			var tags = "header hgroup nav article time footer".split(" "),
				i = 0,
				l = tags.length;
			for (; i < l; i += 1) {
				document.createElement(tags[i]);
			}
		}());
	</script>
<![endif]-->

And that’s all there was to it! It’s great that you can start using basic HTML5 right now without any difficulties, and it will work in all browsers, even IE6! Of course, you can’t just start using the more advanced features like <video> or Web Workers, but basic markup is a great start.

So what do you think? You ready to upgrade your own personal site?

Finding Improper JavaScript Globals

2010-02-12T00:00:00-08:00

When I interview web developers, my first JavaScript question is usually the following:

What is the difference, in JavaScript, between x = 1 and var x = 1. Feel free to answer in as much or as little detail as you feel comfortable.

Most people would give an answer about how the var keyword makes something a local variable, omitting it makes it a global variable. While I'd love to hear about scope chains, the window object, and hear the term "implied global" in an answer, that basic answer is good enough. It might not show a thorough knowledge of JavaScript, but at least it shows some level of understanding of the most common dangerous feature.

There are three basic ways to make a global variable in JavaScript. You can use var (declared global), you can just assign to it without having declared it (implied global), or you can set a property on the window object (window global). Here's those three:

var x = 1; // declared global
y = 2; // implied global
window.z = 3; // window global

Implied globals are bad because they're hard to keep track of, and their declarations aren't hoisted. I don't like window globals either, because mixing window.x and x is bad form. In my opinion, all globals should be declared globals. Unfortunately, JavaScript makes this really hard to maintain. There are tools like JSLint that will perform analysis of your code and help you out, but it can't do an entire code base at once, at least not easily.

I've written a tool that performs run-time analysis of your application, and finds all of the improperly declared globals (both implied and window globals). Check out a demo here.

Introducing badglobals.js

badglobals.js is a tool for finding all of the improperly declared global variables in your application. Using it is simple, but don't use it in production code (see "How It Works" below).

Include badglobals.js in your page, before any other scripts.
When you want to do analysis, open Firebug and run BADGLOBALS.check().
You'll see a warning statement in the console for every bad global found. These contain both the name, and the value.

In addition, there are a few slightly more advanced features you might be interested in.

Exclusions

If you have some globals you don't want to be reported about (such as third-party libraries), you can easily exclude them manually. Before you run the check, just call .exclude, like so:

BADGLOBALS.exclude("jQuery", "$", "YAHOO");

Feel free to call this method as many times as you'd like, it always adds, and never removes. By default, all browser built-ins are excluded (these are found when the script is included). Sometimes, the variable _firebug enters after the script include, and shows in the report. You should exclude this.

Report Object

While the warnings are probably enough, badglobals.js also builds a report object, containing more information. Access it by calling BADGLOBALS.report(). This will run .check() if it has not run already. The report object has the following properties:

bad: An array of the names of the bad globals found.
good: An array of the names of the good globals found.
skipped: An array of the names of the globals that were not checked.

And that's all there is to badglobals.js. It's really simple to use, but remarkably effective.

How It Works

badglobals.js works because of one key difference between implied/window globals and declared globals: declared globals cannot be deleted. This is because using var causes the internal property [[DontDelete]] to be set.

var x = 1;
y = 2;
window.z = 3;

delete x; // false
delete y; // true
delete z; // true

x; // 1
y; // undefined
z; // undefined

badglobals.js simply tries to delete every property of window (skipping the built-ins, of course). If the delete succeeds, it was declared wrong. It always puts it back, but I wouldn't trust this to run in production code, because it just seems dangerous. Here's the core section of badglobals.js:

for (prop in window) {
	if (window.hasOwnProperty(prop)) {
		if (skip.indexOf(prop) >= 0 || exclude.indexOf(prop) >= 0) {
			skipped.push(prop);
		} else {
			val = window[prop];
			if (delete window[prop]) {
				console.warn("Found non-var global %o with value %o", prop, val);
				bad.push(prop);
				try {
					window[prop] = val;
				} catch (e) {
					console.error("Oops, there was an error putting %o back :(", prop);
				}
			} else {
				good.push(prop);
			}
		}
	}
}

Browser Support

This script will not work in Internet Explorer, because I use the array indexOf method, among other things. I also think IE doesn't exactly follow the standard when it comes to delete, so the checks might not work. I don't consider this a problem, because this is a developer tool, not production code. You'll find the complete set of bad globals with Firefox or Chrome, so you should not need to check in Internet Explorer as well. I have not tested it in Opera, but the console references will certainly fail.

Thanks to...

I got the idea for this tool from the excellent article on delete by kangax, over at his blog. If you haven't read that article, you really should. The depth and quality is incredible.

Get badglobals.js

Here are the links, one more time:

JavaScript Scoping and Hoisting

2010-02-08T00:00:00-08:00

Do you know what value will be alerted if the following is executed as a JavaScript program?

var foo = 1;
function bar() {
	if (!foo) {
		var foo = 10;
	}
	alert(foo);
}
bar();

If it surprises you that the answer is “10”, then this one will probably really throw you for a loop:

var a = 1;
function b() {
	a = 10;
	return;
	function a() {}
}
b();
alert(a);

Here, of course, the browser will alert “1”. So what’s going on here? While it might seem strange, dangerous, and confusing, this is actually a powerful and expressive feature of the language. I don’t know if there is a standard name for this specific behavior, but I’ve come to like the term “hoisting”. This article will try to shed some light on this mechanism, but first lets take a necessary detour to understand JavaScript’s scoping.

Scoping in JavaScript

One of the sources of most confusion for JavaScript beginners is scoping. Actually, it’s not just beginners. I’ve met a lot of experienced JavaScript programmers who don’t fully understand scoping. The reason scoping is so confusing in JavaScript is because it looks like a C-family language. Consider the following C program:

#include <stdio.h>
int main() {
	int x = 1;
	printf("%d, ", x); // 1
	if (1) {
		int x = 2;
		printf("%d, ", x); // 2
	}
	printf("%d\n", x); // 1
}

The output from this program will be 1, 2, 1. This is because C, and the rest of the C family, has block-level scope. When control enters a block, such as the if statement, new variables can be declared within that scope, without affecting the outer scope. This is not the case in JavaScript. Try the following in Firebug:

var x = 1;
console.log(x); // 1
if (true) {
	var x = 2;
	console.log(x); // 2
}
console.log(x); // 2

In this case, Firebug will show 1, 2, 2. This is because JavaScript has function-level scope. This is radically different from the C family. Blocks, such as if statements, do not create a new scope. Only functions create a new scope.

To a lot of programmers who are used to languages like C, C++, C#, or Java, this is unexpected and unwelcome. Luckily, because of the flexibility of JavaScript functions, there is a workaround. If you must create temporary scopes within a function, do the following:

function foo() {
	var x = 1;
	if (x) {
		(function () {
			var x = 2;
			// some other code
		}());
	}
	// x is still 1.
}

This method is actually quite flexible, and can be used anywhere you need a temporary scope, not just within block statements. However, I strongly recommend that you take the time to really understand and appreciate JavaScript scoping. It’s quite powerful, and one of my favorite features of the language. If you understand scoping, hoisting will make a lot more sense to you.

Declarations, Names, and Hoisting

In JavaScript, a name enters a scope in one of four basic ways:

Language-defined: All scopes are, by default, given the names this and arguments.
Formal parameters: Functions can have named formal parameters, which are scoped to the body of that function.
Function declarations: These are of the form function foo() {}.
Variable declarations: These take the form var foo;.

Function declarations and variable declarations are always moved (“hoisted”) invisibly to the top of their containing scope by the JavaScript interpreter. Function parameters and language-defined names are, obviously, already there. This means that code like this:

function foo() {
	bar();
	var x = 1;
}

is actually interpreted like this:

function foo() {
	var x;
	bar();
	x = 1;
}

It turns out that it doesn’t matter whether the line that contains the declaration would ever be executed. The following two functions are equivalent:

function foo() {
	if (false) {
		var x = 1;
	}
	return;
	var y = 1;
}
function foo() {
	var x, y;
	if (false) {
		x = 1;
	}
	return;
	y = 1;
}

Notice that the assignment portion of the declarations were not hoisted. Only the name is hoisted. This is not the case with function declarations, where the entire function body will be hoisted as well. But remember that there are two normal ways to declare functions. Consider the following JavaScript:

function test() {
	foo(); // TypeError "foo is not a function"
	bar(); // "this will run!"
	var foo = function () { // function expression assigned to local variable 'foo'
		alert("this won't run!");
	}
	function bar() { // function declaration, given the name 'bar'
		alert("this will run!");
	}
}
test();

In this case, only the function declaration has its body hoisted to the top. The name ‘foo’ is hoisted, but the body is left behind, to be assigned during execution.

That covers the basics of hoisting, which is not as complex or confusing as it seems. Of course, this being JavaScript, there is a little more complexity in certain special cases.

Name Resolution Order

The most important special case to keep in mind is name resolution order. Remember that there are four ways for names to enter a given scope. The order I listed them above is the order they are resolved in. In general, if a name has already been defined, it is never overridden by another property of the same name. This means that a function declaration takes priority over a variable declaration. This does not mean that an assignment to that name will not work, just that the declaration portion will be ignored. There are a few exceptions:

The built-in name arguments behaves oddly. It seems to be declared following the formal parameters, but before function declarations. This means that a formal parameter with the name arguments will take precedence over the built-in, even if it is undefined. This is a bad feature. Don’t use the name arguments as a formal parameter.
Trying to use the name this as an identifier anywhere will cause a SyntaxError. This is a good feature.
If multiple formal parameters have the same name, the one occurring latest in the list will take precedence, even if it is undefined.

Named Function Expressions

You can give names to functions defined in function expressions, with syntax like a function declaration. This does not make it a function declaration, and the name is not brought into scope, nor is the body hoisted. Here’s some code to illustrate what I mean:

foo(); // TypeError "foo is not a function"
bar(); // valid
baz(); // TypeError "baz is not a function"
spam(); // ReferenceError "spam is not defined"

var foo = function () {}; // anonymous function expression ('foo' gets hoisted)
function bar() {}; // function declaration ('bar' and the function body get hoisted)
var baz = function spam() {}; // named function expression (only 'baz' gets hoisted)

foo(); // valid
bar(); // valid
baz(); // valid
spam(); // ReferenceError "spam is not defined"

How to Code With This Knowledge

Now that you understand scoping and hoisting, what does that mean for coding in JavaScript? The most important thing is to always declare your variables with a var statement. I strongly recommend that you have exactly one var statement per scope, and that it be at the top. If you force yourself to do this, you will never have hoisting-related confusion. However, doing this can make it hard to keep track of which variables have actually been declared in the current scope. I recommend using JSLint with the onevar option to enforce this. If you’ve done all of this, your code should look something like this:

/*jslint onevar: true [...] */
function foo(a, b, c) {
    var x = 1,
    	bar,
    	baz = "something";
}

What the Standard Says

I find that it’s often useful to just consult the ECMAScript Standard (pdf) directly to understand how these things work. Here’s what it has to say about variable declarations and scope (section 12.2.2 in the older version):

If the variable statement occurs inside a FunctionDeclaration, the variables are defined with function-local scope in that function, as described in section 10.1.3. Otherwise, they are defined with global scope (that is, they are created as members of the global object, as described in section 10.1.3) using property attributes { DontDelete }. Variables are created when the execution scope is entered. A Block does not define a new execution scope. Only Program and FunctionDeclaration produce a new scope. Variables are initialised to undefined when created. A variable with an Initialiser is assigned the value of its AssignmentExpression when the VariableStatement is executed, not when the variable is created.

I hope this article has shed some light on one of the most common sources of confusion to JavaScript programmers. I have tried to be as thorough as possible, to avoid creating more confusion. If I have made any mistakes or have large omissions, please let me know.

Search/Replace in the DOM with jQuery

2010-02-03T00:00:00-08:00

Ever had a need to to a text search/replace through the DOM? For articles in drafting on this blog I often use the form (TODO: something) as notes for myself to do some more research. I don't want them sticking around in the text I publish, so I highlighted them red (only for myself). Here's the script I used to accomplish that:

jQuery(function () {
	jQuery(":contains(TODO)").not(":has(:contains(TODO))").each(function () {
		var that = $(this),
			html = that.html();

		html = html.replace(/(\(TODO:.*?\))/g, "<span class=\"todo\">$1</span>");
		that.html(html);
	});
});

The trick is to find only the nodes that contain the actual text, not ancestors of such nodes, which is what you would get with just jQuery(":contains(TOOD)"). Filtering with .not(":has(:contains(TODO))") ensures you've got the bottom most ones. I tried doing this with the ":not()" filter, but it didn't work. I guess that might be a jQuery bug, but the .not() method is a fine alternative.

Why JavaScript's "new" Keyword Sucks

2010-02-01T00:00:00-08:00

>>> function F() { return function inner() { return "inner invoked"; }; }
>>> F()
inner()
>>> F()()
"inner invoked"
>>> new F
inner()
>>> new F()
inner()
>>> (new F)
inner()
>>> (new F)()
"inner invoked"
>>> new (F())
{ }
>>> new ((F)())
{ }
>>> (new F())()
"inner invoked"

Preloading JS and CSS as Print Stylesheets

2010-01-29T00:00:00-08:00

UPDATE: This technique has turned out to be dangerous in Chrome. It seems that Chrome will load the JS files into the cache, but then set an implied type="text/css" on them. This means that it will refuse to re-use them as JavaScript on future pages, until they have left the cache. I can no longer recommend this technique, but hope that it was at least interesting. I'll be working on a follow-up post about alternatives.

One of Yahoo's Best Practices for Speeding Up Your Web Site is "Preload Components". Most people are already familiar with doing this for images. Code for that usually looks like this:

var img = Image();
img.src = "/path/to/something.jpg";

This code works in all browsers, and causes that image to be downloaded and placed in the browser's cache.

But what if we want to go further? For a lot of today's big applications, images are no longer the only bandwidth hog. An application I work on has more than 2MB worth of JavaScript, CSS, and data (stored in JavaScript files). We found that users on slow connections could take up to 60 seconds to load our application for the first time! Obviously, we could benefit from intelligently pre-loading this data in the background, on an earlier page where they don't need it yet. But how?

My first attempt was just to use the Image() method above, but point at our text files instead. This didn't work in Firefox. I'm not sure why, but those files never appeared in the cache with that method. So our next idea was to use <iframe> tags. This worked great, until we tried it in Internet Explorer. IE chose to warn the user that our page was trying to download data they didn't ask for, and blocked the downloads until they approved them. Obviously, this wouldn't work either.

This left us scratching our heads, but we eventually had a breakthrough. There is one other tag that is used to include text files in a web page. That's the <link rel="stylesheet"> tag. But this couldn't possibly work with JavaScript, could it? It turns out it could!

var preload = function (file) {
	var elem,
		tag = "link",
		attr = "href",
		extra = " rel=\"stylesheet\" media=\"print\" ",
		target = "head";

	elem = jQuery(["<", tag, extra, attr, "=\"", file, "\"<>/", tag, ">"].join(''));
	elem.load(function () {
		elem.remove();
	});
	jQuery(target).append(elem);

	return elem;
};

This function creates a temporary <link> tag in the <head> of your page, pointing at the requested file. To work across all browsers, we needed to give it the rel="stylesheet" attribute. To not cause preloaded CSS files to try and apply themselves to the current page, or trigger a re-flow, we added media="print". Once the element has loaded, it removes itself from the DOM, but the file remains in the browser's cache. While this example relies on the jQuery library, there is no reason it could not be easily re-written without such a dependency, or written as a jQuery plugin (providing jQuery.preload()).

This technique cut our aforementioned page load from 60s to 20s, over a slow connection. I think this is a complete solution, and seems to work in all major browsers. What do you think?

Consul.js, Simple Logging Abstraction

2009-11-28T00:00:00-08:00

I've been reading Coders At Work by Peter Seibel, and one question he asks in every interview is "How do you debug - print statements, debugger like gdb, or something else?". For me, print statements are my first tool for any situation. That's why Firebug is so crucial to my work on the web, because writing and debugging advanced JavaScript would be a nightmare without it. While working on code, my files are littered with console.log() statements and the like. Unfortunately, shipping code with these statements is a no-no, so I usually pull them out. Frustrated with this process, I decided to build an alternative and throw some new features in along the way.

consul.log("This is a log message");
consul.warn("I even support fancier %s features like %o.", "Firebug", "format strings");
consul.turnOff();
consul.assert(false, "I'm off, so you won't see this error fire.");
consul.turnOn();
consul.info("See this message in Firebug, Chrome, Safari, Firebug Lite, IE8, and Opera.")

Meet consul.js, which acts as a wrapper for the standard window.console features from Firebug. Using it is simple, as you see above. Just include the file, and start using consul instead of console. You can easily turn it off for your live code, but have easy access to the messages by simply turning it back on from the command line.

In addition to what you'd expect, using consul.js ensures that you can view your messages and assertions in any browser. Currently, it supports the latest versions of Firebug, Chrome, and Safari (you'll need the JS console for the latter two). I also built in support for most commands in Internet Explorer 8, provided you've got the developer tools open. Opera Dragonfly support was harder, since it doesn't offer much logging features. However, most things work through window.opera.postError(). Regardless, for IE and Opera (as well as any other browser), it checks for Firebug Lite and outputs there if it finds it, instead of the built-in dev tools.

For a list of features and browser support, look no further than the code itself:

var consoleFuncs = {
	log         : b("firebug", "firebuglite", "chromium", "safari", "ie", "opera"),
	debug       : b("firebug", "firebuglite", "chromium", "safari"),
	info        : b("firebug", "firebuglite", "chromium", "safari", "ie", "opera"),
	warn        : b("firebug", "firebuglite", "chromium", "safari", "ie", "opera"),
	error       : b("firebug", "firebuglite", "chromium", "safari", "ie", "opera"),
	assert      : b("firebug", "firebuglite", "chromium", "safari", "ie", "opera"),
	dir         : b("firebug", "firebuglite", "chromium", "safari"),
	dirxml      : b("firebug", "firebuglite", "chromium", "safari"),
	trace       : b("firebug", "firebuglite", "chromium", "safari"),
	group       : b("firebug", "firebuglite", "chromium", "safari"),
	groupEnd    : b("firebug", "firebuglite", "chromium", "safari"),
	time        : b("firebug", "firebuglite", "chromium", "safari"),
	timeEnd     : b("firebug", "firebuglite", "chromium", "safari"),
	count       : b("firebug", "firebuglite", "chromium", "safari")
};

If a function is not supported in a particular browser, it will simply be ignored, rather than throwing an error.

So give it a shot, and let me know what you think! This tool helped me build this blog's front-end code, and has already been ported into the code I use at work, so I can take advantage of the same tool. I think it's a big win for JavaScript developers who rely on the Firebug console, and introduces a lot of new flexibility without degrading simplicity.

JS Find-and-Replace with Split/Join

2009-11-05T00:00:00-08:00

Update: This advice is now totally out-of-date and wrong. Just use replace. As recent Browserscope results show, all browsers have optimized replace to be much faster than split/join. Thanks to Luigi van der Pal for pointing this out in the comments!

When trying to do a find-and-replace in Javascript with static search and replace strings, it’s always faster to use String.split() and String.join() than to use String.replace(). Compare:

var myString = '......some long text.....'; 
mystring.replace(/,/g,"; "); // Replace the commas with semi-colon-space

with

mystring.split(",").join("; "); // Replace the commas with semi-colon-space

The latter split/join method will always be faster. Note that this is only useful for static find-and-replace. For instance, you could not do anything like the following with splits and joins:

mystring.replace(/^[0-9]+([A-Za-z]*)/g, "$1");

Of course, you could also use this method to strip characters out of a string:

mystring.split(",").join('');

Good luck!

Javascript Pseudo-threading

2009-11-05T00:00:00-08:00

This post has been migrated from my old, defunct blog at bcherry.net. The links may not work, and the formatting may be wonky.

I’ve been playing around with asynchronous Javascript for repeated large-set actions, in the hopes of generating some useful techniques for real applications. I’ve narrowed down a successful technique that I call “simplethreading”. Check out a demo here or the resulting source code here. The demo lets you play with the size of the data set, and also the operation queue size. I found that queuing operations in batches was quite a bit faster than queuing them individually, so that each fired event will process a few data objects instead of just 1. I’d guess the reason for this is that Javascript only supports timeouts at the millisecond level, when most operations are shorter than that. By grouping operations into larger chunks, we can make much better use of the time we’re given.

Here’s the code for both the blocking (I call it “singlethreaded”) and non-blocking (I call it “simplethreaded” because it’s not really multi-threaded, but it’s not really single-threaded either) version. Note that ST.s and ST.n refer to the querystring parameters ‘s’ and ‘n’.:

ST.functions = [
	function (n, context) {
		$(context).append($("<div>").addClass("result").attr("id","r"+n).text(n));
	}, function (n, context) {
		$("#r" + n + ".result", context).addClass("dark");
	}, function (n, context){
		$("#r" + n + ".result", context).remove();
	}
];

/** singlethreaded approach **/
ST.singleStage = 0;
ST.SingleGenerator = function(n) {
	var i = 0;
	this.next = function() {
		if (i &gt; n || i &gt; ST.singlelimit) {
			return null;
		}
		return i++;
	};
};
ST.startsingle = function() {
	var context = $("#singlethreaded .output");
	var gen = new ST.SingleGenerator(ST.n);
	var n;
	var workFn = ST.functions[ST.singleStage];
	ST.singleStage++;
	ST.singleStage = ST.singleStage % ST.functions.length;
	while ((n = gen.next()) !== null) {
		workFn(n,context);
	}
};

/** simplethreaded approach **/
ST.simpleStage = 0;
ST.SimpleGenerator = function(n) {
	var i = 0;
	this.next = function() {
		if (i &gt; n) {
			return null;
		}
		return i++;
	};
};
ST.startsimple = function() {
	var context = $("#simplethreaded .output");
	var gen = new ST.SimpleGenerator(ST.n);
	var n;
	var workFn = ST.functions[ST.simpleStage];
	ST.simpleStage++;
	ST.simpleStage = ST.simpleStage % ST.functions.length;
	var fn = function() {
		var i = 0;
		while (i++ &lt; ST.s &amp;&amp; (n = gen.next()) !== null) {
			workFn(n,context);
		}
		if (n === null) {
			clearInterval(threadID);
		}
	};
	var threadID = setInterval(fn,1);
};

The resulting code is actually pretty straightforward. The key is to maintain the ID returned by the setInterval() call so it can be killed when you’re done with your work. Normally this would get stuffed into a global, but by wrapping the whole lot into a function and then sending a local function reference into setInterval(), I can take advantage of Javascript’s closure to keep the locals I need around for every asynchronous function call, which is really awesome.

Also, using a generator is a pretty neat way to produce the data I need, and lets me work with arbitrary data pretty easily. In a real application where the data is coming from the server asynchronously, I can have the generator returning data from a queue as it comes in, while returning flags like “all done” so the thread knows it won’t see more data or “wait for it” so the thread can stick around and poll for new data when it comes in.

I think that if I were to bring this into a real application, I would have it decide between singlethreaded or simplethreaded dynamically, because it’s obvious that singlethreaded is much better for smaller sets.

jQuery Micro-templates

2009-11-05T00:00:00-08:00

John Resig wrote a neat micro-templating Javascript function a while back. I’ve been playing with this, and have discovered two things:

These templates are totally nestable. Just make an element to be templated within a template, and call the sub-template code beneath the main call.
I found myself tending to do lines like $("...").html(tmpl("...",{...})), which seemed clumsy. It was very easy to extend jQuery to include this templating engine (just add this underneath the existing code):

jQuery.fn.tmpl = function(str,data){ return this.html(tmpl(str,data)); }

Then you can just do the following to templatize any number of elements:

$(".myElems").tmpl("myTemplate",myDataObj);

I’m still playing with these templates, and working on a new project, so I hope to have more to write about it soon.

Managing CSS Through JavaScript

2009-11-05T00:00:00-08:00

It’s often very difficult to keep track of what CSS goes where. Especially when you load up on Javascript controls and files in a large application, which render markup that depends on some CSS being there. One solution is to “inline” the CSS in the JavaScript, by adding style to each element as it’s created. This works, but it’s messy. Also, browsers are heavily optimized to apply CSS, faster than any Javascript solution could be. But when you’ve got a fancy script manager like LABjs, remembering to statically link all of the important CSS is a pain. Especially so if you’re not sure at page load whether certain CSS will be needed. Here’s a simple function to do this for you (note the XHR stuff is pretty basic, and I’d recommend changing it to use whatever JS library you use in your applications):

(function() {
  var loadedFiles = {}; 
  this.$css = function(filename) { 
    if (loadedFiles[filename]) { 
      return; 
    } 
    loadedFiles[filename] = true; 
    var xhr = new XMLHttpRequest(); 
    xhr.open("GET", filename, true); 
    xhr.onreadystatechange = function() { 
      if (xhr.readyState === 4) { 
        var head = document.getElementsByTagName("head")[0]; 
        var styleTag = document.createElement("style"); 
        var style = document.createTextNode(xhr.responseText);
        styleTag.appendChild(style); 
        head.appendChild(styleTag);
      } 
    }; 
    xhr.send(null); 
  };
})();

Now, from any Javascript, just do the following:

$css("styles/main.css"); 
$css("styles/controls.css");

The $css() function keeps track of what’s already been loaded, so that you’ll never load the same stylesheet twice. So don’t be afraid to spell out each CSS dependency your Javascript files have.

Building a Better Friend Selector

2009-11-05T00:00:00-08:00

This post has been migrated from my old, defunct blog at bcherry.net. The links may not work, and the formatting may be wonky.

Working in social entertainment, one of the lynchpins of the entire business is the friend selector. Without it, there is virtually no way to grow your customer base. Having a simple, effective friend selector is absolutely critical. On Facebook, they provide you with either the multi-friend-selector or the condensed multi-friend-selector. If you’ve ever played a Facebook game like Mafia Wars or Farmville or SuperPoke! Pets, then you’re probably already familiar with both of these controls. However, the situation is much more sparse on other networks. And if you’re talking about implementing a network independent game, like SuperPokePets.com, then you’ll eventually need to write your own.

I’ve begun a project in my free time to make a truly great friend selector control, for use in our own applications and elsewhere. You can find the result here, and the source code is available on GitHub. I’m constantly working on improving it. What I’ve built is not useful just for friends lists, but could feasibly serve as a good control for any sort of list selection. It began as a simple exercise in good Javascript, seeing how quickly I could replicate every feature of the Facebook condensed multi-friend-selector (the answer was about 2 hours). But as I worked on it more, I realized that I was capable of producing something of real value to our business. I hope to tackle advanced problems like performance with our so-called “whale users”, who have tens of thousands of friends. My initial attempt starts to break down over 1000 friends, and crashes the browser as I reach above 10000.

You can use it by including the Javascript and CSS files (found in Git), then making a simple call to the constructor. The data is expected to be a list of objects, each of which has the properties ‘name’, ‘id’, and ‘tabs’ (optional). If you don’t want the tabs, just don’t pass a ‘tabs’ list, and don’t include it on your dataset. In the future, the object returned from the constructor will have some useful public methods.

var bfs = new BetterFriendSelector({
  action:"index.html",
  method:"GET",
  submit_text:"Send Friend Request",
  data:fs_data,
  limit:10,
  elem:$("#bfs"),
  tabs:[{key:"hasapp",name:"Friends With App"},          {key:"nonapp",name:"Friends Without App"}],
  extra_form_params:extra_form_params
});

Anyways, I’d like to go through the implementation of what I have so far.

Disappearing Checkboxes

This is the best feature of the condensed selector on Facebook. When a user clicks to select one of their friends, the entry moves to the "selected" area, but with an 'X' instead of a checkbox. This makes it easy to see who's been selected, even while filtering or tabbing the results. Here's how I did that:

// Friend selected event
selector.find("input[type=checkbox]").change(function(){
  var count = selector.find("input[type=checkbox][checked=true]     length;
  if (count > limit) {
    $(this).attr("checked",false);
    return false;
  }
  $(this).parents(".friend").addClass("hidden");
  var selected_elem = $("<div id="">").addClass("selected_frien    ;
  selected_elem.tmpl(BFS.templates.selected_elem,{"name":$(this    siblings("label").text(), "id":$(this).attr("id")});
  selector.find(".selected").append(selected_elem);
});
// Friend unselected event
selector.find(".remove").live("click",function(){
  selector.find(".unselected #" + $(this).attr("id")).att    "checked",false).parents(".friend").removeClass("hidden");
  $(this).parents(".selected_friend").remove();
});

I think this is fairly straightforward, so I'll just list a few points to help understand it:

The correct event to bind is "change". My first attempt had me binding to the "click" event on the checkbox, which was flawed. You can modify the checkbox by clicking it, with your keyboard, or clicking on the label, not to mention other Javascript. "change" catches all of these options
I use the friend's id to index the items. Checkboxes have an id of 'cbXXX' and the corresponding 'selected' element has an id of 's_cbXXX'.
I'm using John Resig's micro-templates, so that's what the .tmpl() call is. The template data is stored in a static collection of strings.
I allow the developer to specify a selection limit ("select up to X friends"), and enforce that in the 'change' event. Returning false prevents the checkbox from becoming selected.
My first attempt used .hide() and .show(), which seems to be natural. I changed it to a class with 'display:none;'. This is because there are a number of reasons a friend element might be hidden, such as being filtered or toggled. Using classes for these ensures they stay hidden when I want them to be.

Typeahead Name Filter

There are plenty of solutions for typeahead to be found online, but I decided to implement my own. It turned out to be really easy, so I'll just show the code and a few points. The key feature of my implementation is that it filters on the start of the first and last name simultaneously, so entering 'be' will keep both 'Ben Franklin' and 'David Beckham'.

var filter = selector.find(".filter input");
var reset_filter = function(){
filter.addClass("empty").val(filter.siblings(".__empty_text").tex    ));
};
// Filter changed event
var do_filter = function() {
var filter_text = filter.val();
if (filter_text == filter.siblings(".__empty_text").text()) {
filter_text = "";
}
selector.find(".unselected label").each(function(){
with($(this)) {
if(!text().match("^" + filter_text) && !text().match("[- \t]+"     filter_text)) {
parents(".friend").addClass("filtered");
} else {
parents(".friend").removeClass("filtered");
}
}
});
};
filter.keyup(do_filter);
// Filter preloaded text stuff
reset_filter();
filter.focus(function() {
if ($(this).val() == $(this).siblings(".__empty_text").text()) {
$(this).removeClass("empty").val('');
}
});
filter.blur(function() {
if ($(this).val() == "") {
reset_filter();
}
});
selector.find(".filter .clear_filter").click(function(){
reset_filter();
do_filter();
});

I'm using a class instead of .hide() and .show() for the same reason as above
The textbox is watermarked using the text in a hidden element next to the textbox

What's Next?

I hope someone finds this control or it's code to be useful. I'm going to keep working on it, and I hope to create something that becomes useful at Slide. My next task is to create a mechanism for handling our "whale users", probably some sort of smart pagination. I'll also be adding some public methods like selectRandom() to make the control easily programmable. Another hurdle will be allowing friend data to be loaded dynamically, by passing some sort of generator function to my control. Again, I hope this was useful, and let me know if you're using or modifying this!

Adequately Good

Replacing `setTimeout` Globally

Failed Approaches

Another Attempt

A Solution

Update! A Better Solution

Thoughts on the Hashbang

It’s Not About the Hashbang

Why Applications on the Web?

The Tradeoff

The Hashbang is a Good Thing

Saner HTML5 History Management

Hashchange

History Management

What’s the Use Case?

So What’s the Problem?

Enter “pathchange”

Writing Testable JavaScript

Avoid Singletons

Avoid Closure-based Privacy

Write Tight Functions

Write Lots of Tests

Conclusions

Spying Constructors in JavaScript

Debugging Closures and Modules

Intro to debugger

Debugging Modules

Safety and Configuration

Problems With Minifiers

Conclusions

Object-to-Primitive Conversions in JavaScript

Types and Primitives

Basic Usage

toString

valueOf

How It Works

Confusion About the + Operator

Bad Stuff

How About Performance?

How About Browser Support?

Conclusions

JavaScript-Style Objects in Python

JavaScript: Better and Faster

JavaScript Module Pattern: In-Depth

The Basics

Anonymous Closures

Global Import

Module Export

Advanced Patterns

Augmentation

Loose Augmentation

Tight Augmentation

Cloning and Inheritance

Cross-File Private State

Sub-modules

Conclusions

Performance of === vs. ==

For Science!

Results

Conclusion

Minimum Timer Intervals in JavaScript

The Results

The Source Code

This Blog - Now With HTML5

Finding Improper JavaScript Globals

Introducing badglobals.js

Exclusions

Report Object

How It Works

Browser Support

Thanks to...

Get badglobals.js

JavaScript Scoping and Hoisting

Scoping in JavaScript

Declarations, Names, and Hoisting

Name Resolution Order

Named Function Expressions

How to Code With This Knowledge

What the Standard Says

Search/Replace in the DOM with jQuery

Intro to `debugger`