Nifty Snippets

catch...from

noreply@blogger.com (T.J. Crowder) — Tue, 09 Feb 2010 13:03:00 +0000

A brief post off my usual JavaScript track for a digression into Java and related languages (although come to think of it, the concept below applies just as much to JavaScript as Java).

Exception handling features in languages are fantastic. They let us (if we're careful and thorough) express the mainline logic of our code uninterrupted by the niggly little detail that just about anything can fail at just about any time, and then let us handle those failures in clearly-defined "...and if that goes wrong, do this" blocks. In pseudocode:

try
{
    doThis();
    doThat();
    doTheOther();
    return result;
}
catch (ExceptionalCondition ec)
{
    // Handle it
}

Lovely and wonderful. We were able to express what we're really trying to do (this, that, and the other) clearly, but then handle the fact that this, that, or the other can fail. The equivalent in, say, C would be to use the convention that returning 0 means everything was fine and returning !0 means something went wrong:

if (doThis() == 0)
{
    if (doThat() == 0)
    {
        if (doTheOther() == 0)
        {
            return result;
        }
     }
}
// Handle the fact that something failed

(Although you probably wouldn't write it quite like that.)

So for obvious reasons I'm a big fan of exception constructs, but there's a refinement I've been wanting for a good 10+ years now (maybe it's about time I got around to submitting a JSR or something; they don't seem to be getting my telepathic messages):

catch..from

This is probably best expressed with an example:

FileOutputStream    output;
Socket              socket;
InputStream         input;
byte[]              buffer;
int                 count;

// Not shown: Opening the input and output, allocating the buffer,
// getting the socket's input stream

try
{
    while (/* ... */)
    {
        count = input.read(buffer, 0, buffer.length);
        output.write(buffer, 0, count);

        // ...
    }
}
catch (IOException ioe from input)
{
    // Handle the error reading the socket
}
catch (IOException ioe from output)
{
    // Handle the error writing to the file
}

As you can see, the goal here is to separate the logic that handles errors reading from the socket from the completely unrelated logic that handles errors writing to the file.

A very basic idea, but I've never heard of anyone implementing it. This may be because I'm so shockingly brilliant that no one else has ever thought of this. (A kind reader will not immediately consider the converse argument.) Or, more likely, there are a lot of subtleties involved that make it tricky.

For instance, exceptions thrown by other objects being used under-the-covers by the socket or file output stream instance need to be handled as though thrown by the socket or output stream instance once they get to the level where we've caught them. But that shouldn't be that hard, although it would require that the stack trace record instance information.

Another wrinkle comes from checked exceptions: Does my code above throw IOException, or not? I've caught it if it's thrown by input or output, and those are the only two instances I've used methods on that may throw the exception, so no, my code doesn't throw IOException. But it'll require enhancing compilers so they can figure that out. But that, too, doesn't seem that complicated.

It's very hard to do the above (in a general way) without language support, hence the suggestion that it may be worth adding a construct to the language.

Happy coding.

GET information, POST changes

noreply@blogger.com (T.J. Crowder) — Thu, 17 Dec 2009 13:53:00 +0000

A micro-post, but this isn't said often enough: Don't use GET requests to change data server-side, use POST (for example, when using Ajax to modify things server-side). GETs are supposed to be idempotent, which is a fancy way of saying that doing them repeatedly has the same effect as doing them once. So, GET the contents of a message on a message board, but POST new messages to the board.

More (in incredibly turgid prose!) in Section 9.1.2 of the HTTP spec.

A better way to hide .htaccess, WEB-INF

noreply@blogger.com (T.J. Crowder) — Tue, 13 Oct 2009 11:59:00 +0000

If you use Apache as your web server of choice, you may wish to have files or directories that Apache pretends are not there. For me, this is because I like to have Apache proxy a servlet container backend, but I'm too lazy to separate out the files, and so I just point Apache and the servlet container at the same directory and tell Apache to pass on the relevant requests to the servlet container (e.g. JSPs, servlets, etc.). The only problem is, that common directory will tend to contain things like the WEB-INF directory and its subdirectories, which I kind of don't want Apache to serve up to the public! If you use .htaccess files, you'll have the same sort of situation.

The usual answer is to simply deny access to the file or directory in question, as in this default rule from the Apache2 config:

<Files ~ "^\.ht">
    Order allow,deny
    Deny from all
</Files>

That sends a 403 (Forbidden) reply when someone tries to access .htaccess. And the same sort of thing can be applied for WEB-INF using the DirectoryMatch directive:

<DirectoryMatch "(^|/)WEB-INF($|/)">
    Deny from all
</DirectoryMatch>

And that's great, but I'm a bit paranoid -- why does the outside world need to know that the thing is there at all? I'd much rather it sent a 404. And huzzah, mod_rewrite to the rescue!

<DirectoryMatch "(^|/)WEB-INF($|/)">
    RewriteEngine on
    RewriteRule .* - [L,R=404]
</DirectoryMatch>

Since the RewriteRule is already inside the DirectoryMatch that will only match what we want, its own regex can just match everything. The L flag says this is the last rule, but the R flag is the magic: It forces a redirect, and if you use the R=xyz form, it redirects with the given code; in this case, a 404. This does the right thing even if you have a custom error document (and you have custom error documents, right?).

Voilà! As far as the outside world is concerned, there just isn't a WEB-INF directory there at all.

If you like, you can have a general rule that works whether mod_rewrite is loaded or not:

<DirectoryMatch "(^|/)WEB-INF($|/)">
    <IfModule mod_rewrite.c>
        RewriteEngine on
        RewriteRule .* - [L,R=404]
    </IfModule>
    <IfModule !mod_rewrite.c>
        Deny from all
    </IfModule>
</DirectoryMatch>

This will do a 404 if mod_rewrite is loaded, but fall back to a 403 of not.

Final note: If you're letting Apache generate directory listings for you by not including a directory file, the above won't hide the WEB-INF diretory (or whatever you're hiding) in those listings. It's easy enough to do it, though: Inside the relevant directive for the directory being listed, just use IndexIgnore to tell the mod_autoindex module to ignore it:

IndexIgnore WEB-INF

In my case, since I want WEB-INF to be hidden everywhere, I can just include it inside my main Directory directive. I never let Apache generate listings for me anyway, but it's nice to have a backstop if I blow away my index file.

Private Methods in JavaScript

noreply@blogger.com (T.J. Crowder) — Sun, 13 Sep 2009 14:21:00 +0000

The question of truly private methods comes up intermittently in JavaScript mailing lists. Most respondents point to Crockford's solution, but then also point out how much memory it takes (each instance gets its own copy of methods; but on the up side, it also gives you truly private member variables as well as methods).

Recently I've been switching to the module pattern for each class, thanks in part to kicking around the Prototype source code a fair bit (and also having read Juriy Zaytsev's article on named function expressions). Since private scope is an intrinsic part of the module pattern, I wondered whether it could help us define truly private methods. And the answer is: Of course it can.

The Pattern

For those not familiar with it, the module pattern is basically:

Use an enclosing function to provide scope
Use named functions within your function
Export the ones you want to have as public methods by returning them from the function (usually as an object hash)

E.g., something like this:

var CoolModule = (function() {
    function foo() { ... }
    function bar() { ... }
    function cool() { ... }

    return {foo: foo, bar: bar};
})();

The anonymous function is the scoping function, which we define and then run immediately, assigning its return value to the CoolModule variable. We end up with a CoolModule object with the properties foo and bar that refer to named functions with the same names. Note that in this example, I did not include cool in the returned object hash. So what's cool for?

The cool function is accessible anywhere within the scoping function, but no where outside of it. So foo can use it, as can bar, but nothing else can. It's a truly private function for the CoolModule module. That's already really useful, but we can also apply it to classes:

Defining Classes

Okay, so let's look at building a class that way. Most toolkits provide some means of creating a "class" based on an object hash of methods. For instance, using Prototype's Class.create, we can define a class like so:

var Thingy = Class.create({

    initialize: function(name) {
        this.name = name;
    },

    doSomething: function() {
        // ...
    }

});

That passes an object literal with named properties for each of our "methods", which Class.create uses to produce a "class" for us (well, more technically a constructor function with the given methods as members on the prototype; JavaScript doesn't have classes, of course, it's a prototypical language — but that's a different subject).

Defining Thingy that way has a problem: All of its functions are anonymous. They're assigned to properties, and those properties have names, but our functions do not. This is a problem any time any tool wants to help us out &mdash for instance, if the browser wants to tell us what function an error occurred in, or if a debugger wants to show us the call stack. So we're better off using the module pattern here, like so:

var Thingy = Class.create((function(){

    function initialize(name) {
        this.name = name;
    }

    function doSomething() {
        // ...
    }

    return {initialize: initialize, doSomething: doSomething};
})());

Now our functions have proper names. (They can have different names from the properties that reference them, which is sometimes handy, but let's keep them the same for now.)

Doing that gives us a truly private scope (our anonymous function), which surely helps with giving us truly private methods, right?

Right.

Class Methods

You remember that cool function from earlier, the one we defined but didn't export. The easiest and most obvious benefit we get from our scoping function is that we immediately have class methods with no need for any other work: We just define a function and don't export it in the return value. It's available to all of our other functions, but it's completely closed off from the outside. It's not tied to any instance, but class methods are quite useful things.

var Thingy = Class.create((function(){

    function initialize(name) {
        this.name = name;
    }

    function doSomething(param) {
        var transmogrified = transmogrify(param);
        // ...
    }

    function transmogrify(str) {
        return str.toUpperCase();
    }

    return {initialize: initialize, doSomething: doSomething};
})());

Our public method doSomething uses our private class method transmogrify to do something to its parameter, presumably something other methods will also need to do but which shouldn't be a public feature of the class. Very handy!

But can this help us get private instance methods, too, without massive memory overhead?

Yup. Remember that JavaScript doesn't have methods, it has functions (very powerful ones). And how a function falls into our somewhat inappropriate class-based terminology ("function", "instance method", "class method") depends entirely on how it's called; there's nothing intrinsic to the function that makes it one of those things. So if we have truly private class methods, that means we also have truly private instance methods, it's just how we call them. In particular, to get an instance method, all we do is call it in a way that we set its context (its "this" value). JavaScript provides three ways to do that:

By assigning it to a property on an object, and then calling it via method notation: obj.method()
By using Function#call
By using Function#apply

The only difference between the last two is how they accept arguments to pass to the function.

Based on the above, so far, I've come up with four ways to get truly private instance methods (at least two of which I know I've seen elsewhere [one after I came up with it, one before]; and I can't imagine the other two are really original, either, even if I haven't seen them). They have varying trade-offs between speed, convenience, and maintainability.

Calling All Instances

The first way is really quite obvious: Using Function#call or Function#apply. Here's an example:

var Thingy = Class.create((function(){

    function initialize(name) {
        this.name = name;
    }

    function doPublic() {
        doPrivate.call(this);
    }

    function doPrivate() {
        alert(this.name);
    }

    return {
        initialize: initialize,
        doPublic: doPublic
    };
})());
var t = new Thing("Fred");
t.doPublic(); // Alerts "Fred" via private instance method

Our doPublic function calls our doPrivate function via Function#call; our truly private instance method then shows the name of the Thingy. doPrivate's context ("this") is set to the instance, and so it has normal access to the instance's properties, including name. Voilà! (Sort of.)

Going Procedural

If, like me, you thought "doPrivate.call(this)" looked vaguely familiar, we may have something in common: A background in procedural languages. Using Function#call to call a function and set its context looks very much like just calling a function and passing it a parameter. So even though it's cheating a bit, let's just give a nod to procedural programming here:

var Thingy = Class.create((function(){

    function initialize(name) {
        this.name = name;
    }

    function doPublic() {
        doPrivate(this);
    }

    function doPrivate(self) {
        alert(self.name);
    }

    return {
        initialize: initialize,
        doPublic: doPublic
    };
})());
var t = new Thing("Fred");
t.doPublic(); // Alerts "Fred"

Here, we just have doPublic call doPrivate in a procedural fashion, passing in the "this" reference. Since there are no private properties in JavaScript, doPrivate has access to just as much information as it did when called in a more instance-like way.

Okay, but really the holy grail has to be calling the method in the "normal" way: this.method()

Leveraging Prototypes

As you probably know, when an object is created via a constructor function, its prototype gets set to the prototype property of the constructor function. All of the members on that prototype are inherited by the object. They're not copied, they're inherited. That opens the door to a means of getting the holy grail syntax with truly private methods, but at a cost: What if we create a new object with "this" as its prototype? Then we can put the private methods on it, and no one else can see them. Well, this does work, but the costs are pretty high. Let's walk through it.

First, how do we create a new object and assign "this" as its prototype? Well that part's actually quite easy: We just have a constructor function lying around, set its prototype property to the instance we want to be the prototype, create a new instance with it, and then restore the constructor's prototype (just to be tidy, we shouldn't keep a reference to the other instance anywhere). (Some browsers let us directly manipulate the prototype of an actual object via a __proto__ property, but others don't and it hasn't been standardized, so we'll stick to the constructor way.) Here's what that looks like:

var Helper = (function() {

    // A factory for setting up our private instances
    function factory() {
    }

    // Our public makePrivate function
    function makePrivate(instance, privateMethods) {
        var rv, name;

        // Set the prototype of our factory
        factory.prototype = instance;

        // Get our private instance
        rv = new factory();

        // Reset the factory prototype (just to be tidy)
        factory.prototype = Object.prototype;

        // Copy over the private methods
        for (name in privateMethods) {
            rv[name] = privateMethods[name];
        }

        // Done
        return rv;
    }

    // Publish the "makePrivate" function
    return {makePrivate: makePrivate};
})();

And we can use it like this:

var Thingy = Class.create((function() {
    var privateMethods;

    // The private methods for our class
    privateMethods = {
        doPrivate: doPrivate
    };

    function initialize(name) {
        this.name = name;
    }

    function doPublic() {
        var self;

        // Get a version of "this" that has our private stuff
        self = Helper.makePrivate(this, privateMethods);

        // Use it
        self.doPrivate();
    }

    function doPrivate() {
        alert(this.name);
    }
    return {
        initialize: initialize,
        doPublic: doPublic
    };

})());
var t = new Thingy("Fred");
t.doPublic(); // Alerts "Fred"

Leaving aside the setup overhead, we've reached the holy grail, right?

Hmmm, not quite. This is all very well for if we read properties, but if we write them, we run into a bit of a problem: Setting a property on our private instance will only set it there, not on its prototype, so when we discard our private instance (as we do at the end of doPublic), any changes are lost with it &mdash e.g., if doPrivate changes anything:

    function doPrivate() {
        alert(this.name);
        this.name = this.name.toUpperCase();
    }

...the changes are lost:

var t = new Thing("Fred");
t.doPublic();  // Alerts "Fred"
alert(t.name); // Also alerts "Fred", not "FRED"

Not good. Now, your first thought (if you're like me) is that all we need is a method that sets the property on the underlying instance, and all private methods need to be sure to use it. But no, if you think about it a little longer, you realize that not only the private methods but also the public ones have to use it (since if we call them, this will refer to our private instance). Worse, not only your own methods, but all superclass methods also have to use it. Now, if you're an adherent to the philosophy that all property access should be through getters and setters anyway, that will work, and your setter would look like this:

function setName(name) {
    var inst = this.hasOwnProperty('_original')
               ? this._original
               : this;
    inst.name = name;
}

...assuming you've modified makePrivate to store the original instance on the private instance as the property _original. If you're not an adherent of that getter/setter philosophy, though (and I've never seen any substantial JavaScript written by anyone who was), it's a problem.

There's another way: Just as we have to do something when we "go private", we just add a step for when we're "done" with the private thing. The "done" function takes any properties explicitly assigned to the private instance and copies them to the original. Like so:

var Helper = (function() {

    // A factory for setting up our private instances
    function factory() {
    }

    // Our public enterPrivate function
    function enterPrivate(instance, methods) {
        var rv, name;

        // Set the prototype of our factory
        factory.prototype = instance;

        // Get our private instance
        rv = new factory();

        // Restore the factory prototype (just to be tidy)
        factory.prototype = Object.prototype;

        // Remember the private methods and the original
        // object, we'll want them for later
        rv._privateMethods = methods;
        rv._original = instance;

        // Copy over the private methods
        for (name in methods) {
            rv[name] = methods[name];
        }

        // Done
        return rv;
    }

    // Our public exitPrivate function
    function exitPrivate(instance) {
        var name, original;

        original = instance._original;
        for (name in instance) {
            if (name != '_original' &&
                instance.hasOwnProperty(name) &&
                !(name in instance._privateMethods)
               ) {
                original[name] = instance[name];
            }
        }
        return original;
    }

    // Export our public functions
    return {enterPrivate: enterPrivate, exitPrivate: exitPrivate};
})();

..and then we can use it like this:

var Thingy = Class.create((function() {
    var privateMethods;

    // The private methods for our class
    privateMethods = {
        doPrivate: doPrivate
    };

    function initialize(name) {
        this.name = name;
    }

    function doPrivate() {
        alert(this.name);
        this.name = this.name.toUpperCase();
    }

    function doPublic(name) {
        var self;

        // Public->Private boundary, need to "enter" private mode
        self = Helper.enterPrivate(this, privateMethods);

        // ...do things...
        self.doPrivate();

        // Private->Public boundary, need to "exit" private mode
        Helper.exitPrivate(self);
    }

    return {
        initialize: initialize,
        doPublic: doPublic
    };

})());
var t = new Thing("Fred");
t.doPublic();  // Alerts "Fred"
alert(t.name); // Also alerts "FRED"

Now we're safe from losing our updated properties when we're done with the private instance (provided we don't forget to call exitPrivate!).

Naturally the enterPrivate and exitPrivate functions mean we incur a lot of runtime overhead; but at least once we're "in," we're able to do things with completely normal syntax and it Just Works. And using prototypes in this way is very cool. Unfortunately, though, all of that copying back and forth and object instantiation really does kill performance.

Enh, Forget Prototypes

If we're going to be doing a lot of copying of things when "entering" and "exiting" private code, we can try to improve performance by ditching object instantiation and fancy prototype stuff and just mucking about with the original instance directly:

var Helper = (function() {

    // Our public enterPrivate function
    function enterPrivate(instance, methods) {
        var old, name;

        old = {};
        for (name in methods) {
            if (instance.hasOwnProperty(name)) {
                old[name] = instance[name];
            }
            instance[name] = methods[name];
        }
        if (instance.hasOwnProperty('_oldStuff') && !('_oldStuff' in old)) {
            old._oldStuff = instance._oldStuff;
        }
        if (instance.hasOwnProperty('_methods') && !('_methods' in old)) {
            old._methods = instance._methods;
        }
        instance._methods = methods;
        instance._oldStuff = old;
        return instance;
    }

    // Our public exitPrivate function
    function exitPrivate(instance) {
        var methods, old, name;

        methods = instance._methods;
        delete instance._methods;
        old = instance._oldStuff;
        delete instance._oldStuff;

        for (name in methods) {
            delete instance[name];
        }
        for (name in old) {
            instance[name] = old[name];
        }
        return instance;
    }

    // Export our public functions
    return {enterPrivate: enterPrivate, exitPrivate: exitPrivate};
})();

And then we use it like so:

var Thingy = Class.create((function() {
    var privateMethods;

    // Our private methods
    privateMethods = {
        doPrivate: doPrivate1
    };

    function initialize(name) {
        this.name = name;
    }

    function doPrivate() {
        alert(this.name);
        this.name = this.name.toUpperCase();
    }

    function doPublic(name) {

        // Public->Private boundary, need to "enter" private mode
        Helper.enterPrivate(this, privateMethods);

        // Do some stuff
        this.doPrivate();

        // Private->Public boundary, need to "exit" private mode
        Helper.exitPrivate(this);
    }

    return {
        initialize: initialize,
        doPublic: doPublic
    };

})());
var t = new Thingy();
t.doPublic();  // Alerts "Fred"
alert(t.name); // Alerts "FRED"

Well, like the prototype solution, it works, but it has some pretty ugly performance.

I should also mention that both the prototype and the overlay solutions will wreak havoc if you have a subclass with a private method that has the same name as a public method on a base class. You Have Been Warned.

Performance Matters

Okay, so bringing it all together, we've looked at four mechanisms (one with two variations) that avoid the problem of creating a copy of every method for every instance. We've mentioned that a couple of them incur quite a bit of overhead. But...how much overhead, really? I mean, do they cut speed by 10%, 20%, 30%? Enquiring minds want to know!

Well, I put together basic test for six options:

Don't worry, just use public methods and tell people not to call them (e.g., by naming convention or some such).
Use the Function#call variety.
Use procedural programming.
Use the prototype trick, with an "exit" that finds and copies back properties set since going private.
Use the prototype trick, being careful to always use "set" methods.
Using the overlay trick (updating the instance in place).

You can grab the source or run it for yourself here; here's the skinny from some popular browsers on Windows:

Firefox:

IE7 (I don't have IE8 on my little netbook):

Chrome:

Safari:

Opera:

As you can see, our "holy grail" solutions (the last three in each set) perform abysmally. It varies by browser quite a bit, but across the board they're really dramatically slow. Mind you, my implementations of them may well be sub-optimal (I dashed them off), but the implementation would have to improve hugely to make performance acceptable. Combine that with the inevitable bugs related to forgetting an "enter" or "exit" call, and I think you'd be hard-pressed to find a good use case for them.

The results for the first three are interesting. Chrome and IE7 are apparently faster at finding a function object by looking at the properties of the instance than they are by looking on the scope chain; Firefox, Safari, and Opera are faster when we ask for the function from the scope chain. The theme that definitely emerges, though, is that if performance matters to you, just use public methods people are supposed to leave alone, or make your truly-private functions procedural (although the cost of calling them via Function#call instead isn't very high, if you really prefer that).

As a side-note, although I didn't test it, I would expect Crockford's pattern (closures for each instance, each instance gets its own copy of all functions) to have call times in line with the procedural timings above, as that's effectively what it does, just at the instance rather than class level.

Does Performance Matter?

So the next question is, okay, so some of the mechanisms impact call time by huge amounts &mdash more than an order of magnitude in some cases. But...does it matter? Computers are crazy fast these days, right?

Chrome, the fastest in this test (and many others) varied from performing a private call in 0.000003418396717245264495027600135 seconds (three hundreths of a millisecond) at its fastest to doing it in 0.00013514974591847767326197426749 seconds (1.3 milliseconds) at its slowest. IE7, the slowest in this test (and many others) varied from 0.000092537755404204915605567071365 seconds (9/10ths of a millisecond) per call down to 0.48923679060665362035225048924 seconds (just under half a second) per call. And these tests were done on a netbook-class machine (wonderful little unit, don't let the picture in the WP article put you off, it's a color-balance problem in the photo).

Everyone has to make their own decision about this. My stake in the ground is that it does matter, yes, particularly as long as the majority of web users are still using the slowest web browser out of the big players: IE. In our modern web applications, a click by a user might kick off 10-20 underlying JavaScript function calls (at least!). If we say 10 and half of those are "private," it makes the difference between IE in responding in no less than 5 milliseconds to taking at least 2.4 seconds. That's a big drop in user-perceived performance. So for my money, within an application I probably won't worry and will just make things public; in library work (either internal libraries used by multiple apps, or publicly-released libraries) I'll probably go with truly private class methods and pass in instances when necessary (e.g., procedural use).

Whatever you end up doing, I hope this roundup of private method options is useful.

Happy coding,

-- T.J.

Simple, Efficient Supercalls in JavaScript

noreply@blogger.com (T.J. Crowder) — Wed, 09 Sep 2009 16:19:00 +0000

In this post, I'll be outlining a simple means of doing supercalls in JavaScript without causing unnecessary runtime overhead, running into maintenance issues, or falling afoul of the new "strict" mode of ECMAScript 5.

The Short Version

The mechanism in brief is:

Have a "make a class" function (most libraries do).
In that function, detect that a subclass is overriding a superclass function and, if so, put a reference to the superclass function on the subclass function instance as a property.
To call a superclass function from a subclass function, authors use Function#call or Function#apply on the $super property of the subclass function instance (being sure to get the function directly, not via this). This is simplest and fastest with named functions, but possible with anonymous ones too.
Allow for mixins, if you're into that sort of thing (and why not, they're cool).

That's it. Dead simple, and dramatically more efficient than the wrapping subclass functions in closures, doing function decompilation (which has never been standardized), etc. It leads to more maintainable (and readable) code than having subclasses refer to their parents by name, and even encourages the use of named functions, which is a good idea for lots of other reasons.

Okay, on to the details, including a full sample implementation, and discussion around the various choices and issues.

What Lead To This

Let me step back a moment and explain how this came about. I have a project requiring a fair bit of server-side JavaScript and also object hierarchy. Not wanting to use "raw" JavaScript hierarchy or roll my own, I naturally reached for my favorite JavaScript library, Prototype, since this is one of the many things it does. Now, Prototype is currently tied to web browsers (something the core team are fixing even as I write this), so I figured until Prototype 2 comes out, I'd just copy the relevant parts (the Class class) and then switch to using Prototype 2 later when it comes out.

So I started doing that, but then remembered that I'm not a big fan of how Prototype does supercalls — you know, when a subclass overrides a superclass method but then wants to call the superclass's version. My initial issue was mostly with the API it provides (the superclass's function is a special parameter to the subclass function, which seems odd to me), but as I was copying Class over and reading through the code, and realized that what Prototype does under the covers to make supercalls work (which includes function decompilation, wrapping in closures, and creating a new function on every call to a supercall-enabled method) is an awful lot of runtime overhead.

So I thought: "Surely I could just tweak this a bit to..." Yes, that's right, I fell right into the anti-pattern of reinventing the wheel. But I'm glad I did, because it taught me a lot I didn't know before, and lead me (with some help from my friends) to come up with this dead-simple mechanism for supercalls. (Caveat: I can't imagine this mechanism is unique, but it was new to me.)

Hierarchy In JavaScript

I won't go into a thorough discussion of hierarchy in JavaScirpt, but suffice to say that whether you use Prototype or something else, it probably provides something that lets you create "classes" by calling a special helper function, passing in an object that has properties referencing the instance methods you want in your class, and getting back a constructor you can call:

// Defining the class
var SpiffyClass = Helper.makeClass({
    initialize: function() {
        // ...initialize an instance...
    },
    nifty: function() {
        // ...do something nifty...
    }
});

// Creating an instance
var spiffy = new SpiffyClass();

This pattern — passing an object literal into a helper function — has been very successful in recent years, and for good reason: It's terse but expressive. (There's a problem with it we'll come back to later, but it gets the job done.)

Inside The Helper

Let's look into the helper to see what it's doing. Here's a first-cut on what Helper.makeClass might look like — the code comments basically tell the story (download it here if the below is awkward to read; I hate Blogger!):

// Take I, doesn't help with supercalls.
// Inspired by Prototype's Class class (http://prototypejs.org)
// Copyright (C) 2009 by T.J. Crowder
// Licensed under the Creative Commons Attribution License 2.0 (UK)
// http://creativecommons.org/licenses/by/2.0/uk/
var Helper = (function(){

    // Build and return a constructor; we do this with a separate function
    // to minimize what the new constructor (a closure) closes over.
    function makeConstructor() {

        // Here's our basic constructor function (each class gets its own, a
        // new one of these is created every time makeConstructor is called).
        function ctor() {
            // Call the initialize method
            this.initialize.apply(this, arguments);
        }

        return ctor;
    }

    // This function is used when a class doesn't have its own initialize
    // function; since it does nothing and can only appear on base classes,
    // all instances can share it.
    function defaultInitialize() {
    }

    // makeClass: Our public "make a class" function.
    // Arguments:
    // - base: An optional constructor for the base class.
    // - ...:  One or more specification objects containing properties to
    //         put on our class as members. If a property is defined by more
    //         than one specification object, the last in the list wins.
    // Returns:
    //     A constructor function for instances of the class.
    //
    // Typical use will be just one specification object, but allow for more
    // in case the author is drawing members from multiple locations.
    function makeClass() {
        var base,       // Our base class (constructor function), if any
            argsIndex,  // Index of first unused argument in 'arguments'
            ctor,       // The constructor function we create and return
            members,    // Each members specification object
            name;       // Each name in 'members'

        // We use this index to keep track of the arguments we've consumed
        argsIndex = 0;

        // Do we have a base?
        if (typeof arguments[argsIndex] == 'function') {
            // Yes
            base = arguments[argsIndex++];
        }

        // Get our constructor
        ctor = makeConstructor();

        // Apply the base class if any by assigning an instance of it to
        // our constructor's prototype
        if (base) {
            ctor.prototype = new base();
        }

        // Set our constructor property so "this.constructor" resolves
        // correctly
        ctor.prototype.constructor = ctor;

        // Assign the members from the specification object(s) to the prototype
        // Again, typically there's only spec object, but allow for more
        while (argsIndex < arguments.length) {
            // Get this specification object
            members = arguments[argsIndex++];

            // Copy its members
            for (name in members) {
                ctor.prototype[name] = members[name];
            }
        }

        // If there's no initialize function, provide one
        if (!('initialize' in ctor.prototype)) {
            // Note that this can only happen in base classes; in a derived
            // class, the check above will find the base class's version if the
            // subclass didn't define one.
            ctor.prototype.initialize = defaultInitialize;
        }

        // Return the constructor
        return ctor;
    }

    // Return our public members
    return {makeClass: makeClass};
})();

So far, nothing new, this is just the kind of thing you'll find in most libraries.

Supercalls

The helper as written is already really useful:

var Parent = Helper.makeClass({
    nifty: function() {
        return "Nifty!";
    }
});
var Child = Helper.makeClass(Parent, {
    spiffy: function() {
        return "Spiffy!";
    }
});
var c = new Child();
alert(c.nifty());  // Alerts "Nifty!" using Parent#nifty
alert(c.spiffy()); // Alerts "Spiffy!" using Child#spiffy

But what if we want to override #nifty in Child to do some processing on Parent's return value? Well, we can do that by going direct to the parent and using Function#call (or Function#apply), but the syntax is quite awkward:

var Child = Helper.makeClass(Parent, {
    nifty: function() {
        var rv = Parent.prototype.nifty.call(this);
        return rv.toUpperCase();
    },
    spiffy: function() {
        return "Spiffy!";
    }
});
var c = new Child();
alert(c.nifty());  // Alerts "NIFTY!"

That works, but there are a few good reasons not to do it:

It's long-winded, inviting irritating mistakes.
It's makes re-basing a class difficult (every method that does this has that superclass name in it).
It makes moving code between classes difficult (ditto).

In short, it's a maintenance problem. Manageable, but a problem. And this is why most libraries try to address it in various ways, often involving adding closures and decompilation and indirection and whatnot. On the "up" side, the above is quite direct and performs very quickly. But what if we could get that directness and speed without getting all of that other stuff, and with brevity and maintainability?

We can.

With a Little Help(er) From My Friends

Our makeClass function has all of the information it needs to help us out: It can easily tell when we're overriding a base class function, and it can give us a reference to that function without our having to know the base class name. How? By putting it on our function instance. Remember that functions are first-class objects, we can put properties on them.

This turns out to be really easy. In makeClass, when copying the members from the specification object(s) to the new constructor's prototype, if a member we're copying is a function and we have a base and the base value is also a function, we stick it on the override function as $super. E.g., we replace these lines from earlier:

for (name in members) {
    ctor.prototype[name] = members[name];
}

...with:

for (name in members) {
    value = members[name];
    if (base && typeof value == 'function') {
        baseValue = base.prototype[name];
        if (typeof baseValue == 'function') {
            value.$super = baseValue;
        }
    }
    ctor.prototype[name] = value;
}

(Also add the 'value' and 'baseValue' declarations at the top.)

So now we can get at our superclass's function by looking at the $super property on our own function. But how do we get that? In our Child#nifty function earlier, say, how do we get to Parent#nifty? If your first thought (like mine) is this.nifty.$super, I'm afraid you're not thinking it through — that will work only for one level of hierarchy, it will fail if anything (say, GrandChild) derives from Child and has its own #nifty function, because with a GrandChild instance, this.nifty is always GrandChild#nifty. If Child#nifty called this.nifty.$super in that situation, it would be calling itself, not Parent#nifty, recursing until the engine gave up on it. So this is out.

And here we run into an issue. We want to get at our current function object, but so far we haven't been giving our functions names. They're just anonymous functions we've assigned to properties on objects. The properties have names, but the functions do not, and we've just concluded we can't use the property (because we can only get to it via this, and that means it'll always point to the bottommost — GrandChild — function). So now what?

Well, there is a way, but it's a bit ugly (on a couple of fronts): arguments.callee. The arguments object, I'm sure you know, is an array-like object defined within functions that contains the arguments passed into the function. That object also has a property on it called callee that is a reference to the actual function object. So we can get at our super function using arguments.callee.$super. And like all other functions, we can call it and pass in the value to use as this by using the Function#call function. So:

var Child = Helper.makeClass(Parent, {
    nifty: function() {
        var rv = arguments.callee.$super.call(this); // Blech
        return rv.toUpperCase();
    },
});
alert(c.nifty());  // Alerts "NIFTY!"

That works, and is more efficient than most other schemes, but it has some issues:

It's long-winded, inviting irritating mistakes (wait, where have I heard that before?)
The new ECMAScript standard, 5th edition, has a "strict" mode in which arguments.callee (amongst other things) is disallowed (many thanks to John-David Dalton and Juriy Zaytsev — kangax — for pointing this out to me)
It's slow (although faster than many other ways of doing this), most browsers don't set up arguments.callee unless you use it, and they're pretty slow at setting it up (thanks again, kangax, for that one)

Still, it is functional (outside of strict mode). But what if we could get there another way without those problems?

You guessed it, we can. And in fact, we get a lot of other benefits when we do.

Named Functions

The problem is anonymous functions, and it's not the only problem with them either. Anonymous functions are tools-hostile. Our browsers can't tell us the name of the function in which an error occurred if it's anonymous. Our debuggers can't show us meaningful call stacks with anonymous functions (we just see a lot of question marks, usually). Etc.

So if the problem is anonymous functions, well, let's just give our functions names! That way, we can use the name directly, since a function's name is defined throughout the scope in which the function is defined (including within the function itself); that's how we usually write recursive functions, after all.

So when defining classes, how can we give our functions names? One's first thought might be to simply do this:

var Child = Helper.makeClass(Parent, {
    nifty: function nifty() { // <= PROBLEMATIC
        // ...
    }
});

In some ways, that should work, but it has ramifications. The name "nifty" gets defined in the enclosing scope, which in our case above is global — not a good idea! And it flat out doesn't work with Internet Explorer (or anywhere you're using JScript) or Safari. (For the full story, check out Juriy's excellent article on the subject, it's a good read.)

No, it would be better to contain the scope a bit (not to mention working around bugs), which is easy enough: Wrap it up in a function:

var Child = Helper.makeClass(Parent, (function(){
    function nifty() { 
        // ...
    }
    function spiffy() { 
        // ...
    }
    return {
        nifty:  nifty,
        spiffy: spiffy
    };
})());

That probably wants breaking down a bit:

We define an anonymous function for scoping
Within that scoping function, we define our various methods for Child — as named functions
Our function names are in scope throughout the scoping function
Our scoping function returns an object with the properties mapped to our named functions (I added a "spiffy" to this example just to make clear how it works with multiple functions)
We execute the scoping function immediately and pass the result (the object) into makeClass

No scope bleed, no issues with IE or Safari, we're all set.

So okay, but so far I've been leaving out the good bit — the supercall in Child#nifty! Here we go:

var Child = Helper.makeClass(Parent, (function(){
    function nifty() { 
        var rv = nifty.$super.call(this);
        return rv.toUpperCase();
    }
    function spiffy() { 
        // ...
    }
    return {
        nifty:  nifty,
        spiffy: spiffy
    };
})());
var c = new Child();
alert(c.nifty()); // Alerts "NIFTY!"

Within #nifty, we refer to the function by name — no this, that's very important! — which always refers to the right one. We get its $super property, and use Function#call on it. (We could use Function#apply if we wanted.) We pass in this as the context parameter so that this is correct within the supercall.

Revisiting makeClass

I floated this supercalls idea on the Prototype Core development mailing list, and Allen Madsen pointed out that if we're going to be using functions to create enclosing scope when defining classes, makeClass should just support our passing those functions in directly rather than making us execute them first. It simplifies the syntax markedly:

var Child = Helper.makeClass(Parent, function(){
    function nifty() { 
        var rv = nifty.$super.call(this);
        return rv.toUpperCase();
    }
    return {nifty: nifty};
});
var c = new Child();
alert(c.nifty()); // Alerts "NIFTY!"

We just define the function rather than defining and calling it; makeClass will call it for us. So let's go support that in makeClass (we'll still support providing the specification object directly) by marking constructor functions (so we know whether we have a base class, since everything might be a function now — this was also Allen's idea) and then just invoking any functions we find to get their specification objects. (Code below under "Bringing It All Together".)

Mixins

A fairly common pattern these days it to have "mixins" that can be mixed into classes. A "mixin" is a collection of functions that coordinate to provide some functionality, intended to be added to a class rather than to be a class in its own right. One mixin might be used by several different classes. Mixins are sometimes seen as a replacement for derivation, but to me they're an adjunct — there are times you derive, and other times you mix in.

Mixins are the reason that Helper.makeClass allows multiple specification objects. Suppose we have a mixin that provides semantics around getting and setting a "cool" property:

var CoolMixin = (function(){
    function setCool(cool) {
        if (typeof cool != 'string') {
            throw "Hey, man, that ain't cool.";
        }
        this.cool = cool;
    }

    function getCool() {
        return this.cool;
    }

    return {getCool: getCool, setCool: setCool};
})();

Now we can mix that into any class:

var CoolChild = Helper.makeClass(Parent, CoolMixin, function() {
    function spiffy() {
        // ...
    }
    return {spiffy: spiffy};
});
var cc = new CoolChild();
alert(cc.nifty());  // Alerts "Nifty!" via Parent#nifty
cc.setCool(1);      // throws the uncool exception

CoolChild inherits from Parent, but also gets the cool stuff from CoolMixin. By convention (and in order to ensure that a subclass's own members take precedence), mixins are always included after the parent and before the child's own members.

Mixins raise an issue for makeClass: Normally, it modifies function instances, setting a $super property on them if they override a base class function (it leaves them alone if they don't). But mixins are used in multiple classes, so we shouldn't touch them. While in practice it would be fairly harmless to set $super on the mixin function if the mixin function didn't call $super (which is definitely an edge case), it's still not right and in certain very edgy edge cases could cause class cross-talk, which is a Bad Thing(tm).

To handle this, we can provide a means of marking methods as being mixin methods and telling makeClass to leave them alone. (I actually mark the methods rather than doing it by context because some implementations — like Prototype — allow modifying hierarchies after initial derivation, in which case it's really important to know which functions are mixin functions and which aren't). It may seem to be pandering to an edge case, but there's effectively no runtime cost and the code is trivial, so let's do it — see Helper.makeMixin below.

Alternately, we can take a page from previous implementations and — in this one very specific case — wrap the mixin function so we can set the $super property on the wrapper. I haven't done that here, but it's a viable solution if you think mixins calling $super is a common enough use case for you.

Anonymouses Anonymous

Not everyone will want to use named functions; perhaps they have a large amount of legacy code using specification objects directly. Rather than leave them with the ugly syntax:

arguments.callee.$super.call(this, ...);

...let's give them a helper method to take on some of that complexity:

this.callSuper(arguments, ...);

But we don't want to do that all the time. Hey, that's the case for a mixin! See the CallSuperMixin below.

Bringing It All Together

Okay, taking all of the above, mixing in an IE workaround (thank you, Prototype!), and adding in Helper.makeMixin, we end up with this:

// Take IV: Explicitly handle mixins, provide a mixin for calling super when
// working with anonymous functions.
// Inspired by Prototype's Class class (http://prototypejs.org)
// Copyright (C) 2009 by T.J. Crowder
// Licensed under the Creative Commons Attribution License 2.0 (UK)
// http://creativecommons.org/licenses/by/2.0/uk/
var Helper = (function(){
    var toStringProblematic,    // true if 'toString' may be missing from for..in
        valueOfProblematic;     // true if 'valueOf' may be missing from for..in

    // IE doesn't enumerate toString or valueOf; detect that (once) and
    // remember so makeClass can deal with it. We do this with an anonymous
    // function we don't keep a reference to to minimize what we keep
    // around when we're done.
    (function(){
        var name;

        toStringProblematic = valueOfProblematic = true;
        for (name in {toString: true, valueOf: true}) {
            if (name == 'toString') {
                toStringProblematic = false;
            }
            if (name == 'valueOf') {
                valueOfProblematic = false;
            }
        }
    })();

    // Build and return a constructor; we do this with a separate function
    // to minimize what the new constructor (a closure) closes over.
    function makeConstructor() {

        // Here's our basic constructor function (each class gets its own, a
        // new one of these is created every time makeConstructor is called).
        function ctor() {
            // Call the initialize method
            this.initialize.apply(this, arguments);
        }

        return ctor;
    }

    // This function is used when a class doesn't have its own initialize
    // function; since it does nothing and can only appear on base classes,
    // all instances can share it.
    function defaultInitialize() {
    }

    // Get the names in a specification object, allowing for toString and
    // valueOf issues
    function getNames(members) {
        var names,      // The names of the properties in 'members'
            name,       // Each name
            nameIndex;  // Index into 'names'

        names = [];
        nameIndex = 0;
        for (name in members) {
            names[nameIndex++] = name;
        }
        if (toStringProblematic && typeof members.toString != 'undefined') {
            names[nameIndex++] = 'toString';
        }
        if (valueOfProblematic && typeof members.valueOf != 'undefined') {
            names[nameIndex++] = 'valueOf';
        }
        return names;
    }

    // makeClass: Our public "make a class" function.
    // Arguments:
    // - base: An optional constructor for the base class.
    // - ...:  One or more specification objects containing properties to
    //         put on our class as members; or functions that return
    //         specification objects. If a property is defined by more than one
    //         specification object, the last in the list wins.
    // Returns:
    //     A constructor function for instances of the class.
    //
    // Typical use will be just one specification object, but allow for more
    // in case the author is drawing members from multiple locations.
    function makeClass() {
        var base,       // Our base class (constructor function), if any
            argsIndex,  // Index of first unused argument in 'arguments'
            ctor,       // The constructor function we create and return
            members,    // Each members specification object
            names,      // The names of the properties in 'members'
            nameIndex,  // Index into 'names'
            name,       // Each name in 'names'
            value,      // The value for each name
            baseValue;  // The base class's value for the name

        // We use this index to keep track of the arguments we've consumed
        argsIndex = 0;

        // Do we have a base?
        if (typeof arguments[argsIndex] == 'function' &&
            arguments[argsIndex]._isConstructor) {
            // Yes
            base = arguments[argsIndex++];
        }

        // Get our constructor
        ctor = makeConstructor();
        ctor._isConstructor = true;

        // Apply the base class if any by assigning an instance of it to
        // our constructor's prototype
        if (base) {
            ctor.prototype = new base();
        }

        // Set our constructor property so "this.constructor" resolves
        // correctly
        ctor.prototype.constructor = ctor;

        // Assign the members from the specification object(s) to the prototype
        // Again, typically there's only spec object, but allow for more
        while (argsIndex < arguments.length) {
            // Get this specification object
            members = arguments[argsIndex++];
            if (typeof members == 'function') {
                members = members();
            }

            // Get all of its names
            names = getNames(members);

            // Copy the members
            for (nameIndex = names.length - 1; nameIndex >= 0; --nameIndex) {
                name = names[nameIndex];
                value = members[name];
                if (base && typeof value == 'function' && !value._isMixinFunction) {
                    baseValue = base.prototype[name];
                    if (typeof baseValue == 'function') {
                        value.$super = baseValue;
                    }
                }
                ctor.prototype[name] = value;
            }
        }

        // If there's no initialize function, provide one
        if (!('initialize' in ctor.prototype)) {
            // Note that this can only happen in base classes; in a derived
            // class, the check above will find the base class's version if the
            // subclass didn't define one.
            ctor.prototype.initialize = defaultInitialize;
        }

        // Return the constructor
        return ctor;
    }

    // makeMixin: Our public "make a mixin" function.
    // Arguments:
    // - ...:  One or more specification objects containing properties to
    //         put on our class as members; or functions that return
    //         specification objects. If a property is defined by more than one
    //         specification object, the last in the list wins.
    // Returns:
    //     A specification object containing all of the members, flagged as
    //     mixin members.
    function makeMixin() {
        var rv,         // Our return value
            argsIndex,  // Index of first unused argument in 'arguments'
            members,    // Each members specification object
            names,      // The names in each 'members'
            value;      // Each value as we copy it

        // Set up our return object
        rv = {};

        // Loop through the args (usually just one, but...)
        argsIndex = 0;
        while (argsIndex < arguments.length) {
            // Get this members specification object
            members = arguments[argsIndex++];
            if (typeof members == 'function') {
                members = members();
            }

            // Get its names
            names = getNames(members);

            // Copy its members, marking them as we go
            for (nameIndex = names.length - 1; nameIndex >= 0; --nameIndex) {
                name = names[nameIndex];
                value = members[name];
                if (typeof value == 'function') {
                    value._isMixinFunction = true;
                }
                rv[name] = value;
            }
        }

        // Return the consolidated, marked specification object
        return rv;
    }

    // Return our public members
    return {
        makeClass: makeClass,
        makeMixin: makeMixin
        };
})();

Which lets us do things like this:

var Parent = Helper.makeClass(function(){
    function hierarchy() {
        return "P";
    }
    return {hierarchy: hierarchy};
});
var Child = Helper.makeClass(Parent, function(){
    function hierarchy() {
        return hierarchy.$super.call(this) + " < C";
    }
    return {hierarchy: hierarchy};
});
var GrandChild = Helper.makeClass(Child, function(){
    function hierarchy() {
        return hierarchy.$super.call(this) + " < GC";
    }
    return {hierarchy: hierarchy};
});
var gc = new GrandChild();
alert(gc.hierarchy()); // Alerts "P < C < GC"

And I believe I promised a sample mixin to help those anonymouses:

// Define our CallSuper mixin
Helper.CallSuperMixin = makeMixin(function() {
    function callSuper(ref) {
        var f,          // The function to call
            args,       // Arguments to pass it, if we have any
            len,        // Length of args to pass
            srcIndex,   // When copying, the index into 'arguments'
            destIndex,  // When copying args, the index into 'args'
            rv;         // Our return value

        // Get the function to call: If they pass in a function, it's the
        // subclass's version so look on $super; otherwise, they've passed
        // in 'arguments' and it's on arguments.callee.$super.
        f = typeof ref == 'function' ? ref.$super : ref.callee.$super;

        // Only proceed if we have 'f'
        if (f) {
            // If there are no args to pass on, use Function#call
            if (arguments.length == 1) {
                rv = f.call(this);
            } else {
                // We have args to pass on, build them up.
                // Note that doing this ourselves is more efficient on most
                // implementations than applying Array.prototype.slice to
                // 'arguments', even though it's built in; the call to it
                // is expensive (dramatically, on some platforms).
                len = arguments.length - 1;
                args = new Array(len);
                srcIndex = 1;
                destIndex = 0;
                while (destIndex < len) {
                    args[destIndex++] = arguments[srcIndex++];
                }

                // Use Function#apply
                rv = f.apply(this, args);
            }
        }

        // Done
        return rv;    // Will be undefined if there was no 'f' to call
    }

    return {callSuper: callSuper};
});

Conclusion

The goal of this post is not for people to run off and start using the Helper provided here. The goal is to demonstrate a really simple, highly efficient means of implementing supercalls in these sorts of libraries, and discussing some edge cases around doing so. If you use Prototype, you might want to take a look at my patch for Prototype 1.6.1 (I have no idea whether that will make it into Prototype), which handles dynamic redefinition and such (although that patch doesn't — yet — handle mixins).

We've talked about a lot of things, so let's remind oursselves of what the mechanism is at its roots:

Have a "make a class" function.
In that function, detect that a subclass function (Child#nifty) is overriding a superclass function (Parent#nifty) and, if so, put a reference to the superclass function on the subclass function instance as a property.
To call a superclass function from a subclass function, authors use Function#call or Function#apply on the $super property of the subclass function instance (being sure to get the function directly, not via this). This is simplest and fastest with named functions, but possible with anonymous ones too.
Allow for mixins, if you're into that sort of thing (and why not, they're cool)
Oh, and of course:
Profit!

Many thanks again to JDD, kangax, Allen, and others who've helped me hammer this out!

Happy Coding,

-- T.J.

COMPLETELY off-topic...

noreply@blogger.com (T.J. Crowder) — Fri, 14 Nov 2008 10:22:00 +0000

...but if this doesn't gob-smack you, you're dead inside:
http://news.bbc.co.uk/1/hi/sci/tech/7725584.stm
I mean, we've known they were there, in theory, at least since Newton; we've known they were there, by observation (star wobbles, etc.), for 20 years. But this is direct observation. We can see them. To me, this is a Neil Armstrong moment.

Unobtrusiveness

noreply@blogger.com (T.J. Crowder) — Sun, 19 Oct 2008 12:00:00 +0000

Another link-post today (although again a link to a snippet I've written, so not completey OT) since most of my writing time is going to the Unofficial Prototype & script.aculo.us wiki at the moment: How To - Using Unobtrusive JavaScript

Minimizing Download Times

noreply@blogger.com (T.J. Crowder) — Thu, 16 Oct 2008 14:52:00 +0000

Hello all,

That's right, first post since...wow, since April. And it's not even a post, it's sort of a link-post.

I've been doing some work helping build the Prototype user community (moderating the user discussion group, creating an unofficial wiki, that kind of thing) and as part of that I've been doing little mini-articles, much like the ones I expected to do here.

So if you're writing web applications or web pages and you're interested in minimizing the download times for your scripts, check out this article I posted over there: Tip - Minimizing Download Times

You must remember 'this'

noreply@blogger.com (T.J. Crowder) — Tue, 01 Apr 2008 16:30:00 +0000

Of all the tech blogs in all the sites in all the worldwide web, you walk into mine...

If you hang out in JavaScript-oriented newsgroups like these for any length of time, you will eventually see some variation of this question:

Hey, why doesn't this work?
function MyWidget(name)
{
    this.name = name;
    this.element = null;
}
MyWidget.prototype.showName = function()
{
    alert('The name is ' + this.name);
}
MyWidget.prototype.hookElement = function(element)
{
    this.element = element;
    Event.observe(this.element, 'click', this.showName);
}
function test()
{
    var widget;

    widget = new MyWidget('Test Name');
    widget.hookElement(document.getElementById('testDiv'));
}
"testDiv" is a div in the document, and I know that I'm not calling the test() function before the DOM is loaded, so why is it when I click the div I get the message "The name is undefined"?!

(As always, I'm using some convenience syntax in the above for hooking up the event handler.)

The OP (original poster) might even follow on with:

I even tried changing the observe() line to this:
Event.observe(this.element, 'click', function() {
this.showName();
});
because I heard somewhere that you have to do that, but that's even worse, it causes an error saying this.showName() isn't a function?!

The issue here is that the OP hasn't quite grokked "this" and its special role in the JavaScript world.

I talked a bit about 'this' over here, but I wanted to do a post focussing on the specific pitfall the OP above, like so many of us, has fallen into (forgetting 'this') and how you deal with it.

Let's look at what's wrong with this line first:

Event.observe(this.element, 'click', this.showName); // Wrong

JavaScript doesn't have methods (see link above), and so this.showName just returns a function reference with absolutely no connection to the instance the OP wanted to bind to the element. It's just a function. (Used properly, this is a powerful feature, but in this situation it's causing the OP some trouble.) Recall that showName is defined like this:

MyWidget.prototype.showName = function()
{
    alert('The name is ' + this.name);
}

Within the code, 'this' is determined not by where the event handler is set up, but by how the function gets called. Most likely, 'this' will be a reference to the element that was clicked ('testDiv'), because modern browsers use the element related to the event as 'this' within event handlers. Consequently, this.name is undefined unless the element in question happens to have a name attribute.

So to get the intended effect, you have to wrap the call to this.showName() so that 'this' is the MyWidget instance when the code gets executed -- you must remember 'this'. Which is probably what the OP heard about when he tried this:

Event.observe(this.element, 'click', function() {
    this.showName();
}); // Still wrong

This is getting closer, and in fact it would work if we were using a variable to reference the widget rather than 'this', but because it's 'this', we actually still have exactly the same problem we had before: When the event handler gets called, 'this' is the element, not the widget, and so there's no showName() function to call.

So how do we deal with this? Well, here's one approach I've seen to rewriting the hookElement function:

MyWidget.prototype.hookElement = function(element)
{
    var self;

    this.element = element;
    self = this;
    Event.observe(this.element, 'click', function() {
        self.showName();
    });
}

This works because we're no longer using 'this' within the event handler, we're using 'self' (the event handler has access to 'self' because it's a closure; more here). So that solution works. I can't say I like it, though. It just feels...hacky, I guess. But still, it works, and although it looks a bit funny the first time, if you're familiar with the idiom you read right past it thereafter. You just need to be sure the closure isn't unnecessarily preserving some other big amount of data from elsewhere in the function.

Personally, though, I prefer using a reusable "binding" function. Many JavaScript toolkits have these (such as Prototype's bind() and bindAsEventListener()), but it's not complicated:

function bind(f, obj)
{
    return function() {
        return f.apply(obj, arguments);
    };
}

This is a function factory: It creates functions that, when called, will call the given function with the given object set as 'this' (using JavaScript's convenient apply() function; insert your own "The fundamental things apply" joke here). Now we can rewrite the OP's hookElement function like so (changes from the original at the top in bold):

MyWidget.prototype.hookElement = function(element)
{
    this.element = element;
    Event.observe(this.element, 'click',
        bind(this.showName, this)
    );
}

You might be wondering why we have to specify 'this' twice. Remember that this.showName just returns a function reference, with nothing about the instance (we could replace this.showName in the above with MyWidget.prototype.showName if we liked). If we want bind() to know what instance we want to bind the function to, we have to specify it -- the this at the end.

And that's it! Now the event handler works as the OP expected it to.

What's in a name?

noreply@blogger.com (T.J. Crowder) — Sat, 29 Mar 2008 12:59:00 +0000

Micro-post today, folks:

You see a fair bit of confusion from newbies in mailing lists around the "name" attribute vs. the "id" attribute. For instance, I recently saw a "how do I do this" -style post asking how to deal with a form, with the sample being:

<form name='form1'>
<input type='checkbox' id='cb1'>Checkbox 1
</form>

The poster wanted to know how to retrieve the form and submit it (along with some further information) via Prototype's Ajax.Updater. So he wanted to use the serialize() method Prototype adds to form elements when it extends them (e.g., when you retrieve the element using $()).

Consequently, the use of "id" and "name" attributes in his example was exactly backward: He wanted to be able to retrieve the form using $(), which is a general-purpose routine that retrieves elements by their unique ID, and then have the form fields submitted -- but the form field had no name.

Here's the mantra:

Elements have IDs; form fields have names.

IDs are unique in the document; form field names are not (necessarily).

Two further notes:

1. Form fields can also have IDs if you need to refer to their elements in your code (to enable/disable them, etc.), but in terms of sending in a form, fields have names.

2. You don't have to use an ID to get at the element for a form; you can use a name and then get at the form element via document.YourFormName in your JavaScript code. But nowadays we mostly look elements up by their unique IDs; and in the specific case of the question from the poster above, since he was going to want to use $(), he would want an ID.

Mythical methods

noreply@blogger.com (T.J. Crowder) — Mon, 24 Mar 2008 22:00:00 +0000

We frequently talk about JavaScript objects having methods. This is just a convenient myth. JavaScript has functions, but it doesn't have methods. It doesn't need them. Its functions, combined with some syntactic sugar, are more than up to the job.

What is a "method"? I'd have to say that the definition given on Wikipedia is pretty good. As of this writing, it says a method is

...a subroutine that is exclusively associated either with a class...or with an object...

Yeah, JavaScript doesn't have any of those. Granted it seems to have them. For example:

function Guess(killer, location, weapon)
{
    this.killer = killer;
    this.location = location;
    this.weapon = weapon;
}
Guess.prototype.accuse = function()
{
    alert('It was ' + this.killer
        + ' in the ' + this.location
        + ' with the ' + this.weapon
        + '!');
}

function testGuess()
{
    var mustardStudyLeadPipe;
    var plumHallRope;

    mustardStudyLeadPipe = new Guess(
        'Colonel Mustard',
        'study',
        'lead pipe');
    mustardStudyLeadPipe.accuse();

    plumHallRope = new Guess(
        'Professor Plum',
        'hall',
        'rope');
    plumHallRope.accuse();
}

Running testGuess does indeed show us

"It was Colonel Mustard in study with the lead pipe!"

and then

"It was Professor Plum in hall with the rope!"

so that looks an awful lot like accuse is a method of Guess objects.

Except it isn't. Let's add a bit to our testGuess function (new bits in bold):

function testGuess()
{
    var mustardStudyLeadPipe;
    var plumHallRope;
    var accuse;

    mustardStudyLeadPipe = new Guess(
        'Colonel Mustard',
        'study',
        'lead pipe');
    mustardStudyLeadPipe.accuse();

    plumHallRope = new Guess(
        'Professor Plum',
        'hall',
        'rope');
    plumHallRope.accuse();

    accuse = mustardStudyLeadPipe.accuse;
    accuse();
}

What does the final call to accuse show? Does it accuse Colonel Mustard? No, all we've done is get a reference to the accuse function into our local variable, there's nothing there (or in the function definition) that refers to the mustardStudyLeadPipe instance or indeed anything related to Guess. No, what this shows will depend on the HTML document in which it's found, but it'll be something along these lines:

"It was undefined in the http://blog.niftysnippets.org with the undefined!"

Why? Because we didn't do anything to define "this" within the function. (I'll come back to details on why we'd get seemingly-odd alert that includes a URL in a bit.)

There are three main things about JavaScript that make it seem to have methods (leaving aside prototypes and "classes" for the moment): The "this" keyword, the fact that object properties can refer to functions (since they are, after all, just objects), and the fact that when you execute a function using "dotted" notation (e.g., object.function()), the object is set automatically as "this" within the function call.

Let's look at each of those.

The "this" keyword: This keyword looks familiar if you're coming to JavaScript from a background in C++, Java, C#, and the like, where "this" within a method is guaranteed to refer to an instance of the class defining the method (or a subclass). And in JavaScript, "this" does refer to an object instance, but that's where the similarity ends. Other than the name being the same and it referencing an object, it bears no relation to the "this" keyword in class-based languages like C++, Java, or C#. In fact, in many ways, "this" is actually just a function argument that is supplied in a non-obvious (but convenient!) way.

Functions as properties: Let's look again at one line from the code above:

mustardStudyLeadPipe.accuse();

If we were talking about that line of code, we'd probably say "...the accuse method of the mustardStudyLeadPipe object...", which is a useful and convenient way to put it. A more painstakingly-geeky-accurate way of putting it, though, would be "...the function referenced by the accuse property of the mustardStudyLeadPipe object..." Not that anyone's going to say that, but that's really what's going on. Objects don't have methods, they have properties; it's just that a property can refer to a function, since functions are objects like everything else.

Functions called using dotted notation get "this" set for them: This is the part that really makes it seem like JavaScript has methods: The "this" reference gets set automagically to the object instance if you call a method using dotted notation. Let's look at that call again:

mustardStudyLeadPipe.accuse();

This does three completely distinct things: Firstly, it identifies a function to call by getting the function reference from an object property; secondly, it says to execute the function and return its result rather than just get a reference to it (the parentheses do that); and thirdly, it says that within that call, use the object in question as "this". These completely distinct things are combined in that notation for our convenience. Getting the function reference from an object property doesn't link it in any way to the object the property came from (as our accuse test above confirmed), it just gets a reference the function; the JavaScript engine treats dotted notation calls as special and sets up "this" accordingly, but it has nothing to do with the function being called.

Lets prove that another way:

function testGuess2()
{
    var plumHallRope;
    var fakeGuess;

    plumHallRope = new Guess(
        'Professor Plum',
        'hall',
        'rope');
    plumHallRope.accuse();

    fakeGuess = {};
    fakeGuess.location = 'library';
    fakeGuess.killer = 'Mrs. Peacock';
    fakeGuess.weapon = 'revolver';
    fakeGuess.demo = plumHallRope.accuse;
    fakeGuess.demo();
}

This accuses Professor Plum as before, and then:

"It was Mrs. Peacock in the library with the revolver!"

The exact same function produces the alerts in both cases, it's purely the way it was called that defined "this". The fakeGuess object isn't even really a Guess -- "fakeGuess instanceof Guess" returns false -- but it has all of the properties the function expects (killer, location, weapon), and so it works just fine (this is something we'll come back to in a later post). Essentially, "this" is just a function argument that's passed into the function as an implicit feature of the dotted notation.

We can also do it explicitly. JavaScript gives us the call and apply methods on function instances, with which we can say explicitly what we want "this" to be. (They do the same thing, they just provide different ways to specify the function's arguments.) If you say myfunction.call(myobject), you're saying "call myfunction and use myobject as 'this'", which is what the dotted notation does implicitly for you. In fact, this:

plumHallRope.accuse();

equates to

plumHallRope.accuse.call(plumHallRope);

Now all three of the parts we identified above are shown distinctly: Getting the function reference from the property (plumHallRope.accuse) is distinct from the fact we're calling it (call) is distinct from what "this" should be ((plumHallRope)). The fact that these are distinct can be made clearer:

var f;
f = plumHallRope.accuse;
f.call(plumHallRope);

Alternately, here's a more dramatic example:

function testGuess3()
{
    var plumHallRope;
    var fakeGuess;

    plumHallRope = new Guess(
        'Professor Plum',
        'hall',
        'rope');
    plumHallRope.accuse();

    fakeGuess = {};
    fakeGuess.location = 'library';
    fakeGuess.killer = 'Mrs. Peacock';
    fakeGuess.weapon = 'revolver';

    plumHallRope.accuse.call(fakeGuess);
}

Not to bang on about it, but the call at the end accuses Mrs. Peacock, not Professor Plum. The plumHallRope instance was only used to get the function reference, it wasn't used within the function call. In our original testGuess function, we could even do this if we wanted to:

mustardStudyLeadPipe.accuse.call(plumHallRope);

Which accuses Professor Plum. But it's more convenient, isn't it, to say plumHallRope.accuse?

So if "this" is really just a sort of obscure function argument, why have it? Why not just write everything as global functions and pass in the object to act on as the first argument? We could do that (and in fact I did it for years, as a C programmer), but it's more cumbersome. You have to know what functions are meant to be used with what kinds of objects, there's all sorts of stuff littering up the global namespace, etc., etc. By passing around object references, which have properties on them (usually inherited from their prototype; again the topic of an upcoming post) that reference functions intended for use with them, it's just so much more convenient.

Okay, but what was that about that URL showing up earlier when we called accuse? You'll remember earlier when we ran this code:

    accuse = mustardStudyLeadPipe.accuse;
    accuse();

we got the alert with the URL in it. So why did that happen, and where did the URL come from? It was because we didn't give an object to use for "this", and so the function call got the default, which is the JavaScript global object. In browser implementations, the global object is the window object -- so we were showing the values of window.killer, window.location, and window.weapon. In a typical situation, window.killer and window.weapon will be undefined, and of course window.location is the URL of the document in the window.

I chose that example intentionally, because it's a pitfall people fall into a lot: Losing "this". Usually, people lose "this" in the context of an event handler -- e.g., they have an instance (plumHallRope, perhaps), and want to hook up a "method" on it (say, accuse) to an event (a click handler for a button, maybe?), and so understandably they do something like this (I'm again using convenience syntax, as I describe here):

Event.observe('theButton', plumHallRope.accuse); // WRONG

The problem being, that just references the function, nothing about its context. Its context will be determined by how it's called. Earlier when we called accuse directly, because we didn't use dotted notation or call or apply, "this" was the global object; event handlers like this one will usually get "this" set to the element (although not if you're using old DOM0 -- onclick attribute -- events or IE's attachEvent function). To maintain its context, you have to do something like this:

Event.observe('theButton', function() { plumHallRope.accuse(); });

The function that gets called by the event handler then turns around and calls the accuse function such that plumHallRope is "this", so you've preserved "this" using a wrapper (which is also a closure; details). (Note that thanks to a bug in Internet Explorer, that may well cause a memory leak for your IE users. Prototype's Event.observe() does some things to minimize the issue; other frameworks will have other helpers along those lines.)

In Conclusion:

I've seen people deride JavaScript for having "fake" methods, but I think that's missing the point. It doesn't have fake methods, it just doesn't have methods at all. What it does have is very powerful, flexible functions and some convenient syntactic sugar that lets us express the 90% case -- calling a function related to an object instance passing in that object instance as an argument -- in a compact and expressive way, but without limiting our use of the functions involved. By not limiting our use, we can use just about anything as a mixin, we can use duck typing, easily create wrapper objects to enforce or verify contracts, etc.

Like so many things about the language, it's confusing until you grok just how simple it is, and then you start appreciating that it's simple but powerful.

Convenience Syntax in Examples

noreply@blogger.com (T.J. Crowder) — Mon, 24 Mar 2008 18:15:00 +0000

This is a "meta" post, e.g., it's about the blog itself.

In many of my posts, I give code examples to illustrate a point. To avoid cluttering the examples up with browser-specific stuff, I'm using a couple of calls from the Prototype library.

Actually, right now, just the one: Event.observe(). This function hooks up an event handler in a way that is consistent across browsers, even Internet Explorer, which is good because IE doesn't support the standard addEventListener() function, and its own attachEvent() function behaves slightly differently (within the handler, "this" is the global object, like in old DOM0 handlers; whereas with addEventListener it's the element you're hooking the event to). Prototype's Event.observe() ensures that "this" is always the element, even on IE, and also does some housekeeping to try to mitigate an IE bug where it tends to leak memory if you use closures as event handlers (which is incredibly common).

If I use more Prototype stuff, I'll list it here as well, but this is only about trying to keep the examples simple and clear, not about using Prototype or any other specific library...

The horror of implicit globals

noreply@blogger.com (T.J. Crowder) — Tue, 18 Mar 2008 14:40:00 +0000

This code should cause an error, right?

function doSomething()
{
    var x;

    x = 10;
    y = 20;

    alert('x = ' + x);
    alert('y = ' + y);
}

Well, you'd think so, wouldn't you? (There's no declaration for y.) It doesn't, though. It also probably doesn't do what the author intended. Welcome to the horror of implicit globals. The good news is that there's something we can do about it.

So what did the code really do? Well, in a simple test, it would seem to do what the author intended, in that it shows two alerts:

x = 10
y = 20

But x and y are completely different; x is a local variable within the function, whereas y is created as a "global variable". Said "global variable" is not declared anywhere at all, it exists purely because it was assigned to by this function. Functions can create global variables willy-nilly with no advance declaration or indeed, with no declaration at all.

This is clearly a Bad Thing, so why would the language designers let it happen? Well, in some sense I think it's a side-effect. To see why we have the possibility of implicit globals, we have to delve into objects and properties.

If you've done any JavaScript stuff with objects, you know that you can assign a property to an object simply by, well, assigning it:

myobject.myprop = 5;

This sets the property myprop on the object myobject, creating a new property if the property didn't exist before. This is simple, it's convenient, and it's part of the power of JavaScript that objects can just acquire properties without any preamble.

But what does this have to do with implicit global variables? Well, you see, JavaScript doesn't have global variables. No, seriously. It has a global object (just the one), and that object has properties. Those properties are what we think of as global (or page-scope) variables. You might think that this is a distinction without a difference, but if it were, the code shown at the beginning of this post would fail because y isn't declared. It doesn't fail, because by assigning y a value, we created a property on the global object.

"Now hang on," you're saying, "there's nothing in that code referencing some kind of 'global object'." Ah, but there is. And that brings us to the scope chain.

I've mentioned the scope chain before. The scope chain is how JavaScript resolves unqualified references: In any given execution context (e.g., global code or function), there is a chain of objects that the JavaScript engine looks to when resolving an unqualified reference: It first checks to see if the top object has a property with the given name, and uses that property if it does; if not, it checks the next object in the chain, etc. The global object is always the last object in the chain. If the engine gets all the way down to the global object and we're assigning a value to the unqualified reference, the reference is assigned as a property of the global object. And since properties don't have to be declared in advance, voilá, implicit globals.

We can see this in practice. The JavaScript specification says simply that there is a global object and that it's always the root of the scope chain, but it doesn't say what it is -- which makes sense as JavaScript is a general-purpose language. In browser-based implementations, though, the global object has a name: window. Technically, the symbol window in browser-based JavaScript is a property of the global object that refers to the object itself. I mention this not only because it's useful to know, but because it lets us prove to ourselves that "global variables" are really properties of the global object. Let's add a line to our code from above:

function doSomething()
{
    var x;

    x = 10;
    y = 20;

    alert('x = ' + x);
    alert('y = ' + y);
    alert('window.y = ' + window.y);
}

This starts out by showing the same two alerts we had before:

x = 10
y = 20

...and then it shows a third:

window.y = 20

...because y and window.y both refer to the same property of the same object, the global object.

All of this holds true even if we do declare y at global scope:

var y;
function doSomething()
{
    var x;

    x = 10;
    y = 20;

    alert('x = ' + x);
    alert('y = ' + y);
    alert('window.y = ' + window.y);
}

This shows the same three alerts shown earlier. (There is a very slight technical difference between declared and undeclared globals that relates to the delete statement, but the distinction isn't important here, it's just a flag on the property.)

Now, some readers will be thinking "Cool! This means I don't have to have 'var' statements for my globals! I can save a bit of space on the downloaded script files!" Well, true, you could. It's an awfully bad idea, though. Not only would it be a bit unfriendly to anyone else trying to read your script (I suppose you could address that somewhat with comments that get stripped out before download), but it would also mean you couldn't use any lint tools, and you really, really want to. Because the ramification of all of this is that a simple typo, perhaps:

thigny = 10;

instead of

thingy = 10;

can create a new global variable in your page, introducing a bug that's awfully hard to find. Fortunately, though, people have created various tools to do lint checking on JavaScript, tools that can find that error for you before you even get to the testing stage, much less production.

And so there we are, the horror of implicit globals -- and the relief of knowing that we have a defense against them!

Now at blog.niftysnippets.org

noreply@blogger.com (T.J. Crowder) — Mon, 17 Mar 2008 13:32:00 +0000

Just briefly, this blog is now at http://blog.niftysnippets.org, rather than http://niftysnippets.blogspot.com. It's still hosted by Blogger (for now), and Blogger is kind enough to forward you from the old address, but if for some reason you're a regular reader of this blog, it's worth updating your Atom/RSS links just in case someday I end up hosting it myself or moving to Wordpress or something...

When you absolutely, positively need new scope

noreply@blogger.com (T.J. Crowder) — Mon, 17 Mar 2008 11:52:00 +0000

Just a quick one today, one that in many ways is a solution in search of a problem... ;-)

As you probably know, in most of the languages with vaguely C-like syntax (C, C++, Java, C#, ...), a new block introduces new scope. E.g., this Java code:

String myJavaMethod()
{
    String s;

    s = "outer string";

    {
        String s;

        s = "inner string";

        // (Presumably do something with 's' in here)
    }

    return s;
}

...while strange, is perfectly valid. It returns a String with the text "outer string". The s variable declared within the inner block is scoped only to the inner block.

A similar-looking JavaScript function has a completely different result:

function myJavaScriptMethod()
{
    var s;

    s = "outer string";

    {
        var s;

        s = "inner string";

        // (Presumably do something with 's' in here)
    }

    return s;
}

It's still valid, but it returns the string "inner string" rather than "outer string". Blocks do not introduce new scope in JavaScript, only functions do that. (If you're thinking that the second var declaration should be an error, well, I agree -- but it isn't, details here.)

But what if you absolutely, positively need new scope within a function? It's actually pretty easy to achieve; just use a closure. If I really needed my JavaScript function above to behave like the Java version, I could rewrite it like this:

function myJavaScriptMethodUpdated()
{
    var s;

    s = "outer string";

    (function()
    {
        var s;

        s = "inner string";

        // (Presumably do something with 's' in here)
    })();

    return s;
}

Looks weird, eh? What I've done is create an anonymous inline function and then immediately execute it. Since it's a closure (all JavaScript functions are closures), it has access to all of the arguments and variables defined in its containing function except for the ones it's masked by declaring its own (which it's done with s).

Naturally, as this is defining and calling a function, there's a performance aspect although setting up a function call should be fairly quick in any decent JavaScript implementation -- and again, you're only doing this when you absolutely, positively need new scope, right?

So, okay, how is this useful? Well, as I say, this may well be a solution in search of a problem. It mostly gave me an excuse to highlight the fact that blocks don't introduce new scope in JavaScript. And frankly, in most cases if you need to set up new scope within a function like that, it probably means you haven't broken things up into small enough pieces; if you do that, you'll probably find that you don't absolutely, positively need new scope because your pieces are small enough that things are scoped properly.

That said, though, think in terms of sandboxing and namespacing -- if you wrap an entire external library in a closure and then execute that closure, any globally-declared vars in that library become local variables within the closure and can't conflict with others defined at global scope. So there may be something useful there, although it's far from a real sandboxing or namespacing solution...

A simple point about design

noreply@blogger.com (T.J. Crowder) — Sun, 16 Mar 2008 11:09:00 +0000

A good friend of mine pointed me at this comic and I just had to share it. I've rarely seen this point put more...simply.

Closures by example

noreply@blogger.com (T.J. Crowder) — Sat, 15 Mar 2008 15:00:00 +0000

In an earlier post, I promised a follow-up with a few examples of closures. Before we get to the examples, a quick note: To avoid cluttering things up with browser-specific stuff, I'm using the Event.observe() method from Prototype. Event.observe() hooks up an event handler, allowing for differences between browser implementations (see the link for more detail).

I have to say that I found this post to be a real challenge, and it took me a while to figure out why: I kept trying to boil things down to their essence, and the fact is that if you do that, you end up with a single example, one that demonstrates that a closure function has access to intrinsic data. And let's face it, that's not very interesting. :-)

So rather than boil things down, I'll give a few examples of where closures frequently get used, even though they all pretty much demonstrate the same fundamental point (about the functions having intrinsic data).

Okay, to the examples:

A Simple Bound Event Handler
Enduring References
Enduring References to What
Private Properties
Callbacks
The Inadvertent Closure (an anti-pattern)
Your Examples

#1: A Simple Bound Event Handler

One of the most common uses of closures is to bind a function and some data to an event handler:

function wireUpMessage(element, msg)
{
    Event.observe(
        element,
        'click',
        function()
        {
            alert(msg);
        }
    );
}

This wireUpMessage function creates a closure (the bit in bold) and hooks it up to the "click" event of the given element, having it show the given message. The closure keeps a reference to the arguments and variables in scope where it was defined, so it has access to the msg argument we passed the wireUpMessage function, even though that function has returned.

Assuming we have a button with the ID "btnSayHey", we can use this as follows:

wireUpMessage('btnSayHey', 'Hey there');

Try it out:

#2: Enduring References

You could easily get the idea from the previous example that the closure we created somehow had the text "Hey there" bound into it literally. That's not true. It's the reference to the context containing the msg argument that's bound to the closure, not the value of that argument. The msg argument's value is evaluated when the closure is executed, not when it's defined. This is important, powerful, and frequently misunderstood. ;-)

Consider this:

function setupCounterButtons(startBtnName, showBtnName, stopBtnName)
{
    var counter;
    var intid;

    counter = 0;
    intid = undefined;

    function startCounter()
    {
        stopCounter();
        counter = 0;
        intid = window.setInterval(function() { ++counter; }, 200);
    }
    function stopCounter()
    {
        if (intid !== undefined)
        {
            window.clearInterval(intid);
            intid = undefined;
        }
    }
    function showCounter()
    {
        var msg;

        msg = "Counter = " + counter + " (";
        if (intid === undefined)
        {
            msg += "not ";
        }
        msg += "running)";
        alert(msg);
    }

    Event.observe(
        document.getElementById(startBtnName),
        'click',
        startCounter
    );
    Event.observe(
        document.getElementById(showBtnName),
        'click',
        showCounter
    );
    Event.observe(
        document.getElementById(stopBtnName),
        'click',
        stopCounter
    );
}

This function has the local variables counter and intid, three named functions (which are closures), and a fourth anonymous closure we've passed into window.setInterval. The named functions are: startCounter, which starts a 200ms repeating update of the counter local variable; showCounter, which displays the current counter value; and stopCounter, which stops the 200ms update. It hooks these functions up to the buttons whose names we pass into the function. So if we hook up the buttons 'btnStartCounter1', 'btnShowCounter1', 'btnStopCounter1' with this code:

setupCounterButtons('btnStartCounter1', 'btnShowCounter1', 'btnStopCounter1');

...we get to try it out:

This demonstrates two important things: Firstly, the closures have an ongoing reference to the counter variable, not its literal value where they're defined. Secondly, all of the closures are referencing the same context: When startCounter sets the counter and intid values, showCounter sees those changes; when the closure we've enabled via window.setInterval updates counter, we see that update. All closures defined in the same context share access to that same context.

#3: Enduring References to What?

Look again at the previous example and think for a moment about what it is that the closures have access to, which endures even after the setup function has completed. Is it something unique attached to the setup function? Well, if that were true, what would happen if I called the function again and hooked it up to different buttons; would those buttons control and access the same counter that the first set of buttons do? If your impulse is to say "yes," consider that counter is a local variable within the function. Local variables aren't specific to the function for all eternity, right? They only relate to a specific time you've called that function. So if we call it again...yup, that's it, we get a new context, and the closures within that context access that new context, not the previous one.

And so if we hook up a new set of buttons:

setupCounterButtons('btnStartCounter2', 'btnShowCounter2', 'btnStopCounter2');

We get a second counter. Try it out and see whether there's any interaction between the counter controlled by these buttons:

...and the one controlled by the buttons in the previous example.

Right, there isn't any, because they're referencing different counters.

#4: Private Properties

Lots of JavaScript apps these days make extensive use of object orientation. One key OOP principle is information hiding. There are several good reasons for information hiding. One example is to defend against an object instance having an invalid state. Frequently this is done by having private data members and only allowing access to them via accessor methods that can prevent invalid values. Since all properties of JavaScript objects are public, how can we have private data members? The answer, of course, is to use closures, much as we did with the counters above. (Crockford is probably the first to document how you can do this, here.)

Here's an example, a Circle class that ensures that its radius is never allowed to become negative:

function Circle(radius)
{
    this.setRadius = function(r)
    {
        if (r < 0)
        {
            throw "Radius cannot be less than zero.";
        }
        radius = r;
    };
    this.getRadius = function()
    {
        return radius;
    };

    this.setRadius(radius);
}

Note that we don't have a "radius" property on this object at all. This code:

var c;
c = new Circle(10);
alert("Radius: " + c.radius); // undefined!!

Shows "Radius: undefined" because the property simply doesn't exist. Try it:

Instead, the setRadius and getRadius methods of the object are closures, both of which have a reference to the context in which they were defined, and they share that context as they were defined in the same scope. So rather than having a "radius" property, they simply use the radius parameter given to the constructor, which they keep a reference to even after the constructor has returned. Consequently, this code works just fine:

var c;
c = new Circle(10);
alert("Radius before update: " + c.getRadius());
c.setRadius(20);
alert("Radius after update: " + c.getRadius());

Try it:

(I should mention that there is a downside to this. When you do this, all Circle objects [for example] have their own copies of the setRadius and getRadius functions, rather than sharing copies on an underlying Circle.prototype object. That's a whole different topic, though.)

#5: Callbacks

Closures get used as callbacks a lot. Of course, event handlers are callbacks and so we've already talked about this, but it's useful to remember non-event-handler callbacks.

One very common example is container objects that offer a method (usually called "each" or "forEach") that will call a callback once for each contained object. Prototype provides this (Enumerable.each) on arrays and several other objects; Dojo has something similar (dojo.forEach); JavaScript 1.6 defines this for arrays (Array.forEach). So if we have an array (say) of Person objects, we can act on each element like this (using Prototype syntax):

personArray.each(
    function(person)
    {
        // Do something with the 'person' object
    }
);

Suppose we wanted to build a sublist of only the people who are 65 and over:

var sixtyFivePlus;
sixtyFivePlus = [];
personList.each(
    function(person)
    {
        if (person.age >= 65)
        {
            sixtyFivePlus.push(person);
        }
    }
);

That uses a closure to add each 65-and-over person to the sixtyFivePlus array.

In practice, closures are used like this in modern JavaScript all over the place.

#6: The Inadvertent Closure (an anti-pattern)

In Example #1, I said that a very common use of closures was to bind a function and some data to an event handler. Unfortunately, while it may literally be true that that's a very common use, I suspect many times it's not the intent of the programmer creating the closure. I suspect much of the time, the programmer just wanted to bind the function, not any data, to the event handler, but didn't realize they were creating a closure and binding data as well.

Unless a page author is using DOM Level 0 event handling (e.g., onclick attributes in the HTML markup), usually there's a setup function called on page load that hooks up event handlers to the buttons and such using some analog of the Event.observe function I've been using in this post. Sometimes those setup functions are dealing with large data sets, perhaps a big JSON or XML document retrieved via an XMLHttpRequest query, referencing that data with local variables. Then the event handlers are defined within the setup function, which means they're closures, which means...right, they keep a reference to the setup function's context, and so the large data set is retained in memory.

Now, if that's on purpose -- the page is referring back to it periodically, etc. -- that's fine. But if the data set was just to be used for setup, it's a waste of memory. I call this the "Inadvertent Closure" anti-pattern.

Here's an example:

function mySetup()
{
    var setupData;
    var n;

    // Set 'setupData' to some large amount of data used only for setup;
    // the below is just a contrived array.
    setupData = [];
    for (n = 0; n < 1000; ++n)
    {
        setupData[n] = "Item " + n;
    }

    // (Presumably use the setup data for something here)

    // Hook up some event handlers
    Event.observe(
        'someButton',
        'click',
        function()
        {
            // Do something interesting
        }
    );
    Event.observe(
        'someOtherButton',
        'click',
        function()
        {
            // Do something else interesting
        }
    );
    // (Etc.)
}

Here, a bunch of setup data is allocated and referenced via the setupData array, and then we hook up a couple of event handlers that don't need a reference to the setupData array.

Well, they may not need it, but they have it, which means that all of that data is sitting around eating up memory unnecessarily.

The good news is that it's easy to fix. There are at least two ways to fix it; the way I don't like, and the way I like. ;-)

The way I don't like: Just "null out" the big data set when you're done with it; in our example, we'd just add this line to the end of the mySetup function:

setupData = undefined;

Personally, though, I prefer modularity: Break out the bit that needs to deal with the big data set into its own function, and the setup of the event handlers in their own function:

function myBetterSetup()
{
    doTheBigDataThing();
    setupEventHandlers();
}
function doTheBigDataThing()
{
    var setupData;
    var n;

    // Set 'setupData' to some large amount of data used only for setup;
    // the below is just a contrived array.
    setupData = [];
    for (n = 0; n < 1000; ++n)
    {
        setupData[n] = "Item " + n;
    }

    // (Presumably use the setup data for something here)
}
function setupEventHandlers()
{
    Event.observe(
        'someButton',
        'click',
        function()
        {
            // Do something interesting
        }
    );
    Event.observe(
        'someOtherButton',
        'click',
        function()
        {
            // Do something else interesting
        }
    );
    // (Etc.)
}

This give us a clear separation of what we're doing, and as a nice side-effect, prevents the closures from keeping our data set around unnecessarily.

At this point, a couple of you might be thinking "OMG! But that means any time I'm dealing with a big temporary data set, I need to be sure no closures are defined within that context! What if I want to act on the data set with an iteration function like in Example #5?!" Don't worry. Remember that the context is kept around because it has a reference from the closure; when the closure is released and cleaned up by the garbage collector, the context is also released and can be cleaned up by the GC. This only matters if you're keeping an enduring reference to the closure, as in the case of event handlers.

#7 and on: Your Examples!

I've tried to give some overview of closures based on examples of usage, and I hope it's been useful, but I've really only scratched the surface here; what are your examples? Post away!

Poor misunderstood 'var'

noreply@blogger.com (T.J. Crowder) — Mon, 03 Mar 2008 13:00:00 +0000

It seems most programmers coming to JavaScript from C, C++, Java, and the like equate the var statement with variable declaration statements in the languages they come from and use it the same way. And at some casual level that's reasonable; but it can lead you down a misleading path...

Consider this code:

function foo()
{
    var ar;

    // ...do some stuff, create an array in 'ar'...

    for (var index = 0; index < ar.length; ++index)
    {
        doSomethingWith(ar[index]);
    }
}

This is a common idiom, but a misleading one. You might think that index is only defined within the for loop (that's certainly the impression we're giving in the code). But it's not true: In fact, index is defined throughout the function -- within the loop, outside the loop, above the loop, and below the loop. The var statement defines a variable within the current scope (all of it, not just "from here on"), and unlike some other languages, in JavaScript blocks don't have any effect on scope; only functions introduce a new scope.

Consequently, the above function can be written as it is above, but also with the index declaration...

...at the top:

function foo()
{
    var ar;
    var index;

    // ...do some stuff, create an array in 'ar'...

    for (index = 0; index < ar.length; ++index)
    {
        doSomethingWith(ar[index]);
    }
}

...at the bottom:

function foo()
{
    var ar;

    // ...do some stuff, create an array in 'ar'...

    for (index = 0; index < ar.length; ++index)
    {
        doSomethingWith(ar[index]);
    }

    var index;
}

...anywhere in the middle:

function foo()
{
    var ar;

    // ...do some stuff, create an array in 'ar'...

    for (index = 0; index < ar.length; ++index)
    {
        var index;
        doSomethingWith(ar[index]);
    }
}

...or even all of them!

function foo()
{
    var ar;
    var index;

    // ...do some stuff, create an array in 'ar'...

    for (var index = 0; index < ar.length; ++index)
    {
        doSomethingWith(ar[index]);
    }

    var index;
}

We can get away with that last one because a var statement defining a variable that already exists in the current scope does not replace the variable (this is what keeps you from accidentally masking your function's arguments, or even the arguments array that's provided for you).

This seems like an odd way to define the var statement until you get into the plumbing of JavaScript and how it sets up calls to functions. You can get into some of that by reading my earlier post, Closures are not complicated, but the net effect of the plumbing is that all var statements are treated as though they were at the top of the function (if they have initializers, those become assignments and stay where they are).

So does that mean that the common idiom of declaring an indexer within the loop statement is "wrong"? Well, that's a matter of perspective, and ~~the older I get~~ the more experience I accumulate, the less I think in terms of absolutes like right and wrong. The language spec allows it, so in that sense it's not "wrong". In some ways, it's sort of a shorthand way of telling the next person reading the code that you're going to use it for the loop (and only for the loop, right?), so in that sense perhaps it's not "wrong".

But the further your code gets from expressing what's really happening, the easier it is for someone reading the code later (perhaps you!) to get the wrong end of the stick and introduce a problem. For example, suppose you have a 30-some-odd-line function and the loop appears in within the body of a conditional about two-thirds of the way down:

function foo(someArray)
{
    var thingy;
    var otherThingy;

    // ...20 lines of code...

    if (thingy > otherThingy)
    {
        for (var index = 0; index < someArray.length; ++index)
        {
            doSomethingWith(someArray[index]);
        }
    }

    // ...10 more lines of code...
}

Six months after you write this, Mike edits the function and needs to remember the index of something at the top so he can do something with it at the bottom; he declares an "index" variable, sets index at the top, and then uses it at the bottom, having missed the loop:

function foo(someArray)
{
    var thingy;
    var otherThingy;
    var index;

    index = findSomething(someArray);

    // ...20 lines of code...

    if (thingy > otherThingy)
    {
        for (var index = 0; index < someArray.length; ++index)
        {
            doSomethingWith(someArray[index]);
        }
    }

    // ...10 more lines of code...

    restoreSomething(someArray, index);
}

Mike's introduced a bug, an irritating, intermittent bug. Sometimes the restoreSomething call at the end fails for some reason; not always, mind, but sometimes. (Because index gets set by the loop, but only when thingy > otherThingy.)

Obviously, this bug could have been avoided if Mike had read through the entire function carefully before making his mods. Or if you'd chosen a different name for your index variable (in hopes of reducing the odds of Mike using it). Or it could have been caught by thorough unit tests that explore all conditions (and then Mike would have to go back and fix it).

But let's throw Mike a bone, eh? If we declare the variable in the text in the same place it's defined by the interpreter at runtime, we help him avoid making the mistake in the first place. And we like Mike, we don't want to trip him up...right?

Regardless of your decision about how to write your code, though, understanding what var is really doing can help you get that code doing what you want it to do.

Closures are not complicated

noreply@blogger.com (T.J. Crowder) — Fri, 29 Feb 2008 21:00:00 +0000

Closures seem to frighten people a bit. Partially I suspect this is down to the academic nature of the term "closure". It sounds like something Californians look to achieve, not something to help you write software. (Disclosure: I mostly grew up in San Francisco, so I'm allowed to poke fun at my fellow Californians. Don't Try This At Home.) I suspect it's also because if you don't know the rules for them, they seem mysterious, and the mysterious is often frightening.

First off, what is a closure? I'll leave a thorough definition to my academic betters, but let's put it this way for my fellow plodders and I: A closure is a function with data intrinsically bound to it.

Consider this code:

function updateDisplay(panel, contentId)
{
    var url;

    url = 'getcontent?id=' + contentId;
    jsonRequest(
        url,
        function(resp)
        {
            if (resp.err)
            {
                panel.addClassName('error');
                panel.update(
                    'Error retrieving content ID '
                    + contentId
                    + ' from "'
                    + url
                    + '", error: '
                    + resp.err);
            }
            else
            {
                panel.removeClassName('error');
                panel.update(resp.json.content);
            }
        }
    );
}

This updates a panel element based on the results of a call to retrieve some JSON data. The call to the jsonRequest function accepts two parameters: The URL that will return the JSON data, and a callback function to trigger when the request completes (one way or the other). If the data was returned successfully, the callback sets the content of the panel from the JSON data and makes sure that the "error" CSS class is not set on the element; if there's an error, it shows details about the error and sets the "error" CSS class. (In this example, we happen to be using Prototype to extend our panel element with the nifty class name and update methods, but that's as much to keep the example simple as anything else.)

The callback above is an example of a closure: A function with data bound to it, in this case the panel and contentId arguments we passed into the updateDisplay function, and also updateDisplay's url local variable. It's useful to have this information bound to the callback, because the jsonRequest function doesn't know anything about panel or contentId, it just knows about triggering the callback -- but the callback has the information it needs to do its job.

Magic? Nah. I can't speak for closures in other languages, but there's nothing complicated about closures in JavaScript. Seriously. They're dead easy. You need to know, say, three things and you're good to go. Well, four things, but that's only because someone has been misinforming you. Here's a quick list of those four things, after which we'll delve in a bit deeper, and then I'll tell you something at the end that will surprise you for three seconds before you go "Oh, of course":

In JavaScript, everything is a data structure, even functions, and -- critically -- even the context in which functions run. One aspect of that context is a "call object" (that's Flanagan's term, the ECMA specification uses "activation object" -- I'll use "call object" because ~~I'm lazy and it's less typing~~ I find it clearer).
JavaScript variable names are resolved on the basis of a "scope chain", which includes (among other things) the globlal object and all of the current call objects in scope.
Functions in JavaScript are lexically scoped -- for the plodders like me out there, that means that the things a function can access (the things "in scope" for it) are determined by the context in which the function is defined, not the context in which it's executed.
Closures do not create memory leaks. Someone probably told you that they did at some point (perhaps they thought that's what Microsoft was trying to say here).

Okay, let's see how those things come together to make closures easy.

#1: Everything is a data structure

When we call a JavaScript function, the context of that call to the function -- the parameters we've used, the variables within the function itself -- are part of a data structure called the call object (or, again, "activation object" in ECMA parlance). The interpreter creates a call object for this particular execution of the function and sets some properties on that call object before passing it into the function's code. The properties are:

A property called arguments which is an array (of sorts, in most implementations it's not actually an Array object) of the actual arguments we called the function with, plus a callee property which is a reference to the function being called.
A property for each of the arguments defined by the function declaration. The values of these properties are set to the values passed into the function. If any arguments weren't specified (because JavaScript lets you call functions with however many arguments you want, regardless of the definition), properties for any missing arguments are still created on the call object -- with the value undefined.
A property for every declared variable within the function (e.g., every "var" statement). (These start out with the value undefined.)

Let's look again at selected bits of the updateDisplay function from earlier:

function updateDisplay(panel, contentId)
{
    var url;

    // ...
}

And let's assume we call it with a reference to a 'leftPanel' div and a contentId of 123:

updateDisplay(leftPanel, 123);

That creates a call object for this execution of updateDisplay that essentially looks like this:

call.arguments = [leftPanel, 123]
call.arguments.callee = updateDisplay
call.panel = leftPanel
call.contentId = 123
call.url = undefined

This object is then used within the body of the function when the code uses the arguments panel or contentId, or the local variable url, etc.

"But wait," you say. "I don't refer to an object when I use the arguments or variables in my function, I just use their names." Indeed -- the use of the call object is implicit. How do we end up using the call object when we just write "contentId" (for instance)? Because once the call object is created, it's put at the top of the scope chain for the duration of the function call. Which takes us nicely to...

#2: Variable names are resolved using a "scope chain"

If you've written JavaScript in a browser environment, you've probably used the global document object, and perhaps the global navigator object as well. Here's the thing: Those aren't global objects. Those are properties of a global object -- in fact, of the global object. The document and navigator properties are set up for you by the browser, but they're just properties of an object.

So how do you get away with just giving the property name, rather than giving the object name as well? Because of the scope chain, an ordered series (indeed, a chain) of objects. When the JavaScript interpreter sees an unqualified variable reference, it checks the top object on the scope chain to see if it has a property by that name: If so, it gets used; if not, the interpreter checks the next object down the chain. The global object is the bottom of the chain, so if you just type document.writeln("Blah blah blah"), eventually the document property is found on the global object and used.

(The global object is an abstract entity in the generic JavaScript definition, but in the specific case of a web browser you know it by another name: window. In browsers, the window object is the global object; it just also has a property, "window", that refers to itself.)

So quick: Within a function, how do variable names get resolved? Right! When the function is being executed, the call object with all those nifty properties for the function's arguments and variables is put at the top of the scope chain. So during our call to updateDisplay from earlier, the call object for the call is at the top of the scope chain, followed by the global object, like this:

When the interpreter sees a variable reference, say contentId, it looks on the call object: If there's a contentId property on the call object, it gets used; otherwise, the interpreter looks at the next object in the scope chain, which in this case is the global object.

(There's more to know about the scope chain than I've described here; the astute reader will be wondering, for instance, how objects and their instance members come into play, or what the with statement does to things. Alas, we can't tackle everything all at once...)

But how can I know when I'm writing updateDisplay that the scope chain will look like that? I mean, doesn't it depend on who's calling the function? Nope. And that brings us to our next point:

#3: Functions in JavaScript are lexically scoped

Okay, so we get the concept of the call object, which is created when we call a function; and we get the global object, which sits at the bottom of the scope chain to handle globals. But what's this "lexically scoped" thing? It's just a fancy way of saying that a function's scope is determined by where it's defined (e.g., the text defining it; léxis is Greek for "word" or "speech"), not where it's called.

Let's put that another way: When a function is defined, the code defining it is in some kind of context -- a function is running, or the page itself is running if you've done the code at the top level. That context has an active scope chain when the function is defined. So when creating the function object, the interpreter creates a property on it (called [[Scope]] in the ECMA spec, but you can't access it directly) with a reference to the active execution context's scope chain. Even when the context goes away (e.g., the function returns), because the function object created by the definition has a reference to the scope chain, the scope chain isn't garbage collected -- it's kept alive by the active reference to it from our function object. (Assuming that we've kept a reference to the function object, as we did in our example by passing it into the jsonRequest method; otherwise, the function and the scope chain are both garbage-collected since nothing references them.)

Remember our closure at the beginning of this post? It gets a [[Scope]] property pointing to the scope chain in effect when updateDisplay was called with panel = leftPanel, etc.:

So when we call it, first its scope chain is put in place, then the call object for this particular call to the function is put on top of the scope chain, and the function is executed with this chain:

And there we are, the closure can access panel and all of the other properties of the call object for the call to updateDisplay because they're on the scope chain. They're on the scope chain because that was stuck onto the closure's function object when it was created. No magic. Just data structures, the scope chain, and lexical scoping all working together.

#4: Closures don't cause memory leaks

So why do people think closures cause memory leaks? A couple of reasons, I suspect, but chief among them being ~~a bug in~~ an issue with Internet Explorer. IE's DOM objects are not garbage-collected in the same way that JavaScript objects are; instead, IE shows its COM roots by using reference counting, a mechanism that is, unfortunately, prey to issues with circular references. Event handlers can easily end up being circular references, and so in IE it's easy to "leak" the memory associated with a DOM element and its event handler because they're referring to each other. (JavaScript's garbage collector doesn't have an issue with circular references; it works on the "reachability" principle rather than reference counting.) This is an issue with event handlers, rather than closures, but since many event handlers are written as closures, people associate it with closures. Fortunately, it's not that hard to kick IE into shape (in this particular way); Crockford talks about the problem and its solution here and many toolkits [like Prototype] will do this for you if you ask them nicely.

Lest I be accused of Microsoft-bashing, I should point out that IE is not the only browser that sometimes loses track of things; it's just by far the worst. Firefox 2 has a bad habit of leaking a bit of memory on XmlHTTPRequest calls, which also frequently involve closures. The good news there is that Firefox 3 beta 3 is looking awfully good indeed on this front.

Separate from browser issues, though, if you're not really clear on how closures work, particularly with regard to the scope chain, you'll miss the fact that a closure keeps a reference to all of the variables and arguments in scope where it's created, not just the ones it uses; and so if you have (say) a massive temporary array allocated in that scope that the closure doesn't use, you might be tempted to say that the closure is causing the array's memory to leak. (The answer there is simple: Clear the array's variable when you're done with it.)

In Conclusion

There are lots of nifty things you can do with closures. I'll post a follow-up in a couple of days, "Closures By Example", demonstrating a few of them and just generally walking through some seemingly-complicated examples. But until then, I'll just reiterate my theme: Closures in JavaScript are not complicated. They're powerful, but if you understand that 1. Everything is a data structure, 2. Variables -- I should probably call them unqualified references -- are resolved according to a scope chain, and 3. The scope chain for a function is defined by where the function is defined lexically, you'll be golden.

Oh! I said I'd tell you something at the end that would surprise you for about three seconds before you said "Oh, of course." Here it is:

All JavaScript functions are closures.

Happy coding!

JavaScript's Curiously-Powerful OR Operator (||)

noreply@blogger.com (T.J. Crowder) — Mon, 18 Feb 2008 17:30:00 +0000

The "or" operator (||). It seems innocuous enough: A binary logical operator that returns true if either of its params is true, usually used in the context of a conditional statement:

    if (a || b)

Perhaps if you've been programming in any of several other high-level languages for a while, you expect that it won't evaluate the second param if the first one is true, something called short-circuiting. And you'd be right. But there's a lot more to the JavaScript || operator than that!

Consider this statement:

    x = a || b;

If the || operator were purely a logical operator, then x would receive one of two values: true or false. But that's not what happens. Instead, x gets the value of a if a is true or can be converted to being "true", otherwise it gets the value of b. And that's another kettle of fish entirely.

First off, there's this question of "...or can be converted to being "true"..." What is truth? (And whither beauty? Never mind, that's a different blog...) I won't go into the ins and outs of JavaScript's type coercion here (refer to Flanagan's book), but for our purposes today, a can be converted to "true" if it's not zero, an empty string, null, or undefined. So 1 is considered true, as is the string "Fred", as is an object reference.

But lots of languages do at least some level of type coercion; what marks the JavaScript operator out as special is the result of the expression: What you get back isn't a true or false, but rather the value of one of the operator's parameters. Now that's interesting, and it's something you may not be expecting if you're relatively new to the language. It can make it difficult for a newcomer to read some of the more dense JavaScript found in sample code, toolkits, etc.

Here's an example I ran across a few weeks back (the names have been changed for simplicity):

    function sort(comparator)
    {
        comparator = comparator || this.defaultComparator;
        // ...
    }

Here, the author provides a sort method which takes an optional comparator function reference; if you don't supply one, the class's default comparator is used instead. Let's break that down a bit in each scenario.

If you call the method without supplying a comparator function reference:

comparator is undefined.
The expression evaluates comparator, sees the undefined value, and converts it to a boolean: false.
Because the first parameter is false, the operator returns the second parameter's value.

And now let's look at it if you do supply a comparator function reference:

comparator is a reference to a function.
The expression evaluates comparator, sees the function reference, and converts it to a boolean: true.
Because the first parameter is true, the operator returns the first parameter's value (it never evaluates the second parameter at all).

So the author's line

    comparator = comparator || this.defaultComparator;

sets up the comparator variable with the default if it didn't already have a value.

You might be asking, why not just do this:

    if (!comparator)
    {
        comparator = this.defaultComparator;
    }

Well, for one thing, it doesn't demonstrate l33t skilz, which is just as important to the current set of twentysomething coders as it was to the previous set, and the set before that (as it is now, has it ever been). But they would probably also point out that even after being minified, the result is in the region of six characters longer -- or nearly 12% of the total. (And they both beat out the version using the ternary operator.) Size counts with downloaded JavaScript. Some would also probably argue that it's more "expressive". Plodders like myself might raise counter-arguments about clarity, but that's the thing, to a day-in, day-out JavaScript coder, the statement isn't unclear.

Let's take our example one step further: Suppose our hypothetical class with the sort method only sets its defaultComparator on the instance if it's explicitly set, and relies on a class-wide default otherwise? The || operator lets us write the choice of the comparator function with a single line (split into three lines here for word-wrap):

    comparator = comparator
              || this.defaultComparator
              || NiftyClass.classComparator;

The first || tests comparator and, finding it false-ish, takes the value of this.defaultComparator; then the second || evaluates this.defaultComparator and, finding it false-ish, takes the value of NiftyClass.classComparator. Putting it more simply: It selects the first one that's defined. As a good friend of mine said "...it is kind of a selection operator more than a logical one..." Indeed, it's almost a specialized version of the ternary operator where the test is for "truth" or "existence" (and I figure that's got to account for 80% or more of the use of the ternary operator).

So, do we all want to see the || idiom for selection rather than if statements? There we enter the realm of style, I tend to doubt that either way is markedly better than the other in terms of execution speed (which probably varies from engine to engine anyway), the size difference is minimal, and it's personal choice which is more expressive. But if you're going to be reading any significant amount of modern JavaScript, you'll see this

    x = a || b || c;

idiom a lot, so be prepared for it. And if you're an old plodder like me, well...it does kinda grow on you. ;-)

[I should point out that in early versions of JavaScript, the || operator did evaluate to true or false, rather than providing this nifty return-the-value-of-the-param behavior. But the JavaScript engines in all major browsers (including IE6) use the behavior described above.]

Nifty Snippets

noreply@blogger.com (T.J. Crowder) — Thu, 14 Feb 2008 12:17:00 +0000

Welcome to Nifty Snippets, my new blog for capturing little snippets of code and techniques (as well as engineering, business, and teamwork "lessons learned"). Initially the focus will be on Ajax stuff, and that means lots of JavaScript.

Now, JavaScript is about 18 times more interesting than most people think. This is not a toy language, this is an incredibly rich, powerful, expressive language that until recently has been dramatically under-used. Partially that's because of how it was introduced to the world (scripting events in Netscape Navigator), and partially I suspect it's down to a bias many of us had toward hierarchical, class-based languages like Java and C++. JavaScript is neither hierarchical nor class-based; it's a prototype language.

This first post is just meant to say "hi". (Hi!) But I'll also throw out a few handy links for those with an interest in JavaScript, Ajax, and rich web applications:

If you're doing any serious work in JavaScript, read Crockford. If there's something powerful and interesting about the JavaScript language, the odds are Douglas Crockford has written about it in detail. Don't let the depth of the language daunt you, you can ease your way into it.
Similarly, if you're doing Ajax apps, you need JavaScript: The Definitive Guide by David Flanagan [O'Reilly, Amazon].
Lots of people are really getting a lot out of Prototype, which is basically a bunch of nifty stuff for JavaScript wrapped up into a toolkit.
Many of those same people are enjoying the various effects and other goodness available via script.aculo.us.
Alternately, perhaps you'd like to peruse one of the dozens of other JavaScript toolkits out there.

Those'll get you started, anyway. Back soon with our first snippet!