Recursive

The Hikikomori's Guide to JavaScript

2013-04-27T00:00:00-07:00

“The Answer to the Great Question… Of Life, the Universe and Everything… Is… Forty-two,' said Deep Thought, with infinite majesty and calm.”

— Douglas Adams in The Hitchhiker's Guide to the Galaxy.

Why am I quoting Douglas Adams' most awesome book? Why am I using a wordplay on my blog post's title? Why are JavaScript's modules such a mess? How do brains work? Well, my dearest basement dwellers, I am afraid I do not have all the answers, but none the less I shall try to provide some tips on writing useful, simple and composable applications for the Great Good™ of the JavaScript community here.

Here's a TL;DR:

Start with an API to manipulate your data, not with an Application.
Extract the most basic, orthogonal concepts, and provide primitives to encode them.
Provide “glue” code to compose concepts together and create big things.
Work with structured data.
Write the actual application using your API.

Introduction

There was a time when no one knew how to write JavaScript, but those times are long since gone. People know how to write JavaScript now (some), and that's good!!1ELEVEN! Unfortunately, there's still a large portion of people who don't know how to write applications¹.

As a result of this, you often end up with applications that do too much, or applications that do too little. But the worst problem of all is when you end up with applications that you can only use through some human interface of sorts, and can't easily manipulate the stuff you're interested in with different things. Mind you, programmatic extensions matter a lot!

Thus, in this blog post I'll try to provide a few hints on how to achieve small, composable and extensible applications. Stick with me!

Start with an API

Don't start with an Application, start with the API. This might seem weird, but it actually makes a lot of sense, because the API defines how other computer things can interact with the data your application manipulates. But hey, why should we leave that as “other computer things”? Let me rephrase:

The API defines how your application (and other computer things) can interact with the data you're interested in.

My usual design approach is to examine the inputs and outputs of the application, and then design an API that handles such data. Later I'll stack the user experience and the “glue” code on top of such API to form an actual application.

This fits nicely with the "One Module Does One Thing" approach because then the API defines which kind of data you're dealing with, and all the meaningful transformations you can apply to such data. Transformations in turn do one thing, and might be grouped logically. Modules are really just a logical group of computations, that it happens to be a file in many languages is just an accident.

For example, let's say we want to write an application to display Tweets to the user. First and foremost, we examine the inputs and outputs, and draft some types to encode the data in our application (I'm using a type notation inspired by Haskell, but this should be nonetheless pretty straightforward):

-- | A TwitterTweet is something Twitter gives us (the input)
-- (note that this is a stripped down version)
type TwitterTweet
  favorited     :: Boolean
  retweeted     :: Boolean
  retweet_count :: Number

  created_at :: String  -- ^ Serialised Date
  source     :: String
  user       :: User
  id_str     :: String

  in_reply_to_user_id_str   :: String
  in_reply_to_status_id_str :: String
  in_reply_to_screen_name   :: String

  entities   :: { "urls"          :: [Entity]
                , "hashtags"      :: [Entity]
                , "user_mentions" :: [Entity]
                }

  text :: String

So, the data Twitter gives us is quite a mess, and it'd be really difficult to manipulate that kind of data in our application. We can do better, so let's define a better type to encode a Tweet:

type User
  name :: String -- ^ The users' screen name
  id   :: String -- ^ Twitter's user ID

type Entity
  url          :: String
  expandedUrl  :: String
  displayUrl   :: String
  indices      :: (Number, Number)


type Text
  plain    :: String
  entities :: { "urls"          :: [Entity]
              , "hashtags"      :: [Entity]
              , "user_mentions" :: [Entity]
              }

type Tweet
  id            :: String -- ^ Unique identifier for this tweet
  user          :: User   -- ^ The user that tweeted this
  inReplyTo     :: Tweet
  source        :: String -- ^ Which application was used to compose this
  date          :: Date   -- ^ When this tweet was crafted
  favourited    :: Boolean
  retweeted     :: Boolean
  retweetCount  :: Number
  text          :: Text

So, now User and Text are separate types, this is because they make sense outside of the context of a Tweet and we might want to manipulate them separately. There's no reason to provide a complected type to a function that only needs to know the name of a user, for example.

Once we're done with the types our application needs to manipulate, we can draft an API that provides the primitives to manipulate these types, given the operations we'll be applying to them and the output.

-- * Type conversions

-- | We need to convert from Twitter format to ours
normaliseTweet :: TwitterTweet -> Tweet

-- | Convert Twitter Date serialisation to actual DateTime
parseDate :: String -> Date

-- | Extract the User that composed the tweet
twittedBy :: TwitterText -> User

-- | Extract reply information
repliedToUser :: TwitterText -> User
repliedToTweet :: TwitterText -> Tweet

-- | Extract the Text
textFor :: TwitterText -> Text


-- * Display transformations
-- (These are application-specific because they only make sense in the
-- context of user-facing HTML pages)

-- | We want to display a Tweet as HTML
renderTweet :: Tweet -> HTML

-- | We want to display a Text as HTML
textToHTML :: Text -> HTML

-- | We want to know the relative time since the tweet
fromNow :: Date -> String

-- | We want to display a link to a User
linkToUser :: User -> HTML

-- | We also want to display a link to a Tweet
linkToTweet :: Tweet -> HTML

If there's one hint I can provide when doing the initial API design, it would be:

Extract the most basic, orthogonal concepts, and provide primitives to encode them.

You can always add combinators on top of those minimal and simple primitives to let them do more stuff. Working with reeeeally small set of primitives and a lot of combinators means you get to write simple code that actually scales! But then, picking the right primitives can be really hard at times, so you need to have a good deal of knowledge about the domain you're trying to encode in your API.

Provide “glue” code to compose concepts

Compositionality is a big thing. Compositionality is what you want in a big application. Compositionality is what will save your bacon when you have to actually maintain all the shit you've written. This is one of the reasons we don't put them in the first API draft, we want to get the primitives right first, and make sure they don't overlap!

Back to our Twitter example, when you retrieve data from Twitter, you usually get a List of tweets. Notice that nothing in the previous API allows you to take a list of Tweets and spits back a list of HTMLs, but it can take a single tweet and spit back a single HTML. We also have baked right into the standard library a function that takes a List of things, a function that transforms a thing A into thing B, and returns a list of things B. Well, this is enough to derive our transformation for lists of Tweets:

// Renders a list of Tweets
// renderTweetList :: [Tweet] -> [HTML]
function renderTweetList(tweets) {
  return tweets.map(renderTweet)
}

// Or we can use a better version of Map (if you know functional
// programming) 
var map = curry(2, Function.call.bind([].map))
var renderTweetList = map(renderTweet)


// -- An aside: ------------------------------------------------------

// If you don't know what `curry` is, well... A minimal explanation
// would be that functions in JavaScript actually takes a List of
// arguments. You should think about:
function add(a, b) { return a + b }

// As being actually:
function add(arguments){ return arguments[0] + arguments[1] }

// And when you're calling it as: add(1, 2) you're actually saying
// add([1, 2]).

// Currying takes a different route. Functions takes only one
// argument:
function itself(a) { return a }

// If you need to create a function that takes more than one argument,
// you use closures:
function add(a){ return function to(b) { return a + b }}

// And when you're calling it as: add(1, 2) you're actually saying
// add(1)(2).

// You can see an implementation of currying here:
// https://github.com/killdream/athena/blob/master/src/higher-order.ls#L56-L81

But this doesn't display anything in the screen yet, mostly because that's not the job of renderTweetList — it already does everything it needs to do. A thing that displays tweets on the screen should be something that takes an HTML, a container and adds that HTML to the container:

// addTo :: HTML, HTML -> HTML
function addTo(container, html) {
  $(container).append(html)
  return container
}

Now we can derive a simple function that will take a list of HTML things, and add them to a container:

// addAllTo :: HTML, [HTML] -> HTML
function addAllTo(container, htmls) {
  htmls.map(function(html){ addTo(container, html) })
  return container
}

// Or, we can go use our Curry friend and make it better-er
var addTo = curry(2, addTo)
var addAllTo = curry(2, function(container, htmls) {
  htmls.map(addTo(container))
  return container
})

Work with structured data

I can't stress this point enough! If you want people to actually use your API in a meaningful way, you must work with structured data. Please don't “but strings are easy!” me. Strings might be easy, but we don't want easy when designing an API, we want simple ². Simple stuff sometimes means you get to write more, but also means that you get something that's more meaningful overall, that's extensible and that composes well with other things without randomly breaking for no good reason. When you pass Strings around for other people to parse you lose all the guarantees that they'll agree with each other on the structure your API (and external APIs) expect.

In the case of our API example, it would mean passing around Tweet types, rather than the HTML representation of them!. All of the central points of the API should accept one of our types (Tweet, User, Text), not arbitrary HTML or plain text strings, because then everyone can encode that slightly different.

“So, what if I want to send it over to someone else over the wire? Wouldn't it be better if I just use the representation that the other side will use to display the thing?”

Well, think about the following scenario: You have your application sending tweets to a logger that will display them. You want to “Keep It Easy”, and so decides it's a good idea to just send the way you want tweets to be displayed on the other side, so people don't need to write anything besides console.log.

A few weeks later someone comes up to those guys and say, “Hey, we're going to log only stuff that got retweeted at least 50 times.” The other-side guys quickly hack together a regular expression that looks for /(\d+) retweet/ and call it a day.

Some days later you decide retweet is too long and it's taking valuable space on the screen of your application now that you're porting it over to mobile devices. Then you decide to shorten that to rt. Guess who just got all their system's screwed?

If you pass over structured data, then it's simple. They wouldn't even need to touch their main system if they didn't want to, just put on a proxy in front of the service with this code:

next(tweets.filter(function(tweet){ tweet.retweetCount > 50 }))

If you need to communicate data with other services, you should just encode a structured representation using the best serialisation format for the job. JSON everywhere won't cut it, as won't XML. JSON is a generic data serialisation format as plain text, and XML is a document serialisation format as plain text. They're cool if they fit your data, and you don't care about the additional bandwidth/encoding time. Otherwise there are other stuff like Protocol Buffers to take a look at.

Warning: Please please please please please please! For the love of God, don't use XML to encode general data. XML is a document serialisation format, it's something you use to serialise TREE STRUCTURES. Mind you, Lists are not the best case for XML, dictionaries aren't either. Use a general data serialisation format for everything that isn't a tree.

Write your application using your API

You've gone through great lengths to create a minimal and polished API, now it's finally time to use it by writing your Application on top of it. Why, you might ask? Well, because Applications are the human-facing interface to your data. Applications talk to humans, and only ever to humans, because they choose a format that is difficult or impossible to use to talk to other application. APIs on the other hand talk only to applications using structured data, which is not the best format to present to the user for most types of data.

Say we want an application that will get the timeline of a given user and display it on a webpage. This can be encoded in a simple way using our API:

var dataP = twitter.statuses('notSorella')

dataP.then(function(data) {
  var tweets = data.map(normaliseTweet)
  addAllTo(twitterContainer, renderTweetList(tweets))
}).done()

If we then are tasked with displaying the same set of Tweets on the command line, we can just use the primitives, which are not HTML-specific!

var dataP = twitter.statuses('notSorella')

function renderTweet(tweet) { 
  return '@' + tweet.user.name + ': ' + tweet.text.plain
}

dataP.then(function(data) {
  var tweets = data.map(normaliseTweet)
  tweets.map(compose(print, renderTweet))
}).done()

Conclusion

This is it, me dears. This is the key to write large applications, this is the key to write extensible applications, and the key to write easily maintainable applications: compositionality.

You start with an idea, extract the key components of that idea (primitives), provide combinators to compose ideas together, and only then provide additional transformations for the user-facing interface.

Footnotes:

¹ : I am, of course, referring to my own notion of How Applications Should Be Written™, which might be fairly arbitrary.

² : For a much better explanation of why we should value simplicity over easiness, Rich Hickey (the guy from Clojure) has a most awesome presentation on the topic, called Simple Made Easy.

Trans people should not be allowed to pee!

2013-03-23T00:00:00-07:00

Let's face it, trans people should not be allowed to use public bathrooms for straight, cisgendered people. Trans people are weird. Like, really weird. And they do make normal, straight people immensely uncomfortable. As such, I am forced to agree with John Kavanagh, and reinforce his claim that If you think someone is using a public bathroom that's the wrong gender for them, THEY SHOULD FACE THE JAIL!

Now, unlike John, I hold a more extremist view on this subject. Why stop at trans people? There are lots of other things that make normal, straight people likewise uncomfortable. For example, this scenario:

You're a white, christian, straight man, making your way over to the public bathroom in a STEM convention. You decide to use the urinals, like any sane person would in this occasion. At this point, another man (if we ever could call that… disgusting thing that), looks at you and stops at the urinal at your side. He pulls out his dick, still staring at you, making faces at you. HOLY MOTHER OF FUCKS HE'S MAKING FACES AT YOU! You feel violated, wishing all this awkward moment will end as soon as possible. You feel an urge to punch him, but decide not to: that would cause a huge commotion, and what if people think you are gay too?! Oh, God forbid that!

There's no denying that this sort of thing should not happen to normal, white, straight men. Life is already way too difficult as it is. And these things, these feelings of being violated! Oh my dearest God, I can not even think about these! As such, I propose we don't stop with just potentially forbidding trans people from using public bathrooms, but also homosexuals. Those disgusting, vile people!

However, by taking such actions, the international community and the "human" rights community might be upset at America¹. People might say we are not "inclusive" for people. Or that we're not a truly "free" country. Or even utter bullshit like "We're not making progress, but regressing instead!" Those fools! The 19th century was clearly more developed in manners and society values as this century. We truly valued masculinity back then. And now? Now we have these… oh I can't even finish this phrase. It disgusts me so.

None the less, we want the Americans who might support this kind of things to think we're being "inclusive", so I propose that besides forbidding those weird fucks to use public bathrooms, we should do so under the pretence of "We want to make Americans not feel violated when using a public place.". This would mean that the rules for using public bathrooms would have to change to the following:

Public bathroom usage would be defined by gender and sexual/romantic orientation. That is, you would only be able to use the public bathroom that fits your gender (with a distinction on cis/trans people), and your sexual orientation.
So, homosexual cis-men would have to use a separate bathroom than an heterosexual cis-men, which is only reasonable. A homosexual trans-men would not use the same public bathroom as a homosexual cis-men, even more if that trans people haven't fully completed the procedures for sex reassignment yet.

For genderfluid people, this mean we'd need an additional division, since those fuckers might feel like a man one day and a woman the other day. Disgusting!

Public places, if providing public bathrooms, would be required by law to provide bathrooms for every combination of gender and sexual/romantic orientations. Since we don't have a fixed number of these, this could potentially grow to infinity.
This would, of course, need to have a flaw in the law's writing such that a decent lawyer could use it to say "Oh, we're still trying to adapt to the changes, just give us a little more time." Of course, as long as the place has public bathrooms for straight cis-men and straight cis-women. If the place doesn't provide these, we should just fuck them up.

People who are reported as using the wrong public bathroom for their gender or sexual/orientation will face a penalty according to how weird they are. Thus, straight cis people would just have to spend the night on the jail. Weird people would spend a lot more.

Conclusion

I really think this is the way America should go. To deny that people who don't conform with the gender roles for their biological sex are even people. This is a huge improvement from the 19th century, and I believe perhaps even more than the Dark Age. It's likely to also be more inclusive than the Nazi Germany, if we keep at it with all our strength.

Let's together make America a better place for straight, white, cis-men, who are already way too bullied by their social position, and worse, threatened by this scourge of weird asses who insist in labelling themselves "human".

About this piece

I'm a genderfluid, asexual, panromantic person who was just way too shocked with mister John Kavanagh's proposed law. If you don't get it, this blog post is riddled with blatant sarcasm, and I'm just expressing my support to the petition linked a few times throughout this post (https://www.allout.org/en/actions/arizona), by showing how absurd this law really is :3

Footnotes:

¹ : I don't even know why it's called human rights, they defend the rights of gay and trans people, who clearly are not human.

A Case for Prototypes

2013-02-28T00:00:00-08:00

Two days ago I started working on my own implementation of QuickCheck, since none of the implementations I saw for JavaScript did quite what I wanted. So, I sat down to outline the major goals of Claire:

Provide the means for property-based testing. For pure functionality.
Provide the means for random program generation. For complex, state-dependant functionality.
Provide core primitives and combinators for random-data generation. This way domain-specific generators can be easily derived by combining primitives.
Be a nice DSL for use in JavaScript.

So I started drafting the initial API that would satisfy those goals (ignoring program generation for now, since I decided to leave that for later), and decided on two basic data types:

-- | Describes a property and its specifications
type Property
  arguments    :: [Gen a]
  classifiers  :: [(a... -> Maybe b)]
  implications :: [(a... -> Bool)]
  invariant    :: (a... -> Bool)

  -- | The DSL
  satisfy  :: @Property => (a... -> Bool) -> Property
  classify :: @Property => (a... -> Maybe b) -> Property
  given    :: @Property => (a... -> Bool) -> Property


-- | Describes a way of randomly generate a series of values
type Gen a
  size   :: Int
  next   :: () -> a

-- | A semantic way of getting a Property than Property.make()
forAll :: Gen a... -> Property

I'll use these types as a basis for the rest of the article. If you don't know Haskell's type annotation syntax, it suffices to say that :: describes the type of the thing on the left, [a] describes a list of a's, and a -> b describes a function that takes a value a, and returns a value b. I've added @a => b for describing a function that, when applied to type a (as this), yields a value of type b. And the ellipsis to denote variadic functions.

Maintaining purity

First of all, I wanted the DSLs of properties to be pure. That means that, even though you'll be writing things like:

var sqrt_p = forAll( Int )
             .satisfy(function(n){ return Math.sqrt(n * n) == n })

The satisfy function should not change the Property instance that it was applied to. Instead, it should always return a new Property instance. Thus, when you write things like:

var all_positive_ints = forAll( Int )
                        .given(function(n) { return n > 0 })

var sqrt_p = all_positive_ints
             .satisfy(function(n) { return Math.sqrt(n * n) == n })

Even though satisfy modifies the invariant field of a Property, all_positive_ints and sqrt_p should be two different data structures. It could be obviously achieved by creating a new object, copying all fields of the previous object in the new one, and changing the invariant field, but prototypes make that both dead simple and efficient (in terms of processing power and memory usage).

So, turns out the implementation for this is just a simple and straight-forward one-line (the actual implementation is simpler due to both LiveScript's syntax and the fact that Property inherits from Boo's Base object):

var derive = require('boo').derive

var Property = {
  /* (...) */
  satisfy: function(f) { return derive(this, { invariant: f }) }
}

note: So, derive here is just a thin abstraction over Object.create that creates a new Object which delegates to another object (this, in the case above), and extends that new object with the given properties.

If you inspect the values of invariant, you'll get:

all_positive_ints.invariant
// (Function) => function (){
//                 throw Error('unimplemented');
//               }

sqrt_p.invariant
// (Function) => function (n){
//                 return Math.sqrt(n * n) === n;
//               }

all_positive_ints.isPrototypeOf(sqrt_p)
// (Boolean) => true

So, this is a cheap trick, and not really new (I've used it on a parser combinator library I was writing ages ago). But it's a cheap trick that gets the work done extremely well, in a simple way (both to write and to reason about) and still manages to be fairly efficient. We couldn't really ask for more ;3

The real deal

Things start getting a lot more awesome when we take a look at how Generators were modelled, specially when we take into account the combinators defined for creating new Generators. Here's the deal:

We want generators to follow the pure model Properties do.
We want combinators to be expressive, but still easy enough to write.

I could have used functions, in which case those would be relatively easy to achieve due to the higher-order facility in JavaScript, but a Generator actually encodes more than just a way of generating the next value. It also knows how to shrink a value it has generated, and how to display itself to the user in a human-readable and concise way for debugging purposes.

So, I was stuck with objects, and I still had to satisfy the two constraints above. Well, meet the redemption: Prototypes. Prototypes are a really great fit in this case because, even though all generators obey the basic Gen a structural type, they can be based in any kind of generator that the user may come up with. Since we have prototypes rather than classes, it's entirely possible to encode this complex and dynamic hierarchy as simple functions as combinators. Yes, we get all we need from combinators, and still have combinators as plain functions (that yield new Generators).

The `asGenerator' combinator

So, the first and most basic generator is the asGenerator. This is a function that takes any value a, and returns a Generator that will always yield that value (or the result of applying a function, if a is a function). On the other hand, if the function receives a generator as value, it works like the identity function. This allows us to lift a regular primitive value, like a Boolean or a String, into a generator:

var True = asGenerator(true)
True.next() // => (Boolean) true

The core Generator object inherits from Boo's Base object, so it's rather easy to write a function like asGenerator, and in fact all other generators take advantage of this sugar. The following is a slightly simplified version of asGenerator:

function asGenerator(a) {
  return generatorP(a)?   a
       : /* otherwise */  Generator.derive({
                            next: function(){ return a }
                          })
}

In a class based language, specially those that are less expressive, like Java, you would have to encode the asGenerator function as a whole new Class, making the implementation unnecessarily complicated.

The `choice' and `frequency' combinators

On top of the lifting generator, a clear second step is to provide people with a way to alternatively select between different generators to create a new value. These roles are satisfied by the choice combinator, which takes several generators and generates a value using one of them in an uniformly distributed pseudo-random choice. And then there's the frequency combinator, which does almost the same as choice, but uses a weighted distribution instead.

So, what's the big deal here? Ain't those other two cases that you could encode as a class? Well, sure. You can always encode things using something in computer science, but that doesn't mean it'll be easy, simple or straight-forward. And I want the three of those characteristics.

Using these combinators looks like this:

var vowels = choice('a', 'e', 'i', 'o', 'u')
var coins = frequency([3, true], [5, false])

Since both choice and frequency leverage asGenerator, you can pass either simple values or generators over to them. In fact, one of the nice things about using objects as generators is that checking if something is a generator is easy: you just check the conformance to the structural type Gen a.

This is a simplified implementation of choice and frequency:

function choice() {
  var gs = toArray(arguments).map(asGenerator)
  return Generator.derive({
           next: function() { return pickOne(gs).next() }
         })
}

function frequency() {
  var gs = toArray(arguments).reduce(weightedConcat, [])
  return choice.apply(null, gs)

  function weightedConcat(r, x) {
    return r.concat(replicate(x[0], x[1]))
  }
}

And finally, the `repeat' combinator

Once we have all this in place, we can also generate lists of things, and this is easily done by the repeat combinator. In fact, the List core generator is a thin abstraction on the repeat combinator, just so you don't have to type repeat(choice(...)).

Now, this is where the magic of prototypes truly shine. I didn't tell you until now, but all generators need to maintain all of the properties of the previous generator. All of them! This is because other generators might depend on those properties (for example, size). And in fact, the size field plays a huge and important role in the repeat combinator, since it will only generate lists of values up to that length!

Now, enough talk, this is how the repeat combinator looks for reals:

var numbers = repeat(function(){ return randInt(0, 10) })

Pretty straight forward, huh? So is the simplified implementation:

function repeat(g) {
  return asGenerator(g).derive({
           next: function() {
             var range = Array(randInt(0, this.size)).join(0).split(0)
             return range.map(g.next.bind(g))
           }
         })
}

So, now we're taking any kind of Generator, an instance, and making a new kind of Generator that efficiently shares all the properties defined in the previous generator, but also defines its own properties. This is the true beauty of prototypical inheritance: instances inheriting directly from instances.

Conclusion

There's much more to Claire than what I've shown above, but those are out of the scope of this blog post, which was to outline how you can take advantage of prototypical inheritance to create awesome things, since you're already coding in a language that provides it for you. Dynamic delegation chains are sweet for lots of things, these are just some of them.

Perhaps I might write a new blog post on some other aspect of Claire (for instance, shrinking or program generation, when those are done). But I don't know, I've got some other quite hot and sweet topics to talk about as well! :3

Anyways, hope you guys can see now why inheriting directly from objects is a Darn Good Thing™. And why I still think that Function.prototype is an anti-pattern in the language, which should be avoided (although it's undeniably fast).

Acknowledgements

Thanks to Theo for pointing out that abcde aren't vowels ;3

Unfancy API Documentation

2013-02-23T00:00:00-08:00

I've been fiddling with different approaches to API documentation over the last years, and found a sweet spot that actually fits my needs. However I still faced a problem: there were no tools to support my approach to documentation. There were lots of tools out there that took different approaches for documentation, but I couldn't use any of them because they were either monolithic or language-specific and not that easy to bend my way.

So, in this blog post I describe that approach better and my way of trying to solve not just my problem, but alleviating the cost of coming up with different approaches to documentation, which should probably benefit more people (specially those who disagree with my dictatorial rules :3)

An overview of the documentation landscape

I work with a few different languages in a daily basis. Sometimes I'll code in Python, then hack up some stuff in Clojure, or invoke some clever computer spirits with JavaScript spells. Still, most of my time these days is spent working with LiveScript. At the end of the day, though, this context switching between a dozen of different languages leads to a really, really annoying thing: they use completely different tools and environments.

Some of those tooling is specific to a language because it needs to be specific to a language, for example editor modes, or the runtime library, static checkers and linters, etc. Others aren't really specific to a language, but still, like editors, debugger interfaces, build tools or documentation toolkits. Except that more often than not, people tend to make non-language-specific tools language specific, which kills the idea of reusing a otherwise generic tool in different environments.

I really mean it, reusing generic tools for different languages is a big thing. And it'll be even more of a big thing as people start working in more than one environment at the same time — front-end developers usually work with at least three.

I lied. Documentation isn't actually generic. There are plenty of language-specific parts. For example, language A might have run-time macros, whereas language B has a different concept of modules, and you want to capture those differences in the documentation in order to help the user grasp the meaning of your code quicker. However, these "language-specific" features are more often than not based upon years-old programming language fundamental building blocks, and those can be well generalised up to a point — at least, as long as it concerns the programmers' understanding of what an API does, why it does such, how it does such and how you should use it.

And please, let's not forget that API documentation can be "consumed" and visualised in lots of ways, using the same basic meta-data, whether it's generalised or not. You could say, documentation is made of several steps, and each of these steps produce something useful (or "consumable").

Even with this possibility of generalising things, I've found that the majority of the main-stream tools for documentation are language specific or totally overlook the step part, or yet only lets you change how the basic meta-data is visualised in a handful of ways. This is true for JSDoc, Docco, Marginalia, PyDoc, etc. Tools that can give you the meta-data they gather from the source code, like Doxygen, are often monolithic and generate this meta-data in annoying formats, like XML. Tools that don't rely on the structure of the language don't allow you to make that meta-information available by other tools, and there isn't a "visualisation only" format (well, you could kind of call Sphinx one, but then you're not really using meta-data).

For example, if you have a project that uses Scala and Java, but you want to present their API together because it actually makes sense, you can't easily do that. Or if you want to include the documentation directly on your project's README because you have only one or two public API methods, you also can't do that easily because there's no tool that just does: "hey, take this documentation meta-data, and generate something I can use in a Markdown-formatted README." You have to actually reinvent the wheel all the time by parsing the whole source and generating the thing in the format you want, or be forever tied to a particular tool by writing a plugin.

You also can't say, "Oh, I want to parse the documentation in this language using its structure, because that's easily to statically determine, but for this other language I want a generic thing because it's all too dynamic and you can only take guesses about what everything means. But I still want to display them using the same format." If that thing happens, you're basically out of luck.

Separating the concerns in documentation tools

Most of this can be fixed by separating the documentation in some well-defined steps, and creating the whole picture by mixing-and-matching tools that fit those steps. Below is a list of concerns that could be addressed by any documentation tool, but are at the same time independent of all other concerns:

Analysing the entities that occur in a code-base (classes, modules, etc.)
Resolving types in the code by way of type-inference.
Resolving relationships between entities (multi-methods, inheritance, etc.)
Analysing or guessing contextual relationships or abstractions.
Merging documentations from different projects into one.
Searching a particular entity in a code-base.
Visualising documentation.

This list is in no way authoritative, there might be plenty of concerns that aren't listed here depending on the particular needs of a project. Or those concerns might be too much for a particular project. But again, this just shows how we could all benefit from separating all those concerns in tools that can be easily mixed and matched to achieve a greater goal: a nice way of documenting your APIs.

So, I did say that we could separate a documentation tool in a fixed number of steps. There are four steps in documenting an API, to be more precise:

A developer annotates his code following certain rules. This helps analysis tools get a better sense of the code, and users of the API to see faster how to use or what's the intention of a particular part of the API.
A tool analyses and extracts meta-information about API entities. Entities are all the things that are visible or usable in some way by the end user. So, classes, modules and functions are examples of entities in most contexts, whereas syntactical constructions like `for` or `if` are not — but more on that later.
Meta-information including structure, relationships and annotations is put together in a single place, in some easy to parse and easy to use format.
A visualisation tool uses the previous meta-information to display the structure, relationships, annotations, usage examples and any other kind of information that might be associated with the entities in a way that makes sense in the context of the visualisation tool.

The graph below shows one way the concerns could fit in the steps above:

So, why would this work? Simple (despite what my failures at graphs would tell you), the annotated source is the canonical reference for everything, it's what tells us which Entities we care about to handle in the next stages.

Then, several tools analyse this annotated source from different angles, one tool may extract entities using a simple structural analysis of the sources, another can extract type annotations, another can make sense of annotation meta-data in the comments, or here-docs if the language supports it. At the end of the day, however, all of those tools are able to communicate between each other and collaborate because they use a standard format.

Then, all of those (up until now) disjoint information about the entities in your code base moves on to the merging stage. At this point, all of the entity information that we gathered is put together as a single thing, and tools collaborate between each other to resolve relationship graphs, they could also provide better contextual organisations, solve links between entities, automatically "suggest" similar or more abstract methods, load examples, etc.

At last, all the information that was put together in the merging phase passes to the last phase, which is visualisation. Again, since everything uses an standard format, the visualisation tools don't need to care about which tools were ran, they just care about the information they were given. At this stage, the information can be used to display the API documentation inline in README files, if it is small enough, or as an interactive web application, or even as a searchable resource within your text editor (Emacs has this).

Meet Dollphie, Calliope, Kalamoi and Papyr°

As an experimental implementation of the concept defined above, I quickly hacked together these three tools (Kalamoi, Papyr°, Calliope) over the course of this week. I needed a generic documentation format since I want to use the same toolkit for documenting all the languages I work with. But I also needed to take into account the differences in each language's semantics and fundamentals behind each kind of entity. And I wanted to use different kinds of visualisation depending on the context, thus small modules with one or two public functions would have all their API documentation in a README.md, whereas more complex ones would be displayed in an interactive web application.

These tools are far from production ready (for example, there are no support for class hierarchies right now, nor field visibilities, nor multi-methods), and they need some serious refactoring (for example, kalamoi does everything from analysis to merging), but they're usable. Calliope is an opinionated wrapper over the other two tools for JavaScript projects that are already using CommonJS packages, and it's a really thin wrapper.

The basic idea of this toolkit revolves around a simple and generic structural annotation format (Dollphie) to describe entities in a non-language-specific way. This alone gets us around the problem of relying on a language's structure in overly dynamic languages like JavaScript, and works as a fallback for providing a common ground for languages that don't have specialised tools. Then there's a parser and analyser (Kalamoi) for this format, which spits out a standard documentation JSON with the entities meta-data, and finally the visualiser (Papyr°) takes that JSON and displays to the user in an interactive way.

The generic structural annotation

Since most of my work is in a overtly dynamic language like JavaScript (granted I use LiveScript now, but same semantics), parsing the source and trying to make sense of it isn't always easy. Say you have the following:

/// Given an ID, sanitises it, otherwise generates an unique ID.
var id = function() { var guid = 0

                      return function(name) {
                               return name?    sanitise(name)
                               :      /* _ */  (++guid).toString(16) }

                      // Sanitises an ID
                      function sanitise(name) {
                        return name.replace(/\s/, '-')
                                   .replace(/\W/, '')
                                   .toLowerCase() }}()

Even though this use of the "Module Pattern" is way too common, it's not obvious right away that the documentation refers to the inner function rather than the immediately invoked function expression that surrounds it. Or say you have something like this:

/// Takes the first `n` items of a list.
var take = curry(function(n, xs) {
                   return xs.slice(0, n) })

Which is also fairly common (not just the currying, but decorating a function in general), not just in JavaScript, but also in Python. This doesn't make the type of take immediately obvious, unless you happen to have figured out the type of curry already, or if your documentation tool evaluates a certain sub-set of JavaScript to figure out these things.

So, you have to annotate your code anyways with types, but there might be cases where your documentation tool will have some trouble figuring out the structure of your code anyways. Not to say you'd need extremely language-specific tools to extract this information. In the end I ditched this approach and decided for a generalisation of all languages I worked mostly with (which at the time were mainly JavaScript and Python), not just on the way entities are annotated, but also in the way structure itself is annotated.

The documentation format I came up with (and evolved informally through the course of this year) is heavily inspired by ReStructuredText and Markdown. The basic idea is that you have sections with varying depth levels (nested sections yay!), and each section can have a range of information belonging to it, including other sections. This way, entities are just special kinds of sections. The main difference from systems like Docco, etc. is that there are subtle annotations that make it possible to figure out the structure of the code and extract meta-data concerning a particular entity.

The whole parser fits in under 100-loc of LiveScript code. You can get a feeling of the documentation format by reading Kalamoi, Papyr°, Calliope or Boo source code. Boo is written in JavaScript, the others are all LiveScript. Adapting the parser to other languages can be done in three lines of code, but until I refactor stuff this is still a pain:

// in parser.ls
exports.luaSyntax = boo.derive(baseSyntax, { comment: '--' })

// in index.ls
var languageFor = function(extension){ ...
  case '.lua': return 'Lua'

var syntaxFor = function(language){ ...
  case: 'lua': return parser.luaSyntax

You probably guessed the parser relies on single-line comments, and yes. That's because the parser is line-based for simplicity. Basically, every line that starts with <comment-character> is treated as special meta-data for the parser, and the rest is treated as code. Lines that start with a repetition of the last <comment-character> are treated as sections, and the number of <comment-character> defines the level of that section (just as Markdown headings).

## Module foo
# Description of `foo`.

### -- Internal functions

#### Function id
# Description of `id`
id = (x) -> x

### -- Public functions

#### Function k
# Description of `k`
k = (x) -> (y) -> x

As you can probably guess, the special "kind" of section is defined by the first <word> after the comment character. A <word> here is more in the sense of Lisp, except the only thing that works as a word boundary is white-space. Sections where the kind is -- are special headers, that have no special entity meaning, but can be used to logically group things within a source code to make it easier to navigate and understand the structure.

There are only two other special constructs that can be used inside of a section to declare special meta-data about an entity. The <generic meta data>, in the form of :keys: optional value, and the <type signature>, in the Haskell-veins of :: type, although the format of the type signatures themselves is not specified — not a job for this particular language:

### Function id
# :internal, deprecated:
# The identity function.
#
# :: a -> a
id = (x) -> x

This all works well for well structured source code, and probably somewhat well for pseudo-literate code.

A format to exchange entity meta-data

I said before that all the benefits of this approach derive from a standard and simple format to exchange meta-data about entities. And there isn't anything easier nowadays for this kind of structured data than JSON. It's available everywhere, easy to parse, easy to write, and it usually maps nicely to lots of language's core data structures (at least dynamic ones) since it uses Maps, Vectors, Strings and Numbers as types, rather than a document tree .

This format is devised from an overt generalisation of all entities, with basis on what users of the API might be looking for when they browse through API references. This includes ideas that are common in other documentation systems, and kind of obvious, like an entity's name, its type or a description about what it means:

type URI = String

type EntityKind = Data
                | Module
                | Function
                | Object
                | Type

type Author = { name    :: String
              ; email   :: String
              ; website :: String
              }

type Entity = { id         :: URI  -- An unique ID for the entity
              ; parent     :: URI -- The entity that contains this entity
              ; kind       :: EntityKind -- The type of the entity being represented
              ; name       :: String -- The name of this entity
              ; signatures :: [String] -- The types of the entity
              ; text       :: String -- A text describing this entity
              ; code       :: String -- The source code associated with this entity
              ; meta       :: { String -> String | Bool } -- Additional metadata
              ; language   :: String -- The language of the source code
              ; line       :: Number -- Where the source code starts
              ; end-line   :: Number -- Where the source code ends
              ; file       :: String -- The file where the entity appears
              ; copyright  :: String -- Short copyright notice (if any)
              ; repository :: String -- The canonical repository for the code
              ; authors    :: [Author] -- Who contributed to the code
              ; licence    :: String -- The terms in which the code is released
              }

type Entities = [Entity]

Note: Yes, I do realise it makes more sense to create a new data structure to store all information related to code, I'll likely do it when refactoring these tools. I'll probably make code a List as well, so it can handle multi-methods defined in different parts of the code and such.

This, however does not cover several kinds of entities, specially entities that have intrinsic relationships with others in a way that it changes their behaviour, like Classes and SuperClasses/Mixins/Traits/Delegative & Concatenative Prototypes, etc.. It could just well be encoded in the type annotation, but then you lose the ability of using these relationships to convey additional information for the user when visualising the documentation, and encoding it as an additional but non-specified meta-data means that, while other tools can easily make sense of the data, we don't have a common-ground for them to collaborate with every other tool in this environment, so that's a no-go.

As it current stands, these things would need to be devised on top of the current standard and be added for the specification, so we don't have the risk of conflicts between different notations, and tools that don't understand that "extension" can just ignore the additional meta-data.

Visualising entity meta-data

Well, meta-data is good for applications to manipulate them, but not good for the end user, thus we are left with the problem of "how do we visualise this meta-data". Turns out, there might be many different cases where we want to do different visualisations. The goal of Papyr° is to cover one of them: finding some entity that fits the problem the user is trying to solve in a potentially large and complex API, and discovering how to use that entity. In the current form, Papyr° manages to achieve almost none of the goals outlined above, although it is a starting point.

First, helping the user find the right component to use for a certain problem involves first knowing how the user searches for information about that particular problem. In some cases it's easy to guess, for example, the user might be looking for a stable sort, and as such he'll type "stable sort" in the search box and expect to see classes, functions, procedures, methods, whatever that will do the thing he wants. But perhaps the user isn't really looking for a stable sort, perhaps in his case some other sorting algorithm will do the job better or faster because he doesn't have the constraints he gave in the first place. In this case, the system has to guess the intentions of the user from the provided context, and suggest things that fit the context but provide different constraints, or different benefits.

This also kind of asks for a more expressive search. One where the user can define his constraints better. For example, if he knows he's working with a list of Elements, he might be looking for a sorting function that is efficient for those, so he could provide a basic type, say: sort :: [Element] -> [Element], and the search engine would attempt to match sorting functions that provide an approximate type to that. Hoogle does this kind of approximate type searches. Other constraints might involve platforms, mutability, etc. Ranking the search results by favouring the ones that fit the constraints the user provided better is also an interesting addition.

Currently Papyr° does not provide automatic contextual hints for potentially useful components, but this is definitely a feature I want to work on, and that could perhaps be branched in other projects. So, for example, an analysis tool could be ran on the entities meta-data earlier on to determine some possible logical tags for each entity, which could be used by the visualisation tool for relating different entities and ranking them. Approximate type searches are something that are planned but not yet implemented, this is however something I need to give a little more thought since the type signatures are not defined, so it's not something as straight-forward as Hoogle's type search. The problem is also more with analysing the user input and determining the types rather than having different type annotations used by the entities, since you could probably determine that by some meta-data or language.

So, the second goal of Papyr° is helping the user to use a particular entity to solve his or her problem. This is partially done by automatically loading Markdown literate examples and displaying them alongside the source code and other details for an entity. The literate examples are, however, static. There could be some use in defining a format for interactive examples, or even examples that would be ran by a particular engine when building the documentation JSON, for the cases where fully interactive examples in the browsers aren't possible.

The only other visualisation format I plan on writing is Reedox, a minimal visualisation format for overly small APIs (one or two functions), where it doesn't make much sense to have the user load an entire new page to search for something where the whole API fits in two paragraphs in your project's README. And this is the case for quite a few packages on NPM, including many I plan on writing in order to refactor the projects outlined above.

Conclusion and future works

This is just the beginning, there's lots of stuff to do, not only in standardisation, so we can get better processing and analysis tools to extract the meta-data we need, but also in visualisation tools so we can actually use that meta-data in a way that it's useful for the user.

Again, none of the projects above is really finished, but they can work for you right now, and the core interfaces are unlikely to change. Calliope will surely have the same interface and it'll be updated to reflect any changes that might happen in the other projects. It's also quite easy to integrate with your existing NPM package, so you can give it a try :3

None the less, I'll be actively working on these and other tools (testing and building, mostly). More crazy stuff to come ;3

On semicolons, pseudo-technical and subjective arguments, revisited

2012-04-16T00:00:00-07:00

The JavaScript community is, indeed, very mature — just like any other programming community. We always find time to properly and politely (and heatedly!) discuss the most important kinds of topics: like whether to use semicolons or not, or whether to use tabs or spaces. Or yet, whether to use Emacs or Vi{,m}. Of course, everyone knows you should use Emacs, so I don't even know why those discussions surface.

In any case, I decided to write a post to shed some light on the subject ~~and let the flames begin~~. I tried to just shake this whole discussion away, because it started out too subjective… but I can't. This is an important post. Someone is wrong on the internet!

A bit of history

Discussions on the rules of Automatic Semicolon Insertion (henceforth ASI) aren't a new thing in the community. In fact, we've been seeing these for as long as the community exists — well, perhaps not as much, but definitely the discussions did grow as the community as a whole grew.

On the one side, there are people who argue that ASI is harmful, and you should be ashamed of using such a excuse for "poor coding style". Of course, the qualities of a coding style are, usually, extremely subjective, and make absolute no sense taken without context — that is, without stating what concerns the particular coding style chosen tries to address. Other arguments involve blatant (and sometimes personal) attacks on people who don't use semicolons in their code, misconceptions about when ASI applies, and things like "we've told ASI is harmful, so it must be!" or "JS is not a semicolon-less language" — I wonder if they would also consider Python and Haskell as non-semicolon-less languages, since they also have semicolons and ASI (albeit much more significant line breaks and white space handling).

On the other side, there are people who argue that semicolons are harmful, and we're much better off without them. Arguments against semicolons vary from a simple "Yes, yes we can!", passing through the "But I use other languages that don't require semicolons!", to the "Omitting semicolons make errors more apparent by transforming them in non-usual things" — needless to say those are not the actual quotes, but they are close enough to the crux of the arguments.

In this post, despite the hints for flamewar here and there, I'll try to lay out the discussion from a not-so-biased standpoint. I am on the Yay ASI!, side, though, so you might see some bias here and there — I'll try to make those as overt as possible, so you can just laugh it off (or rage) and move on.

What is ASI?

We can't start this whole discussion without first clarifying what we're talking about. That's why, we first need to define what in the Lord's name ASI is.

Let's start with the specification definition of this whole mess:

Certain ECMAScript statements (empty statement, variable statement, expression statement, do-while statement, continue statement, break statement, return statement, and throw statement) must be terminated with semicolons. Such semicolons may always appear explicitly in the source text. For convenience, however, such semicolons may be omitted from the source text in certain situations. These situations are described by saying that semicolons are automatically inserted into the source code token stream in those situations.

The specification then goes on to define all of the rules by which semicolon insertion takes place.

If you still don't grasp what ASI is after reading this part of the specification, don't worry, most people don't. We could summarise it as: "You need semicolons to end the statements cited above, but for convenience you can omit them in some situations, if you want to."

As Brendan Eich noted in his post, ASI is an syntactic error-correction procedure, that got implemented in JavaScript engine parsers and then made it into the specification. This means that, technically speaking, all code written without semicolons has syntactic errors in them. There's no denying it, because that's how the specification describes these procedures.

Practically speaking, though, the parsing rules and ASI rules — should we even call it that, since parsers don't need to actually add that token, — for the ECMAScript language are pretty deterministic, and ambiguities in terms of Automatic Semicolon Insertion can be resolved in a single or double look-ahead — if we ignore comments and horizontal white space.

In fact, if we looked at the practical matter of the subject, we could categorise famous languages' semicolon insertion rules in the following groups:

White-space strict (aka Statements ends as early as possible)

With Python as the most known example of this particular style. In a language where "statements ends as early as possible", you can kind of define the language's statements' (or expressions) rules as the following:

<statement>        ::= ( lots of junk ) <end-of-statement>
<end-of-statement> ::= NEWLINE | ";"

Which, at a first glance, might look quite easy to remember, right? Unfortunately, in real world code, there are quite some more intricacies regarding these rules. Sticking with Python, since you want a little more expressiveness on how to lay out your long statements, Python offers you optional implicit and explicit statement continuations. So, a simple example with a string spanning multiple lines (and yes, I am aware of the additional constructs for multi-line strings in the language), would look like the following:

foo = "a string"    \
    + "in multiple" \
    + "lines."

Notice how each line has to be terminated with an explicit continuation. Worse! If your editor can't highlight invisible characters, you risk running into the following error:

$ python test.py
  File "test.py", line 1
     foo = "a line" \
                      ^
SyntaxError: unexpected character after line continuation character.

An alternative to explicit continuations are implicit line joins, which happen on parenthesised expressions:

foo = ( "a string"
        "in multiple"
        "lines." )

Note that, in Python's particular case, the concatenation operator isn't necessary when two chunks of strings are separated by just white-space. This style saves you from worrying about whether there's white-space after your explicit continuations, but then your simple <end-of-statement> rules stop being just NEWLINE | “;”.

Non-white-space strict (aka If we can't parse, insert a damn semicolon!)

ECMAScript is actually the only language I know who fits this bill — Haskell having even more complex statement rules, but managing to be overtly white-space strict.

In ECMAScript, your statements always ends up with a semicolon. Whether you need to explicitly spell one in your source code, though, is another matter entirely. In fact, you can write large JavaScript applications without needing to write any semicolon in your source code at all, (comments excluded), as long as you stay away from old-style for loops.

In JavaScript, the same example above would look like this:

var foo = "a string"
        + "in multiple"
        + "lines."

ECMAScript's handling of statement continuation, imho, feels much more natural to read and write than Python's one. Therefore, they are, of course, TEH BESTEST — if my feelings would constitute actual arguments, sadly they do not.

How ASI actually works?

If you've been reading the comments from this whole ~~flamewar~~ discussion, you'll see that many people describe ASI in JavaScript as a guessing game. They couldn't be further from the truth. In fact, there's no guessing at all, everything is extremely deterministic from the grammar of the language itself.

And what's more, to check if an statement ends, a parser would need to use at most 2 look-aheads, if we don't consider comments and white-space. So, let's first define what the usual rule for ending a statement in JavaScript is:

<end-of-statement> ::= ";"
                     | <horizontal-space>* NEWLINE blank*
                       (and STATEMENT_ENDED? (not <continuation>))
<horizontal-space> ::= SPACE | TAB | <comment>
<continuation>     ::= "(" | "[" | <infix-operator>

Well, actually, it's not as simple as that. You have a few special cases in the language to handle prefix operators and a few others, like the return statement, which the specification calls restricted productions.

For prefix statements, it just suffices to say that they require an argument succeeding then, and so can't end earlier due to a line break. Thus, the following is valid JavaScript:

var a = 1
~
a
// => -2

In the case of restricted productions, however, we have the "awesome" <no line-break here> restriction in the grammar itself. Which means that, while these productions may have something succeeding the token, they do not require that such a case happens, and as such, a line break would indicate that we want to end the statement early. A good example of restricted productions, and that is widely (and mistakenly¹) used to indicate how you should always end your statements with a semicolon, is the return statement, which accepts an optional return value:

<return-stmt> ::= "return" [no line break here] <expression>

(function(){ return 1 })()
// => 1

;(function(){ return
              2 }()
// => undefined

Of course, inserting a semicolon after 2 in the second example won't prevent the return statement from always returning undefined.

For more in-depth articles on the intricacies of ASI in JavaScript, please refer to the awesome people that have written about it over and over and over again. They'll probably dive into more practical (and non-grammar-ish) details than I have here. Inimino's post is particularly awesome.

Is ASI safe?

We've seen that all of the ASI rules are pretty deterministic, and they don't rely on any kind of Black Magic from hell that only parsers and parser-writers know. They are also pretty manageable for a human brain to keep on his head while reading the code — or at least, I consider JavaScript's ASI rules, despite all of the complexity, in the same level of cognitive overhead as Python's, if only because it feels more natural to me.

But do all of the engines implement it properly? Is it safe? Will my code suddenly break?! Well, if you've been blindly following the Cult of Crockford, you might have been lead to believe that the answer to all of those questions would be: no, NO!, BET THE HELL IT WILL.

Of course, you should apply the not predicate from higher-order logic to all of those answers to get the actual, useful, answers. So, addressing the concerns in order:

Do all of the engines implement it properly?

Well, as inimino says in his post about JavaScript semicolons, there's no reason to fear any incompatible behaviour between browsers in regards to the "feature" (or misfeature, the definitions vary in the community). I have not been able to verify this claim with empiric proofs yet to date, but from my experiments in code, it seems to hold perfectly valid.

If any browser implementer would like to clarify the matter, it would be an interesting (and welcome) addition, indeed.

Another misconception is that bugs in browser JavaScript engines mean that using semicolons everywhere is safer, and will protect the developer from compatibility issues between browsers. This is simply not the case. All extant browsers implement the specification correctly with regard to ASI, and any bugs that may have existed are long since lost in the mists of early Web history. There is no reason to be concerned about browser compatibility in regard to semicolon insertion: all browsers implement the same rules and they are the rules given by the spec and explained above.

Is it safe?

No. I mean yes. I mean, it depends on what you mean by safe. From a technical stand-point, it is perfectly safe — after all, it is deterministic. You just need to know the rules.

Will my code suddenly break?!

As David Herman, a TC39 member, kindly enough clarified TC39 won't just break working code for no good reason. All the more when there's such a large body of working code relying on this particular feature all the way around.

Brendan Eich also discussed the issue with compatibility with older versions of ECMAScript and legacy code as they move forward to extend the language, several times (in his blog, in es-discuss, in twitter, in…).

It can all be summed up as: "if something is working and in use in the JavaScript community, we can't just break it."

So, no, your code will not suddenly break if you start omitting semicolons here and there. Now, I can't guarantee people won't want to break you.

Reasons to use or not ASI

So, this all said, there are any actual arguments in favour of omitting semicolons from your code, aside from the "well, duh, we can!" non-argument argument? Turns out, there are a few, but they're all overtly subjective, and not in any sense technical:

Consistency! I write code in other languages that use no semicolons, so avoiding them in JavaScript makes it easier for me to switch back and forth, without inserting them in the wrong places, in the wrong languages.
Semicolons are not necessary, thus removing them promotes them from ordinary statement terminators/separators to special tokens that should be present in certain situations. It's expected that this change in the role of semicolons would highlight certain kinds of bugs, by making lacks of semicolons in places where they matter more apparent.
Removing semicolons reduces the overall noise of a source code, granted the indication given by semicolons is duplicated by line breaks and indentation, this means that we don't have extraneous and needless symbols to distract you from the actual code.

Of course, there are reasons for not relying on ASI, and the valid ones are also overtly subjective, and not in any sense technical:

Consistency! I write code in other languages that use no ASI, so putting them everywhere in JavaScript makes it easier for me to switch back and forth, without omitting them in the wrong places, in the wrong languages.
Avoiding semicolons means that you have a higher cognitive load, and you must always scan much more of the source code than you should, just to be sure that the next line doesn't imply a continuation.
I use tools that don't support ASI (like JSMin, or JSLint).

Okay, the last one of those is not a subjective reason not to rely on ASI, but is not a technical reason on why other people should do the same. After all, you're just relying on a broken tool.

Conclusion

You should omit all semicolons from all your JavaScript code. Semicolons everywhere are insanely stupid code, and I'm not going to change my coding style because some punks dislike it… wait, that wasn't really the topic in discussion. Well, whatever.

Footnotes:

¹ : Citation needed.

Yay for Sugary JavaScript OO

2011-11-19T00:00:00-08:00

So, JavaScript is an object oriented language and all that, we have discussed it previously to length, and I have quite expressed how confusing and rough all the primitives the language provides are when it comes to dealing with object composition and behaviour sharing — which is just no good for a prototypical language.

Well, the good thing is that we can provide our own abstractions for this overtly dynamic object orientation semantics through libraries, which is what I have been doing for a while. And now that Boo has gained some shape and use, it's time to release it officially.

Why? Oh god WHY?

So, what's this Boo thing I'm talking about?

Well, my dearest friend, Boo is my attempt at bringing some sugar and sort-of declarative syntax for defining objects in JavaScript, which makes the structuring of programs easier to follow and reason about.

So, instead of the ugly, factory-ish approach:

function Animal() { /* ... */ }
function Cat()    { /* ... */ }
Cat.prototype = Object.create(Animal.prototype)
Cat.prototype.say = function() { /* ... */ }

You can have a sort-of declarative one that works entirely around objects:

var Base   = boo.Base
var Animal = Base.derive({ /* Animal's properties here */ })
var Cat    = Animal.derive({ /* Cat's properties here */ })

What Boo provides?

Boo is structured in three layers: mixins, inheritance and syntactical-sugar. Each of these serve to a particular purpose, and build upon the previous one — although you could say that the concepts themselves are orthogonal, which makes them composable.

Mixins

On the basic level, Boo provides the developer with mixins and data-objects — which are a specialised kind of mixins.

Mixins are parent-less objects that can be included in any object, at any time. Data-objects do the same thing, but they're also a factory for yielding the objects that will be included, which makes them a neat thing for default objects and such.

The primitives extend and merge account for all of these:

var ring_data = {
  items: [], max: 3,

  toData: function() {
    return { items: [], max: this.max }}
}

var ring = {
  push: function(item) {
    this.items.unshift(item)
    if (this.items.length > this.max)
      this.items.pop() }
}

var spells = boo.merge(ring, ring_data, {
  max: 2
})

spells.push('Watera')
spells.push('Blizarra')
spells.push('Firaga')
spells.items
// => ['Firaga', 'Blizarra']
ring_data.items
// => []

boo.extend(spells, [ring_data])
spells.items
// => []
spells.max
// => 3

Inheritance

For inheritance, Boo just defines a thin layer over the standard Object.create method, such that it accepts mixins rather than property descriptors — which are way too verbose to be practical.

The primitive derive is used for coupling inheritance and extension:

var Collection = {
  pop: function(){
    return this.items.pop() },

  push: function(item){
    this.items.push(item) },

  each: function(fn) {
    this.items.forEach(fn) },

  map: function(fn) {
    return this.items = this.items.map(fn) }
}

// Clones the `collection' behaviours and extends the clone with the
// `ring' behaviours
var RingCollection = boo.derive(Collection, ring)
var spells         = boo.derive(RingCollection, ring_data)

RingCollection.isPrototypeOf(spells)
// => true

spells.push('Agi')
spells.push('Dia')
spells.push('Patra')
spells.push('Recarm')
spells.map(function(spell){ return spell.toUpperCase() })
// => ['RECARM', 'PATRA', 'DIA']

Syntactical sugar

Last but not least, Boo gives the developer nice object-oriented syntactical sugar to write all of this in a nice sort-of declarative syntax, which is easier to read and reason about.

The base object is Base, which can be cloned through derive and instantiated with make, although both use cloning in the prototypical sense:

var Enum = boo.Base.derive({
  each: function(fn) {
    this.items.forEach(fn) },

  filter: function(fn) {
    return this.items = this.items.filter(fn) },

  map: function(fn) {
    return this.items = this.items.map(fn) },

  fold: function(fn, start) {
    return this.items.reduce(fn, start) }
})

var Coll = Enum.derive({
  push: function(item) {
    this.items.push(item) },

  pop: function(item) {
    this.items.pop(item) },

  has_p: function(item) {
    return this.find(item) != -1 },

  find: function(item) {
    return this.items.indexOf(item) }
})

var my_coll = Coll.make()
my_coll.items = [1, 2, 3]
my_coll.map(function(n){ return n * n })
// => [1, 4, 9]
my_coll.find(4)
// => 1

What's next?

Boo's source code is all on Github:

$ git clone http://github.com/killdream/boo.git

Though, if you want a quick'n'dirty install, just get it from NPM:

$ npm install boo
node> var boo = require('boo')
node> var Stuff = boo.Base.derive({ ... })

Future developments?

Next release of Boo will include traits, and should be up around December. Traits are more interesting for structuring larger programs than mixins are, but I have yet to experiment with them more until I can stick with a nice API :3

Acknowledgements

Thanks for AdamR in the comments and xivix on Freenode's ##javascript channel for pointing out the issue with Object.create on the first snippet.

Understanding JavaScript OOP

2011-10-09T00:00:00-07:00

JavaScript is an object oriented (OO) language, with its roots in the Self programming language, although it's (sadly) designed to look like Java. This makes the language's really powerful and sweet features stay covered by some pretty ugly and counter-intuitive work-arounds.

One such affected feature is the implementation of prototypical inheritance. The concepts are simple yet flexible and powerful. It makes inheritance and behaviourism first-class citizens, just like functions are first-class in functional-ish languages (JavaScript included).

Fortunately, ECMAScript 5 has gotten plenty of things to move the language in the right way, and it's on those sweet features that this article will expand. I'll also cover the drawbacks of JavaScript's design, and do a little comparison with the classical model here and there, where those would highlight the advantages or disadvantages of the language's implementation of prototypical OO.

It's important to note, though, that this article assumes you have knowledge over other basic JavaScript functionality, like functions (including the concepts of closures and first-class functions), primitive values, operators and such.

1. Objects

Everything you can manipulate in JavaScript is an object. This includes Strings, Arrays, Numbers, Functions, and, obviously, the so-called Object — there are primitives, but they're converted to an object when you need to operate upon them. An object in the language is simply a collection of key/value pairs (and some internal magic sometimes).

There are no concepts of classes anywhere, though. That is, an object with properties name: Linda, age: 21 is not an instance of any class, the Object class. Both Object and Linda are instances of themselves. They define their own behaviour, directly. There are no layers of meta-data (i.e.: classes) to dictate what given object must look like.

You might ask: "how?"; more so if you come from a highly classically Object Orientated language (like Java or C#). "Wouldn't having each object defining their own behaviour, instead of a common class mean that if I have 100 objects, I will have 100 different methods? Also, isn't it dangerous? How would one know if an object is really an Array, for example?"

Well, to answer all those questions, we'll first need to unlearn everything about the classical OO approach and start from the ground up. But, trust me, it's worth it.

The prototypical OO model brings in some new ways of solving old problems, in an more dynamic and expressive way. It also presents new and more powerful models for extensibility and code-reuse, which is what most people are interested about when they talk about Object Orientation. It does not, however, give you contracts. Thus, there are no static guarantees that an object X will always have a given set of properties, but to understand the trade-offs here, we'll need to know what we're talking about first.

1.1. What are objects?

As mentioned previously, objects are simple pairs of unique keys that correspond to a value — we'll call this pair a property. So, suppose you'd want to describe a few aspects of an old friend (say Mikhail), like age, name and gender:

Objects are created in JavaScript using the Object.create function. It takes a parent and an optional set of property descriptors and makes a brand new instance. We'll not worry much about the parameters now.

An empty object is an object with no parent, and no properties. The syntax to create such object in JavaScript is the following:

var mikhail = Object.create(null)

1.2. Creating properties

So, now we have an object, but no properties — we've got to fix that if we want to describe Mikhail.

Properties in JavaScript are dynamic. That means that they can be created or removed at any time. Properties are also unique, in the sense that a property key inside an object correspond to exactly one value.

Creating new properties is done through the Object.defineProperty function, which takes a reference to an object, the name of the property to create and a descriptor that defines the semantics of the property.

Object.defineProperty(mikhail, 'name', { value:        'Mikhail'
                                       , writable:     true
                                       , configurable: true
                                       , enumerable:   true })

Object.defineProperty(mikhail, 'age', { value:        19
                                      , writable:     true
                                      , configurable: true
                                      , enumerable:   true })

Object.defineProperty(mikhail, 'gender', { value:        'Male'
                                         , writable:     true
                                         , configurable: true
                                         , enumerable:   true })

Object.defineProperty will create a new property if a property with the given key does not exist in the object, otherwise it'll update the semantics and value of the existing property.

By the way

You can also use the Object.defineProperties when you need to add more than one property to an object:

Object.defineProperties(mikhail, { name:   { value:        'Mikhail'
                                           , writable:     true
                                           , configurable: true
                                           , enumerable:   true }

                                 , age:    { value:        19
                                           , writable:     true
                                           , configurable: true
                                           , enumerable:   true }

                                 , gender: { value:        'Male'
                                           , writable:     true
                                           , configurable: true
                                           , enumerable:   true }})

Obviously, both calls are overtly verbose — albeit also quite configurable —, thus not really meant for end-user code. It's better to create an abstraction layer on top of them.

1.3. Descriptors

The little objects that carry the semantics of a property are called descriptors (we used them in the previous Object.defineProperty calls). Descriptors can be one of two types - data descriptors or accessor descriptors.

Both types of descriptor contain flags, which define how a property is treated in the language. If a flag is not set, it's assumed to be false — unfortunately this is usually not a good default value for them, which adds to the verbosity of these descriptors.

writable: Whether the concrete value of the property may be changed. Only applies to data descriptors.
configurable: Whether the type of descriptor may be changed, or if the property can be removed.
enumerable: Whether the property is listed in a loop through the properties of the object.

Data descriptors are those that hold concrete values, and therefore have an additional value parameter, describing the concrete data bound to the property:

value: The value of a property.

Accessor descriptors, on the other hand, proxy access to the concrete value through getter and setter functions. When not set, they'll default to undefined.

get (): A function called with no arguments when the property value is requested.
set (new_value): A function called with the new value for the property when the user tries to modify the value of the property.

1.4. Ditching the verbosity

Luckily, property descriptors are not the only way of working with properties in JavaScript, they can also be handled in a sane and concise way.

JavaScript also understands references to a property using what we call bracket notation. The general rule is:

<bracket-access> ::= <identifier> "[" <expression> "]"

Where identifier is the variable that holds the object containing the properties we want to access, and expression is any valid JavaScript expression that defines the name of the property. There are no constraints in which name a property can have¹, everything is fair game.

Thus, we could just as well rewrite our previous example as:

mikhail['name']   = 'Mikhail'
mikhail['age']    = 19
mikhail['gender'] = 'Male'

⁣

Note: All property names are ultimately converted to a String, such that object[1], object[⁣[1]⁣], object['1'] and object[variable] (when the variable resolves to 1) are all equivalent.

There is another way of referring to a property called dot notation, which usually looks less cluttered and is easier to read than the bracket alternative. However, it only works when the property name is a valid JavaScript IdentifierName², and doesn't allow for arbitrary expressions (so, variables here are a no-go).

The rule for dot notation is:

<dot-access> ::= <identifier> "." <identifier-name>

This would give us an even sweeter way of defining properties:

mikhail.name   = 'Mikhail'
mikhail.age    = 19
mikhail.gender = 'Male'

Both of these syntaxes are equivalent to creating a data property, with all semantic flags set to true.

1.5. Accessing properties

Retrieving the values stored in a given property is as easy as creating new ones, and the syntax is mostly similar as well — the only difference being there isn't an assignment.

So, if we want to check on Mikhail's age:

mikhail['age']
// => 19

Trying to access a property that does not exist in the object simply returns undefined ³:

mikhail['address']
// => undefined

1.6. Removing properties

To remove entire properties from an object, JavaScript provides the delete operator. So, if you wanted to remove the gender property from the mikhail object:

delete mikhail['gender']
// => true

mikhail['gender']
// => undefined

The delete operator returns true if the property was removed, false otherwise. I won't delve into details of the workings of this operator, since @kangax has already written a most awesome article on how delete works.

1.7. Getters and setters

Getters and setters are usually used in classical object oriented languages to provide encapsulation. They are not much needed in JavaScript, though, given how dynamic the language is — ~~and my bias against the feature~~.

At any rate, they allow you to proxy the requests for reading a property value or setting it, and decide how to handle each situation. So, suppose we had separate slots for our object's first and last name, but wanted a simple interface for reading and setting it.

First, let's set the first and last names of our friend, as concrete data properties:

Object.defineProperty(mikhail, 'first_name', { value:    'Mikhail'
                                             , writable: true })

Object.defineProperty(mikhail, 'last_name', { value:    'Weiß'
                                            , writable: true })

Then we can define a common way of accessing and setting both of those values at the same time — let's call it name:

// () → String
// Returns the full name of object.
function get_full_name() {
    return this.first_name + ' ' + this.last_name
}

// (new_name:String) → undefined
// Sets the name components of the object, from a full name.
function set_full_name(new_name) { var names
    names = new_name.trim().split(/\s+/)
    this.first_name = names[⁣'0'] || ''
    this.last_name  = names['1'] || ''
}

Object.defineProperty(mikhail, 'name', { get: get_full_name
                                       , set: set_full_name
                                       , configurable: true
                                       , enumerable:   true })

Now, every-time we try to access the value of Mikhail's name property, it'll execute the get_full_name getter.

mikhail.name
// => 'Mikhail Weiß'

mikhail.first_name
// => 'Mikhail'

mikhail.last_name
// => 'Weiß'

mikhail.last_name = 'White'
mikhail.name
// => 'Mikhail White'

We can also set the name of the object, by assigning a value to the property, this will then execute set_full_name to do the dirty work.

mikhail.name = 'Michael White'

mikhail.name
// => 'Michael White'

mikhail.first_name
// => 'Michael'

mikhail.last_name
// => 'White'

Of course, getters and setters make property access and modification fairly slower. They do have some use-cases, but while browsers don't optimise them better, methods seem to be the way to go.

Also, it should be noted that while getters and setters are usually used for encapsulation in other languages, in ECMAScript 5 you still can't have such if you need the information to be stored in the object itself. All properties in an object are public.

1.8. Listing properties

Since properties are dynamic, JavaScript provides a way of checking out which properties an object defines. There are two ways of listing the properties of an object, depending on what kind of properties one is interested into.

The first one is done through a call to Object.getOwnPropertyNames, which returns an Array containing the names of all properties set directly in the object — we call these kind of property own, by the way.

If we check now what we know about Mikhail:

Object.getOwnPropertyNames(mikhail)
// => [ 'name', 'age', 'gender', 'first_name', 'last_name' ]

The second way is using Object.keys, which returns all own properties that have been marked as enumerable when they were defined:

Object.keys(mikhail)
// => [ 'name', 'age', 'gender' ]

1.9. Object literals

An even easier way of defining objects is to use the object literal (also called object initialiser) syntax that JavaScript provides. An object literal denotes a fresh object, that has its parent as the Object.prototype object. We'll talk more about parents when we visit inheritance, later on.

At any rate, the object literal syntax allows you to define simple objects and initialise it with properties at the same time. So, we could rewrite our Mikhail object to the following:

var mikhail = { first_name: 'Mikhail'
              , last_name:  'Weiß'
              , age:        19
              , gender:     'Male'

              // () → String
              // Returns the full name of object.
              , get name() {
                    return this.first_name + ' ' + this.last_name }

              // (new_name:String) → undefined
              // Sets the name components of the object,
              // from a full name.
              , set name(new_name) { var names
                    names = new_name.trim().split(/\s+/)
                    this.first_name = names['0'] || ''
                    this.last_name  = names['1'] || '' }
              }

Property names that are not valid identifiers must be quoted. Also note that the getter/setter notation for object literals strictly defines a new anonymous function. If you want to assign a previously declared function to a getter/setter, you need to use the Object.defineProperty function.

The rules for object literal can be described as the following:

<object-literal>  ::= "{" <property-list> "}"
                    ;
<property-list>   ::= <property> ["," <property>]*
                    ;
<property>        ::= <data-property>
                    | <getter-property>
                    | <setter-property>
                    ;
<data-property>   ::= <property-name> ":" <expression>
                    ;
<getter-property> ::= "get" <identifier>
                    :       <function-parameters>
                    :       <function-block>
                    ;
<setter-property> ::= "set" <identifier>
                    :       <function-parameters>
                    :       <function-block>
                    ;
<property-name>   ::= <identifier>
                    | <quoted-identifier>
                    ;

Object literals can only appear inside expressions in JavaScript. Since the syntax is ambiguous to block statements in the language, new-comers usually confound the two:

// This is a block statement, with a label:
{ foo: 'bar' }
// => 'bar'

// This is a syntax error (labels can't be quoted):
{ "foo": 'bar' }
// => SyntaxError: Invalid label

// This is an object literal (note the parenthesis to force
// parsing the contents as an expression):
({ "foo": 'bar' })
// => { foo: 'bar' }

// Where the parser is already expecting expressions,
// object literals don't need to be forced. E.g.:
var x = { foo: 'bar' }
fn({foo: 'bar'})
return { foo: 'bar' }
1, { foo: 'bar' }
( ... )

2. Methods

Up until now, the Mikhail object only defined slots of concrete data — with the exception of the name getter/setter. Defining actions that may be performed on a certain object in JavaScript is just as simple.

This is because JavaScript does not differentiate how you can manipulate a Function, a Number or an Object. Everything is treated the same way (i.e.: functions in JavaScript are first-class).

As such, to define an action for a given object, you just assign a function object reference to a property. Let's say we wanted a way for Mikhail to greet someone:

// (person:String) → String
// Greets a random person
mikhail.greet = function(person) {
    return this.name + ': Why, hello there, ' + person + '.'
}

After setting the property, we can use it the same way we used the concrete data that were assigned to the object. That is, accessing the property will return a reference to the function object stored there, so we can just call.

mikhail.greet('you')
// => 'Michael White: Why, hello there, you.'

mikhail.greet('Kristin')
// => 'Michael White: Why, hello there, Kristin.'

2.1. Dynamic `this`

One thing that you must have noticed both in the greet function, and the functions we've used for the name's getter/setter, is that they use a magical variable called this.

It holds a reference to the object that the function is being applied to. This doesn't necessarily means that this will equal the object where the function is stored. No, JavaScript is not so selfish.

Functions are generics. That is, in JavaScript, what this refers to is decided dynamically, at the time the function is called, and depending only on how such a function is called.

Having this dynamically resolved is an incredible powerful mechanism for the dynamism of JavaScript's object orientation and lack of strictly enforced structures (i.e.: classes), this means one can apply a function to any object that meets the requirements of the actions it performs, regardless of how the object has been constructed — hack in some custom multiple dispatcher and you have CLOS.

2.2. How `this` is resolved

There are four different ways of resolving the this variable in a function, depending on how a function is called: directly; as a method; explicitly applied; as a constructor. We'll dive in the first three for now, and come back at constructors later on.

For the following examples, we'll take these definitions into account:

// (other:Number[, yet_another:Number]) → Number
// Returns the sum of the object's value with the given Number
function add(other, yet_another) {
    return this.value + other + (yet_another || 0)
}

var one = { value: 1, add: add }
var two = { value: 2, add: add }

2.2.1. Called as a method

If a function is called as an object's method, then this inside the function will refer to the object. That is, when we explicitly state that an object is carrying an action, then that object will be our this inside the function.

This is what happened when we called mikhail.greet(). The property access at the time of the call tells JavaScript that we want to apply whatever actions the greet function defines to the mikhail object.

one.add(two.value) // this === one
// => 3

two.add(3)         // this === two
// => 5

one['add'](two.value) // brackets are cool too
// => 3

2.2.2. Called directly

When a function is called directly, this will be resolved to the global object in the engine (e.g.: window in browsers, global in Node.js)

add(two.value)  // this === global
// => NaN

// The global object still has no `value' property, let's fix that.
value = 2
add(two.value)  // this === global
// => 4

2.2.3. Explicitly applied

Finally, a function may be explicitly applied to any object, regardless of whether the object has the function stored as a property or not. These applications are done through a either the call or apply method of a function object.

The difference between these two methods is the way they take in the parameters that will be passed to the function, and the performance — apply being up to 55x slower than a direct call, whereas call is usually not as bad. This might vary greatly depending on the engine though, so it's always better to do a Perf test rather than being scared of using the functionality — don't optimise early!

Anyways, call expects the object that the function will be applied to as the first parameter, and the parameters to apply to the function as positional arguments:

add.call(two, 2, 2)      // this === two
// => 6

add.call(window, 4)      // this === global
// => 6

add.call(one, one.value) // this === one
// => 2

On the other hand, apply lets you pass an array of parameters as the second parameter of the function. The array will be passed as positional arguments to the target function:

add.apply(two, [2, 2])       // equivalent to two.add(2, 2)
// => 6

add.apply(window, [ 4 ])       // equivalent to add(4)
// => 6

add.apply(one, [one.value])  // equivalent to one.add(one.value)
// => 2

Note

What this resolves to when applying a function to null or undefined depends on the semantics used by the engine. Usually, it would be the same as explicitly applying the function to the global object. But if the engine is running on strict mode, then this will be resolved as expected — to the exact thing it was applied to:

window.value = 2
add.call(undefined, 1) // this === window
// => 3

void function() {
  "use strict"
  add.call(undefined, 1) // this === undefined
  // => NaN
  // Since primitives can't hold properties.
}()

2.3. Bound methods

Aside from the dynamic nature of functions in JavaScript, there is also a way of making a function bound to an specific object, such that this inside that function will always resolve to the given object, regardless of whether it's called as that object's method or directly.

The function that provides such functionality is bind. It takes an object, and additional parameters (in the same manner as call), and returns a new function that will apply those parameters to the original function when called:

var one_add = add.bind(one)

one_add(2) // this === one
// => 3

two.one_adder = one_add
two.one_adder(2) // this === one
// => 3

one_add.call(two, 2) // this === one
// => 3

3. Inheritance

Up to this point we have seen how objects can define their own behaviours, and how we can reuse (by explicit application) actions in other objects, however, this still doesn't give us a nice way for code reuse and extensibility.

That's where inheritance comes into play. Inheritance allows for a greater separation of concerns, where objects define specialised behaviours by building upon the behaviours of other objects.

The prototypical model goes further than that, though, and allows for selective extensibility, behaviour sharing and other interesting patterns we'll explore in a bit. Sad thing is: the specific model of prototypical OO implemented by JavaScript is a bit limited, so circumventing these limitations to accommodate these patterns will bring in a bit of overhead sometimes.

3.1. Prototypes

Inheritance in JavaScript revolves around cloning the behaviours of an object and extending it with specialised behaviours. The object that has it's behaviours cloned is called Prototype (not to be confounded with the prototype property of functions).

A prototype is just a plain object, that happens to share it's behaviours with another object — it acts as the object's parent.

Now, the concepts of this behaviour cloning does not imply that you'll have two different copies of the same function, or data. In fact, JavaScript implements inheritance by delegation, all properties are kept in the parent, and access to them is just extended for the child.

As mentioned previously, the parent (or [⁣[Prototype]⁣]) of an object is defined by making a call to Object.create, and passing a reference of the object to use as parent in the first parameter.

This would come well in our example up until now. For example, the greeting and name actions can be well defined in a separate object and shared with other objects that need them.

Which takes us to the following model:

We can implement this in JavaScript with the following definitions:

var person = Object.create(null)

// Here we are reusing the previous getter/setter functions
Object.defineProperty(person, 'name', { get: get_full_name
                                      , set: set_full_name
                                      , configurable: true
                                      , enumerable:   true })

// And adding the `greet' function
person.greet = function (person) {
    return this.name + ': Why, hello there, ' + person + '.'
}

// Then we can share those behaviours with Mikhail
// By creating a new object that has it's [[Prototype]] property
// pointing to `person'.
var mikhail = Object.create(person)
mikhail.first_name = 'Mikhail'
mikhail.last_name  = 'Weiß'
mikhail.age        = 19
mikhail.gender     = 'Male'

// And we can test whether things are actually working.
// First, `name' should be looked on `person'
mikhail.name
// => 'Mikhail Weiß'

// Setting `name' should trigger the setter
mikhail.name = 'Michael White'

// Such that `first_name' and `last_name' now reflect the
// previously name setting.
mikhail.first_name
// => 'Michael'
mikhail.last_name
// => 'White'

// `greet' is also inherited from `person'.
mikhail.greet('you')
// => 'Michael White: Why, hello there, you.'

// And just to be sure, we can check which properties actually
// belong to `mikhail'
Object.keys(mikhail)
// => [ 'first_name', 'last_name', 'age', 'gender' ]

3.2. How `[⁣[Prototype]⁣]` works

As you could see from the previous example, none of the properties defined in Person have flown to the Mikhail object, and yet we could access them just fine. This happens because JavaScript implements delegated property access, that is, a property is searched through all parents of an object.

This parent chain is defined by a hidden slot in every object, called [⁣[Prototype]⁣]. You can't change this slot directly⁴, so the only way of setting it is when you're creating a fresh object.

When a property is requested from the object, the engine first tries to retrieve the property from the target object. If the property isn't there, the search continue through the immediate parent of that object, and the parent of that parent, and so on.

This means that we can change the behaviours of a prototype at run time, and have it reflected in all objects that inherit from it. For example, let's suppose we wanted a different default greeting:

// (person:String) → String
// Greets the given person
person.greet = function(person) {
    return this.name + ': Harro, ' + person + '.'
}

mikhail.greet('you')
// => 'Michael White: Harro, you.'

3.3. Overriding properties

So, prototypes (that is, inheritance) are used for sharing data with other objects, and it does such in a pretty fast and memory-effective manner too, since you'll always have only one instance of a given piece of data lying around.

Now what if we want to add specialised behaviours, that build upon the data that was shared with the object? Well, we have seen before that objects define their own behaviours by means of properties, so specialised behaviours follow the same principle — you just assign a value to the relevant property.

To better demonstrate it, suppose Person implements only a general greeting, and everyone inheriting from Person define their own specialised and unique greetings. Also, let's add a new person to our case scenario, so to outline better how objects are extended:

Note that both mikhail and kristin define their own version of greet. In this case, whenever we call the greet method on them they'll use their own version of that behaviour, instead of the one that was shared from person.

// Here we set up the greeting for a generic person

// (person:String) → String
// Greets the given person, formally
person.greet = function(person) {
    return this.name + ': Hello, ' + (person || 'you')
}

// And a greeting for our protagonist, Mikhail

// (person:String) → String
// Greets the given person, like a bro
mikhail.greet = function(person) {
    return this.name + ': \'sup, ' + (person || 'dude')
}

// And define our new protagonist, Kristin
var kristin = Object.create(person)
kristin.first_name = 'Kristin'
kristin.last_name  = 'Weiß'
kristin.age        = 19
kristin.gender     = 'Female'

// Alongside with her specific greeting manners

// (person:String) → String
// Greets the given person, sweetly
kristin.greet = function(person) {
    return this.name + ': \'ello, ' + (person || 'sweetie')
}

// Finally, we test if everything works according to the expected

mikhail.greet(kristin.first_name)
// => 'Michael White: \'sup, Kristin'

mikhail.greet()
// => 'Michael White: \'sup, dude'

kristin.greet(mikhail.first_name)
// => 'Kristin Weiß: \'ello, Michael'

// And just so we check how cool this [[Prototype]] thing is,
// let's get Kristin back to the generic behaviour

delete kristin.greet
// => true

kristin.greet(mikhail.first_name)
// => 'Kristin Weiß: Hello, Michael'

3.4. Mixins

Prototypes allow for behaviour sharing in JavaScript, and although they are undeniably powerful, they aren't quite as powerful as they could be. For one, prototypes only allow that one object inherit from another single object, while extending those behaviours as they see fit.

However, this approach quickly kills interesting things like behaviour composition, where we could mix-and-match several objects into one, with all the advantages highlighted in the prototypical inheritance.

Multiple inheritance would also allow the usage of data-parents — objects that provide an example state that fulfils the requirements for a given behaviour. Default properties, if you will.

Luckily, since we can define behaviours directly on an object in JavaScript, we can work-around these issues by using mixins — and adding a little overhead at object's creation time.

So, what are mixins anyways? Well, they are parent-less objects. That is, they fully define their own behaviour, and are mostly designed to be incorporated in other objects (although you could use their methods directly).

Continuing with our little protagonists' scenario, let's extend it to add some capabilities to them. Let's say that every person can also be a pianist or a singer. A given person can have no such abilities, be just a pianist, just a singer or both. This is the kind of case where JavaScript's model of prototypical inheritance falls short, so we're going to cheat a little bit.

For mixins to work, we first need to have a way of combining different objects into a single one. JavaScript doesn't provide this out-of-the box, but we can easily make one by copying all own property descriptors, the ones defined directly in the object, rather than inherited, from one object to another.

// Aliases for the rather verbose methods on ES5
var descriptor  = Object.getOwnPropertyDescriptor
  , properties  = Object.getOwnPropertyNames
  , define_prop = Object.defineProperty

// (target:Object, source:Object) → Object
// Copies properties from `source' to `target'
function extend(target, source) {
    properties(source).forEach(function(key) {
        define_prop(target, key, descriptor(source, key)) })

    return target
}

Basically, what extend does here is taking two objects — a source and a target, — iterating over all properties present on the source object, and copying the property descriptors over to target. Note that this is a destructive method, meaning that target will be modified in-place. It's the cheapest way, though, and usually not a problem.

Now that we have a method for copying properties over, we can start assigning multiple abilities to our objects (mikhail e kristin):

// A pianist is someone who can `play' the piano
var pianist = Object.create(null)
pianist.play = function() {
    return this.name + ' starts playing the piano.'
}

// A singer is someone who can `sing'
var singer = Object.create(null)
singer.sing = function() {
    return this.name + ' starts singing.'
}

// Then we can move on to adding those abilities to
// our main objects:
extend(mikhail, pianist)
mikhail.play()
// => 'Michael White starts playing the piano.'

// We can see that all that ends up as an own property of
// mikhail. It is not shared.
Object.keys(mikhail)
['first_name', 'last_name', 'age', 'gender', 'greet', 'play']

// Then we can define kristin as a singer
extend(kristin, singer)
kristin.sing()
// => 'Kristin Weiß starts singing.'

// Mikhail can't sing yet though
mikhail.sing()
// => TypeError: Object #<Object> has no method 'sing'

// But mikhail will inherit the `sing' method if we
// extend the Person prototype with it:
extend(person, singer)

mikhail.sing()
// => 'Michael White starts singing.'

3.5. Accessing overwritten properties

Now that we're able to inherit properties from other objects and extend the specialised objects to define their own behaviours, we have a little problem: what if we actually wanted to access the parent behaviours that we just overwrote?

JavaScript provides the Object.getPrototypeOf function, that returns the [⁣[Prototype]⁣] of an object. This way, we have access to all properties defined within the prototype chain of an object. So, accessing a property in the parent of an object is quite simple:

Object.getPrototypeOf(mikhail).name    // same as `person.name'
// => 'undefined undefined'

// We can assert it's really being called on `person' by
// giving `person' a `first_name' and `last_name'
person.first_name = 'Random'
person.last_name  = 'Person'
Object.getPrototypeOf(mikhail).name
// => 'Random Person'

So, a naïve solution for applying a method stored in the [⁣[Prototype]⁣] of an object to the current one, would then follow, quite naturally, by looking the property on the [⁣[Prototype]⁣] of this:

var proto = Object.getPrototypeOf

// (name:String) → String
// Greets someone intimately if we know them, otherwise use
// the generic greeting
mikhail.greet = function(name) {
    return name == 'Kristin Weiß'?  this.name + ': Heya, Kristty'
         : /* we dunno this guy */  proto(this).greet.call(this, name)
}

mikhail.greet(kristin.name)
// => 'Michael White: Heya, Kristty'

mikhail.greet('Margareth')
// => 'Michael White: Hello, Margareth'

This looks all good and well, but there's a little catch: it will enter in endless recursion if you try to apply this approach to more than one parent. This happens because the methods are always applied in the context of the message's first target, making the [⁣[Prototype]⁣] lookup resolve always to the same object:

The simple solution to this, then, is to make all parent look-ups static, by passing the object where the current function is stored, rather than the object that the function was applied to.

So, the last example becomes:

var proto = Object.getPrototypeOf

// (name:String) → String
// Greets someone intimately if we know them, otherwise use
// the generic greeting.
//
// Note that now we explicitly state that the lookup should take
// the parent of `mikhail', so we can be assured the cyclic parent
// resolution above won't happen.
mikhail.greet = function(name) {
    return name == 'Kristin Weiß'?  this.name + ': Heya, Kristty'
         : /* we dunno this guy */  proto(mikhail).greet.call(this, name)
}

mikhail.greet(kristin.name)
// => 'Michael White: Heya, Kristty'

mikhail.greet('Margareth')
// => 'Michael White: Hello, Margareth'

Still, this has quite some short-commings. First, since the object is hard-coded in the function, we can't just assign the function to any object and have it just work, as we did up 'till now. The function would always resolve to the parent of mikhail, not of the object where it's stored.

Likewise, we can't just apply the function to any object. The function is not generic anymore. Unfortunately, though, making the parent resolution dynamic would require us to pass an additional parameter to every function call, which is something that can't be achieved short of ugly hacks.

The approach proposed for the next version of JavaScript only solves the first problem, which is the easiest. Here, we'll do the same, by introducing a new way of defining methods. Yes, methods, not generic functions.

Functions that need to access the properties in the [⁣[Prototype]⁣] will require an additional information: the object where they are stored. This makes the lookup static, but solves our cyclic lookup problem.

We do this by introducing a new function — make_method — which creates a function that passes this information to the target function.

// (object:Object, fun:Function) → Function
// Creates a method
function make_method(object, fun) {
    return function() { var args
        args = slice.call(arguments)
        args.unshift(object)        // insert `object' as first parameter
        fn.apply(this, args) }
}


// Now, all functions that are expected to be used as a method
// should remember to reserve the first parameter to the object
// where they're stored.
//
// Note that, however, this is a magical parameter introduced
// by the method function, so any function calling the method
// should pass only the usual arguments.
function message(self, message) { var parent
    parent = Object.getPrototypeOf(self)
    if (parent && parent.log)
        parent.log.call(this, message)

    console.log('-- At ' + self.name)
    console.log(this.name + ': ' + message)
}

// Here we define a prototype chain C -> B -> A
var A  = Object.create(null)
A.name = 'A'
A.log  = make_method(A, message)

var B  = Object.create(A)
B.name = 'B'
B.log  = make_method(B, message)

var C  = Object.create(B)
C.name = 'C'
C.log  = make_method(C, message)

// And we can test if it works by calling the methods:
A.log('foo')
// => '-- At A'
// => 'A: foo'

B.log('foo')
// => '-- At A'
// => 'B: foo'
// => '-- At B'
// => 'B: foo'

C.log('foo')
// => '-- At A'
// => 'C: foo'
// => '-- At B'
// => 'C: foo'
// => '-- At C'
// => 'C: foo'

4. Constructors

Constructor functions are the old pattern for creating objects in JavaScript, which couple inheritance with initialisation in an imperative manner.

Constructor functions are not, however, a special construct in the language. Any simple function can be used as a constructor function; just like this, it all depends on how the function is called.

So, what's it about constructor functions, really? Well, every function object in JavaScript automatically gets a prototype property, that is a simple object with a constructor property pointing back to the constructor function. And this object is used to determine the [⁣[Prototype]⁣] of instances created with that constructor function.

The following diagram shows the objects for the constructor function function Person(first_name, last_name):

4.1. The `new` magic

The prototype per se is not a special property, however it gains special meaning when a constructor function is used in conjunction with the new statement. As I said before, in this case the prototype property of the constructor function is used to provide the [⁣[Prototype]⁣] of the instance.

// Constructs a new Person
function Person(first_name, last_name) {
    // If the function is called with `new', as we expect
    // `this' here will be the freshly created object
    // with the [[Prototype]] set to Person.prototype
    //
    // Of course, if someone omits new when calling the
    // function, the usual resolution of `this' — as
    // explained before — will take place.
    this.first_name = first_name
    this.last_name  = last_name
}

// (person:String) → String
// Greets the given person
Person.prototype.greet = function(person) {
    return this.name + ': Harro, ' + person + '.' 
}

var person = new Person('Mikhail', 'Weiß')


// We could de-sugar the constructor pattern in the following
// Taking into account that `Person' here means the `prototype'
// property of the `Person' constructor.
var Person = Object.create(Object.prototype)

// (person:String) → String
// Greets the given person
Person.greet = function(person) {
    return this.name + ': Harro, ' + person + '.' 
}

// Here's what the constructor does when called with `new'
var person = Object.create(Person)
person.first_name = 'Mikhail'
person.last_name  = 'Weiß'

When a function is called with the new statement, the following magic happens:

Create a fresh Object, inheriting from Object.prototype, say { }
Set the [⁣[Prototype]⁣] internal property of the new object to point to the constructor's prototype property, so it inherits those behaviours.
Call the constructor in the context of this fresh object, such that this inside the constructor will be the fresh object, and pass any parameters given to the function.
If the function returns an Object, make that be the return value of the function.
Otherwise, return the fresh object.

This means that the resulting value of calling a function with the new operator is not necessarily the object that was created. A function is free to return any other Object value as it sees fit. This is an interesting and — to a certain extent — powerful behaviour, but also a confusing one for many newcomers:

function Foo() {
    this.foo = 'bar'
}

new Foo()
// => { foo: 'bar' }


function Foo() {
    this.foo = 'bar'
    return Foo
}

new Foo()
// => [Function: Foo]

4.2. Inheritance with constructors

We've covered inheritance with plain objects through Object.create, inheritance with constructors follow quite naturally from there, the difference being that instead of the main actor being the target of the inheritance (the constructor function, in this case), the prototype property is:

// new Person (first_name:String, last_name:String)
// Initialises a Person object
function Person(first_name, last_name) {
    this.first_name = first_name
    this.last_name  = last_name
}

// Defines the `name' getter/setter
Object.defineProperty(Person.prototype, 'name', { get: get_full_name
                                                , set: set_full_name
                                                , configurable: true
                                                , enumerable:   true })

// (person:String) → String
// Greets the given person
Person.prototype.greet = function(person) {
    return this.name + ': Hello, ' + (person || 'you')
}


var proto = Object.getPrototypeOf

// new Mikhail (age:Number, gender:String)
function Mikhail(age, gender) {
    // Find the parent of this object and invoke its constructor
    // with the current this. We could have used:
    //   Person.call(this, 'Mikhail', 'Weiß')
    // But we'd lose some flexibility with that.
    proto(Mikhail.prototype).constructor.call(this, 'Mikhail', 'Weiß')
}

// Inherits the properties from Person.prototype
Mikhail.prototype = Object.create(Person.prototype)

// Resets the `constructor' property of the prototype object
Mikhail.prototype.constructor = Mikhail

// (person:String) → String
Mikhail.prototype.greet = function(person) {
    return this.name + ': \'sup, ' + (person || 'dude')
}


// Instances are created with the `new' operator, as previously
// discussed:
var mikhail = new Mikhail(19, 'Male')
mikhail.greet('Kristin')
// => 'Mikhail Weiß: \'sup, Kristin'

5. Considerations and compatibility

The functions and concepts presented up until now assumed that the code would be running in an ECMAScript 5 environment, since the new additions make prototypical inheritance more natural, without the initialisation and inheritance coupling provided by constructor functions.

However, obviously this means that code using these functions will not work everywhere. @kangax has a most awesome compatibility table for the implementations that follow ECMAScript 5.

This section provides fallbacks to some of the functionality, and point to libraries that implement these fallbacks so you don't get to reinvent the wheel. Note that this section only exists to highlight how the functionality works, and how the core part of those behaviours could be reproduced in legacy code, it's not meant to provide ready-to-use fallbacks. Use libraries for that :3

5.1. Creating objects

In ECMAScript 5 we have got Object.create to handle inheritance, but constructor functions can also be used to set the [⁣[Prototype]⁣] link for the constructed object — which is what we're interested about.

A clone function could be defined such that it would create a new object based on the given prototype:

// (proto:Object) → Object
// Constructs an object and sets the [[Prototype]] to `proto'.
function clone(proto) {
    function Dummy() { }

    Dummy.prototype             = proto
    Dummy.prototype.constructor = Dummy

    return new Dummy()
}

var mikhail = clone(person)
// Equivalent to `var mikhail = Object.create(person)'

5.2. Defining properties

Object.defineProperty and it's batch cousin Object.defineProperties are also new additions, and they allow properties to be defined with internal tags, like writable, configurable and enumerable. It's not possible to get this behaviour in the older versions of the language.

All properties defined otherwise will inevitable have writable, configurable and enumerable set to true, which is usually not really that much of a problem — still, not compatible with full ES5 code.

In regards of getters and setters, they are supported to a certain extent with non-standard syntax — the __defineGetter__ and __defineSetter__ methods, — but are also not available everywhere. Most notably, such methods have never been present in IE.

// (target:Object, key:String, descriptor:Object) → Object
// Defines a property in the target object.
// Getters and Setters are handled through the fallback
// calls, whereas values are set directly. Tags are
// ignored.
function defineProperty(target, key, descriptor) {
    if (descriptor.value)
        target[key] = descriptor.value
    else {
        descriptor.get && target.__defineGetter__(key, descriptor.get)
        descriptor.set && target.__defineSetter__(key, descriptor.set) }

    return target
}


var x = { }
defineProperty(x, 'foo', { value: 'bar' })
defineProperty(x, 'bar', { get: function() { return this.foo }
                         , set: function(v){ this.foo = v    }})

x.foo
// => 'bar'

x.bar
// => 'bar'

x.bar = 'foo'
x.foo
// => 'foo'

x.bar
// => 'foo'

5.3. Listing properties

We have seen how it's possible to list the properties of an object with Object.getOwnPropertyNames, and list only the enumerable properties through Object.keys. Well, prior to ECMAScript 5, listing the enumerable properties is the only thing one can do.

This is achieved through the for..in statement, which iterates through all the enumerable properties of an object, either directly set in the object, or in the prototype chain. Object.prototype.hasOwnProperty may be used to filter the properties to include only the ones set directly in the object.

// (object:Object) → Array
// Lists all the own enumerable properties of an object
function keys(object) { var result, key
    result = []
    for (key in object)
        if (object.hasOwnProperty(key))  result.push(key)

    return result
}

// Taking the mikhail object whose [[Prototype]] is person...
keys(mikhail)
// => [ 'first_name', 'last_name', 'age', 'gender' ]

keys(person)
// => [ 'greet', 'name' ]

5.4. Bound methods

Bound methods in JavaScript do much more than just assert the value of this inside a function, they can also be used for partial function applications and behave slightly different when called as a constructor. For this, we'll just focus on the first two.

Basically, when calling the bind method of a function, we're creating a new function object that has a defined thisObject and perhaps a defined initial list of arguments. This can be just as well achieved with a closure to store the given state, and a explicit function application, through the apply method.

var slice = [].slice

// (fun:Function, bound_this:Object, args...) → Function
//  --> (args...) → *mixed*
// Creates a bound method from the function `fn'
function bind(fn, bound_this) { var bound_args
    bound_args = slice.call(arguments, 2)
    return function() { var args
        args = bound_args.concat(slice.call(arguments))
        return fn.apply(bound_this, args) }
}

5.5. Getting the `[⁣[Prototype]⁣]`

For accessing overriden properties, we need to get the a reference to the [⁣[Prototype]⁣]. In environments that expose such link (like Firefox's SpiderMonkey or Chrome's V8), it's easy and reliable:

function proto(object) {
    return object?            object.__proto__
         : /* not object? */  null
}

However, in environments that don't expose the [⁣[Prototype]⁣] link, things aren't quite as reliable. The only way of getting the prototype of an object, in this case, would be by the constructor's prototype property, but we can only access that from the object given the constructor property is kept intact.

A fallback covering most cases would look like this:

function proto(object) {
    return !object?                null
         : '__proto__' in object?  object.__proto__
         : /* not exposed? */      object.constructor.prototype
}

Note that the actual Object.getPrototypeOf throws a TypeError when you pass something that is not an object to it.

5.6. Libraries that provide fallbacks

ES5-shim attempts to implement fallbacks for ECMAScript 5 functionality that can be done in pure JavaScript, whilst adding minimal support for the other ones. It's important to note, however, that the fallbacks are intended to provide equivalent functionality that is close to the ones defined in the specs, it's not guaranteed that they will work exactly the same way.

To quote the README:

"As closely as possible to ES5" is not very close. Many of these shims are intended only to allow code to be written to ES5 without causing run-time errors in older engines. In many cases, this means that these shims cause many ES5 methods to silently fail. Decide carefully whether this is what you want.

6. Wrapping it up

The object orientation model chosen for JavaScript is definitely one of the things that makes the language expressive and powerful, however the really poor semantics from the before-ES5 age quite killed all the fun about it.

With ECMAScript 5, we have got better ways to deal with objects and inheritance, but most of the API is pretty verbose and awkward to use out of the box, so abstracting them is the only sane way of exploring all the power of the first-class inheritance model provided by the language.

Once you dwell on the depths of JavaScript's prototypical object orientation, however, you will find it lacking on aspects that would otherwise seem like the obvious thing to do — like multiple inheritance and message resending, but also basic features like an easier object extension functionality.

Luckily most of these issues manage to have a solution, albeit not necessarily a satisfactory one in some cases — i.e.: manual mixins. Being able to reproduce semantics that are not provided straight away on the language by patterns leveraging the built-in constructs is an important part of the language, and this all is made easier because of the way functions are treated in JavaScript.

7. Things worth reading up next

Brendan Eich's "Prototypical vs Closure" rant: Although not really a reading, this particular podcast from Brendan Eich is a must listen for anyone working with object oriented JavaScript. it delves on the performance of engines in regards of object construction, highlighting how the prototypical pattern stands against the Closure pattern, and discussing the specifics of how browsers handle prototypical code so they run fast.

Organizing Programs Without Classes (paper): Albeit not specific to JavaScript, this white-paper dwells on how the structuring of programs differ from the classical object orientation approach to the prototypical take on the subject. It provides lots of Self code to go with it, but they are more or less easily translated to JavaScript code.

8. Acknowledgements

Thanks to @hughfdjackson, Alex and Taylor for the additional revision of the article.

Thanks to -- for pointing out (in the comments) that properties expect an IdentifierName, rather than a plain Identifier, which would allow reserved words (like foo.class or foo.null) to be used unquoted.

Footnotes:

¹ : Some implementations have magical names, like __proto__, which may yield undesired and unwanted results when set. For example, the __proto__ property is used to define the parent of an object in some implementations. As such, you wouldn't be able to set a string or number to that.

² : While an IdentifierName also allows reserved words, you should be aware that ECMAScript engines that aren't fully compliant with the ECMAScript 5 specs may choke on reserved words when they're used for property access unquoted.

³ : It should be noted that, while ECMAScript-defined native objects don't throw an error when you try to access a non-existing property, it's not guaranteed that the same will hold true for a host object (i.e. an object defined by the engine implementor). After all, host object semantics are not defined, they are dependent on the particular run-time implementation.

⁴ Some engines do expose the [⁣[Prototype]⁣] slot, usually through a property like __proto__, however no such thing is described in the specifications for the language, so it's recommended that you avoid using it, unless you're well aware that all platforms you code must run on will have such means of setting the [⁣[Prototype]⁣] object directly. It should also be noted that messing with the prototype chain might defeat all look-up optimisations in the JS engine.

The Dark-Side of JavaScript

2011-08-15T00:00:00-07:00

Black is an extension to JavaScript's standard objects, providing missing functionality, a few aliases and making using generic functions sweeter.

It supports both functional style:

black.unpack_all(['utils', 'generic'])
each(map(filter(links, selectedp)
        ,text)
    ,display)

And Object Oriented style:

black.unpack_all(['own'])
black.list.to_array(links).filter(selectedp)
                          .map(text)
                          .each(display)

And it can use either the global scope or patch the native object's prototype directly, if you decide to do so.

If you're on Node.js, you can just install it from npm:

$ npm install black

Otherwise, clone the github repository and follow the installation instructions.

$ git clone htp://github.com/killdream/black.git

# Or if you use hg-git, like me:
$ hg clone http://github.com/killdream/black.git

The issues

Black didn't just sprung out of nowhere, it came into existence to fix several issues I've had while writing JavaScript code.

Lack of functionality and generic sugar

The main issue was that I often had to write the same functions over and over again, because the standard library didn't provide them.

That goes from the simple object handling, that is, merging two objects, from sequence handling, which I do use a lot. On this release, I've tried to include the features I miss the most, for numbers, functions, objects and sequences.

Yes, sequences, not Arrays. Since JavaScript is weakly-typed to the very core, mostly functionality is intentionally generic. However, using those generic functions on non-arrays is usually a huge. pain. in. the. ass:

var seq = {0: 1, 1: 2, 2: 3, length: 3}
Array.prototype.map.call(seq, function(x){ return x + 1 })
// => [2, 3, 4]

That unnecessary verbosity really annoys me. Other thing that does so is the inconsistency in own and generic methods. You can't just have Array.slice(sequence), even though the method is supposed to be generic, you need to apply an Array's own method to another object type if you need it. For me, it just makes no sense.

Verbosity and hatefulCamelCaseIsHateful

The second major issue I have is the naming convention used in JavaScript. Sorry guys, but camelCase sucks way too much to be taken seriously. And on top of that, lots of functionality have unnecessary and overly verbose names.

Note that when I say verbose I don't necessarily mean long. There's a huge difference here. Verbosity is the unnecessary extension of a name just for the sake of having a long name. Descriptive and concise names are your friends when you actually want to read a piece of code.

Aliasing hell

One thing I noticed a lot lately, is that all my JavaScript files were starting with the following:

var map    = [].map
  , each   = [].each
  , some   = [].some
  , every  = [].every
  , reduce = [].reduce
  , filter = [].filter
  , slice  = [].slice
  , hasp   = {}.hasOwnProperty
  , ( ... )

Things were coming to a point that I had written a template in Yasnippet to include all those lines for me. But well, even so, it was still quite unwielding and I still raged every time I saw that shitload of aliases at the start of every file (things only got worse when the file used external libraries, btw).

If only current ECMAScript had destructuring assignment support, things wouldn't be as bad:

var {map, each, reduce, filter} = Array.prototype

Which is not that much different from Python's:

from module import x, y, z

Except you can import anything you want from any object. Having written some code for SpiderMonkey-only projects before, I can say that JavaScript 1.7 introduces quite a lot of awesomeness I missed in the language for a long time (and still miss…)

What Black provides

Extensions

The main point of Black is to include extensions for working with the standard objects. Most of the extensions are for working with sequences, and objects. Extensions are logically divided in modules, and can be used separately.

type: Provides type, interface and functionality testing.
obj: Handles objects as sets of key/values. This includes extracting lists of keys and values, cherry-picking and transforming them.
list: Handles iteration and manipulation of sequence's structure and items.
str: Provides generic aliases for built-in utilities and sugar for JS/CSS interop.
num: Provides a few mathematical functions for boundary handling and padding.
fn: Provides a few functional utilities. Currently not much useful.

Sane API naming

Black's API uses a naming convention derived from Python and Lisp. Words are written in all lower-case and separated by underscores. Names are kept concise and descriptive.

From Common Lisp, it uses the p-suffix naming convention for predicate functions (like nullp for null? or isNull). And n-prefix for destructive functions when a pure alternative exists.

For functions that construct objects, a make_ prefix is used.

Patching

The actual thing about Black, though, is how it handles the aliasing hell issue. The core module provides unpacker functions that copies properties from an Object to another one. Modules decide which functionality they want to export and where they want it exported by default.

The unpackers separate functionality into utils, which are exported to the global object; generic, which are exported directly into the constructor function; and own, which are exported into the [⁣[Prototype]⁣].

This patching is carried by two functions, the just-do-the-right-thing(tm) one, unpack_all, which will follow the defaults the modules have provided:

black.unpack_all(['utils', 'generic', 'own'])
var factor = 2
first(range(1, 10).filter(function(x){
    return x > 3 * factor }))
// => 7

And the fine-tuned one, if you want control over where you want your stuff, unpack:

var numbers = [1, 2, 3, 4, 5]
black.unpack(['own'], null              // global target
                    , null              // generic target
                    , Array.prototype   // own
                    , black.list )      // source

function squared(x){ return x * x      }
function evenp(x)  { return x % 2 == 0 }
numbers.map(squared).filter(evenp).first()
// => 4

Wrapping it up

This is still an early release of the library, it's usable, but not thoroughly tested (and all tests have been done only on Node.js). Though, it has some documentation and examples, and most functionality I wanted for now, so I'm just making an early release.

I'm planning checking for set difference in the unpackers to avoid silently writing existing functionality, associative sequence handling, and other stuff, so well, still lots of work to do :3

Microframeworks and the state of the web

2011-05-22T00:00:00-07:00

Micro-"frameworks"¹ have been quite a hot topic lately on the JavaScript community. And by hot, I mean that it has lead to some pretty "heated" discussion (either good ones and "bad" ones), and some pretty interesting replies.

But what's all this fuzz about? Well, the microlibs introduce a shift from the monolithic libraries (think jQuery, Prototype, etc), to self-contained and "modular" ones.

In this post I'll try to explain what's good and bad about either approaches, delve a little on modular design and sum up what I think that's wrong with most JS libs, and where we should go from here.

Note:

I'm going to use microlibs thorough this article to refer to what people call micro-"frameworks"¹ as I think the name fits better what they actually are.

What's wrong with JS frameworks

So you have a new web app to develop, and you're looking for some building blocks because you don't want to write everything from the scratch?

Toughest luck, my dear.

Welcome to the "Reproducing desktop's UI toolkit mess in the JavaScript world" show, only worse. A thousand times worse!

1. Feature bloat

If you take jQuery, which weights around 230kb (31kb minified AND gzipped), you will get all the following goodies out of the box:

CSS Selector engine
DOM abstraction API
Deferreds
Ajax/XMLHttpRequest abstractions
JSONP
CSS helpers
Animation/Effects
Event abstraction
Some other small utilities to fill in some standard lib "gaps"
Lots of cross-browser hacks

That is, obviously, overwhelming. People usually don't use all of the features the library provide, and the overhead of this "dead code" feels even larger because you have tons of them.

The problems are also solved at a reduced scope, as you'd otherwise have an unwielding codebase/filesize. For example, the effects included in jQuery only cover the most basic use-cases, whereas other features have a lot more of coverage — DOM abstraction.

2. Large filesize

Having more code equals having a larger filesize, which is a problem in web apps because you'll be sending a lot of dead code over the wire.

This is bad from both the end-user's perspective: pages will take more time to load, and this hurts a good user experience. No one likes waiting 2 minutes to see a web site.

And from an efficiency's perspective. You should be very worried about sending lots of code that will never be used, sir. Seriously.

3. Higher learning curve

As you have more features, you'll take longer to master them. This is not really that much of a problem on the end-user dev, because you would probably have to learn the same amount of things by stacking lots of small libraries together.

However, the thing is that it'll take far more time to understand all the implications of the source code to extend it. Whereas with a simpler and focused library, it would be quite straight forward.

4. Framework-centric code

But all these minor annoyances aside, there's one big issue with the way most JavaScript frameworks are designed: they encourage you to write Framework-centric code.

Want to write a date handling library? You should tie the core workings to the jQuery's DOM manipulation functions. What about a library that abstracts browser storage? TIE IT TO THE FRAMEWORK'S DOM HANDLING LIKE A BOSS!

Mind you, the browser is not just about the DOM, or a framework. Writing framework-centric code when your library does NOT extend on the problem the framework is trying to solve is a silly thing.

The community can't benefit from your contribution as a whole because they can't include the DOM framework you're using in their projects. Either because doubling the overhead with code that solves the same problem should warrant you a death sentence, and because mixing DOM libraries may break everything in some cases (Prototype + jQuery, for example).

Microlibs to the rescue!

As a response to that, wild microlibs appeared. The main design goal of these libraries is to focus on doing just a single thing — and doing that well, while also "doing the simplest thing that could possibly work".

As such, there's little to no overlap on the features offered from one microlib to another (given they don't try to solve the same problem, of course) and you get only what you really need. No more, no less.

In fact, microlibs would fit the Unix philosophy quite well (the text streams aside, usually):

Write programs that do one thing and do it well. Write programs to work together. Write programs to handle text streams, because that is a universal interface.

…but microlibs are broken too

Reality is a bit harsh, though. There are a few problems with the current state of microlibs, both because they're pretty new and because somehow this whole size-limit thing has gotten quite messed up.

Some of these problems are quite a show stopper.

1. Lack of documentation

Documentation is good and shouldn't be overlooked. If you want people to use your code, you should tell them how they should use it. No, dumping a weird API reference on a README file won't cut it. Random words scrawled over the source won't either.

I'm talking about good documentation on the library btw, not just some API reference. And most microlibs I know (including my Ekho lib) have no examples, no use cases, no how to get started, no anything!

But there are worse cases (imo). All @ded's libraries have almost no comments on the source code, and rely rather on dumping a little of introductory text on an obscure README.md file.

Mind you, I want to know what are the purposes and implications of using some function if I'm going to extend the library. I don't want to shoot myself on the foot by accidentally calling a function that happens to have side-effects which are not apparent from reading the function alone.

2. Lack of tests

Some libraries do have a small amount of test cases, but as @jDalton showed on his screen cast, some don't really have that good of a coverage.

Some libraries (Ekho) have absolutely NO unit tests, which is something you should find outrageous² (and, maybe, help fixing).

3. Small community

This is more related to how recent all these libraries are. They don't have a large user base like the monolithic frameworks. So it may be quite more difficult to find an answer to problems you're facing.

It's also likely that the library will have problems that people using more mature and larger frameworks have already reported. And these would take a bit to fix as well.

4. Trying to solve problems they're not meant to

According to the buzz on the internets, microlibs are supposed to do only one thing and do it well™, but in practice, that's not true. Almost all of these libraries will provide fallbacks for features that are not related to the design goals of the library, and worse, for features that are described in the ECMAScript 5 specs.

Yes, I'm looking at all of the libraries that implement Array#indexOf, Array#filter, Function#bind and such fallbacks, when the library tries to solve an entire different problem.

This creates two problems. The first is that now instead of having dead code, you have duplicated code. It's even worse if you're only targeting platforms which are ES5-compliant, because now you have duplicated dead code.

The second problem is that some of these libraries implement things that don't conform with the specs. And I'm not too keen on having 20 different implementations of Array#indexOf which don't work for sparse arrays, thank you.

5. API inconsistencies

You can't expect to get API consistencies when using libraries from 10 different authors if they're not designed to work together, which you'd get for free on JavaScript frameworks (or would you?³).

This can be really bad when you need to have two libraries to work together, say a CSS selector engine and a DOM manipulation library. Of course you can abstract all of them and create a DSL on top of them, just as you could read a minified source rather than a properly commented one…

6. Dependency hell

Browsers still don't offer native support for modules, although it's being discussed for ES.Next, nor can you use jspm install microlib and have all the dependencies sorted out nicely for you.

The thing is that using lots of small libraries will make your building and testing process quite more complex. Where you had to include a simple script tag to load everything you need, you'll need to include loads of them (in the right order too), or have something that automagically concatenates those files in the right order.

Either way, it's quite some hassle.

7. Focus on arbitrary filesize

Having a soft-limit for filesize isn't necessarily a bad thing. It helps you to know when you're over-engineering things. However, not all the problems are the same that you can happily have a 5kb limit for everynyan.

With the microlibs, things have somehow gotten screwed up and the focus that should be into "reducing the scope of the library for modularity" (think minimalism), turned into a battle for who could fit an entire DOM library in a tweet.

I find this whole premature and reckless optimisation for filesize pretty stupid, of course. As Sussman says:

Programs must be written for people to read, and only incidentally for machines to execute.

Programs should also be written for correctness and to effectively solve the problems they propose to solve, but I guess everyone knows that already…

What can we do?

So, large and bloated frameworks are a bad idea for web apps, but the current state of microlibs is also pretty bad. That's no fun, and definitely not good for the community, right?

The obvious answer to these problems is to focus on writing modular libraries. And when I say modular, I'm referring to libraries that fit the Unix philosophy.

They should be designed to solve just one problem, and solve it well, so people can decide exactly which problems they want to solve instead of getting a large black box that tries to solve everything.

They should also be designed to work nicely with other libraries, otherwise what's the point in having modular stuff anyways? You'll most likely never have to solve just a single problem when writing a web app.

A new design philosophy

In fact, we can summarise it nicely in a list of few points to keep in mind when designing a library. Mostly stolen from Eric Raymond's The Art of Unix Programming book:

Modularity: Write simple parts connected by clean interfaces.
Composition: Design libraries to be connected to other libraries.
Simplicity: Design for simplicity; add complexity only where you must.
Correctness: If it doesn't fully solves the problem it's designed to solve, it's broken.
Lazyness: Don't try to solve what's already solved; stop writing broken ES5 fallbacks.
Parsimony: Write big libraries only when it's clear by demonstration that nothing else will do.
Optimisation: Prototype before polishing. Get it working before you optimise it (be it for size, speed or anything else).
Extensibility: Design for the future, because it will be here sooner than you think (modules are a good thing, and they'll be added to the language!)

Problems with this philosophy

Obviously, these points are not anything near perfect. And it gets worse if you take the current state of browser platforms into account. The web is quite heterogeneous. Even on browsers who do conform with the standards, you still have some quite funny discrepancies on the inner workings — and that affects your code.

To make things yet worse, you don't have much things to solve the dependency hell, aside from writing a custom build tool. And even so, you'll likely need to download and build a set of different libraries to get everything working.

This is not exactly my definition of fun.

For this to really work, we'd need a good package manager; like what npm is to Node.js. And we would also need either some support for loading these external modules in a sane way (you do have some libraries for that), rather than having to do with build scripts.

Wrapping it up

There's no such thing as one true way. I don't think modular libraries/frameworks solve all the problems, but they do address quite some of the problems we have with the current models of libraries.

And it would be nice having a healthier environment for developing new awesome apps on the Browser, without having to write everything from scrath again because the thing you need is only included in another library, and it uses a framework-centric approach.

All in all, it's something I think would benefit the JavaScript community as a whole :3

Footnotes:

¹ I hate using the term "framework" to refer to libraries with all my guts. Seriously, people, is `library` **that** hard to spell?

² I'll work on tests and documentation for Ekho and friends when I have time to do so. Probably after I finish some of the core concepts/early prototypes for OrpheOS. Until then, considering the library a beta release may be a good thing.

³ I don't think PrototypeJS has a consistent API, for example. Though Ruby people might say otherwise, it just strikes me as odd if you use bindAsEventListener and gsub randomly on your API (I would never take a Ruby programmer's word on good API design anyways ;P). jQuery's API on the other hand (although I've only used a small subset of it for a short time) is just a mess of counter-intuitive and poorly-named methods. People should stop getting inspired by PHP when designing an API…

Meeting Enlightenment

2011-03-05T00:00:00-08:00

I've used quite a few Window Managers in the past: IceWM, KDE, Gnome-Shell, Metacity, Fluxbox, etc. but they either looked plain horrible, were bloated like hell — yeah, KDE, take that — or were just plain and irritatingly slow.

I've always liked lightweight and tiling window managers — which is one of the reasons I love Emacs — but they all felt too plain to my likings. As an art lover, I need some spark and beauty on my desktop, rather than strictly functional things.

So, here am I ranting about about the features of e17 that got me hooked, and why you should give it a try.

Why you should try it

You probably heard elsewhere that enlightenment is lightweight and sparkles. But I don't think that is enough reason to switch to a Window Manager — all the more when it hasn't seen a stable release yet.

So, here's my little list of reasons:

Lightweight and FAST

I have a kinda old PC with an on-board graphic card, which means that not even OpenGL gets some love here, thus being lightweight is a must for me. Not only this, but Enlightenment manages to be a lot faster than any other window manager I've tried so far — taking the delicious amount of eye-candiness into account, obviously.

I also have a MacPup that runs directly from my thumb-drive. The distro is bundled with IceWM, Enlightenment and another window manager I can't remember the name for life, still it fits nicely in less than 228MB, and runs entirely on the RAM, leaving all the remaining space on my thumb-drive for actual data.

It doesn't makes much of a difference if you have a bleeding-edge PC, or use any KDE application out there — I try to stay clear of them, by the way.

Fully configurable

Which is one of the reasons I use Linux in the first place. I like configuring everything to fit my workflow, and Enlightenment lets me do so quite easily with the settings GUI.

Most of this configuration is quite straight forward, but some of the UI are not that good.

You can also configure everything from the command line, or editing the configuration files — though in this case you have to decompile them and recompile when you're done.

Another good thing about Enlightenment's configuration is the profiles. This allows you to have as many different configurations as you need, and allows you to safely test stuff away, while having your previous configuration safely stored.

For example, I use three different profiles for my MacPup: Netbook, Notebook and Desktop. Each providing a slight different set of keybindings, mouse adjustments and screen/modules. So any time I plug it in a different computer, I just have to select a suitable profile instead of reconfiguring everything.

Modularity

Since Enlightenment is built up on a modular structure, you can choose exactly which features you use instead of wasting RAM on useless stuff. This also means that plugins blend in really nice with the entire Window Manager, which is always a good thing.

Alongside with the configuration profiles, this lets you test modules away without having to worry about breaking the Window Manager — since you can always go back to your previous sane profile.

Shelves and Gadgets

Metacity has desklets and panels, Enlightenment has shelves and gadgets. Gadgets are just small applications you can use on your desktop, and shelves are a way to group a collection of gadgets.

Shelves also have a few interesting features that I like. The most important of them is the ability of hiding a shelf until I actually need it — by either moving my mouse over the edge of the screen, or clicking on it. I just hate clutter and think my screen space is better used to hold the windows of the applications I use, instead of a taskbar or dock.

Another interesting stuff about shelves is that, aside of the screen position, you can also specify the stacking position. So shelves can be placed above or below everything. You can also configure whether the shelves' space should be available for windows or not.

Top, top-left and bottom-right are shelves. The other ones are plain gadgets placed over the desktop.

Charmingly beautiful

Even if you don't have a powerful enough computer to run the compositing stuff (Enlightenment ships with Ecomorph, which is a port of Compiz), there are still plenty of eye-candy for you, including animated backgrounds, animated icons and transitions.

Better still, everything is theme-able, including the third-party modules. And you can build your own theme with parts of various other themes.

For Gtk applications you'll need to install a suitable Gtk theme though.

Currently, I'm using Detour as my base Enlightenment theme, which parts of Simply White, Imago and a few others. For the Gtk part, the Orta theme blends in quite nicely and looks hella sweet :3

To feel the beauty of Enlightenment you have to try it though. It's not something that can be portrayed easily with images. You can have a go at some youtube videos, though.

Everything a few keystrokes away

I loathe going through xyz menus just to launch an application — I hate most menus, actually. Enlightenment comes with a nice QuickSilver-like launcher though, called Everything. If you're not familiar with the Mac world, it's something like Gnome-Do, but with far more levels of awesomeness — and frankly, I've always found Gnome-Do frustrating.

Everything follows the modularity structure of Enlightenment, so you can add any plugin to it to make doing some tasks faster and easier.

You can have a quick calculator, spell checker, web search, file search and any other thing you want with a plugin. It's really handy (even for someone who almost uses Emacs as his OS).

Shortcuts for anything

Another thing that I dislike are mouses. I just can't get used to them and feel that they get too much on my way. Well, when using Enlightenment I don't need it most of the time, because there's keybindings for almost everything.

The input dialogues allows for shortcuts for usual system commands, as you would expect, but also to manipulate virtual desktops and windows. And when I say manipulate windows I mean you can resize, position and switch between these windows using your keyboard in a sane way.

I have almost all possible cute mnemonic combinations with the useless Super key (Super+key, Ctrl+Super+key, Shift+Super+key, Alt+Super+key, …) to do some actual useful stuff, so I guess you can get an idea of the awesomeness :3

Tiling

I believe this is bundled in e17 in the Illume2, but since the Illume stuff is for embed and mobile systems, I haven't dig into this. Instead I use the tiling module that's on the SVN.

The module isn't perfect, but I don't have that many complaints against it.

Handling errors gracefully

Yes, the Enlightenment's window manager (e17) is still in pre-alpha, so it's not all that stable — although I've been using it for months now without much trouble.

However, you're likely to experience some segfaults if you play around too much:

It's not something you should worry that much though, only the window manager is affected, not your data. And it gracefully recovers itself to the previous state in less than a second, which is a pretty nice thing.

The drawbacks

Yeah, Enlightenment is great and all, but it has its drawbacks. The worst issue is really stability. Not that it will crash each minute, but you're likely to experience a few minor issues: like an auto-hide shelf that won't hide and stuff like that.

This can all be solved by restarting the Window Manager — and I mean the WM not your DM, which means that your application state won't be affected. You can restart e17 from the main menu in Enlightenment → Restart. I personally prefer binding a shortcut to it (Super-r) and using that.

You may also miss some features. I think the modules have a great coverage of the features I need — even though the tiling isn't perfect — but they may not cover well your use case.

Getting it up and running

There are quite some ways to try Enlightenment out now. You don't need to compile it out yourself if you don't want to, or are just to lazy to do so.

The easy way

You can grab any Linux distro that comes with Enlightenment as their default Window Manager and run from the LiveCD or install it on a spare HDD partition or a VM. Just get any of the following and try it away:

Unite17 — Unity-based desktop distro
MacPup — Puppy linux distro (uses Ubuntu Lucid Lynx)
moonOS — Ubuntu-based. Neak's main edition uses e17.
Bodhi — Minimalist Ubuntu 10.04-based distro.

The hard way

If you want to try your luck, you can grab the source from the SVN repository and compile it yourself… Okay, it's not so hard, it just takes a little bit of time depending on your internet connection and computer resources.

I'll try to describe a little the steps to get e17 up and running, and while these concepts should apply to any distro, bear in mind that I'm assuming an Ubuntu one as basis for package list and specific commands.

Before installing

e17 depends on quite a few libraries and tools. The svn repository lists most of it, but depending on your distro and the amount of C-coding/compiling you've done, it may vary. At least, make sure you have all of the tools and libraries listed there.

If you happen to be using an Ubuntu distro, you can get away with getting the following from apt (just copy/paste it on the terminal):

$ sudo apt-get install subversion gcc autoconf \
autopoint automake libtool make gettext \
libpam0g-dev libfreetype6-dev libpng-dev \
libjpeg-dev zlib1g-dev libdbus-1-dev \
liblua5.1-0-dev libx11-dev libxcursor-dev \
libxrender-dev libxrandr-dev libxfixes-dev \
libxdamage-dev libxcomposite-dev libxss-dev \
libxp-dev libxext-dev libxinerama-dev \
libxkbfile-dev libxtst-dev libtiff-dev \
librsvg2-dev libgif-dev libcurl4-gnutls-dev \
libgnutls-dev libxml2-dev libudev-dev

Building it

You can just checkout from the svn repository and run autogen.sh && make && sudo make install for every project. BUT THERE ARE BAZILLIONS OF THEM. To automate this annoying chore, I just use the easy_e17.sh script.

So, to get it the easy way:

$ wget http://omicron.homeip.net/projects/easy_e17/easy_e17.sh
$ chmod +x easy_e17.sh
$ sudo ./easy_e17.sh --install

This will install the the most basic functionality of e17 — the core EFL and the WM.

If you want to get more power, you can install all of the extra modules, by providing --packagelist=half to the installer, or simply everything, by providing --packagelist=full. Be warned that some of these additional features may not compile, or may be unmaintained and conflict with your stuff.

To peek on the things you can configure in the install script, just use --help.

If you'd rather have control over all of the build process, or want a sane way of getting the newest features of the repository, you can use the sample script on the svn repository as basis and modify it as you need.

Note that for building the ewheather module you'll need to build and install the libeweather (on trunk/PROTO) first. For the places module you'll need HAL too.

Setting the environment

If all goes well, you'll have enlightenment and its libraries (and perhaps a few extra modules too) fully installed on your system. If you've used a non-standard install path, which is probably the case with running the easy_e17.sh script, you'll need to export the paths so your OS can find all that stuff.

You can just copy the lines easy_e17.sh tells you and paste them in your .bashrc file, then reload it:

$ nano ~/.bashrc
$ source ~/.bashrc

4. Telling your OS to use e17

If you're using a Desktop Manager, you should probably look for how to add more sessions to it. If you just don't care about all that, you can set a simple .xsession file on your $HOME folder and restart X:

$ echo "exec `which enlightenment_start`" > ~/.xsession

if you're using a Desktop Manager, you'll need to logout and select User Defined Session.

Wrapping it up

As you can see, there's much more to Enlightenment than the "lightweight" or "beauty at your fingertips" stuff you hear everywhere, but most of the nice things about it can't be easily (if at all) put into words. It's about the UX — about how you feel while interacting with the system, — and as such it's something you really have to try out to understand.

Don't forget to drop by #e@irc.freenode.org if you have any doubt or problems with EFL or e17.

My workspace in 7 items

2011-02-12T00:00:00-08:00

Cindy, that cutie, invited me for this meme, so here I am, describing the things I use in my little everyday life to get my work done. Those are very important things for my work.

I mean, seriously :3

GNU/Linux: Ubuntu 10.10

I should start with the most important of all these things: my OS. I've been using the Ubuntu distro for quite some time now and I'm quite happy with it. Though I still dislike some of its organisation, most of the things are nicer than the other GNU/Linux distros I've tried so far.

And yes, I'm saying that being noob friendly (as in user friendly) isn't a bad thing — au contraire, it's one of its strengths. At any rate, it's still a GNU/Linux at its core, and you can customise it to your likings, just like any other distro. Which, mind you, it's the important part.

Metacity sucks balls though, so I've switched to using Enlightenment e17 as my Window Manager. Despite being a dev version, it's actually pretty stable and the core libraries just hit a stable release a while ago. The thing I like about it are the customisable key bindings for everything, so I almost don't need to use the mouse for anything. Not to mention it's hella fast and beautiful :D

Emacs

My second OS… I mean, my text editor of choice! Emacs, like GNU/Linux is pretty customisable and robust. Vim people like saying it lacks a good text editor — I beg to disagree. I love most of Emacs text editing features, and I have gotten so used to its shortcuts that they just make sense (and I keep pressing C-s to search texts in Chrome now)

Kill ring and mark ring are one of my favourite features. The first one allows you to cycle through previous text copied to the clipboard, the second lets you save positions in the buffer and cycle through it.

For programming, I use js2-mode with yasnippet and auto-complete-mode for JavaScript, and python-mode with pymacs/ropemacs for Python. I use the standard modes for most of the other languages, where available.

Obviously, I don't use Emacs just for programming. As a good Emacs hipster, I try to do everything from it, so I use ERC for chatting with people on IRC, Twittering-mode for posting to twitter and stalking people, Tumble-mode for posting to my Tumblelog, aHG for versioning my files through Mercurial, and a few other things.

For e-mail I decided I'm better off sticking with Thunderbird though…

Mercurial

Simply the best VCS. Ever. Period. Take that, git!

Erm, anyways, I use Mercurial for my versioning. I used to rely on ol' SVN pal back then, and the transition to Mercurial was really pretty smooth. Everything in Hg feels pretty natural if you have some background in version control, and when it doesn't feel that intuitive, the great help is there to save the day.

Since Mercurial's design involves having each command focused in doing just one thing — and doing that well — you can quickly discover what the command does by reading an one-line summary about it.

Those are basically the things that made me fall in love with Hg. And yeah, I've used Git, but it's just not my thing. Each command in Git is like a swiss army knife, and it's not anything near intuitive until you get familiar with the git-way of doing things (which I never did).

For interacting with Git repositories I prefer using the hg-git extension and sticking with Mercurial for my local versioning.

Headphones

Now these are more important than my OS. REALLY, I'm one to listen to music all day long. I just can't live without it. When I'm not listening to music on the 'puter, I'm doing so on my mobile, so yeah, I'm a bit of a music freak :3

Bad mobile pictures plus messy hair for the win :3

Compass

Compass is a framework on top of Sass to take out a bit of the DRY from CSS. I can't work with CSS without using it anymore, it provides lots of useful mixins that you can use to workaround including lots of vendor prefix properties in your stylesheets.

In short, it keeps everything cleaner. And as a clean code junkie, there's just no way I wouldn't love it :3

Dropbox

I love Dropbox. I really do. I keep there lots of private backups and use it to share content with the doujin group I'm helping with.

Cute pictures folder

This is also a pretty important item on my workspace. I have slight more than 10GB of cute anime pictures, Japanese idols (most of which are from the sweet Nozomi Sasaki) and cute fashion clothes.

And what do I do with them? Well, I just stare and go d'awww, of course, then get motivated to do work :3

It goes without saying that all my pictures include properly dressed girls (and traps)…

Wrapping it up

That's about it. But now that I've gone through it all, I'm curious to see how my fella programmers' workspace looks like.

No, this is not an implicit invitation for a meme. In fact, it's not implicit at all, so go write it up now :D

Recursive

The Hikikomori's Guide to JavaScript

Introduction

Start with an API

Provide “glue” code to compose concepts

Work with structured data

Write your application using your API

Conclusion

Footnotes:

Trans people should not be allowed to pee!

Conclusion

About this piece

Footnotes:

A Case for Prototypes

Maintaining purity

The real deal

The `asGenerator' combinator

The `choice' and `frequency' combinators

And finally, the `repeat' combinator

Conclusion

Acknowledgements

Unfancy API Documentation

An overview of the documentation landscape

Separating the concerns in documentation tools

Meet Dollphie, Calliope, Kalamoi and Papyr°

The generic structural annotation

A format to exchange entity meta-data

Visualising entity meta-data

Conclusion and future works

On semicolons, pseudo-technical and subjective arguments, revisited

A bit of history

What is ASI?

How ASI actually works?

Is ASI safe?

Do all of the engines implement it properly?

Is it safe?

Will my code suddenly break?!

Reasons to use or not ASI

Conclusion

Footnotes:

Yay for Sugary JavaScript OO

Why? Oh god WHY?

What Boo provides?

Mixins

Inheritance

Syntactical sugar

What's next?

Future developments?

Acknowledgements

Understanding JavaScript OOP

1. Objects

1.1. What are objects?

1.2. Creating properties

1.3. Descriptors

1.4. Ditching the verbosity

1.5. Accessing properties

1.6. Removing properties

1.7. Getters and setters

1.8. Listing properties

1.9. Object literals

2. Methods

2.1. Dynamic this

2.2. How this is resolved

2.2.1. Called as a method

2.2.2. Called directly

2.2.3. Explicitly applied

2.3. Bound methods

3. Inheritance

3.1. Prototypes

3.2. How [⁣[Prototype]⁣] works

3.3. Overriding properties

3.4. Mixins

3.5. Accessing overwritten properties

4. Constructors

4.1. The new magic

4.2. Inheritance with constructors

5. Considerations and compatibility

5.1. Creating objects

5.2. Defining properties

5.3. Listing properties

2.1. Dynamic `this`

2.2. How `this` is resolved

3.2. How `[⁣[Prototype]⁣]` works

4.1. The `new` magic

5.5. Getting the `[⁣[Prototype]⁣]`