blog. Ivan De Marino

Web dev: do we need to raise the bar?

2012-02-21T18:16:33Z

Articles like this one and initiatives like "pre-fix the Web", aim to highlight to web-developers how important is to build a cross-browser web. The importance of such a point has been explained in so many places, there is no point in me saying anymore. People like Tim Huegdon can give you a far better lesson on this.

But if you are after a "one sentence to blame them all":

If you write a website caring only to test it on WebKit based browser (worse, only your iPhone!!!), you are a MORON.

Recently I happened to re-read the "How Browsers Work: Behind the scenes of modern web browsers" and precisely the section about the forgiving HTML Parser. I ask myself: why did we (the web people) cheer at HTML5 and killed XHTML2, if not for the forgiveness? Didn't we do this to "lower the bar", to allow "your young cousin to write website for his dad's business"?

We do spend LOADS of time pushing our self and colleagues to adopt best practice, validate code and so forth (and that is great!). Why do we fail to make the logic step of thinking:

The guy that I swear at for using -webkit-* only prefixes, is the same guy that uses/abuses the HTML parser!

On one side we help, and sometimes even endorse stupid people, because we want the web to be accessible (I guess) to young and non-experienced (more content on the web == better web ???). But when they use it to do proprietary shit, things we would have not expected or hoped for, then we go on berserk (I'm the first to do it!) and start talking like grandpas recalling Vietnam when their nephew Is preparing to leave for Iraq:

... It's Internet Explorer monopoly all over again! ... Stop WebKit from taking over the web! ...

A question: what about enforcing HTML parser to be less forgiving? What about a shame-banner built in the browser appearing on the top of the viewport, saying:

This site is shite! Who made this is a moron!

I'm only half joking: either we want stupid people writing bad websites, or we don't. Either we want an easy to access, low-bar web, or we want a web of experts, computer scientists even, that know what-the-fuck they are doing.

What's your take?

My Rhinoceros likes WebSocket, not leafs

2012-02-14T10:03:43Z

Last week I started implementing W3C Sockets API into a Rhino based JavaScript runtime.

It seems like plumbing API from one environment to another is one of the activity I do the most. And if you dare doing a joke about my Italian-ness and Super Mario... you will be pointed in the direction of the door.

Here I want to share a couple of findings.

Rhino is not bad at all

Yes, I said it. And who has worked in the past with me, knows how my opinion are hard to change. But they do sometimes, fortunately.

a big one...

I'm accustomed to QtScript and QtWebKit: those are the javascript runtime API/environment I used the most. But I have to give credit to this very good piece of software from the Mozilla Foundation.

In facts, Rhino is not bad at all. I was initially very dubious of his quality (Java, ehm...) but I have to slap my own hand: it not only provides good execution time, but it also has a very, VERY nice API. And Neustar Webmetrics runs it's whole business on it!

What I liked the most? 2 things:

One, the fact that to create a JavaScript read-only property named, say, prop you just have to provide getProp method. And if you want the property to be writable, provide a setProp. The engine will take care of mapping L-VALUES or R-VALUES usages to the correct getter/setter.
Two, the NativeFunction Class is neat and easy to use: extremely simple to allow the scripter to provide an event handler. Almost brain-dead-simple.

Java-WebSocket

Nathan Rajlich has made available on GitHub a good (not yet complete, though) implementation of WebSockets in Java. The API is simple, reflects more or less the W3C specs, and does the job.

the fun of WebSocket handshake

I used it with a couple of Node WebSocket implementations (miksago's and einaros'), and it's the only one that managed to negotiate a connection and transmit stuff without issues.

There are not many implementation of WebSockets out there yet (particularly for Java). I believe the main reason being the unstable and still-changing specs (take a look at this table).

So, KUDOS to Nathan!

P.S. A beer if you can write in the comment where the name Rhino originally comes from. A hint: you must have studied JavaScript BEFORE jQuery was cool and XHR was yet to become a superstar.

Scope 'this', and scope 'that'

2012-02-13T16:22:19Z

As a guy that tries to live his professional life in the middle of the dichotomy "C/C++ - JavaScript", I often find myself discussing the discrepancies, differences and implementation details of the latter (an half-assed prototypal language) using the former.

The other day I was chatting with Luca (@lucabox) about scope in JS, and we were describing the different situations you can end up with... and how we work our way out of those culprits.

What I did later on, was picking up JS-Bible and go to Chapter 4, Functions. I strongly advice you get yourself one and read that chapter again and again, as I did now for the 3rd time. Every time there is a little bit of news to grow your knowledge. That book has to stay on your desk, at any time.

After that I did the obvious: git checkout master && qmake && make my beloved PhantomJS and started fiddling around.

In Chapter 4 of Crockford's there is a section called "Invocation": there The Man highlights the 4 Invocation Patterns that can be used in JavaScript. What I wrote was a practical demonstration of those patterns: this way it would be an easy documentation of how functional scoping is such a tricky bitch to handle.

The "subject"

First, I wrote this function:

var foo, obj, that, Constructor, ClosuredScope;
aVariable = "I'm a String in a global (attached to 'window')";

// 'foo', but no 'bar'
foo = function(callMessage) {
    console.log("\n\033[0;32m" + callMessage + " {\033[0m");    //< console.log(callMessage + "{");

    console.log(aVariable);
    console.log(this.aVariable);
    console.log(window.aVariable);
    if (typeof(that) != "undefined" && that.aVariable) {
        console.log(that.aVariable + " (passed via \033[0;31m'that'\033[0m)");
    }
    if (this === window) {
        console.log("Here \033[0;31m'this'\033[0m resolves to \033[0;31m'window'\033[0m");
    }

    console.log("\033[0;32m}\033[0m\n");                        //< console.log("}");
};

What this those is reading the value of aVariable, taking it implicitly from the current scope and explicitly from this (i.e. pointer to the current scope) and from window (i.e. the gloval scope). Also, it does a couple of extra bits:

Checks if there is that.aVariable in the current scope (in case we are using the famous that = this trick)
Highlights when the magical JS Runtime is using the global scope (window) as local scope (this)

Yes, and I added some bash colouring to my print-out: I like to have the key stuff highlighted to me.

Function Invocation Pattern

I started with something very easy:

1 2	// Function Invocation Pattern foo("* Proving the 'Function Invocation Pattern' *");

that will print out:

*** Proving the 'Function Invocation Pattern' *** {
I'm a String in a global (attached to 'window')
I'm a String in a global (attached to 'window')
I'm a String in a global (attached to 'window')
Here 'this' is equal to 'window'
}

What's happening here is obvious: the scope in which the invocation is executed, is parent of the internal scope of the callee. Hence, if a variable is not found in the local scope, will be looked-up in the parent scopes(s).

But we all knew that, didn't we?

Method Invocation Pattern

Now, let's start to have some fun:

// Method Invocation Pattern
obj = {
    aVariable : "I'm a String in an object (i.e. scope/'this')",
    foo : foo
};
obj.foo("*** Proving the 'Method Invocation Pattern' ***");

the above will print:

*** Proving the 'Method Invocation Pattern' *** {
I'm a String in a global (attached to 'window')
I'm a String in an object (i.e. scope/'this')
I'm a String in a global (attached to 'window')
}

The object obj to which we added a reference to foo, now acts as the scope parenting the internal function scope. Hence, aVariable is found in obj, set to something different. In other words, during the invocation the function run in a scope where obj === this.

Constructor Invocation Pattern + Prototype Inheritance

// Constructor Invocation Pattern + Prototype Inheritance
Constructor = function(s) {
    this.aVariable = s;
};
Constructor.prototype.foo = foo;
obj = new Constructor("I'm a String in an object (i.e. scope/'this'), assigned via a Constructor");
obj.foo("*** Proving the 'Constructor Invocation Pattern + Prototype Inheritance' ***");

predictably this prints:

*** Proving the 'Constructor Invocation Pattern + Prototype Inheritance' *** {
I'm a String in a global (attached to 'window')
I'm a String in an object (i.e. scope/'this'), assigned via a Constructor
I'm a String in a global (attached to 'window')
}

when I was working at Betfair, I found this kind of inheritance in the hierarchy we used, and all was based on YUI2. Than moved to more Closure-based inheritances with YUI3.

It's not a bad pattern per se, and makes it pretty clear to understand what's going on, but it does not offer the protection of Closures.

Constructor Invocation Pattern + Closure

Here I have to take a bit of "liberty" and make some ugly detour:

ClosuredScope = function(s) {
    var aVariable = s;
    this.aVariable = s;
    this.fooWrapper = function(m) { foo(m); console.log(aVariable + " (from the wrapper)"); };
    this.fooOriginal = foo;
};
obj = new ClosuredScope("I'm a String in an object (i.e. scope/'this'), accessible via Closure");
obj.fooOriginal("*** Proving the 'Constructor Invocation Pattern + Closure' (original) ***");
obj.fooWrapper("*** Proving the 'Constructor Invocation Pattern + Closure' (wrapper) ***");

that prints out:

*** Proving the 'Constructor Invocation Pattern + Closure' (original) *** {
I'm a String in a global (attached to 'window')
I'm a String in an object (i.e. scope/'this'), accessible via Closure
I'm a String in a global (attached to 'window')
}


*** Proving the 'Constructor Invocation Pattern + Closure' (wrapper) *** {
I'm a String in a global (attached to 'window')
I'm a String in a global (attached to 'window')
I'm a String in a global (attached to 'window')
Here 'this' resolves to 'window'
}

I'm a String in an object (i.e. scope/'this'), accessible via Closure (from the wrapper)

In the first invocation (i.e. obj.fooOriginal(...)) the callee acts as expected: same printout as per the Prototype Inheritance. But when invoked by obj.fooWrapper(...) obj's scope is not made parent of the callee scope, as it is invoked as-is (i.e. foo(m);).

A Closure though was indeed created: console.log(aVariable + " (from the wrapper)"); proves just that.

Apply Invocation Pattern

1
2
3

foo.apply({
    aVariable : "I'm a String set via the 'apply' method: scope/'this' is passed to 'apply()'"
}, ["*** Proving the 'Apply Invocation Pattern' ***"]);

obviously produces:

*** Proving the 'Apply Invocation Pattern' *** {
I'm a String in a global (attached to 'window')
I'm a String set via the 'apply' method: scope/'this' is passed to 'apply()'
I'm a String in a global (attached to 'window')
}

this is the easiest one, as the scope manipulation is made very explicit, and controlled by the developer. Useful when you want to tightly control what your callee receives.

`var that = this` Invocation Pattern

If you wrote a lot of JavaScript, you have written one of this. Somewhere in time, you had a piece of code that drove you nuts for a little while, because the scope was just like Frank Abagnale. Hence, to trap it, you created a fixed reference to it, an "alias" some would say.

// 'that' = 'this' Invocation Pattern
ClosuredScope = function(s) {
    this.aVariable = s;
    return {
        aVariable : this.aVariable,
        that : this,
        fooThat : foo
    };
};
obj = new ClosuredScope("I'm a String set in another Closure");
obj.fooThat("*** Proving the 'That = This Invocation Pattern' ***");

and this one comes out with:

*** Proving the 'That = This Invocation Pattern' *** {
I'm a String in a global (attached to 'window')
I'm a String set in another Closure
I'm a String in a global (attached to 'window')
I'm a String set in another Closure (passed via 'that')
}

what's the reason? it's the only way to "catch and hold" this, the scope. that is than a reference to this, to ensure the callee can still get hold of it. And yes, this is a Closure.

What do you say? Too messy? Well, the that = this-for-dummies is:

// ...
var that = this;
 setTimeout(function () {
     that.doStuff();
 }, 1000);
// ...

but the same applies.

Conclusions

Well, after all this the only real conclusion I can give you is: be careful when writing JavaScript, and make sure you really understand what's going on.

Tools like PhantomJS, Firebug, WebKit Inspector or Opera Dragonfly (choose your favourite), are your best friends to workout what's going on. When your code grows, it's not always YOUR code: breakpointing and analysing in details what's going on it's often the only way to solve this kind of issues.

And you, tweeps: please share your wisdom. I'm sure you guys will have loads to say about this ;-) .

Happy scoping!

UPDATE: here is the full gist invocation_patterns.js.

USA: 2 weeks in, another 1 to go

2012-01-31T23:29:54Z

In 2006 I decide that I wanted to leave my home country. I wanted to try hard and make some space for me in the world (well, the tech-world). I was really tired of being paid fuck all, dealing with a country driven by old-and-hard-to-die politicians, not seeing any foreseeable future in front of me. I made a decision, and I started working towards my leaving (I finishing uni, so this required a bit of planning).

I left. And a lot has changed. Friends have changed (for the best, I believe); Love has changed; Family has changed (and still changing).

Before that decision, I looked at this tech-internet-world from far. From the far lands of the south of Italy. Yes, I had Internet... but everything was far from reach. I was scared. I was really scared. How could I, a Neapolitan-breed kid, make it into a world that a) knew nothing about where I come from b) what it knew, was mostly a stereotyped reality c) was moving to a speed for me unimaginable.

What did I learn? That I was wrong. The world didn't assume anything on me. The world was NOT like us: we, my breed, we assume that "if it's from USA, it must be better" or "wow, this comes from LONDON!" or "French people are snob" (well, some actually are!).

The world was just quietly going on without me. And would have continued going on without me: I'm just a freaking drop in a ocean of people. I do have something to contribute? Well, I'm welcome to do so. As everyone else on this galaxy (an maybe beyond) is.

I have met great people, and it took me time to learn their names (who knows me well, understands that is not because I don't care - it's because I have a shitty memory). I met terrible people, and their name stick to my mind like super-glue: funny how certain things work.

The C++ interview at Boundary Row. My terrible English. The cute Scottish engineer. Canning Town Station. The twin room with John. Boundary Row. Bootcamp Donatello. My team. The first year with Majid. Breaking up with Ale. The flat with the colleagues/friends. Mr Vladimir Stoyanov. Going back to Ale. Waterloo. Serage and John. Filling the washing machine with dish soap and the foam covering the floor. Spilling coke over my MacBook Pro, yelling "O'Madonna! O'Madonna!", cleaning it and reselling it on eBay like it never happened. The heavy snow. Me falling down like an idiot while trying to walk on the freaking snow. Orange Labs. Luca. Mobeen, Rafel and the other guys. Losing Vincenzo, Alessandra and Manuela. Betfair. Having gay colleagues (such an honor and a satisfaction to say it back home! - I can't resist to write this down: for me IT IS a BIG deal working with free men). Paul. Matt. Rediscovered the pleasure of Cycling. Losing almost no weight. Still loving it. Maj. Gui. PhantomJS. Finally finding an Open Source project to which I feel like contributing something. The Stanford AI class. Ariya. Neustar. Here.

Well, more or less it's all there. Almost 5 years (will be five this April), and they have passed so fast. A friend of mine just tweeted:

... where did time go?

USA, a totally new place. A place where, apparently, everyone wants to be. A place where everything else is not THIS good. The place where, if you are strong enough, you can actually make it, and make it BIG. Is here the place for me? Could I live in such non-European culture? Don't know, maybe not.

What I can though, is work with great engineer, develop new stuff, have fun, challenge difficult problems, solve some of them, fail at others. Hopefully, keeping low the number of failures: I'm here to succeed.

Yes, I know, I'll be working mostly from UK, but this doesn't matter anymore: I learned that truly I can make something of myself. I just need to have fun! That's all it is: what I do for job is fun. Learning or applying: it doesn't matter. Learning is when I start modelling ideas in my head. Applying is when I prove that those idea were good (or shit, as Jobs would say).

I'm in! And I like it.

PS Sorry, I know the title suggested I'd have talked more about my experience so far in USA. I just pushed the random button this time and blew off some dust off those boxes at the back of my head. Have fun with it.

ai-class.com - Wrapping up

2012-01-04T09:09:17Z

I have been thinking about what to write in this post since the day I took the final exam of the Stanford Online AI Class. I have so many things to say (and still to do), that I have a mess of mixed thoughts in my head.

So, I apologise in advance if thoughts will sound a bit random.

Amazing experience

That's what this class has been for me. I have learnt tons of new things, all thanks to the great work of prof. Thrun and Norvig. I digged into the amazing topic of Artificial Intelligence, and now I feel a bit more "round" as a Computer Scientist.

Cut of my Statement of Accomplishment

The final score? 98.7%. Not bad at all!

Massive class

Probably the most amazing aspect of what those prof. have achieved, is to work with ~160-thousands students, from all around the world. Yes, we had indeed issues with servers going down on deadlines, and we didn't manage to do much programming (checking the code of such a massive class requires a re-thinking in homework-checking-approaches), but we still got lessons being delivered on time, a perfectly working and useful forum, a community of students discussing the lessons and so forth.

Being this the first of a growing number of Teaching-Experiments, I truly feel like having taken part to a tiny piece of History.

A Privilege and a Realisation

I was born in the south of Italy, from a family that was wealthy enough to raise 5 kids (I'm the first), but not enough to send us study farther than few kilometres from home. So I always looked at universities like Stanford (actually, Stanford in particular!) as my dreamland of study.

Being able to take part, even for just 1 class and only online, to a class thought by figures like Norvig and Thrun was a real privilege. I wish one day to drive to Stanford, find their office and just shake their hands (should I bring them a Pastiera? what do you recon?).

But, on the other hand, being able to follow, and perform also quite well makes me see that, after all, my modest University of Studies of Naples "Federico II" was not bad at all: with a smaller budget and less pompous name, it has been able to make of me a Computer Scientist that can compare himself to Stanford Graduates. A big thank you to the world oldest state university.

Sharing

During this class I had discovered that I haven't changed much: to learn, I still need to take lots of notes. This is good, but also consumes tons of paper. And when I started I was not willing to waste paper in 2011 for class notes: I wanted to change approach.

I so decided to start writing my notes in an Evernote Notebook. What I didn't realise, was that I had unconsciously made a commitment!

My notebook "Online Stanford AI Class" has got very quickly popular, and people started to politely (and sometimes less politely) demand for notes about the classes, while the course was underway. Some were just thanking with comments on the blog, others posted corrections or highlighted typos. I quickly found a massive traffic coming through and I was not really expecting that.

I thank everyone of the users of those notes for the kind words, and I'm humbled that my private effort (I wrote them as my own reference after all), written in very poor ital-english, has been helpful to some.

YES, I KNOW: I still haven't published the notes for the last 2 classes about Natural Language Processing. When the Final Exam came out, and I realised no questions were related to that topic, I decide to focus on the Exam. Afterwards, I left for holiday to Italy. Hopefully I'll get around to write them soon. I really want to go through again those last lessons, as Norvig gave a very good basis to do work on concrete NLP problems.

A request to the professors

In the spirit of sharing and global-ness, I think is really important that all the data about the class participation and performance is made publicly available. After proper anonymization (of course!), we should get the details about students location, background and class performance. I'm sure tons of interesting data can be extracted and fact understood.

This was, indeed, one of the biggest experiments involving massive knowledge sharing: I'm sure there is lot to learn from those data about teaching in the modern world.

Just in case you missed, above is the video of a Google+ Hangout that has happened during the course, involving both the profs and Sal Khan) of the Khan Academy: the topic discussed there fit exactly with what I'm mentioning here. I leave Khan to put it in way better words.

Outcome

What I got as most important outcome from all this, is that now I have answers to some of the "curiosity" I got left from my University times, when I had to rush through the Bachelor Degree, and couldn't stay to do the Master (I needed the start working and make money). Yes, there are still advanced topics in the field of Machine Learning and Optimisation that I'd love to go through, but this is a start!

Why I don't study now by myself you say? Because it's not the same! I don't know you, but having someone teaching you a topic gradually, and setting up schedules and deadlines for you, forces you into a study mode that it's just very hard to achieve on your own. Don't you agree?

Job opportunity letter from Thrun

Yes, in addition to the learning, I also got an email from prof. Thrun about the fact that he would like to pass around our CV (i.e. the first 1000 of the class), as they are looking for talents like ours.

That is really great, even thought I just got a new job and I'm not really looking for a new one. Networking is always useful, even more if it's purely based on what I tangibly demonstrated.

Proud

Well, what else to say? I'm proud. That's what I can ultimately take out of this experience. I learnt tons and from the best, I performed well and I helped others do the same. Once in a while, it's good to feel proud of yourself.

What's next?

Well, first of all I need to resume my work on PhantomJS: I have been (almost totally) away from the project in the last 2 months, and I still have stuff I want to do, and new things I want to investigate.

Also, I'm in the middle of a revolution, that is going to make me travel every couple of months across the Atlantic to work in Neustar Webmetrics. It's going to be quite a change and a life-changing experience.

Of course, I do feel like this course opened for me the proverbial Pandora's Box. Professors at Stanford are putting online more, very interesting courses, and I'm thinking about some of them: "The Lean Launchpad" it's probably a great one to follow at my age and career stage; also "Natural Language Processing" attracts me a lot, because we did just a quick dip from prof. Norvig and I want to know more; "Human Computer Interaction" sounds like a great way to compensate for the bad class I got at uni years ago!

So, in other words, I'm not going to roll my thumbs. 2012 is going to be a very busy year, full of new things!

See you around and... happy coding!

Maven, PhantomJS and Jasmine to write your JS Unit Testing

2011-12-22T18:34:40Z

In Betfair we are building all sort of new things, and one of the aspects on which we are trying to focus more is doing Automated Tests for JavaScript. Currently, all non-js code in Betfair passes through different kind of Automated (and Human) Testing (from Unit Testing, to Security, to Performance, to Human and Scripted QA), but the JavaScript language, by its very nature, makes it tricky to test in an automated fashion. It's not impossible, just a bit harder than it should.

A beautiful jasmine...

Lately I have been spending a big chunk of my spare time lending a hand to the PhantomJS project, as some of you that know me might recall.

Don't break the flow

That started because I saw in PhantomJS, and it's headless nature, a great opportunity to contribute to the internal effort around JS Unit Testing. And because I wanted to allow both developers and CI-Servers to run JavaScript Tests with ease.

I believe that if development experience it's easy (and, I dare to say, FUN), code just flows! And if you can do testing with just one command from your CLI, the flow becomes almost beat.

In Betfair we use Maven to drive our build system (yeah, I know, XML orgies are terrible...). So, we type a lot of:

1	$ mvn clean install

even in departments where we mostly do Front End code. Ideally, we wanted to add, as a step to the build, a way to hook in the JavaScript Testing, and having the test result driving the build outcome.

IF JavaScript Test fails THEN Build Fails! Dead simple.

Jasmine, but not necessarily

There are multiple libraries to write Unit Tests for JavaScript out there, and we decided to go for Jasmine, the evolution of JSUnit (now discontinued).

You can go for anything you want really, but in this post/example (and here in Betfair) we picked Jasmine. Feel free to choose your own library: as long as you are running in a browser, anything is good.

Setup

To run the show, I came up with this setup: PhantomJS runs a script that runs a "jasmine-enabled" webpage. The script is then able to extract the result out of the webpage and save it on files. That's how it looks like:

Yes, it's a long chain of delegation. But it works. And it's also "modular": in case you want to run the tests in another Browser, you can just open the ".html" runner! Or, in case you don't want to use Maven, you can just invoke phantom and pass it the "jasmine xml runner script" (plus arguments).

Jasmine has a very great design feature: Reporters. You can implement this kind of plugins to attach to Jasmine, separating the test execution from the test reporting. Based on the reporter you write, you can get any sort of output.

In our case, we need the tests to produce output in JUnit XML format, and so I wrote one. But, if you know your shit, you are probably thinking:

«A web browser is sandboxed: how do your write a file with XML from there?»

Well, read on.

Jasmine `test_runner.html`

As mentioned, Jasmine requires a webpage prepared to run the tests. In our case, something like the following is good:

<!DOCTYPE HTML>
<html>
<head>
  <title>Jasmine Test Runner</title>
  <link rel="stylesheet" type="text/css" href="lib/jasmine-1.0.2/jasmine.css">
  <script type="text/javascript" src="lib/jasmine-1.0.2/jasmine.js"></script>
  <script type="text/javascript" src="lib/jasmine-1.0.2/jasmine-html.js"></script>
  <script type="text/javascript" src="lib/jasmine-reporters/jasmine.phantomjs-reporter.js"></script>

  <!-- include source files here... -->
  <script type="text/javascript" src="../src/tv.js"></script>

  <!-- include spec files here... -->
  <script type="text/javascript" src="test_spec.js"></script>

</head>
<body>

<script type="text/javascript">
  jasmine.getEnv().addReporter(new jasmine.TrivialReporter());
  jasmine.getEnv().addReporter(new jasmine.PhantomJSReporter());
  jasmine.getEnv().execute();
</script>

</body>
</html>

How PhantomJS does the trick

PhantomJS, as you might know, allows to instantiate and manipulate webpages. And webpages are "sandboxed": so, it's not possible to write files on the filesystem from there. So, what I did 2 things:

Implemented a set of API for File I/O and submitted a pull request (see fs object in PhantomJS)
Implemented a Jasmine Reporter that saves the output of the tests in isolated variables in the test runner page context (see test/lib/jasmine-reporters/jasmine.phantomjs-reporter.js)
Wrote a PhantomJS script that is able to extract the test output from the webpage context and save it on filesystem (see test/phantomjs_jasminexml_runner.js)

In this way it's possible to preserve the sanboxing, while saving the desired XML format. And, along the way, give to PhantomJS the ability to manipulate files.

The script looks like this:

// ...
    var htmlrunner = phantom.args[0],
        resultdir = phantom.args[1],
        page = new WebPage();
    
// ...

    page.open(htmlrunner, function(status) {
        if (status === "success") {
            utils.core.waitfor(function() { // wait for this to be true
                return page.evaluate(function() {
                    return typeof(jasmine.phantomjsXMLReporterPassed) !== "undefined";
                });
            }, function() { // once done...
                // Retrieve the result of the tests
                var suitesResults = page.evaluate(function(){
                    return jasmine.phantomjsXMLReporterResults;
                });
                
                // Save the result of the tests in files
                var f = null;
                for ( var i = 0, len = suitesResults.length; i < len; ++i ) {
                    try {
                        f = fs.open(resultdir + '/' + suitesResults[i]["xmlfilename"], "w");
                        f.write(suitesResults[i]["xmlbody"]);
                        f.close();
                    } catch (e) {
                        console.log(e);
                        console.log("phantomjs> Unable to save result of Suite '"+ suitesResults[i]["xmlfilename"] +"'");
                    }
                }
                
                // Return the correct exit status. '0' only if all the tests passed
                phantom.exit(page.evaluate(function(){
                    return jasmine.phantomjsXMLReporterPassed ? 0 : 1; //< exit(0) is success, exit(1) is failure
                }));
            }, function() { // or, once it timesout...
                phantom.exit(1);
            });
        } else {
            console.log("phantomjs> Could not load '" + htmlrunner + "'.");
            phantom.exit(1);
        }
    });

Ah, the Filesystem API (i.e. the fs.open() above) are still stabilising as we are not sure yet how much of the CommonJS/Filesystem specs we want to implement in PhantomJS. If you care about PhantomJS, contribute to the discussion here.

Oh, yes, Maven

Some of you might have reached this page for Maven. Let me say, I'm sorry you have to deal with that thing, but... let's just focus.

Maven has a plugin for running external executable: exec-maven-plugin. It allows to launch any executable and, in case this exit failing (i.e. exit(1); or exit(EXIT_FAILURE);), fail the build. Super!

So, how it looks like?

<?xml version="1.0"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
    http://maven.apache.org/maven-v4_0_0.xsd">
    <!-- ... -->
    <build>
        <plugins>
            <plugin>
                <groupId>org.codehaus.mojo</groupId>
                <artifactId>exec-maven-plugin</artifactId>
                <version>1.1</version>
                <executions>
                    <execution>
                        <id>PhantomJS Unit Testing</id>
                        <phase>test</phase>
                        <goals>
                            <goal>exec</goal>
                        </goals>
                    </execution>
                </executions>
                <configuration>
                    <executable>phantomjs</executable>
                    <workingDirectory>test</workingDirectory>
                    <arguments>
                        <argument>phantomjs_jasminexml_runner.js</argument>
                        <argument>test_runner.html</argument>
                        <argument>${project.build.directory}/surefire-reports</argument>
                    </arguments>
                </configuration>
            </plugin>
        </plugins>
    </build>
</project>

This adds a step in the test phase of the build, running our tests.

It's on GitHub

As every piece of code that a) I think it's of any value, and b) that I can share, it's on GitHub: phantom-jasminexml-example. As the name says, this is just an example. An inspiration. Please customise it as much as you want to fit your purposes. And in case you can, contribute a patch to make it better.

Conclusion

This just started. My experience in JavaScript Unit Testing is not very good, and I'm learning in the process. So, chances are, this solution has a lot of problems, it's rubbish and needs rewrite.

If that's the case, please post a comment with your thoughts and, possibly, suggestions on how this can be improved. For example, it would be great if phantomjs_jasminexml_runner.js was able to generate the test_runner.html automagically, after having scanned a directory containing test specs.

PR time

By the way, if you eat code for breakfast, and you are proud of it, we are hiring: you might join us in trying to build this new piece of the puzzle for Betfair.

Crossing the pond

2011-12-14T01:30:47Z

I borrowed the title from a guy who kickstarted all of this. Thanks Ariya ;)

I'm leaving Betfair. I owe a lot to this company: for a bit longer than a year I have had the pleasure to work with smart, enlightened, knowledgeable people from which I learned loads. Leaving was definitely not an easy choice.

The only thing didn't happen, is that I didn't become a proper punter: I just learned that even something sounding "boring" as betting on sports, can actually be a very big challenge in terms of the technologies involved. Even more, if you try to do it in the Betfair way: with million of customers, all constantly hammering the servers to make the most out of their money.

Where am I going?

On January I'll be joining Neustar, working with the Webmetrics team, based in San Diego.

Google searches about San Diego return only this kind of picture ;)

Webmetrics works in the business of Monitoring and Testing websites. It's a big player in this market and I'm going to help on scaling their infrastructure to use the light-and-awesome PhantomJS.

Also, I'll try to add my bit to the already remarkable amount of work they share with the Open Source community: people like Patrick Lightbody, creator of Selenium RC, works there!

I'll be reporting to Simon Nicoud: folks that navigate the Web Velocity waters must remember him for his talks at Velocity Conf.

I'm not exactly moving to the USA

Love is a very important thing in Life, especially if you have found it. In the name of it, I decided to try to marry love-need with work-aspiration: I'll still be based in UK, but travel on a regular basis to USA. This will require me to become a very frequent flyer, and learn all the tricks of jet lag management.

I'm very excited about this: working in the USA, with USA folks is something I always wanted to experience. And now I finally can. Plus, I'll be working on technologies I'm really into: what better can I ask for?

Plus, San Diego seems like a REALLY nice place to be. So, I'm not going to miss the english weather. Right? ;)

See you downtown!

ai-class.com - My shared Notebook

2011-11-23T08:37:59Z

I decide that is very little point in keep writing posts just to highlight new notes, as I finish to write them. So, I decide to just share with the world all my notes about the ai-class.

You will find them in (sort of) chronological order, and will give also an idea on how I studied so far (as every material I produced is in there).

About the homework, I'll make sure NOT TO PUBLISH THEM BEFORE the respective deadline: I don't want to break the Stanford Honor Code.

By the way, have you done the Midterm Exam? How did it go for you? I got 100% (yeah!). Hopefully I'll manage to maintain this trend ;)

Good luck and happy AI Class!

ai-class.com - Notes of Lesson 7, 8, 9 and 10

2011-11-13T10:14:35Z

I have been very busy at home and in the weekend, so I'm a bit behind with study and sharing my notes.

I just finished going through the very interesting lesson 9, and I now have to start with lesson 10. I plan to do it on Saturday: tomorrow is a special night! ;-)

Instead, regarding lesson 8, I just managed to put down the list of topics discussed, but for lack of time I didn't put anything in writing just yet. Will eventually do it (maybe).

UPDATE 12 Nov 2011: I just finished putting down notes of Lesson 10. Very information dense, but I can see that Prof. Norvig went "easy" on us: Reinforcement Learning could have taken way more time.

Also, those two lessons are all about this guy below: better you get down and really study MDP. This stuff will hit us back in during the exams.

Guess who is this guy?

Here is the links:

Lesson 7
Lesson 8 (for now, topic list only)
Lesson 9
Lesson 10

UPDATE 2: I'd strongly suggest you go through Mostafa Abedi's blog. His notes and examples can really help to "iron out" doubts.

ai-class.com - Notes of Lesson 5 and 6

2011-11-05T21:18:36Z

If the "mood" of the previous lessons was about Inference, I can only say that the current are about "smooth ing", "occam's razor" and "perce ptron". 3 concepts that will remain with you even after having forgotten about all this (check it out below :-P).

Enjoy the notes and, again, do Lesson 5, than complete the Homework, than study Lesson 6.

UPDATE Sat 05 Nov 2011: I just finished putting down the notes of Lesson 6, and you can find the link above.

I must say that Lesson 6, if we exclude Expectation Maximisation, is way less "mathematical" than the other lessons. Prof. Thrun, at a certain point, seems like givings us just an "overview" of some of the Unsupervised Learning concepts. It really makes you mouth-watering!