I’ve seen a day I never thought would come:  the day I turned into a hipster!

If you’ve been following this blog, you’ll have noticed that throughout 2011 I’ve been working more and more in Javascript, more specifically with Node.JS.

The last few months have been especially Node.JS-centric. During the day, I’ve been building architectural improvements into Jabbakam. Evenings and weekends, I’ve been  hacking about on the Live Unsigned mobile site. All of my recent projects revolve around Node.JS, and I’ve been loving it. Node is incredibly interesting and powerful, and it is giving my brain a workout.

However, Javascript itself has a few niggly bits that become more problematic the larger the project gets. Issues with structure and maintainability of code motivate projects such as Joose and Backbone.js. One of the things that has been bugging me is having to hack in class-based inheritance. After a lot of experimentation, I settled on Prototypal Inheritance, which does the job to an extent, but left me dissatisfied .

So, the other day I finally broke down and tried CoffeeScript.

When CoffeeScript first came out I resolved never to bother with it. It is clearly for Ruby people who don’t quite understand Javascript. Since then it’s been getting a lot of press, and it now even has its own book!

Anyway, I was up the other night, waiting for my daughter to wake up in screaming pain as razor-sharp pieces of enamel slowly bored through her gums. With nothing better to do, I decided to play around with CoffeeScript.

I was up and running in no time, installing the Node.JS Module and browsing the Little Book of CoffeeScript. After about an hour I had effortlessly written some additions to Live Unsigned which integrated seamlessly. To be honest, I feel like a bit of a muppet for not trying it sooner.

After just a couple of days playing around with it, CoffeeScript has now become a major part of my toolkit. The syntax is elegant, nicely lending itself to clean coding, and code can be reused easily, improving project structure. CoffeeScript compiles to pretty clean Javascript, and from what I can make out, the better you understand Javascript, the better CoffeeScript you can write.

So yes, I am now coding in CoffeeScript.

I am now a hipster.

There is nothing left for me but to rush out, buy some skinny jeans and a Macbook Air and learn Vim.

Oh, God.

Now, I’m not American or any of that nonsense, but it seems the 2012 US elections are already heating up, especially with all this talk of SOPA, PIPA etc.

After all the hope (or hype?) that was Obama there are some new contenders for the presidency. Out of all of these, Ron Paul seems to be the only one making any sort of sense (caveat, I may very well be wrong about this, but he actually talks about the bad things the US has done!). Mitt Romney sees to have that strange chiselled jaw only Americans aspire. And Senator Rick Santorum? Well, let’s just say in my own tiny way, I’m doing my bit.

In this post, we’re going to make that example slightly more complex by introducing dependencies within the modules. Now the code must be able to handle these dependencies for both the client side and the server side.

In the last post, we built a simple module consisting of a single function that returned a greeting:

var Greetings = function() { }
Greetings.prototype.hello = function(who) {
   return "Hello "+who;
}
module.exports = Greetings;

At this point, it would be very useful to sanitize and validate the parameter passed to the “hello” function. We could obviously build this functionality directly into the code, but this is exactly the sort of thing that should be abstracted into its own module(s).

For the purpose of this example, we’ll simply include the functionality to give the parameter passed a “trim”; after some hacking and snippet-borrowing from node-validator by Chris O’Hara, we can build another simple module called Tidy:

var Tidy = {
   whitespace: '\\r\\n\\t\\s',
 
   trim: function(str) {
      var whitespace =  '\\r\\n\\t\\s';
      str = str.replace(new RegExp('^['+this.whitespace+']+|['+this.whitespace+']+$', 'g'), '')
      return str;
   }
}
module.exports = Tidy;

We can now improve our Greetings module, including the dependency and functionality:

var Tidy = require('./tidy');
 
var Greetings = function() { }
Greetings.prototype.hello = function(who) {
   who = Tidy.trim(who);
   return this.hello(who);
}
module.exports = Greetings;

To see if this will work as expected, let’s add some spaces to the parameter passed in the NodeJS script:

var Greetings = require('./greetings');
var greetings = new Greetings();
console.log(greetings.hello('  Node   ')+"!");

When we run it, this is what we see:

$ node runner.js
Hello Node!

Good, so this works on the server side, but let’s test its behaviour on the client side. Again, we should add some spaces to the parameter passed to ensure that it is working:

<script type="text/javascript"> </script>
<script src="greetings.js" type="text/javascript"> </script>
<script type="text/javascript"> </script>

Unfortunately, we now see the following error:

require is not defined
[Break On This Error] var greetings = new Greetings();

Our attempt to “require” the dependency in the Greetings module has broken the client side. To fix this, we need to define our own require() function, much like we had to for the module object. We do this by creating a new front end-only script, which we’ll call “middle-end.js”.

var require = function(module) {
   document.write('<script src="'+module+'.js" type="text/javascript"> </script>');
}
 
var module = {
   exports: undefined
};

(Obviously we should include much more error checking, but this will be enough for our example).

We modify our html as follows:

<script src="middle-end.js" type="text/javascript"> </script>
<script type="text/javascript"> </script>
....

Now when we reload the page in the browser, all should work as expected. Huzzah!

This is just a very simple example of what can be done when JavaScript code is shared between the client and server sides—the “Middle End”. This concept becomes really exciting when you start thinking about potential approaches to form validation, session management, and even display logic. This is definitely something worth playing around with.

All of the code is available as a Gist.

In my previous post, I talked about Kyle Simpson’s concept of a Middle End—a layer between the front end and back end. Today, I’m going to sketch out my very simple experiments with constructing such a layer.

What I am focusing on here is simply a means of sharing JavaScript code between the client side and the server side.

Let’s start with a very simple NodeJS Module, greetings.js:

var Greetings = function() { }
Greetings.prototype.hello = function(who) {
   return "Hello "+who;
}
module.exports = Greetings;

Next, let’s create a very simple NodeJS script that imports this module, and uses it:

var Greetings = require('./greetings');
var greetings = new Greetings();
console.log(greetings.hello('Node')+"!");

When we run it, this is what we get:

$ node test.js
Hello Node!

All very simple stuff.

Now, suppose we want to use the Greetings module in our front end web app. 

<script src="greetings.js" type="text/javascript"> </script>

Well, it runs and we get the desired output. However, Firebug reports an error within the greetings.js file:

module is not defined
[Break On This Error] module.exports = Greetings;

The solution to this issue is to “mock up” the module object so the error isn’t thrown:

var module = {
   exports: undefined
};

Run it again, and now we see no errors.

Great, so it’s working. We can share simple modules between the client and the server sides.

But what if we want to have a more complex module? One that has some dependencies?

That will be discussed in the next post.

All of the code is available as a Gist.

Again, this is one of those posts more for me, but hopefully other people might find it useful.

Recently I updated my Arch Linux box

# pacman -Syu

and during this update a new kernel was added – which meant that my VirtualBox installation didn’t work any more, chucking out the following message

The VirtualBox Linux kernel driver (vboxdrv) is either not loaded or there is a permission problem with /dev/vboxdrv. Re-setup the kernel module by executing

‘/etc/init.d/vboxdrv setup’

One of the talks was discussing the concept of the Middle End, by Kyle Simpson. This really made an impression on me, and I think I probably bored Kyle in the bar afterwards with my drunken enthusing. When we arrived back home, my aim was to start playing around with the idea of a Middle End as soon as possible. Unfortunately, as is life, events took over (such as discovering one is going to become a dad!) and these thoughts were consigned to the back of my mind.

Until now

The more I hack around with NodeJS the more I’ve been able to experiment with different aspects of it. Well, today I began hacking around with some Middle End concepts.

The idea of the Middle End is quite simple.

In a web application there is traditionally the Front End and the Back End. The front end handles all of the client side processing (such as rendering some whizzy JQuery plugins or whatever). The back end handles the actual hard crunching, such as retrieving and adding data from the storage engine.

However, there are certain aspects that both the front end and the back end need to handle. Some things that spring to mind are: session managment, input validation and possibly some display logic (Kyle’s post goes into some more depth here).

Currently these aspects are handled both by the front end and back end code, but as traditionally front end code is JavaScript where as back end code maybe Ruby, PHP, Python, etc that means there ends up being duplication of code. Two different languages doing the same thing.

This is in direct violation of the DRY principal. Quite simply, this is a waste.

There is a solution to this waste. Using JavaScript on the server side, effectively code can now be reused. But how can you use the same piece of code for the client side as well as the server side? In my next post I’ll be looking at how to do that.

I’ve been researching automated deployment of NodeJS applications, and found this excellent blog post to help me out:

Deploying Node.JS Apps, by Ben Gourley.

Plus this question on stack overflow on nginx proxying should help me out.

That’s it. now I just need some time to play around with this stuff!

Also, if anyone knows of a really simple, easy to organize bookmarking site please let me know!

Three things really stood out for me today.

Firstly, Cloud9 IDE. This is a browser based JavaScript IDE, and it looks fantastic. With no setup or configuration needed it lets you just start writing NodeJS directly from your browser. The lovely guys even gave me a premium account free for 1 year! I am going to sign up and start hacking away as soon as I get a chance.

Secondly, even though the Arduino talk was a bit, er, ramshackled, they did talk about the Raspberry Pi. This is a tiny (and I mean tiny) ARM box running Linux. It should be available in November some time, which will be perfect timing for a birthday present to myself!

Finally, Emscripten is very exciting. Emscripten is a compiler that takes bitcode and turns it into JavaScript. From what I understood, this is what made repl.it possible, which I saw yesterday in Hacker News and thought was so cool I bookmarked it!

As there were two tracks I did miss a bunch of talks, but I’m sure they will be availble on the website soon.

All in all, good stuff, tiring (especially as some little madam decided to wake up at 5am) but brain is now firing off with a bunch of possibilities!

I have high hopes for the conference. Last year I didn’t even attend the main event, just the pre-party on the Friday night and still came away with a load of ideas! One conversation lead me to completely rewrite major parts of the Jabbakam system. Who knows what new thoughts will spark in my head when I actually go to the real thing?

For updates, follow me on Twitter.

What I have been working on is having a single parent process that runs constantly in the background, and this parent is responsible for child processes which actually do the work. One thing I found particularly interesting was how the parent can handle a child crashing, and also what happens to the children when the parent is killed. Below is the culmination of my hackings, it comprises of two simple scripts, a child script, and a parent. I have committed them to my GitHub account, so feel free to clone, fork, hack or whatever.

Note: Node version at time of writing was v0.4.9 and running off a Debian machine.

Child Worker

For this experiment we need a really simple child script. It doesn’t have to do anything apart from tell us it is alive when it is running, and then kill itself so we can see how the parent handles this event.

The main bulk of this script is the loop:

var child_loop = function() {
   // report the current loop
   console.log("Child loop: "+NUMBER_OF_LOOPS);
 
   // to simulate the child exiting itself without just running
   // out of program, if we have completed the desired number of
   // loops, exit this process
   if( NUMBER_OF_LOOPS < 0 ) {
      console.log("Child finished loop");
      process.exit();
   }
 
   // sleep for a bit, then continue through the loops
   setTimeout(function() { child_loop() }, LOOP_SLEEP_TIME);
   NUMBER_OF_LOOPS--;
}
 
// start the loop
child_loop();

As you can see this loops for NUMBER_OF_LOOPS, outputting what loop it is currently on. At the end of the loop it kills itself, informing us whilst it does it.

So, this is fine whilst the child is running, but we also need to try and get the child to clean up after itself if it is killed by an external process – i.e. the parent. After some playing around and reading of the documentation, I ended up with this:

// detect and report if this child exited
process.on("exit", function() {
   console.log("Child exiting");
});
// detect and report if this child was killed
process.on("SIGTERM", function() {
   console.log("Child SIGTERM detected");
   process.exit();
});

I need to go more in depth, but whilst testing, this seemed to correctly handle the child being terminated externally.

So, that’s the child script. You can run this directly from the command line and it does what is expected. Good, so we’re all ready to build the parent so it can spawn multiple children.

Parent

This was the main focus of the experiment. The chief concerns were:

• Ensuring the process ran indefinitely as a daemon
• Spawning the children
• Handling a child that crashed or exited
• Cleaning up the children when the parent was exited

The first point is relatively simple, all we need is an indefinite loop:

loop = setInterval(function() {
   // functionality here
}, LOOP_SLEEP_TIME);

Handling the children was a little more complex. In the end I created an array that would contain the children. Within the main loop, I then ran through that array, checking to see if the child existed, if not then spawning it. Notice how the loop is returning itself into the variable “loop”. This will be important later.

var spawn = require('child_process').spawn;
var loop = setInterval(function() {
   for(i=0; i

This is fine for creating the children, but then what if a child dies? Simple really. The parent needs to listen for the exit event on the child. When detected, ensure that the array element for that child is cleaned out, and then in the next loop, the child will be respawned.

children[i].on('exit', function (code) {
   children[i] = undefined;
});

Next, what if the parent is killed. We need to make sure we don’t leave any orphaned children behind. After some playing around, I found that two events needed to be listened for. Again, the exit event, this time on parent, but also the SIGTERM event on the parent.

When a SIGTERM event is detected, we simply tell the script to exit cleanly:

process.on("SIGTERM", function() {
   console.log("Parent SIGTERM detected");
   // exit cleanly
   process.exit();
});

Now, when the exit event is triggered, we can clean up the children. One thing we must also do, is stop the Interval of the main loop. If we didn’t do this, when the children were killed, the next “tick” of the loop, then they would all be created again. It’s possible that this could happen before the main script has exited, orphaning the children. We can eaily do this as we have the variable that the setInterval call has been saved into.

process.on("exit", function() {
   // ensure no more "ticks"
   clearInterval(loop);
 
   // kill all children!
   for(i=0; i<TOTAL_CHILDREN; i++) {
      if(children[i] != undefined) {
         console.log("Kill child "+i);
         children[i].kill();
      }
   }
});

That is it for the main bulk of the script. As I’m testing and running it on Debian I am using start-stop-daemon to run the parent as a daemon. As such I’ve had to add the functionality for creating a .pid file. One issue I had with this, and I have no idea why, is when I tried to write the process.pid value to a file it filled the file up with nonsense. However, if I converted it to a string (which from what I can tell it already was) before writing it to the file, then it worked fine.

var pid = ""+process.pid; // need to turn into a string
fs.writeFile('/tmp/parent.pid', pid, function (err) {
   if (err) throw err;
});

You can see the complete parent and child scripts on my Github repository.

Running

Now we have our scripts, we need to run them to see if they actually behave as expected. In Debian (and derivatives) this is pretty simple:

To start

start-stop-daemon --start --exec /home/rob/workspace/NodeJS/process/parent.js --pidfile /tmp/parent.js.pid

And to stop

start-stop-daemon --stop --pidfile /tmp/parent.js.pid

As we are not running the process in the background then you will need to have two terminal windows open. Also, you will be able to see the output which explains what is going on.

You should be able to easily convert the script to log to a file (I recommend Vision Media’s Log.js) and then write an init.d script using the start-stop-daemon functionality to create a background daemon process .

You should be able to see that we now have the ability to run a Node.JS daemon process which can spawn multiple children, ensuring it can keep a consistent number of children available, and then handle the cleanup if that daemon process is killed. This opens up lots of possibilities!

Again, feel free to clone, fork, hack, improve, etc the code for this.