REST is now the most common approach used for designing Web APIs. A well-designed REST API is based upon media types that define the message formats used in client-server exchanges. See REST APIs in the REST API Design Rulebook for more on this topic. A media type will document both the structure of a valid message and the hypermedia “controls” used to interact with resources.

Examples of controls are forms or typed links. The HTML media type defines several controls, including:

• The img and a elements use links for transclusion and navigation
• The form elements are used to construct requests to a server

Typed links are the most common form of control and provide the means for clients to traverse the functionality exposed by an API. When we are designing a new API, it’s useful to draw on existing patterns for designing links. It turns out that there is both a common conceptual framework that we can use, as well as an existing registry of link types.

A Conceptual Link Framework

Both HTML and Atom include a generic link element. For more on Atom, read Chapter 7.6: Atom Everywhere in Rest in Practice. In HTML, this is most commonly used to reference stylesheets:

	<link rel="stylesheet" type="text/css" media="screen" href="/stylesheet.css" />

Atom uses the atom:link element in a number of ways, for example, to publish the permalink to a blog post and the URI used to edit it:

   <atom:link href="http://example.org/2003/12/13/atom03" />
   <atom:link rel="edit" href="http://example.org/2003/12/13/atom03/edit"/>

The common aspects of these elements have been standardized as part of RFC 5988 (Request for Comments), which describes a conceptual framework for defining links in media types. This is exactly what we need for creating RESTful APIs.

RFC 5988 describes a number of key aspects of a link:

• The Context URI — the resource to which the link applies, such as a specific Atom feed entry
• The Target URI — the resource that is being referenced in the link, such as a stylesheet
• The Link Relation — the type of the link, such as “edit” or “stylesheet” in the above examples

Optionally, there may be some standard Target Attributes that describe an aspect of the destination resource, like its title or expected media type (text/css).

But the RFC doesn’t define how links should be expressed, just its properties. The HTML and Atom examples show one way to capture that information in an XML format, while a potential JSON encoding might be:

	{
      "link": {
         "href": "http://example.iana.org/resource1",
		 "type": "text/html"
      }
    }

Our media type documentation needs to define how a link will be formatted and, most importantly, what types of links a client might encounter. Client behavior is driven through discovering and following typed links.

Media types that use RFC 5988 style links are easily extended: all we need to do is define new types of link relations.

The IANA Link Registry

We may also be able to re-use existing link types. The IANA (Internet Assigned Numbers Authority) maintains a central registry for common link relations. There are a wide range of relations including:

• A bookmark providing a permalink
• A describes relation that asserts that one resource provides a description of another
• An edit relation refers to a resource that can be used to edit the content of the context URI
• Navigation relationships (previous, first, last, next) for moving within a list of resources

The registry is an excellent point of reference, either for inclusion in a custom media-type or as a source of new relations for an existing type. New types can be added to the registry, and it’s also possible to use a URI as an identifier for a link type, rather than a short name.

Instrumenting Existing APIs

Handily, RFC 5988 also defines a way to publish links via the Link HTTP header. This has a number of advantages:

• Links can be published without changing current formats or clients, making it possible to make existing APIs more RESTful
• Links can be used by clients without the need to parse and understand specific media types, allowing for some generic client interactions
• Where a resource exposes data in multiple formats, it provides a consistent location for exposing the links that are common to all representations

The “edit” link from the earlier Atom example could be included in an HTTP header as:

   Link: <http://example.org/2003/12/13/atom03/edit>; rel="edit"

Similarly, paging links could be exposed like this:

   Link: <http://example.iana.org/collection/1>; rel="first",
      <http://example.iana.org/collection/100>; rel="last",
      <http://example.iana.org/collection/3>; rel="next",
      <http://example.iana.org/collection/2>; rel="previous",

An API could also link to both the license associated with the data it returns and its terms of use:

   Link: <http://opendatacommons.org/licenses/odbl/>; rel="license",
      <http://example.iana.org/about/terms-of-service>; rel="terms-of-service",

Summing Up

Typed links are a core feature of any RESTful API. They provide the means for clients to traverse the functionality exposed by the API. By building on a common conceptual framework, and an existing registry of link relations, we can take advantage of existing experience and design patterns when designing new APIs.

Where we have limited freedom to add links to existing media types, we can still make those APIs more RESTful by exposing links at the HTTP protocol level.

For more details about REST, see the ebooks referenced below.

Safari Books Online has the content you need

	REST continues to gain momentum as the best method for building web services, leaving many web architects to consider whether and how to include this approach in their SOA and SOAP-dominated world. Rest in Practice offers a down-to-earth explanation of REST, with techniques and examples that show you how to design and implement integration solutions using the REST architectural style.
	The basic rules of REST APIs – “many nouns, few verbs, stick with HTTP” – seem easy, but that simplicity and power require discipline to work smoothly. REST API Design Rulebook provides the next steps for implementing complex REST projects on simple and extensible foundations.
	SOA with REST: Principles, Patterns & Constraints for Building Enterprise Solutions with REST is the first comprehensive tutorial and reference for designing and building RESTful services as part of service-oriented solutions and in conjunction with service-oriented architecture (SOA). This book demonstrates that REST is not only a suitable medium for building truly service-oriented solutions, but also that the service-oriented architectural model is a necessary foundation for REST technology architectures to realize their full business potential.

About the author

Leigh Dodds is a freelance consultant specializing in Linked Data, Open Data and API development. Passionate about the web, open standards, and open source he enjoys exploring how to apply new technologies to create interesting products. He’s also quite fond of gaming, Lego, Minecraft and a good beer. You can reach Leigh on Github: https://github.com/ldodds or on Twitter: @ldodds.

Instrumenting APIs with Links in REST Safari Books Online

In simple terms, hypermedia is how people consuming your services discover your system. One of the great things about hypermedia is that it’s dynamic – so the links you provide through hypermedia can change depending upon the state of the resource your system is providing at runtime (this contrasts with traditional SOA where links/endpoints are static). Hypermedia is often forgotten in RESTful systems design, but HATEOS (Hypermedia as the Engine of Application State) is a core component of REST.

Using hypermedia in the field can sometimes be elusive, or daunting. For more background details, read Chapter 6.4: The Role of Hypermedia in SOA with REST. Take, for instance, the user resource in the ubiquitous user admin system. Many RESTful systems support the following:

GET /users 		- Get all users
GET /users/123 	        - Get a specific user
POST /users 		- Create a new user
PUT /users/123 	        - Update a specific user
DELETE /users/123	- Delete a specific user

Your system may support all five of the above actions on the user resource, but access is probably limited. Normal users can probably view themselves, update themselves, and view other people’s information, but probably can’t delete users, list all users or update other users.

So your API, which supports five operations against users, actually only supports those operations in certain circumstances.

This is where hypermedia can make your API easier to understand, and can simplify how your services are consumed.

In a system that does not employ hypermedia, when you GET a specific user you may often serve up some data such as:

{
	“id”:”456”,
	“name”:“Fred Jones”,
	“role”:“Normal”
}

However, this does not let us understand immediately what Dave (an admin) can do in relation to the system.

A better representation of viewing a user (using hypermedia) could be:

{
	“id”:”456”,
	“name”:“Fred Jones”,
	“role”:“Normal”,
	“actions”: [
		{ “ability”: “List all users”, “method”:”GET”, “uri”:”/users”},
		{ “ability”: “Delete Fred”, “method”:”DELETE”, “uri”:”/users/456”},
		{ “ability”: “Edit Fred”, “method”:”PUT”, “uri”:”/users/456”},
		{ “ability”: “View my profile”, “method”:”PUT”, “uri”:”/users/789”},
	]
}

This shows us that Dave can list all users, delete that user, edit that user, and also view their profile.

For a normal user looking at Fred, a representation could instead be:

{
	“id”:”456”,
	“name”:“Fred Jones”,
	“role”:“Normal”,
	“actions”: [
		{ “ability”: “View my profile”, “method”:”PUT”, “uri”:”/users/789”},
	]
}

Note that as a normal user looking at Fred, there are no admin operations like listing all users, or deleting/updating other users. The only action they can do (in terms of the user administration sub system) is view their own profile.

So by using hypermedia, we are dynamically offering up different operations on our system, depending upon the type of user accessing the service.

We’re enabling discovery of our system, and allowing that dynamically driven information to be driven by our state – we are representing the state of our system and transferring that state through our API.

Where this gets useful is in the consumer end of our API.

Often, websites that use a remote service (whether RESTful or not) know far too much about the inner workings of the service they are using.

You often see code like this:

%h2 Dave's profile page
- if current_user.is_admin?
  =link_to '/users', 'List all users'
  =link_to '/users/789/delete', 'Delete Dave'
  =link_to '/users/789/edit', 'Edit Dave'

- if current_user != user_being_viewed
  =link_to '/users/123', 'View my own profile'

%p 
 Name: 
 = current_user.name
%p
 Role:
 = current_user.role

In this small example, the website knows about a few things it doesn’t need to:

• The uri to list the users
• Whether or not a user is an admin
• Whether or not a user can list other users
• Whether or not a user can update/delete the user

By using hypermedia though, we can push the responsibility of that knowledge back to the service, and away from the consuming website:

%h2 Dave's profile page
- current_user.actions.each do |action|
  link_to action.uri, action.name
%p 
 Name: 
 = current_user.name
%p
 Role:
 = current_user.role

Now obviously, you’re never going to be able to build completely dumb websites that consume RESTful services, and there’s always going to be some knowledge of the API that leaks out into the consumer application. Limiting those if statements, however, is a worthy goal, which pushes knowledge away from your consuming application.

Another trick to building APIs that expose state, is to push the state of fields within your resource representation. This isn’t strictly hypermedia, but borrows from its understanding of how applications should represent themselves.

Take a user representation before:

{
	“id”:”456”,
	“name”:“Fred Jones”,
	“role”:“Normal”
	actions[
 		… actions
	]
}

One thing that is missing from this is any indication of whether the person viewing that user can edit any fields (think a profile update page).

Another representation (for an Admin) could be:

{
	field { “name”:“id”, “value”:“456”, “editable”:“false”, “type”:“text” }
	field { “name”:“name”, “value”:“Fred Jones”, “editable”:“true”, “type”:“text” }
	field { “name”:“role”, “value”:“Normal”, “editable”:“true”, “type”:“select”,
		“options”:[“Normal”, “Hacker”, “Admin”]
	}
	actions[
 		… actions
	]
}

And the representation for a normal user:

{
	field { “name”:“id”, “value”:“456”, “editable”:“false”, “type”:“text” }
	field { “name”:“name”, “value”:“Fred Jones”, “editable”:“true”, “type”:“text” }
	field { “name”:“role”, “value”:“Normal”, “editable”:“false”, “type”:“text” }
	actions[
 		… actions
	]
}

So depending upon the type of user, and who they are viewing, the options change.
For example, a normal user cannot edit their role, but for an admin user, the role field is editable, and a list of potential options are given.

This is where APIs get really useful, and start pushing the state of their resources to the client side. By using this kind of pattern, a single client website page can use the same view to render a representation in HTML without having to resort to if/else statements. The logic remains inside the service.

For more details about REST, see the ebooks referenced below.

Safari Books Online has the content you need

	REST continues to gain momentum as the best method for building web services, leaving many web architects to consider whether and how to include this approach in their SOA and SOAP-dominated world. Rest in Practice offers a down-to-earth explanation of REST, with techniques and examples that show you how to design and implement integration solutions using the REST architectural style.
	The basic rules of REST APIs – “many nouns, few verbs, stick with HTTP” – seem easy, but that simplicity and power require discipline to work smoothly. REST API Design Rulebook provides the next steps for implementing complex REST projects on simple and extensible foundations.
	SOA with REST: Principles, Patterns & Constraints for Building Enterprise Solutions with REST is the first comprehensive tutorial and reference for designing and building RESTful services as part of service-oriented solutions and in conjunction with service-oriented architecture (SOA). This book demonstrates that REST is not only a suitable medium for building truly service-oriented solutions, but also that the service-oriented architectural model is a necessary foundation for REST technology architectures to realize their full business potential.

About the author

Ben Keeping is a freelance developer based in the UK. He currently specializes in applications written in .NET and Ruby, using lean and agile techniques. He can be contacted @benkeeping.

Using Hypermedia in Your RESTful API Design Safari Books Online

HBase lets you store big data on commodity machines. It can scale up to millions of rows and columns each, putting Big Data processing, in real time, within reach of even individual developers, which is extremely difficult, if not impossible, with Relational Database Management Systems (RDBMS). For an introduction to HBase, be sure to read HBase – An OpenSource BigTable Database. In this post, I will talk about data migrations, showing you how to migrate your data from MySQL into an HBase data store.

There are three primary ways to migrate data into HBase:

• Put API
• Bulk load tool
• MapReduce Job

The Put API is probably the most straightforward way to import data quickly into an HBase table. It’s not recommended, however, to be used when importing huge amounts of data. You can think of it as the “Hello World” of HBase data migration.

The bulk load tool, on the other hand, runs a MapReduce job behind the scenes to populate data into an HBase table. Once populated, the tool also supports generating the HBase internal file format (HFile), enabling you to feed the freshly imported data directly into a running HBase cluster.

To quickly and reliably load a huge data file, you can use the importTSV tool. Since data migration is a write-intensive task, if you go the custom MapReduce job route, you will have to handle issues like optimal configuration to make sure data writes are not clogged. Let’s see how you can load data from a MySQL table into an HBase table with importTSV.

First, you will have to export your table data from an RDBMS, like MySQL, into a TSV format (remember, we’re using the Import TSV tool). You will have to make sure that there’s a field representing the row key of the HBase table row. There are scripts that can do this for you.

Now, since importTSV uses a MapReduce job in the backend to import data, you will have to spin a MapReduce instance on your cluster. Starting a MapReduce daemon can be done by executing this command on the master node:

hadoop$ $HADOOP_HOME/bin/start-mapred.sh

Follow along with these four steps to load data from a TSV file to HBase table:

1. Since importTSV only reads data from HDFS, you have to copy the TSV file from local file system to HDFS directory:

   hac@client1$ $HADOOP_HOME/bin/hadoop fs -mkdir /user/hac/input/2-1
   
   hac@client1$ $HADOOP_HOME/bin/hadoop fs -copyFromLocal hly-temp-10pctl.tsv
    /user/hac/input/2-1

2. Create a target table in HBase and add a column family to it. If the table already exists, you can alter it and old data will not be changed. I’ve added a ‘t’ column family where data will be populated:

   hac@client1$ $HBASE_HOME/bin/hbase shell
   hbase> create 'hly_temp', {NAME => 't', VERSIONS => 1}
   
   hbase> alter 'hly_temp', {NAME => 't', VERSIONS => 1}
   hbase> enable 'hly_temp'

3. Add hbase-site.xml and HBase dependency JARs in Hadoop classpath:

   hac@client1$ ln -s $HBASE_HOME/conf/hbase-site.xml $HA-DOOP_HOME/conf/hbase-site.xml
   
   hadoop@client1$ vi $HADOOP_HOME/conf/hadoop-env.sh
   export HADOOP_CLASSPATH=/usr/local/zookeeper/current/zookeeper-3.4.3.jar:/usr/ local/hbase/current/lib/guava-r09.jar 

4. Finally, run the importTSV tool by running this script:

   hac@client1$ $HADOOP_HOME/bin/hadoop jar $HBASE_HOME/hbase-0.92.1.jar 
      importtsv \-Dim-porttsv.columns=HBASE_ROW_KEY,t:v01,t:v02,t:v03,t:v04,
      t:v05,t:v06,t:v07,t:v08,t:v09,t:v10,t:v11,t:v12,t:v13,t:v14,t:v15,
      t:v16,t:v17,t:v18,t:v19,t:v20,t:v21,t:v22,t:v23,t:v24 \
   hly_temp \
   /user/hac/input/2-1

What you’re doing here is running importTSV with the following parameters:

• Porttsv.columns, which specifies the row key column. I’ve used HBASE_ROW_KEY that specifies the row key.
• Mapping information between the HBase table column family and the data columns in your TSV file. In this case, the data was arranged in the TSV file as row_key, value01, value02 and so on.
• TSV file path.

You can check the status of your MapReduce data migration job on the Admin page:

http://<master_node_name>:<default_port>/jobtracker.jsp

This is one of the several ways you can migrate MySQL database tables into HBase table. You can find a lot more hands-on examples in the books referenced below.

Safari Books Online has the content you need

Check out these Hbase books available from Safari Books Online:

	HBase: The Definitive Guide shows you how Apache HBase can fulfill your needs. As the open source implementation of Google’s BigTable architecture, HBase scales to billions of rows and millions of columns, while ensuring that write and read performance remain constant. This book provides the details you require to evaluate this high-performance, non-relational database, or put it into practice right away.
	HBase Administration Cookbook provides practical examples and simple step-by-step instructions for you to administrate HBase with ease. The recipes cover a wide range of processes for managing a fully distributed, highly available HBase cluster on the cloud. Working with such a huge amount of data means that an organized and manageable process is key and this book will help you to achieve that.
	HBase in Action has all of the knowledge you need to design, build, and run applications using HBase. First, it introduces you to the fundamentals of distributed systems and large scale data handling. Then, you’ll explore real-world applications and code samples with just enough theory to understand the practical techniques. You’ll see how to build applications with HBase and take advantage of the MapReduce processing framework. And along the way you’ll learn patterns and best practices.

About the author

Salman Ul Haq is a techpreneur, co-founder and CEO of TunaCode, Inc., a startup that delivers GPU-accelerated computing solutions to time-critical application domains. He holds a degree is Computer Systems Engineering. His current focus is on delivering the right solution for cloud security. He can be reached at salman@tunacode.com.

Migrating Data from MySQL into HBase Safari Books Online

If you have been following along, you know that I’ve been writing a series of posts on the .NET Entity Framework, and this post is the last in the series.

The Entity Framework (EF) is an object-relational database mapper that enables easy database access for .NET developers using domain-specific .NET objects that EF can generate for you. As shown in the Using Database Migration post, you can easily use EF to create your database from the classes that are defined in your project code. You can then use Code First Migrations to handle any database changes for the life cycle of your application.

During the course of developing your application (or during the testing phase), you often make changes either to the data collected or to the way data is processed. Sometimes you may have a bug in your code that has let corrupt or incorrect data get into your database. This can give you unpredictable results, and you can end up dropping and creating a database multiple times. When this happens, you are sometimes faced with the undesirable task of getting data back into the database to test your bug fixes. In the past this was a painstakingly slow process, since you had to add all of your test data back into the database.

Well, Entity Framework and Code First Migration has a solution for that. In the last blog post on Data Migration, you saw that running the Enable-Migrations -ContextTypeName SafariCodeFirst.SeminarContext command created a Migrations folder and two C# source files:

As previously described, the Configuration.cs file contains the settings for Migrations to work its magic, which in this case is to seed the database. If you examine Configuration.cs, you will see that Migrations has already created a stub method called Seed as the code below shows. It even contains some commented out code that shows how to go about adding a seed data to the table.

namespace SafariCodeFirst.Migrations
{
    using System;
    using System.Data.Entity;
    using System.Data.Entity.Migrations;
    using System.Linq;

    internal sealed class Configuration : DbMigrationsConfiguration
    {
        public Configuration()
        {
            AutomaticMigrationsEnabled = false;
        }

        protected override void Seed(SafariCodeFirst.SeminarContext context)
        {
            //  This method will be called after migrating to the latest version.

            //  You can use the DbSet<T>.AddOrUpdate() helper extension method 
            //  to avoid creating duplicate seed data. E.g.
            //
            //    context.People.AddOrUpdate(
            //      p => p.FullName,
            //      new Person { FullName = "Andrew Peters" },
            //      new Person { FullName = "Brice Lambson" },
            //      new Person { FullName = "Rowan Miller" }
            //    );
            //
        }
    }
}

Let’s remove the commented out code (or leave it for future reference) and add the code to seed the database as shown here:

        protected override void Seed(SafariCodeFirst.SeminarContext context)
        {

            context.Seminars.AddOrUpdate(
                p => p.Name,
              new Seminar
              {
                  Name = "Seed Seminar One",
                  Speaker = "Marshall Harrison"
              },

              new Seminar
              {
                  Name = "Seed Seminar Two",
                  Speaker = "Marshall Harrison"
              },
              new Seminar
              {
                  Name = "Seed Seminar Three",
                  Speaker = "Marshall Harrison"
              }
            );

        }

You can see how the Context.Seminars.AddOrUpdate helper class is used to create three new seminars if they don’t exist. If the database rows exist, then they will be updated. One thing to note is that you need the line that reads p => p.Name, or you will add all three seminars each time you run Update-Database, since that is used to identify the records.

So now that the code is ready, let’s delete all of the rows in the table and then run Update-Database. Your database should look like the following figure after the command finishes running:

Let’s make one small change to the code and prove that the Seed method will update the existing rows if they already exist. Change the Speaker for “Seed Seminar Three” to read “Jim Smith” and run the Update-Database command again, and then check the results. You will see that this time, instead of adding records, the data was simply updated.

As shown here, you can use Entity Framework and Code First Migrations to seed your database and make your life as a developer a lot easier. Of course, there are other things that configuration.cs can be used for, but I hope I have piqued your interest in Entity Framework so you will start experimenting with it on your own.

Safari Books Online has the content you need

Check out these .NET Entity Framework books available from Safari Books Online:

	Programming Entity Framework: Code First acts as an extension to Programming Entity Framework 2nd Edition. Code First is an additional means of building a model to be used with the Entity Framework and is creating a lot of excitement in the .NET development community. The book begins with an overview of what Code First is, why it was created, how it fits into the Entity Framework and when to use it over the alternatives.
	Entity Framework 4 in Action is an introduction to the Entity Framework for developers working in .NET who possess a working knowledge of ADO.NET. Beginning with a review of the core ideas behind the Object Relational Management (ORM) model, it outlines how the Entity Framework offers a smooth transition from a traditional ADO.NET approach. Featuring numerous examples and in-depth case studies, the book explores the Entity Framework story in clear, easy-to-follow detail.
	Entity Framework 4.1: Expert’s Cookbook holds many examples to help guide you through tough technical decisions and avoid technical landmines. The book will guide you from just using Entity Framework to enhancing the data access wizard. You will be guided through complex mapping scenarios, query definition, reusability, integration with other technologies, and architectural management. The approach is step-by-step and test driven so that it is focused as much as possible on solving problems and getting the most out of the time spent working through the book.

About the author

Marshall Harrison is the founder of GotSpeech.Net and GotUC.net as well as the owner and CEO of GotSpeech Consulting LLC. He holds a B.S. degree in Computer Science and was a Microsoft MVP from 2006 – 2011. He is currently focusing on Microsoft Lync as well as Windows 8 and Windows Phone 8 devleopment. He can be reached through his blog at GotspeechGuy.com.

.NET Entity Framework: Using Database Migration to Seed a Database Safari Books Online

AngularJS is a JavaScript framework that can help you write web applications quickly using less code. AngularJS is not a library, rather it is an entirely different way of thinking about writing web applications. As such, AngularJS has certain opinions about how you should be using it. Whereas in the vanilla JavaScript world, you are given a blank canvas on which to paint, AngularJS provides structure, organization and patterns to work with.

In this post we are focusing on using the ngResource module in AngularJS, which is “a factory that creates a resource object and lets you interact with RESTful server-side data sources.” Read Chapter 9: Practical Applications, Ajax to see an example using ngResource with Ajax.

Why Can’t AngularJS Find the ngResource Module?

I’ve seen this question asked again and again by people starting out with AngularJS. You often see this in the JavaScript console:

Uncaught Error: No module: ngResource

The answer is that you have not included the appropriate JavaScript file. ngResource is not part of the AngularJS core distribution (it used to be), and if you want to use it, you must go download angular-resource.js (or angular-resource.min.js) from the AngularJS repository.

How Can I Make $resource Synchronous?

If you do something like this in a controller:

var User = $resource('/path/to/user/:id');
$scope.user = User.get({id: 1});
$scope.user.name = $scope.user.first_name + ' ' + $scope.user.last_name;

You will find that $scope.user is, in fact, undefined, and you get a lovely JavaScript error. You may believe that you need to make ngResource synchronous so that your $scope.user will be populated when you need it. This is incorrect. What’s happening is: User.get() returns a promise, and when a promise lives in a Scope, it is automatically resolved, but it is undefined until then (more about promises).

What you really want to do is execute code after the User.get() call is finished up. You can do it like this:

var User = $resource('/path/to/user/:id');
$scope.user = User.get({id: 1}, function() {
  $scope.user.name = $scope.user.first_name + ' ' + $scope.user.last_name;
});

You will still get $scope.user eventually, when the request completes. After it is complete, you will be able to concatenate names.

PUTting to URLs

If you need to PUT, you need to PUT. ngResource does not support PUT out of the box, but it is extremely easy to implement:

var res = $resource('/your/url/:id', {foo: 'bar'}, {'put': {method: 
  'PUT', params: {foo: 'baz'}, isArray: false}});

The first parameter is the URL, of course. The second parameter is for default request parameters. For example, if you want to hit /your/url?foo=bar, use the above. The third parameter is where the good stuff happens. This creates a custom method called “put” that executes a PUT request against your URL. In addition, we can supply different default parameters for this particular method. To use our put, do this:

res.put({id: 1234}, {extra: 'post data goes here'}, 
  function success() {}, function error() {});

This was an example of creating a custom Resource (“class”) method. We could have called our method whatever we wanted to, and supplied default parameters for a shortcut to some commonly used resource. You’ll see another example below.

Basic Authentication

ngResource’s $resource service uses the built-in $http service for its requests. It’s simply an abstraction layer on top of that to make it easy to work with RESTful URLs. $resource itself isn’t very configurable, but $http is. One way to tackle the problem is to implement a solution akin to that found here. Once we’ve included the handy Base64 service, we can go about making that request this way:

function MyCtrl($scope, $resource, Base64, $http) {

  // Use this method for Angular 1.0.x.
  // Logs into a page protected by basic authentication.  Grabs
  // username and password from $scope, which would likely be bound to
  // "text" and "password" <input> tags, respectively.
  $scope.login = function login() {
    // modify the Authorization header to send the username & password
    $http.defaults.headers.common.Authorization = 'Basic ' + 
      Base64.encode($scope.username + ':' + $scope.password);
    // get the Resource object.
    $scope.res = $resource('/your/secured/url/here');
    // need to actually execute the request; do whatever with this
    $scope.res.get();
    // restore old defaults
    $http.defaults.headers.common.Authorization = 'Basic ';
  };

  // use this method if using Angular 1.1.1+
  $scope.unstableLogin = function login() {
    // get the Resource object w/ custom "get" method
    $scope.res = $resource('/your/secured/url/here', {}, 
       {get: {method: 'GET', headers: {Authorization: 'Basic ' + 
       Base64.encode($scope.username + ':' + $scope.password)}}});
    // need to actually execute the request; do whatever with this
    $scope.res.get();
  };
}

The problem with this approach is that browser behavior differs. In Firefox, if you put in incorrect credentials, you get a 401 response and that’s that. If you are in Chrome, however, the user will be prompted (with the default basic auth prompt) for the login and password repeatedly until the user hits “Cancel”, at which point you will receive your 401. In fact, the request is still in a “Pending” state while this occurs. Not only is this annoying and awful, AngularJS doesn’t have access to whatever the user entered.

A workaround, if appropriate, is to tell your web server to return something other than 401 on an authentication failure, and go from there. In AngularJS, a 500 (for example) will cause the $http promise to be rejected and you can handle it however you’d like. This is not recommended if you actually ever need the login prompt to occur!

Hopefully this has been helpful for someone looking to solve ngResource-related problems.

For more details about AngularJS, see the ebooks referenced below.

Safari Books Online has the content you need

	Developing an AngularJS Edge is intended for intermediate JavaScript programmers. No attempt has been made to explain the JavaScript syntax used (except in the cases where AngularJS may introduce a peculiarity), nor do we explain concepts such as closures, function chaining, callbacks, or other common patterns. What we do explain are basic AngularJS concepts, components, and their applications. We provide examples along the way, answer questions, and correct common misconceptions. Together, we’ll build a working single-page weblog application using AngularJS, which will help you become proficient with using AngularJS to go out and create your own applications.
	Develop smaller, lighter web apps that are simple to create and easy to test, extend, and maintain as they grow. AngularJS is a hands-on guide that introduces you to AngularJS, the open source JavaScript framework that uses Model–view–controller (MVC) architecture, data binding, client-side templates, and dependency injection to create a much-needed structure for building web applications.
	Instant AngularJS Starter is designed to get you ramped up on AngularJS as quickly and efficiently as possible. By the end of this book, you’ll possess all of the knowledge you need to make full-featured, real-life applications with AngularJS. The code samples are reusable, and specifically intended to give you a head start on your next project. This book will transform your curiosity about AngularJS into a set of production-ready AngularJS skills, through a broad overview of the framework and deep dives into its key features.

About the author

Christopher Hiller is a full-stack software engineer living and working in Portland, OR. He has over 15 years of web development experience and is a JavaScript zealot. When not coding epic masterpieces, he enjoys being a new dad, gaming, craft beers and woodworking. Chris can be contacted via GitHub, at: http://github.com/boneskull. You can also reach him on Twitter: @misterhiller.

AngularJS Tips and Tricks Using ngResource Safari Books Online

The term monoid frustrates a lot of programmers who otherwise are pretty versatile with higher-order generics, mutexes and even XSLT. This blog post will show how using monoids can be very simple and practical. Monoids are the basis of more complicated algebraic structures in mathematics, and are the underlying entities in many operations that we do while coding. There is a code sample in this post for Scala, which is becoming a lingua franca these days, and subsequent posts will focus on Scala.

When you deal with various types in a programming language you will notice common patterns of behavior for binary operations. Here are some examples:

1. Integers with addition. We know that (a+b)+c == a+(b+c) and 0+n == n+0 == n; same with multiplication: (a*b)*c == a*(b*c) and 1*n == n*1 == n
2. Strings with concatenation. a+(b+c)==(a+b)+c; ""+s==s and s+"" == s, etc.
3. Lists with concatenation, like List(1,2)+List(3,4) == List(1,2,3,4).
4. Sets with their union, like Set(1,2,3)+Set(2,4) == Set(1,2,3,4).

See a common pattern with a data type, a binary operation, and a special instance of the type, and certain rules? This common pattern is called monoid. Read Chapter 12: Monoids for more details about monoids.

Formally

Now, I’ll give a strict formal definition of monoid. Given a type T, a binary operation Op:(T,T) => T, and an instance Zero: T, with the properties that will be specified below, the triple (T, Op, Zero) is called a monoid. Here are the properties:

1. Neutral element: Zero `Op` a == a `Op` Zero == a
2. Associativity: (a `Op` b) `Op` c == a `Op` (b `Op` c)

Now, not every binary operation in the world is associative. For example, this average: avg(10, avg(30, 50)) != avg(avg(10, 30), 50); and for tuples, (a,(b,c)) != ((a,b),c).

So, you may be wondering what’s good about associativity? With an associative operation, you can reorganize a calculation in any order. Say, for instance, that we have the following expression:

(a1 `Op` (a2 `Op` (a3 `Op` (a4...))))

In Scala we have, (Zero /: a) (x `Op` y). This calculation can be regrouped and run in parallel, enabling map/reduce as follows:

(a0 `Op` a1 … `Op` a99) -------> p0 ---\
(a100 `Op` a101 … `Op` a199) ----> p1 ---->  p0 `Op` … `Op` p9	
. . .                                                                      /   
(a900 `Op` a901 … `Op` a999) ----> p9 --/

The definition above is all that you need to check whether you have a monoid. But it is important to see relationships between monoids. If you have just types, without any additional structure, any function, such as f:A=>B, is good for mapping instances of one type to instances of another. But if we have monoids, mappings between monoids should preserve operations.

Mappings Between Monoids

For two monoids, (A, OpA, ZeroA) and (B, OpB, ZeroB), we define a mapping from one monoid to another, such as with a function f: A => B, that f(ZeroA) = ZeroB and f(x OpA y) = f(x) OpB f(y).

Here are some good examples of monoid mappings:

• exp: (Double,+,0) => (Double,*,1); exp(a+b) == exp(a)*exp(b)
• length: (String,+,””) => (Int,+,0)
• sum: (List[Int],++,Nil) => (Int,+,0)

Free Monoids

For any type T we can build a monoid on it. Such a monoid will have an interesting universal property, as discussed next.

Suppose we have a type A, a monoid (B, OpB, ZeroB), and a function f:A=>B. This function just maps instances of A to instances of B. Now we can extend this function to a monoid mapping from List[A] to (B, OpB, ZeroB).

Obviously, if we have f:A=>B, we can apply f to elements of any list of elements of type A, and then fold as follows:

  
f'(Nil) = ZeroB and f'(list1 + list2) = f'(list1) OpB f'(list2)

This monoid mapping is uniquely determined by f, and it defines f if applied to singletons.

List[A] is a free monoid because it’s built out of A with no constraints, in a universal way.

Let’s look at a simple example. Suppose we have the following:

						
object WeekDay extends Enumeration {
  val Mon, Tue, Wed, Thu, Fri, Sat, Sun = Value
}

And the following function:

						
WeekDay=>Int = (Mon->1, Tue->1, Wed->1, Thu->1, Fri->1,
Sat->0, Sun->0)

Having a monoid (Int, +, 0), we can automatically extend this mapping to List[WeekDay]->Int, which counts the number of working days.

More Properties?

Not all monoids are created equal. We can find classes of monoids that have additional properties, reflecting the nature of their operations. In this section we will discuss some of those properties.

Commutative Monoids

To start with, here is the commutativity law: for all a and b, a `Op` b == b `Op` a.

Some monoids are commutative, and some are not:

					
2+3==3+2

But that isn’t true with strings:

"hello" + "world" != "world" + "hello"

You might wonder if we can build a free commutative monoid? Say we have a list of characters: “abracadabra“. Forget about the concatenation order; we can just count the number of each entry – or sort the list, obtaining “aaaaabbcdrr“. Only the number of occurrences of each element matters – meaning, we have what is known as a Multiset or a Bag:

{
  "partridges in a pear tree": 1,
  "turtle doves": 2,
  "french hens": 3,
  "calling birds": 4
}

To count, say, all of the birds, we don’t need to know on which day they were appended to the multiset.

How About Sets?

Sets have an interesting property: a set A joined with itself is still set A. Instances with properties like this are called idempotent:

x `Op` x == x

If we take Bags and add the property for each instance that is idempotent, we have Sets.
Here’s the hierarchy, from Alexander Bunkenburg, “The Boom Hierarchy”.

In Scala

To define a monoid in Scala, we declare the following:

trait Monoid[T] {
 def Zero: T
 def op: (T,T) => T
}

Then, if we define a new class, it can also extend Monoid[T]. This is impossible if the class already exists, like, Int or String. We may need to specify which operation we mean to be the monoidal operation. This is where type class should be used. In Haskell, a type class is defined as just an interface. In Scala; however, it is more complicated. It can depend upon the scope, and involves a so-called witness object. All this will be explained in more detail in my next blog post, so stay tuned.

Safari Books Online has the content you need

Check out these monoid and Scala books available from Safari Books Online:

	Learn You a Haskell for Great Good! begins with an overview of Node.js and then quickly dives into the code, core concepts, and APIs. In-depth coverage pares down the essentials to cover debugging, unit testing, and flow control so that you can start building and testing your own modules right away.
	Real World Haskell takes you through the basics of functional programming at a brisk pace, and then helps you increase your understanding of Haskell in real-world issues like I/O, performance, dealing with data, concurrency, and more. Be sure to look at The Data Structures chapter to learn more about monoids.
	Scala in Depth is a unique new book designed to help you integrate Scala effectively into your development process. By presenting the emerging best practices and designs from the Scala community, it guides you though dozens of powerful techniques example by example.

About the author

Vlad Patryshev was born in Russia, and has an education in Math (including now popular category theory). Since 1998 in the US, he worked at Borland (JBuilder team), then at Google in various teams, and had a 20% project, an onscreen keyboard, now available on various Google pages. He then changed several Bay Area startups, and now is working at HealthExpense.com. A functional programmer and a Scala fan since 2008, he is now an organizer of Scala BASE and Bay Area Categories and Types meetups. As a hobby, he rides his road bike, builds decks, and drives around the US (Key West to the Polar Circle in Alaska). He can be reached at @vpatryshev.

Monoids for Programmers: A Scala Example Safari Books Online

Entity Framework (EF) is an object-relational database mapper that enables easy database access for .NET developers using domain-specific .NET objects that EF can generate for you. As we saw in the first blog post on Entity Framework, you can use EF to create your database from the classes you have defined in your project code. You can add or remove properties from your classes and have EF migrate the changes into your database as well.

As developers, you often face projects where the specifications change over time for many reasons. Sometimes the client isn’t sure exactly what they need or their needs change during the course of the project. The design and analysis may determine that you need to add fields to the database or even remove fields from the database for one reason or another. For example, you may have new requirements to keep track of a user’s email address. Or you may need to add and track a user’s cell phone. A field/column entitled fName may even need to be renamed to firstName, or maybe you need to split a name into separate fields. There are many reasons that database fields/columns may need to change on a given project.

For this blog post, we are going to revisit the Visual Studio project used in The .NET Entity Framework Code First post. If you haven’t created that project, follow along with this post, but we recommend viewing that sample code.

If you open the project from The .NET Entity Framework Code First, you will see the following classes:

   public class Seminar
    {
        public int SeminarId { get; set; }
        public string Name { get; set; }
        public virtual List Topics { get; set; }

    }

    public class Topic
    {
        public int TopicId { get; set; }
        public string Title { get; set; }

        public int SeminarId { get; set; }
        public virtual Seminar Seminar { get; set; }
    }

And our database looked like this:

Now, for the sake of our discussion, let’s say that we already have run our application at least once so our database has already been created, and we have decided that we want to add the keynote speaker’s name to the database. We can do that by just adding it as a property in our SeminarID class and then getting a DBA to write the necessary SQL. If you were allowed, you could write the SQL to add the field/column to the database table. That would just modify the database, however, and that is not the best approach since you may need to change it later. The best approach is to automate tracking and manage the migrations as the project grows. To do that we will use Code First Migrations.

First, you need to enable Code First Migrations using the Package Manager Console. If it isn’t already open, select Tools->Library Package Manager->Package Manager Console. Now run the command shown below. Notice that we used the Namespace of our application and the name of our DbContext class to create a fully qualified context:

Enable-Migrations -ContextTypeName SafariCodeFirst.SeminarContext

Visual Studio will process the command and you should get a result that looks something like this:

You will also get a new Migrations folder added to your project that contains two files.

Configuration.cs contains the settings that Migrations uses to do its magic work. We won’t touch it for this blog post, but in a future post we will show you how to use it to provide seed data for your database. The other file is named <timestamp>_IntialCreate.cs and contains the code shown here:

namespace SafariCodeFirst.Migrations
{
    using System;
    using System.Data.Entity.Migrations;

    public partial class InitialCreate : DbMigration
    {
        public override void Up()
        {
            CreateTable(
                "dbo.Seminars",
                c => new
                    {
                        SeminarId = c.Int(nullable: false, identity: true),
                        Name = c.String(),
                    })
                .PrimaryKey(t => t.SeminarId);

            CreateTable(
                "dbo.Topics",
                c => new
                    {
                        TopicId = c.Int(nullable: false, identity: true),
                        Title = c.String(),
                        SeminarId = c.Int(nullable: false),
                    })
                .PrimaryKey(t => t.TopicId)
                .ForeignKey("dbo.Seminars", t => t.SeminarId, cascadeDelete: true)
                .Index(t => t.SeminarId);

        }

        public override void Down()
        {
            DropIndex("dbo.Topics", new[] { "SeminarId" });
            DropForeignKey("dbo.Topics", "SeminarId", "dbo.Seminars");
            DropTable("dbo.Topics");
            DropTable("dbo.Seminars");
        }
    }
}

Closer examination shows that it contains the code to take you from an empty database to one that contains the tables for the Seminar and Topic classes defined in our code.

Now that we have enabled Migrations and the basics are all in place, let’s modify our Seminar class to add a key note speaker so that our Seminar class now looks like this:

   public class Seminar
    {
        public int SeminarId { get; set; }
        public string Name { get; set; }
        public string Speaker { get; set; }
        public virtual List<Topic> Topics { get; set; }

    }

Next, we need to run migrations again using the command Add-Migration AddSpeaker. You can give this any name you want, but it is good to make it descriptive of the changes. Add-Migration will check for changes between the model classes and the last migration that was run, createing a new migration. You will notice that it is called <timestamp>_AddSpeaker.cs, and it has been added to the Migrations folder in our project. The timestamp part of the name is what Migrations uses to keep changes in order to track which changes have been applied. Now we need to run Update-Database so that any pending changes get applied to the database. I like to use the –Verbose switch so that I can see what is going on.

Now if you refresh the database in Server Explorer you will see that your changes have taken place and the new field/column have been added to the Seminars table.

We can go on and on like this, adding and dropping properties from our model classes and running the Add-Migration and Update-Database commands, as our database changes over the life cycle of our application. We still need to manually add code to the application to collect the user’s input or display the data to the user, but as you have seen, all of the hard work in our project has been taken care of for us by the Entity Framework and Code First Migrations.

Safari Books Online has the content you need

Check out these .NET Entity Framework books available from Safari Books Online:

	Programming Entity Framework: Code First acts as an extension to Programming Entity Framework 2nd Edition. Code First is an additional means of building a model to be used with the Entity Framework and is creating a lot of excitement in the .NET development community. The book begins with an overview of what Code First is, why it was created, how it fits into the Entity Framework and when to use it over the alternatives.
	Entity Framework 4 in Action is an introduction to the Entity Framework for developers working in .NET who possess a working knowledge of ADO.NET. Beginning with a review of the core ideas behind the Object Relational Management (ORM) model, it outlines how the Entity Framework offers a smooth transition from a traditional ADO.NET approach. Featuring numerous examples and in-depth case studies, the book explores the Entity Framework story in clear, easy-to-follow detail.
	Entity Framework 4.1: Expert’s Cookbook holds many examples to help guide you through tough technical decisions and avoid technical landmines. The book will guide you from just using Entity Framework to enhancing the data access wizard. You will be guided through complex mapping scenarios, query definition, reusability, integration with other technologies, and architectural management. The approach is step-by-step and test driven so that it is focused as much as possible on solving problems and getting the most out of the time spent working through the book.

About the author

Marshall Harrison is the founder of GotSpeech.Net and GotUC.net as well as the owner and CEO of GotSpeech Consulting LLC. He holds a B.S. degree in Computer Science and was a Microsoft MVP from 2006 – 2011. He is currently focusing on Microsoft Lync as well as Windows 8 and Windows Phone 8 devleopment. He can be reached through his blog at GotspeechGuy.com.

.NET Entity Framework: Using Database Migration Safari Books Online

In our previous Using Streams in Node.js post, we introduced the concept of streams and what abstractions and tools Node.js has for consuming or producing them.

Streams have many usages, ranging from abstracting file access, to streaming Hypertext Transfer Protocol (HTTP) requests and response bodies. In this post, we’re going to see how we can make two or more Node.js processes talk to each other, building a communication protocol on top of streams. Be sure to read Pedro’s Chapter 9: Reading and Writing Streams of Data in Professional Node.js: Building Javascript Based Scalable Software to learn more about Node.js streams.

Using some of these methods, you can build a mesh of Node.js processes that are loosely coupled, talking to each other as they need remote services.

TCP Server

A simple way of having two Node.js processes communicate with each other is by using Transmission Control Protocol (TCP). TCP is a protocol designed to transport data from one network device to another in a reliable and ordered way. It allows two nodes to establish a communication channel and pass a stream of ordered bytes between them. Node.js offers a streaming interface for both connected peers.

To be able to accept TCP connections, you need to create a TCP Server using the net module:

var net = require('net');

var server = net.createServer();

Then we can make this server listen on a certain TCP port:

var port = 3000;

server.listen(port);

server.once('listening', function() {
  console.log('Server listening on port %d', port);
});

Once the server is listening, it accepts connections on that port and emits a connection event:

server.on('connection', function(stream) {
  stream.pipe(stream);
});

Every time a peer connects to our server we get a connection event. In the event listener function call, we get the server socket object, which represents the two-way connection to the other connected peer. This object is a duplex stream that is writable and readable. When you write to it, it sends that data to the remote peer, and when you read to it, you read the data that the remote peer has sent you.

Here we’re simply piping the stream back to itself. All of the bytes that the server gets on this connection are sent back to the peer, effectively making an echo server.

Here is the source code of the entire server:

var net = require('net');

var server = net.createServer();

var port = 3000;

server.listen(port);

server.once('listening', function() {
  console.log('Server listening on port %d', port);
});

server.on('connection', function(stream) {
  stream.pipe(stream);
});

You can save this file with the name 01_tcp_server.js and launch it using the command line (you can access all of the source code found in this article here):

$ node 01_tcp_server.js
Server listening on port 3000

You can then test this server using a command-line TCP client like telnet or nc:

$ nc localhost 3000
hey
hey
what's up?
what's up?

TCP Client

Now we can create a TCP client that connects to our TCP server (02_tcp_client.js):

var net = require('net');

var port = 3000;
var host = 'localhost';

var stream = net.connect(port, host);
stream.setEncoding('utf8');

var interval = setInterval(function() {
  var date = (new Date).toString();
  stream.write('It\'s ' + date);
}, 1000);

setTimeout(function() {
  clearInterval(interval);
  stream.end();
}, 5100);

stream.on('readable', function() {
  var chunk;
  while(chunk = stream.read()) {
    console.log('got from server: %j', chunk);
  }
});

Here we’re using the net.connect function to create a TCP connection to our server. Then we’re setting the connection encoding to UTF-8 so that we can read back strings instead of raw byte buffers. Next, we create an interval, writing the current date to the stream for every second that passes. Shortly after 5 seconds have elapsed, we stop the recurring interval and end the stream, effectively ending the connection.

In the last chunk of code we set up a readable event listener for when we receive data that was sent back from the server, thus printing it out.

You can test all of this by launching this client script while having the server running:

$ node 02_tcp_client.js
got from server: "It's Mon May 06 2013 12:35:34 GMT+0100 (WEST)"
got from server: "It's Mon May 06 2013 12:35:35 GMT+0100 (WEST)"
got from server: "It's Mon May 06 2013 12:35:36 GMT+0100 (WEST)"
got from server: "It's Mon May 06 2013 12:35:37 GMT+0100 (WEST)"
$

Reconnect

If we want to establish a long-lasting connection, we may need to reconnect to the server if the network link goes down. For that, we can use the reconnect module. Let’s then create a package.json manifest where we state that dependency:

{
  "name": "my-connected-node-processes",
  "version": "0.0.1",
  "dependencies": {
    "reconnect": "*"
  }
}

Let’s then download and install this missing dependency:

$ npm install

Here is our reconnecting client (03_reconnecting_tcp_client.js):

var reconnect = require('reconnect');

var port = 3000;
var host = 'localhost';

var reconnector =
reconnect(function(stream) {
  stream.setEncoding('utf8');

  var interval = setInterval(function() {
    var date = (new Date).toString();
    stream.write('It\'s ' + date);
  }, 1000);

  stream.on('readable', function() {
    var chunk;
    while(chunk = stream.read()) {
      console.log('got from server: %j', chunk);
    }
  });

  stream.once('end', function() {
    clearInterval(interval);
  });

}).connect(port, host);

setTimeout(function() {
  reconnector.reconnect = false;
  reconnector.disconnect();
}, 10000);

To test the reconnecting effect of the client, you can start off by having the server down, and then launching the client. You will have to launch it before the client terminates, though, which is 10 seconds after.

Event Emitter

We’ve seen how we can connect two processes together, but all we’ve sent were unstructured string messages. TCP is a byte streaming protocol and does not mark message boundaries. For that, we can use JSON Stream, which JSON-encodes JavaScript objects to be sent down the wire and decodes them once they reach the other side.

So JSON Stream provides an object stream, allowing us to read and write JavaScript objects instead of bytes or strings. On top of the JSON Stream protocol, we’ll add the emit stream module, which transforms an object stream into an event emitter and vice-versa. Actually, and to save us some work, we’re going to use the duplex-emitter module, which makes all the stream connecting for us. Let’s then add duplex-emitter to the package.json manifest:

{
  "name": "my-connected-node-processes",
  "version": "0.0.1",
  "dependencies": {
    "reconnect": "*",
    "duplex-emitter": "*"
  }
}

Now, download and install it:

$ npm install

Let’s then change the server to use duplex emitter to transform the peer stream into an event emitter and then send an arbitrary ping event every second (04_server_events.js):

var net = require('net');
var server = net.createServer();
var port = 3000;
server.listen(port);
server.once('listening', function() {
  console.log('Server listening on port %d', port);
});

var duplexEmitter = require('duplex-emitter');

server.on('connection', function(stream) {
  var peer = duplexEmitter(stream);

  var interval =
  setInterval(function() {
    peer.emit('ping', Date.now());
  }, 1000);

  peer.on('pong', function(myTimestamp, hisTimestamp) {
    console.log('got pong from peer with args %d and %d', myTimestamp, hisTimestamp);
  });
});

Here the server also listens for remote pong events and prints the two arguments that they come with.

Now let’s create a client that also does a similar thing and reacts to remote ping events, sending back a pong event (05_client_events.js):

var reconnect = require('reconnect');
var duplexEmitter = require('duplex-emitter');

var port = 3000;
var host = 'localhost';

var reconnector =
reconnect(function(stream) {
  var peer = duplexEmitter(stream);

  peer.on('ping', function(timestamp) {
    console.log('got ping from peer %d', timestamp);
    peer.emit('pong', timestamp, Date.now());
  });

}).connect(port, host);

You can then launch the server:

$ node 04_server_events.js

And, on a separate command-line window, launch the client:

$ node 05_client_events.js

You should see the server printing lines like:

got pong from peer with args 1367841664066 and 1367841664074
got pong from peer with args 1367841665067 and 1367841665069
got pong from peer with args 1367841666069 and 1367841666070
got pong from peer with args 1367841667070 and 1367841667071
got pong from peer with args 1367841668070 and 1367841668071

And the client printing lines like:

got ping from peer 1367841664066
got ping from peer 1367841665067
got ping from peer 1367841666069
got ping from peer 1367841667070

Summary

Using TCP you can establish a raw stream between two Node.js processes. On top of this stream you can use the JSONStream and the emit-stream modules to turn that stream into an event emitter. By emitting events into that stream, you are transmitting that event to the remote peer. You can also listen for the remove events using the common event emitter API.

Using these techniques, we created a distributed Node.js-based application, making it less coupled and more flexible, scalable and reliable to changes.

Safari Books Online has the content you need

Check out these Node.js books available from Safari Books Online:

	Node.js is a powerful and popular new framework for writing scalable network programs using JavaScript. Professional Node.js begins with an overview of Node.js and then quickly dives into the code, core concepts, and APIs. In-depth coverage pares down the essentials to cover debugging, unit testing, and flow control so that you can start building and testing your own modules right away.
	In just 24 sessions of one hour or less, Sams Teach Yourself Node.js in 24 Hours will help you master the Node.js platform and use it to build server-side applications with extraordinary speed and scalability. Using this text’s straightforward, step-by-step approach, you’ll move from basic installation, configuration, and programming all the way through real-time messaging between browser and server, testing and deployment. Every lesson and case-study application builds on what you’ve already learned, giving you a rock-solid foundation for real-world success!
	This book shows you how to transfer your JavaScript skills to server side programming. With simple examples and supporting code, Node Cookbook talks you through various server side scenarios often saving you time, effort, and trouble by demonstrating best practices and showing you how to avoid security faux pas.
	Node Web Development gives you an excellent starting point straight into the heart of developing server side web applications with node. You will learn, through practical examples, how to use the HTTP Server and Client objects, the Connect and Express application frameworks, the algorithms for asynchronous execution, and use both SQL and MongoDB databases.

About the author

Pedro Teixeira is a geek, programmer, freelancer, and entrepreneur. Author of some Node.js modules, the Node Tuts screencast show, some books about Node.js and overall fervent proclaimer of the Node.js creed. He is also the Co-founder and Partner of The Node Firm, and the organizer of the Lisbon JavaScript Conference.

Making Node.js Talk Safari Books Online

Backbone.js is a client-side MV* framework that provides a simple, but powerful, events API. All Backbone objects (Models, Collections, Views & Routers) trigger events and can listen to events. This architecture is an implementation of the Observer design pattern and using events makes it easier to keep the various parts of your application loosely coupled.

In this post we’ll look at how to set a handler to listen to all of the events on a Backbone object, and why this is useful. Backbone has two main methods for setting up event handlers: .on and .listenTo. For the examples in this post, we’ll be using the .on method. This method works in similar ways to other event libraries, such as jQuery or NodeJS’s EventEmitter.

The .listenTo method was added to Backbone in version 0.9.9 and it is a useful tool to prevent memory leaks. It’s primarily used when a short-lived object needs to observe events in a long-lived object. An example would be if a view needs to listen to changes on a model. The view may be destroyed and created multiple times in the course of running the application, but the model is likely to only be instantiated once – when the application is initialized.

There is a whole catalog of events that are triggered internally by Backbone. It is also possible to trigger and listen to custom events. A useful feature of Backbone Collections is that any event triggered on a model within that collection will also be triggered on the collection itself. For example:

var collection, debug, javascript, python, ruby;

debug = function(name) {
  return function() {
    console.log(name + ":", arguments);
  };
};

ruby = new Backbone.Model({ id: 1, name: "Ruby", score:5 });
javascript = new Backbone.Model({ id: 2, name: "Javascript", score:5 });
python = new Backbone.Model({ id: 3, name: "Python", score:5 });

collection = new Backbone.Collection([ruby, javascript, python]);
collection.on("change", debug("collection - change"));

ruby.set("score", 10);
// Log: collection - change: [Backbone.Model, Object]

In the above code, you can see that when we changed an attribute on the ruby model, the change event was triggered on the collection. This also works with custom events:

collection.on("someCustomEvent", debug("Some Custom Event"));

python.trigger("someCustomEvent", "custom data");

// Log: Some Custom Event: ["custom data"]

So any event that is triggered on a model, whether a built-in Backbone event or a custom event, will be triggered on the collection that the model belongs to. Let’s dive into Backbone’s source to see how this is implemented:

set: function(models, options) {
  // Logic to add to or update models in a collection

  model.on('all', this._onModelEvent, this); 

}

_onModelEvent: function(event, model, collection, options) {
      if ((event === 'add' || event === 'remove') && 
         collection !== this) return;
      if (event === 'destroy') this.remove(model, options);
      if (model && event === 'change:' + model.idAttribute) {
        delete this._byId[model.previous(model.idAttribute)];
        if (model.id != null) this._byId[model.id] = model;
      }
      this.trigger.apply(this, arguments);
    }

When a Model is added to a Backbone Collection, a handler is added to the special “all” event. For instance, the _onModelEvent handler needs to perform custom logic for some events:

• add and remove events are stopped from being re-triggered, since they are usually initiated by the collection.
• destroy events on the model cause the model to be removed from the collection.
• change events to the models idAttribute cause the collection to adjust its internal _byId lookup hash.
• All events, except for add and remove, are re-triggered on the collection.

The Backbone source is well commented and very readable. To see how the events system sets up listeners for the “all” event look at the Backbone Events section.

Now that you’ve seen how the “all” event is used internally by Backbone, let’s see how it can be used in your own application. While plugin authors may use .on(“all”,…”) to implement advanced functionality, most of the time I use this feature for debugging. In a complex Backbone application, multiple events are fired by models and collections, and it can help to log them all to the console. To illustrate how this can be useful, let’s add some all listeners to Hubbub, an example Backbone application, to manage some Github issues. Navigate to the application, open up the console, and enter the following code:

var debug;
debug = function(name) {
  return function() {
    console.log(name + ":", arguments);
  };
};

app.board.issues.on("all", debug("board.issues"));
app.board.repos.on("all", debug("board.repos"));
app.board.on("all", debug("board"));

Now try adding a repo to your board and moving issues around. You will see a steady stream of events being logged to the console. The order of these events can be really useful in debugging an application. For example, if you see unexpected change events on your models after sync events, then you may need to check that the responses from your server are as you expect.

We’ve only touched the surface of what can be done with Backbone Events. The “all” event is incredibly useful for debugging in Backbone, and it would be great if Node’s EventEmitter had the same feature.

For more details about Backbone.js, see the ebooks referenced below.

Safari Books Online has the content you need

	While there are many frameworks that seem to be faster to get started with, Backbone’s lack of surprises, clear documentation, speed and flexibility make it a good fit for all types of applications. Developing a Backbone.js Edge incorporates the best practices and techniques from the combined authors’ experience of developing many Backbone applications. Some of the resources on the web advocate inefficient or inelegant solutions and there are few that cover the whole library. In this book we aim to provide a complete guide to Backbone and equip you to start using the library straight away.
	If you’re looking to use the single-page application model to build your site’s frontend, Backbone.js is an ideal framework for the job. Developing Backbone.js Applications provides all of the information you need to create structured JavaScript applications using Backbone.js and its unique flavor of the model-view-controller (MVC) architecture.
	The enthusiasm behind Node doesn’t just reflect the promise of server-side JavaScript. Developers also have the potential to create elegant applications with this open source framework that are much easier to maintain. Follow author Mike Wilson as he builds a social network application oriented toward real-time updates. Building Node Applications with MongoDB and Backbone shows you how to build complete applications that combine the strengths of Node, the MongoDB scalable storage solution, and the Backbone toolkit for clean application design in the browser.

About the author

Dave Tonge is a Backbone.js contributor and web developer at Simple Creativity. He is a co-author of Developing a Backbone.js Edge, and can be reached at: github.com/davidgtonge.

Debugging Backbone’s Events Safari Books Online

If you’re looking for a cloud platform to build and deploy highly scalable Node.js applications and services, Windows Azure is a good fit. It supports a wide variety of programming tools, frameworks, and development tools, including Node.js.

In this post, we’ll take a look at various options for building and deploying Node applications on Windows Azure. If you’re new to Windows Azure, you can sign up for a free trial to get started. Windows Azure development is a vast topic, and here we’ll just focus on the basics of getting up and running with Node.js. To dive deeper, take a look at Chapter 1: Windows Azure Platform Overview in Windows Azure Platform.

There are three ways to deploy Node applications and services on Windows Azure:

• Windows Azure web sites – highly scalable sites that can be provisioned and deployed quickly. You can deploy your code via FTP, or configure your site for continuous integration with a GitHub repo, Dropbox folder, or TFS repository.
• Infrastructure services – a scalable on-demand infrastructure that you can use to bring existing workloads to the cloud as-is, or spin up new Windows or Linux virtual machines in minutes.
• Cloud Services – a rich PaaS environment with automated provisioning, load balancing, auto-scaling, and OS/security management, so that you can concentrate on writing code.

As a rule of thumb, you’ll want to use Web Sites for simple projects. For more complex scenarios, you can use Infrastructure Services if you want maximum control of configuration details, or Cloud Services if you want to minimize your ongoing operational responsibilities. Regardless of which you choose, you have two options for deploying and managing your site:

• For those who prefer the GUI approach, there is a web interface (the Windows Azure management portal) that provides options for provisioning, starting, stopping, or deleting your applications and services.
• For those who prefer the command line, a set of PowerShell cmdlets is available for managing cloud deployments from Windows. In Mac or Linux environments you can use the cross-platform command line interface, which is written in Node.js and will run on any OS that supports Node. These tools, as well as Node.js SDKs for Windows, Mac, and Linux, are available for free download from the Windows Azure download center.

When you create a Windows Azure Web Site there are two options for Node developers. The Node JS Empty Sites is a pre-built website set up with Node.js, and the Node JS Starter Site also includes the popular Express MVC framework.

For spinning up a virtual machine to host your Node applications on Windows Azure Infrastructure Services, there are several options. One is to simply deploy a base OS and then build your preferred Node environment on top of it. This is the quick create option in the Windows Azure management portal, and current OS options include: Windows Server 2012, Windows Server 2008 R2, openSUSE 12.3, SUSE Linux Enterprise Server 11 SP2, OpenLogic CentOS 6.3, or Ubuntu Server versions 12.04 LTS, 12.10, or 13.04.

An even faster way to get a Node.js development environment up and running on Infrastructure Services is to take advantage of the pre-built virtual machine images available on VM Depot, a community-driven catalog of preconfigured operating systems, applications, and development stacks that can easily be deployed on Windows Azure. Current options on VM Depot include Node.js 10.5-0 on Ubuntu 12.10.

Note that Windows Azure Web Sites and Cloud Services provide automated tools for deploying and managing your applications, but with virtual machines you’ll need to use SSH to manage your deployments. This is another factor to take into account when choosing among the Azure options for Node developers.

To get started with Windows Azure cloud services, see the Build and deploy a Node.js application to a Windows Azure Cloud Service tutorial. The New-AzureServiceProject cmdlet sets up the structure for deploying a Cloud Service, and the Add-AzureNodeWebRole cmdlet generates scaffolding for a basic Node.js application. After editing the generated server.js as desired, you can run your application locally in an emulator, which you launch with the Start-AzureEmulator cmdlet.

The entire dev/test/deploy process is automated through the cmdlets, so that you can concentrate on building your application. Similar options are available for Mac and Linux users via the cross-platform command line interface. And regardless of how you create it, your Node.js-based Cloud Service has full access to Azure’s APIs for services such as compute, storage, caching, and messaging (Service Bus). For an overview of the Azure APIs you can use from a Node application, see the GitHub repo for the SDK.

That’s a high-level overview of the options for getting started with Node.js on Windows Azure. See below for books covering Windows Azure as well as Node.js. In future posts we’ll take a closer look at how to take full advantage of Azure services and how to monitor performance of your Node.js applications on Windows Azure.

You can find a wealth of information on Node.js and Windows Azure in the eBooks referenced below.

Safari Books Online has the content you need

	Smashing Node.js: JavaScript Everywhere, 2nd Edition equips you with the necessary tools to understand Node.js and its uses in developing efficient web applications. With more traditional web servers becoming obsolete, having knowledge on servers that achieve high scalability and optimal resource consumption using Node.js is the key to your app development success. Teaching you the essentials to making event-driven server-side apps, this book demonstrates how you can use less space and take less time for communication between web client and server.
	Professional Node.js: Building Javascript Based Scalable Software is a no nonsense book that begins with an overview of Node.js and then quickly dives into the code, core concepts, and APIs. In-depth coverage pares down the essentials to cover debugging, unit testing, and flow control so that you can start building and testing your own modules right away.
	In Windows Azure Platform is divided into three key parts: Windows Azure, SQL Azure, and Windows Azure AppFabric. Each of these plays a unique role in the functioning of your cloud service. It is the goal of this book to show you how to use these components, both separately and together, to build flawless cloud applications as well as hybrid architectures that fit in alongside your business’ existing systems.
	Windows Azure Programming Patterns for Start-ups is an incremental guide that will take you from the essentials of the Windows Azure platform up to the realization of your own cloud services running on the platform. You will learn how to apply different technologies of the Windows Azure platform with the help of examples all focusing on one single fictitious start-up scenario.

About the author

Doug Mahugh is lead technical evangelist at Microsoft Open Technologies, Inc., a wholly owned subsidiary of Microsoft Corporation responsible for advancing the company’s investments in interoperability, open source, and open standards. He can be reached at dmahugh@microsoft.com, or on Twitter at @dmahugh.

Node.js and Windows Azure: Getting Started Safari Books Online