Nettuts+

Advanced Python Data Structures

Adriana Vasiu — Tue, 18 Jun 2013 00:20:31 +0000

The aim of this tutorial is to show off Python’s data structures and the best way to use them. Depending on what you need from a data structure, whether it’s fast lookup, immutability, indexing, etc, you can choose the best data structure for the job and most of the time, you will be combining data structures together to obtain a logical and easy to understand data model.

Introduction

Python data structures are very intuitive from a syntax point of view and they offer a large choice of operations. This tutorial tries to put together the most common and useful information about each data structure and offer a guide on when it is best to use one structure or another. You can choose different kinds of data structures depending on what the data involves, if it needs to be modified, or if it’s fixed data, and even what access type you would like, such as at the beginning/end/random etc.

Lists

A List represents the most versatile type of data structure in Python. It can contain items of different types and it has no rule against unicity. List indices start from zero, the elements can be sliced, concatenated, and so on. Lists also have a lot of similarities with strings, supporting the same kind of operations but unlike strings, lists are mutable.

How to Construct a List

A list can be built using the keyword list or using square brackets: [], both of which accept comma separated values. Here’s an example:

>>> l = ['a', 'b', 123]
>>> l
['a', 'b', 123]

How to Retrieve an Element From a List

>>> l[0]
'a'
>>> l[10]
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
IndexError: list index out of range

As seen above, in order to access the data within your list, you must know what index position the element is at, otherwise you get an “index out of range” error.

How to Slice a List

All slicing operations return a shallow copy of the list. The slicing indexes are optional and they work in the same way as slicing indexes for strings.

Here’s an example of how to slice a list:

>>> l = ['a', 'b', 123]
>>> l[:]
['a', 'b', 123]
>>> new_l = l[1:]
>>> new_l
['b', 123]
>>> l
['a', 'b', 123]

Let’s take a look at some other common list operations.

Inserting and Removing Elements

>>> l = ['a', 'b', 123]
>>> l.append(234) #inserts an element at the end of the list
>>> l
['a', 'b', 123, 234]
>>> l.insert(2, 'c') #inserts an element into the third position
>>> l
['a', 'b', 'c', 123, 234]
>>> l.insert(-1, 111) #inserts an element into the second from last position of the list (negative indices start from the end of the list)
>>> l
['a', 'b', 'c', 123, 111, 234]
>>> l.remove(111) #removes an element based on value
>>> l
['a', 'b', 'c', 123, 234]
>>> l.remove('does not exist in the list') 
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: list.remove(x): x not in list

Retrieving and Looking Up Elements

Lists can also be used as stacks or queues because of how easy it is to add and remove elements from the beginning or end of the list.

>>> last_element = l.pop() #returns the last element, modifying the list
>>> last_element
234
>>> l
['a', 'b', 'c', 123]
>>> third_element = l.pop(2) #returns the third element, modifying the list
>>> third_element
'c'
>>> l
['a', 'b', 123]
>>> l.index('a') 
0
>>> l.index('does not exist in the list')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: 'does not exist in the list' is not in list
>>> l.count('a') #returns the number of occurrences of an element 
1

Whole List Operations

>>> l.extend ([1, 2]) # concatenates a list on to the existing list
>>> l
['a', 'b', 123, 1, 2]
>>> l.sort()
>>> l
[1, 2, 123, 'a', 'b']
>>> l.reverse()
>>> l
['b', 'a', 123, 2, 1]

As you can see, it’s very easy to extend, sort, and reverse lists using the above methods.

List Comprehensions

A list comprehension means, constructing a list in a way that is very natural from a mathematical point of view. The code for doing this is brief and very easy to read. A simple example of when you would use list comprehensions is when you want to construct a new list based on the elements from another list. Let’s have a look at how you can multiply all of the elements of a numeric list by two, in a simple one line construct:

>>> l = [1, 2, 3]
>>> new_l = [x*2 for x in l]
>>> new_l 
[2, 4, 6]

In the example above, the comprehension is represented by the multiplication expression that will be applied to every x element in the original, l list.

When to Use Lists

As shown in the examples above, lists are best used in the following situations:

When you need a mixed collection of data all in one place.
When the data needs to be ordered.
When your data requires the ability to be changed or extended. Remember, lists are mutable.
When you don’t require data to be indexed by a custom value. Lists are numerically indexed and to retrieve an element, you must know its numeric position in the list.
When you need a stack or a queue. Lists can be easily manipulated by appending/removing elements from the beginning/end of the list.
When your data doesn’t have to be unique. For that, you would use sets.

Sets

A set is an unordered collection with no duplicate values. A set can be created by using the keyword set or by using curly braces {}. However, to create an empty set you can only use the set construct, curly braces alone will create an empty dictionary.

Use the set keyword to create an empty set, curly brackets {} will create an empty dictionary.

How to Construct a Set

The set construct accepts one argument, a list.

>>> l = [1, 2, 3]
>>> s = set(l)
>>> s
set([1, 2, 3])

The {} construct is straight forward as well:

>>> s = {1, 2, 3}
>>> s

Sets are used to eliminate duplicate values from within a list:

>>> l = [1, 2, 3, 3]
>>> s = set(l)
>>> s
set([1, 2, 3])

The way a set detects if a clash between non-unique elements has occurred is by indexing the data in memory, creating a hash for each element. This means that all elements in a set must be hashable.

Here is an example:

>>> set ([1, [1,2]])
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: unhashable type: 'list'

Note that a hashable object doesn’t necessarily mean that an object has the __hash__ method available to it. It is important that the hash of the object doesn’t change during its lifetime, which is obviously not the case with lists, sets, or dictionaries (dictionaries will be discussed later in this tutorial).

Set Operations

The set structure also supports mathematical operations like:

Union
Intersection
Difference
Symmetric Difference

Here are a few examples:

>>> set1 = set([1, 2, 3])
>>> set2 = set([3, 4, 5])
>>> set1 | set2 #union
set([1, 2, 3, 4, 5])
>>> set1 & set2 #intersection
set([3])
>>> set1 -  set2 #difference
set([1, 2])
>>> set1 ^ set2 #symmetric difference (elements that are in the first set and the second, but not in both)
set([1, 2, 4, 5])
>>>

Set Comprehensions

Just like lists, sets also support comprehensions. Here is an example of how to use set comprehensions to find the unique consonants within a word:

>>> vowels = ['a', 'e', 'i', 'o', 'u']
>>> {x for x in 'maintenance' if x not in vowels }
set(['c', 'm', 't', 'n'])

Frozensets

A frozenset is basically just like a regular set, except that is immutable. It is created using the keyword frozenset, like this:

>>> frozen = frozenset([1, 2, 3])

When to Use Sets

You should choose to use a set in the following situations:

When you need a unique set of data: Sets check the unicity of elements based on hashes.
When your data constantly changes: Sets, just like lists, are mutable.
When you need a collection that can be manipulated mathematically: With sets it’s easy to do operations like difference, union, intersection, etc.
When you don’t need to store nested lists, sets, or dictionaries in a data structure: Sets don’t support unhashable types.

Tuples

Tuples are immutable, but can hold mutable objects.

A tuple is represented by a number of values separated by commas. Unlike lists, tuples are immutable and the output is surrounded by parentheses so that nested tuples are processed correctly. Additionally, even though tuples are immutable, they can hold mutable data if needed.

How to Construct a Tuple

Constructing an empty tuple requires parentheses. Here’s an example:

>>> my_empty_tuple = ()
>>> my_empty_tuple 
()

Constructing a tuple with one element requires a trailing comma. Example:

>>> one_elem_tuple = 'a',
>>> one_elem_tuple 
('a',)
>>> one_elem_tuple = ('a',)
>>> one_elem_tuple
('a',)

If the trailing comma is not there when creating a single element tuple, regardless if you use parentheses or not, Python will just interpret the value as a literal and will not create the tuple.

Constructing a tuple with multiple elements requires a list of values separated by commas. Here’s an example:

>>> s = 'a', 'b', [1, 2, 3]
>>> s
('a', 'b', [1, 2, 3])
>>>

Tuples, from a performance point of view are great because of their immutability. Python will know exactly how much memory to allocate for the data to be stored.

When to Use Tuples

When you need to store data that doesn’t have to change.
When the performance of the application is very important. In this situation you can use tuples whenever you have fixed data collections.
When you want to store your data in logical immutable pairs, triples etc.

Dictionaries

Dictionaries are represented by a key:value pair. In other words, they are maps or associative collections. The keys, unlike lists where they are numeric, can be of any immutable type and must be unique. The values can be of any type, mutable or immutable.

How to Construct a Dictionary

There are several ways to construct a dictionary. The most efficient way in terms of performance is to use curly braces {}:

>>> vowels = {1: 'a', 2: 'e', 3: 'i', 4: 'o', 5:'u'}
>>> vowels
{1: 'a', 2: 'e', 3: 'i', 4: 'o', 5: 'u'}

Another way to create a dictionary is by using comprehensions. By using an input collection you can create the key:value pairs in one simple construct.

Here’s an example:

>>> {x:x*x for x in (1, 2, 3)}
{1: 1, 2: 4, 3: 9}

You can also use the keyword dict and get the same result. The dict construct takes as an argument, a list of key:value pairs. Here’s one in action:

>>> dict([(1,'a'), (2,'e'), (3,'i'), (4,'o'), (5,'u')])
{1: 'a', 2: 'e', 3: 'i', 4: 'o', 5: 'u'}

If the keys are strings, you can use the following keyword expression:

>>> dict(a=1, e=2, i=3, o=4, u=5)
{'i': 3, 'u': 5, 'e': 2, 'a': 1, 'o': 4}

However, it does not work the other way around when the keywords are numeric:

>>> dict(1=a, 2=e, 3=i, 4=o, 5=u)
  File "<stdin>", line 1
SyntaxError: keyword can't be an expression

Dictionary Operations

Accessing data in a dictionary is very straight forward, just wrap its key name within square brackets:

>>> vowels = {1: 'a', 2: 'e', 3: 'i', 4: 'o', 5:'u'}
>>> vowels[1]
'a'
>>> vowels[10]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
KeyError: 10
>>>

As seen above, a KeyError occurs if the key doesn’t exist in the dictionary.

Also, a key:value pair can be deleted using the del keyword, like so:

>>> vowels = {1: 'a', 2: 'e', 3: 'i', 4: 'o', 5:'u'}
>>> del(vowels[1])
>>> vowels
{2: 'e', 3: 'i', 4: 'o', 5: 'u'}
>>> del(vowels[10])
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
KeyError: 10
>>>

Built-in Methods

Dictionaries support many built-in methods, but some of the most useful ones are: keys(), values(), iteritems(), itervalues(), and has_key().

Here’s an example:

>>> vowels = {1: 'a', 2: 'e', 3: 'i', 4: 'o', 5:'u'}
>>> vowels.keys()
[1, 2, 3, 4, 5]
>>> vowels.values()
['a', 'e', 'i', 'o', 'u']

The methods iteritems() and itervalues() return iterators, so they can be used in for loops. Here’s an example:

>>> for k, v in vowels.iteritems():
...     print k, v
... 
1 a
2 e
3 i
4 o
5 u
>>> for v in vowels.itervalues():
...     print v
... 
a
e
i
o
u
>>>

When to Use a Dictionary

When you need a logical association between a key:value pair.
When you need fast lookup for your data, based on a custom key.
When your data is being constantly modified. Remember, dictionaries are mutable.

Conclusion

General Python Data Structure Usage

Use lists if you have a collection of data that does not need random access. For random access, you need to have knowledge of the element’s numeric index. Try to choose lists when you need a simple, iterable collection that is modified frequently. Lists are very useful in comprehension expressions for constructing sets or dictionaries.
Use a set if you need unicity for the elements and you don’t need a nested dictionary or list. Also remember that a set cannot hold any unhashable data types.
Use tuples when your data cannot change. Many times, a tuple is used in combination with a dictionary, for example, a tuple might represent a key, because it’s immutable.
Use frozensets if you need both unique data and immutability.

Other Tips

Every data structure has a multitude of built in methods and capabilities. If you go through them carefully you’ll learn how the data is meant to be used. For example, if you call dir and pass in a set object, you will see that it holds a lot of methods that can perform mathematical operations.
Most of the time, your data needs to adapt to the operations that you want to perform. So if you know your object will be hashed, create an immutable structure.
Most data structures have multiple ways of constructing or accessing its data. If you are concerned about the performance of your application always read the documentation or read the source code of the construct implementation to find out more about its performance. For example, creating a dictionary by using dict offers less performance than using the curly braces ({}) syntax.
While debugging, many developers find it useful to learn how to identify and connect the error that they are receiving to a specific data structure. For example, you’ll find that lists throw IndexError and ValueError and dictionaries throw KeyError. This technique should help you with detecting which of your data structures is causing the error.

Building Ribbit in Meteor

Gabriel Cirtea — Sat, 15 Jun 2013 02:01:04 +0000

This is a continuation of the Twitter clone series with building Ribbit from scratch, this time using Meteor.

For this tutorial, please do not expect an in depth explanation of the Meteor framework. You should already have some Meteor experience so that you’ll understand some of the important concepts that will be presented. In order to get a basic understanding of Meteor I recommend Andrew Burgesses course on Tutsplus Premium.

So, let’s get started.

Step 0: Creating and Configuring the Meteor Application

We begin by generating a new Meteor application. Open a command line prompt and navigate to the folder where you want the application to be stored. Then run:

meteor create ribbit

Meteor will create a folder named ribbit containing some files and will ask you to change directory into ribbit and run meteor commands in that folder.

If you step into the ribbit folder and issue a ls -la command in order to see what was generated by Meteor, you will find a hidden folder named .meteor and three files named ribbit.html, ribbit.css and ribbit.js. The .meteor folder contains only one file named packages. The content of the .meteor/packages file informs Meteor about what smart packages are used by the application.

Since we will start developing Ribbit from scratch we will keep only the .meteor folder and delete all the rest.

Let’s see what packages Meteor uses by default: at the command prompt enter

cat .meteor/packages

The output of this command should list the autopublish, insecure, and preserve-inputs packages.

We will discuss the concept of publishing data from the server later, for the moment let’s note that the autopublish package is used on the server side to automatically make data available to the client. Usually this packet is not used in a production environment, so we can remove it by issuing the following command:

meteor remove autopublish

Next, the insecure package allows database writes by default, which can be dangerous, since this allows any user to do any database operations, including deleting data. We will remove this package before deploying the app and add all the necessary code to protect writing to the database.

And lastly, the preserve-inputs package keeps data from all of the form fields with a unique id.

After all of that, we should now have a folder containing just a .meteor sub-folder and this should be the base of our app.

Step 1: Creating the Necessary Folder Structure

There are several rules Meteor uses to serve files and data which we will not being covering in detail here (see Andrew’s course for details). What is important to note, is that Meteor considers all files present in a server folder as being server code. All files in the client and public folders as being client code, which should be served to the client. And lastly, the files outside of these folders are served to both the client and the server. So let’s create the following folders:

client – to store all the client side code.
server – to store all the server side code.
public – containing all the assets such as graphic files, robots.txt
files, favicons etc.

Also, it’s important to know that Meteor considers JavaScript and CSS files as part of the client/server code and not as assets, which should be served from the public folder.

So, we end up with the following folder structure:

Step 2: The Static Version of the Login Page

Now it’s time to download the static version of the Ribbit app.

After unpacking the files, copy the gfx folder into the public folder of our Ribbit app.

Also, copy the home.html and style.less files into the client folder of our app. Now if you would try to start the server by issuing meteor from the command line, in the root of our application, you will see the server crashing and complaining about setting the DOCTYPE in our file.

Meteor expects our HTML files to be composed from only three base elements: the head, the body and the template tags.

So in order to fix the error, we should edit the home.html file and remove the DOCTYPE and html tags, which will be added by Meteor when serving the files.

Now if you refresh the page, you should see its content, but it won’t have any styling since the package we downloaded uses LESS for styling the page.

This is easy to fix by installing Meteor’s LESS smart package. So issue:

meteor add less

then restart the server and reload the page and things should look a little bit better now.

We still have to do some adjustments in the LESS file since not everything looks as it should. Search for all occurrences of gfx in the style.less file and change them by adding a trailing slash before each. This should be done in order to allow files to be loaded from the root of the project.

The next step is to adjust the home.html and style.less files to add in a login button and have everything on one line in the header. The files should look like following:

// client/style.less
input {
  width: 236px;
  height: 26px;
  border: 1px solid @border-color;
  padding: 0 10px;
  outline: none;
  font-size: 17px;
  &amp;:focus {
    background: #FFFDF2;
  }
}

input[type="submit"] {
  height: 26px;
  width: auto;
  border: 1px solid #7BC574;
  border-radius: 2px;
  color: white;
  font-size: 12px;
  font-weight: bold;
  padding: 0 20px;
  cursor: pointer;
  .gradient4f(0%, #8CD585, 23%, #82CD7A, 86%, #55AD4C, 100%, #4FA945);
}

header {
  background: url(/gfx/bg-header.png);
  height: 85px;
  width: 100%;
  div.wrapper {
    padding: 11px 0;
    img {
      position: relative;
      top: 10px;
      margin: 0 15px 0 0;
    }
    span {
      font-size: 18px;
      margin: 0 42px 0 0;
    }
    p {
      display: inline;
      input {
        margin: 0 0 0 14px;
        width: 180px;
        height: 28px;
      }
    }
    #btnLogOut {
      float: right;
      width: auto;
      height: 28px;
      margin: 19px 0 0 0;
    }
  }
}

And here’s what the home.html file should look like:

<!-- client/home.html -->
<head>
</head>
<body>
  <header>
    <div class="wrapper">
      <img src="gfx/logo.png">
      <span>Twitter Clone</span>
      <p>
        <input name="username" placeholder="username" type="text">
        <input name="password" placeholder="password"  type="password">
        <input type="submit" id="btnLogOut" value="Log In">
      </p>
    </div>
  </header>
  <div id="content">
    <div class="wrapper">
      <img src="gfx/frog.jpg">
      <div class="panel right">
        <h1>New to Ribbit?</h1>
        <p>
          <input name="email" placeholder="email" type="text">
          <input name="username" placeholder="username" type="text">
          <input name="fullname" placeholder="fullname" type="text">
          <input name="password" placeholder="password" type="text">
          <input name="password2" placeholder="retype password" type="password">
          <input type="submit" value="Create Account">
        </p>
      </div>
    </div>
  </div>
  <footer>
    <div class="wrapper">
      Ribbit - A Twitter Clone Tutorial<img src="gfx/logo-nettuts.png">
    </div>
  </footer>
</body>

Now take a look at the page in the browser. It should look like the image below:

Well that was pretty easy, wasn’t it? But, what we’ve done up to now is not what we really desired and it’s definitely far from following the Meteor style of doing things. Let’s fix this up!

Step 3: Meteor-ize the Home Page

Meteor constructs the HTML page that is sent to the browser based on three elements, it expects the following to be found: the head element, the body element, and the template element. Since the head and the body are already well known, special attention should be given to the template element.

A template is declared with <template name="foo">...</template> and its content is rendered in the HTML content that is sent to the browser. On the other hand, in the JavaScript file, the template can be accessed using Template.foo, which returns a string of HTML when called.

Now Meteor uses Handlebars as the template framework, but others can be used if needed, such as Jade.

So, let’s see what should be done in order to transform the home.html file into a Meteor compatible page. Let’s take a look at the modified code:

<!-- client/home.html -->
<head>

</head>

<body>
  <header>
    {{> header}}
  </header>
  {{> content}}
  <footer>
    {{> footer}}
  </footer>
</body>

<template name="header">
  <div class="wrapper">
    <img src="gfx/logo.png">
    <span>Twitter Clone</span>
    <p>
      <input id="username" name="username" placeholder="username" type="text">
      <input id="password" name="password" placeholder="password"  type="password">
      <input id="btnLogOut" type="submit" value="Log In">
    </p>
  </div>
</template>

<template name="content">
  <div id="content">
    <div class="wrapper">
      <img src="gfx/frog.jpg">
      <div class="panel right">
        <h1>New to Ribbit?</h1>
        <p>
          <input id="email" name="email" placeholder="email" type="text">
          <input id="newusername" name="username" placeholder="username" type="text">
          <input id="fullname" name="fullname" placeholder="fullname" type="text">
          <input id="newpassword" name="password" placeholder="password" type="text">
          <input id="password2" name="password2" placeholder="retype password" type="password">
          <input id= "btnCreateAccount" type="submit" value="Create Account">
        </p>
      </div>
    </div>
  </div>
</template>

<template name="footer">
  <div class="wrapper">
    Ribbit - A Twitter Clone Tutorial<img src="gfx/logo-nettuts.png">
  </div>
</template>

As you can see, we start with an empty head element. Since Meteor loads all the necessary files for us, we can use the head element to establish the page’s title, but for the moment, it’s blank.

Inside the body element, we render three templates: the header template, which we place in the header tag, the content template, which is placed right after the header, and finally the footer template, inside a footer tag.

What I’ve done for these templates, was to simply copy the old code and place it into the appropriate template. So what was in the header tag, is now in the header template and so on.

I’ve also added some ids to the edit boxes and the buttons, so that later, I’m able to access these from the JavaScript code.

And that’s it. Now our page is finished and works well with the Meteor framework.

The next step will be to create the buddies page which should be displayed when a user logs into the system. We also need to allow the user to sign up and log in to our application.

Step 4: Log Into the Buddies Page

If we follow the same steps from in the previous section, to transform the buddies.html page into a Meteor one, we end with the following code:

<!-- client/buddies.html -->
<head>

</head>
<body>
  <header>
    {{> header}}
  </header>
  {{> content}}
  <footer>
    {{> footer}}
  </footer>
</body>

<template name="header">
  <div class="wrapper">
    <img src="gfx/logo.png">
    <span>Twitter Clone</span>
    <p>
      <input type="submit" id="btnLogOut" value="Log Out">
    </p>
  </div>
</template>

<template name="content">
  <div id="content">
    <div class="wrapper">
      <div id="createRibbit" class="panel right">
        <h1>Create a Ribbit</h1>
        <p>
          <textarea name="text" class="ribbitText"></textarea>
          <input type="submit" value="Ribbit!">
        </p>
      </div>
      <div id="ribbits" class="panel left">
        <h1>Your Ribbit Profile</h1>
        <div class="ribbitWrapper">
          <img class="avatar" src="gfx/user1.png">
          <span class="name">Frogger</span> @username
          <p>
            567 Ribbits<span class="spacing">45 Followers</span><span class="spacing">32 Following</span><br>
            Cras justo odio, dapibus ac facilisis in, egestas Vivamus sagittis lacus vel augue laoreet rutrum faucibus dolor auctor. <a href="#">http://net.tutsplus.com/tutorials/php/ ...</a>
          </p>
        </div>
      </div>
      <div class="panel left">
        <h1>Your Ribbit Buddies</h1>
        <div class="ribbitWrapper">
          <img class="avatar" src="gfx/user2.png">
          <span class="name">Kermit</span> @username <span class="time">15m</span>
          <p>
            Cras justo odio, dapibus ac facilisis in, egestas Vivamus sagittis lacus vel augue laoreet rutrum faucibus dolor auctor. <a href="#">http://net.tutsplus.com/tutorials/php/ ...</a>
          </p>
        </div>
      </div>
    </div>
  </div>
</template>

<template name="footer">
  <div class="wrapper">
    Ribbit - A Twitter Clone Tutorial<img src="gfx/logo-nettuts.png">
  </div>
</template>

As you can see, I took out the long list of ribbits, but other than that, there’s nothing else too impressive about the above code.

However, there are two problems with it:

If you try viewing it in the browser, you will find it’s being displayed twice. There are two reasons for this: Meteor sends files to the browser in alphabetical order, so buddies.html is sent first. We also have a lot of duplication in the code, even the name of the templates are duplicated. So when Meteor wants to display the content template, it displays the first defined one, which is the one in the buddies.html file.
The second problem is also code duplication, which needs to be fixed.

The issues above can be resolved by doing the following: we will create an index.html file containing only the code that will trigger the needed templates: one for the header, one for the footer, one for the home page, and one for the buddies page.

Let’s take a look of each of them:

<!-- client/index.html -->
<head>

</head>

<body>
  <header>
    {{> header}}
  </header>
  {{#if currentUser}}
    {{> buddiescontent}}
  {{else}}
    {{> homecontent}}
  {{/if}}
  <footer>
    {{> footer}}
  </footer>
</body>

This is the main file in which all of the templates are loaded. A very interesting thing can be found when the content template is displayed. We check with a Handlebars if clause whether a user is logged in or not. The Meteor currentUser variable contains the logged in user’s data, if a user is logged in (so we display the buddies template) and is null if no user is logged in (so we display the home template).

<!-- client/header.html -->
<template name="header">
  <div class="wrapper">
    <img src="gfx/logo.png">
    <span>Twitter Clone</span>
    {{#if currentUser}}
      <nav>
        <a id="buddies" href="#">Your Buddies</a>
        <a href="#">Public Ribbits</a>
        <a id='profiles' href="#">Profiles</a>
      </nav>
      <input type="submit" id="btnLogOut" value="Log Out" />
    {{else}}
      <p>
        <input id="username" name="username" placeholder="username" type="text">
        <input id="password" name="password" placeholder="password"  type="password">
        <input id="btnLogOut" type="submit" value="Log In">
      </p>
    {{/if}}
  </div>
</template>

Here we render the header for our pages, and as above, if a user is logged in, we display a navigation bar, otherwise we display a log in form.

<!-- client/footer.html -->
<template name="footer">
  <div class="wrapper">
    Ribbit - A Twitter Clone Tutorial<img src="gfx/logo-nettuts.png">
  </div>
</template>

Then we create the footer template, which only renders the content in the footer.

<!-- client/home.html -->
<template name="homecontent">
  <div id="content">
    <div class="wrapper">
      <img src="gfx/frog.jpg">
      <div class="panel right">
        <h1>New to Ribbit?</h1>
        <p>
          <input id="email" name="email" placeholder="email" type="text">
          <input id="newusername" name="username" placeholder="username" type="text">
          <input id="fullname" name="fullname" placeholder="fullname" type="text">
          <input id="newpassword" name="password" placeholder="password" type="password">
          <input id="password2" name="password2" placeholder="retype password" type="password">
          <input id= "btnCreateAccount" type="submit" value="Create Account">
        </p>
      </div>
    </div>
  </div>
</template>

Now we’ve already saw this template before, it just contains a form to allow the users to register for the application.

<!-- client/buddies.html -->
<template name="buddiescontent">
  <div id="content">
    <div class="wrapper">
      <div id="createRibbit" class="panel right">
        <h1>Create a Ribbit</h1>
        <p>
          <textarea name="text" class="ribbitText"></textarea>
          <input type="submit" value="Ribbit!">
        </p>
      </div>
      <div id="ribbits" class="panel left">
        <h1>Your Ribbit Profile</h1>
        <div class="ribbitWrapper">
          <img class="avatar" src="gfx/user1.png">
          <span class="name">Frogger</span> @username
          <p>
            567 Ribbits<span class="spacing">45 Followers</span><span class="spacing">32 Following</span><br>
            Cras justo odio, dapibus ac facilisis in, egestas Vivamus sagittis lacus vel augue laoreet rutrum faucibus dolor auctor. <a href="#">http://net.tutsplus.com/tutorials/php/ ...</a>
          </p>
        </div>
      </div>
      <div class="panel left">
        <h1>Your Ribbit Buddies</h1>
        <div class="ribbitWrapper">
          <img class="avatar" src="gfx/user2.png">
          <span class="name">Kermit</span> @username <span class="time">15m</span>
          <p>
            Cras justo odio, dapibus ac facilisis in, egestas Vivamus sagittis lacus vel augue laoreet rutrum faucibus dolor auctor. <a href="#">http://net.tutsplus.com/tutorials/php/ ...</a>
          </p>
        </div>
      </div>
    </div>
  </div>
</template>

And then the last template, containing information about the current user, that user’s ribbits, and the people the current user is following.

Up until now, we have two pages of our application which can be displayed, but both are static and we can display only one at a time. So next, let’s add some JavaScript to our application to make it more dynamic.

Registering and Logging In

Meteor gives us several smart packages to handle the process of registering and logging in to an application. For our Ribbit app, we will use following packages: accounts-base for basic support for accounting and accounts-password for getting password support for accounting. To install these packages, run the following commands:

meteor add accounts-base
meteor add accounts-password

Now, let’s create a client.js file in the client folder to hold our client side code. For the home page, we need to handle events for clicking on the Log In button and Create Account button.

Events in Meteor are associated with a specific template, so in order to handle clicking on the Log In button we will add an event handler for the header template. To handle the click event for the Create Account button, we have to add an event handler associated with the homecontent template.

Let’s take a look at the code:

Clicking on the Log In Button

// client/client.js
// handling click event on the Log In button
Template.header.events({
  'click #btnLogOut': function (event, template) {
    if (Meteor.userId()) {
      Meteor.logout();
    } else {
      var userName     = template.find('#username').value,
        userPassword = template.find('#password').value;
      Meteor.loginWithPassword(userName, userPassword, function (error) {
        if (error) {
          console.log(error);
        }
      });
    }
  }
});

On the first line, we attach an events object to the header template. Then we handle the click event on the button with an id of btnLogOut (which we’ve created in our HTML template). Then if a user is logged in, simply log the user out. If Meteor.userId returns null, that means no user is logged in, so we perform a log in with the supplied username and password. Also, notice the values that are inserted in the edit boxes: the template object has a find function which takes as a parameter, the ID of the edit box for which we want to get the value of. If there is an error during logging in, we just display it in the console. For now, this is a temporary approach since we will implement displaying information to the user a little bit later.

Clicking on the Create Account Button

// client/client.js
// hnadling click event on the Create Accounts button
Template.homecontent.events({
  'click #btnCreateAccount': function (event, template) {
    var userEmail = template.find('#email').value,
      userName  = template.find('#newusername').value,
      password  = template.find('#newpassword').value,
      password2 = template.find('#password2').value,
      name      = template.find('#fullname').value;

    Accounts.createUser({
      username: userName,
      email:    userEmail,
      password: password,
      profile: {
        name: name
      }
    }, function (error) {
      if (error) {
        console.log("Cannot create user");
      }
    });
  }
});

Just like before, we attach an events object to the homecontent template, then we get the values submitted by the user from the form fields. To create an account we simply use the Accounts.createUser function, which takes as a parameter an object with the username, password, email and profile components. In case of an error while adding the user, the error is displayed in the console.

Step 5: Making the Buddies Page Dynamic

The next step in developing our application is to make the buddies page dynamic. At the moment, everything displayed on our page is statically taken from the template files. We want to take the information from a database and also to allow the user to create ribbits and store them in a database on the server side.

Changing the Your Ribbit Profile Section

This section displays the full name of the user, their username, the number of ribbits the current logged in user has, the number of followers and following people, and the last ribbit posted by the user.

We will store the code for this part in a main.js file. Note that Meteor loads files with the word main in their name, after the page is loaded.

Let’s see how to display them in the template:

<!-- client/buddies.html -->
...
  <div id="ribbits" class="panel left">
<h1>Your Ribbit Profile</h1>
<div class="ribbitWrapper">
<img class="avatar" src="gfx/user1.png">
<span class="name">{{fullName}}</span> @{{userName}}
<p>
{{noOfRibbits}}<span class="spacing">45 Followers</span><span class="spacing">32 Following</span><br>
{{lastRibbit}}
</p>
    </div>
  </div>
...

What we want to do in this template is to use Handlebars’ variables for the items we’d like to pull from the database. The fullname and the username are taken from the users collection and the noOfRibbits and lastRibbit should be taken from the ribbit collection.

The above variables are returned to the template by the following JavaScript code in the main.js file:

// client/main.js
Ribbits = new Meteor.Collection('ribbits');

Template.buddiescontent.helpers({
  fullName: function () {
    return Meteor.user().profile.name;
  },

  userName: function () {
    return Meteor.user().username;
  },

  noOfRibbits: function () {
    var ribbits = Ribbits.find({user_id: Meteor.userId()}),
      retVal;
    if (ribbits.count() === 1) {
      retVal = "1 Ribbit";
    } else {
      retVal = ribbits.count() + " Ribbits";
    }
    return retVal;
  },

  lastRibbit: function () {
    var lastRibbit = Ribbits.findOne({user_id: Meteor.userId()}, {sort: {created_at: -1}}),
      retVal;

    if (lastRibbit) {
      retVal = lastRibbit.ribbit;
    } else {
      retVal = 'This user has no Ribbits';
    }

    return retVal;
  }
});

Two of Meteor’s principles state that access to the database is done using the same API on both the server and the client and on the client side there is a model simulation, which should look like a zero-latency connection to the database (latency compensation). These principles can be seen in the above code, on line one. We create a Ribbits instance of a Meteor collection, which is the client side database. Since on the server side, we don’t have a ribbits collection, the one defined in our code will be empty.

Next, a helpers function is defined, which takes as a parameter, an object with a function for each Handlebars variable in the template. As you can see, for each one, the appropriate data is returned.

The fullName and userName are both taken from the users database profile object.

The noOfRibbits and the lastRibbit should be taken from the Ribbits collection. Since this is empty at the moment, we expect the functions above to return zero as the number of ribbits and the message, ‘This user has no Ribbits’, instead of the last ribbit.

Changing the Your Ribbit Buddies Section

Let’s take a look at the code. First the template:

<!-- client/buddies.html -->
<div class="panel left">
<h1>Your Ribbit Buddies</h1>
{{#each ribbits}}
<div class="ribbitWrapper">
  <img class="avatar" src="gfx/user2.png">
  <span class="name">{{buddyFullName}}</span> @{{buddyUserName}} <span class="time">15m</span>
  <p>
  {{ribbit}}
  </p>
</div>
{{/each}}

The new thing here is that we use Handlebars’ each to loop through all the ribbits returned from the JavaScript function. The rest is pretty similar to the code inside of the profile section.

Now for the JavaScript code:

// client/main.js
...
buddyFullName: function () {
  Ribbits.find().forEach(function (ribbit) {
    var theUser = Meteor.users.findOne({_id: ribbit.user_id});

    return theUser.profile.name;
  });
},

buddyUserName: function () {
  Ribbits.find().forEach(function (ribbit) {
    var theUser = Meteor.users.findOne({_id: ribbit.user_id});

    return theUser.username;
  });
},

ribbits: function () {
  return Ribbits.find();
}

In this area we want to display all of the ribbits belonging to the current logged in user and all of the users which are followed by the current logged in user. We will implement on the server, a mechanism for returning only the ribbits to the client, that fulfill the above condition. In the client, the Ribbits collection is scanned and for each, a user with the same id as the one stored in the ribbit collection is searched. Then the full name and the username are returned to the template. As for the ribbits, since we already have the ones we are interested in, they can just be returned entirely to the template.

Creating a Ribbit

When adding a ribbit, there are two main tasks that should be completed:

First we should insert a record into the ribbits collection.
Next, update the buddies page with the following information: number of
ribbits, the last ribbit in the “Your Ribbit Profile” section, and the last
ribbit, which could have been posted by any user, along with its author and the time elapsed since it was posted.

It’s now time to write a little bit of server code. Let’s start with the following:

// server/server.js
Ribbits = new Meteor.Collection('ribbits');
Meteor.publish('ribbits', function () {
  return Ribbits.find({});
});

For the above code, we create a Meteor collection instance for the ribbits collection and we publish all the ribbits data to the client.

Now for the client side code:

// client/client.js
Ribbits = new Meteor.Collection('ribbits');
Meteor.subscribe('ribbits');

...

// handling the click event on the Ribbit button
Template.buddiescontent.events({
  'click #createTheRibbit': function (event, template) {
    var ribbitContent= template.find('.ribbitText').value;

    Ribbits.insert({
      user_id: Meteor.user()._id,
      ribbit: ribbitContent,
      created_at: new Date()
    });
    template.find('.ribbitText').value = "";
  }
});

Just like the server side, the client also holds a Ribbits instance of the ribbits collection, in order to support the concept of “Data everywhere”. Additionally, it must also subscribe to the data published by the server in order to make sure that modifications done on one instance of the application are replicated everywhere.

After that, the click event for the Ribbit! button is handled like so: the data inserted into the text field is read and inserted into the ribbits collection. Afterwards, the content of the text field is set to an empty string.

// client/main.js
ribbits: function () {
  return Ribbits.find({}, {sort: {created_at: -1}});
},

buddyFullName: function (ribbitUserId) {
  var theUser = Meteor.users.findOne({_id: ribbitUserId});
  return theUser.profile.name;
},

buddyUserName: function (ribbitUserId) {
  var theUser = Meteor.users.findOne({_id: ribbitUserId});
  return theUser.username;
},

elapsedTime: function (text) {
  var currentDate = new Date(),
    ribbitDate,
    minutes_elapsed,
    hours_elapsed,
    days_elapsed,
    retVal,
    record = Ribbits.findOne({ribbit: text});

  ribbitDate = new Date(record.created_at);
  minutes_elapsed = (currentDate - ribbitDate) / 60000;
  if (minutes_elapsed > 60) {
    hours_elapsed = minutes_elapsed / 60;
    if (hours_elapsed > 24) {
      days_elapsed = hours_elapsed / 24;
      retVal = parseInt(days_elapsed, 10) + "d";
    } else {
      retVal = parseInt(hours_elapsed, 10) + "h";
    }
  } else {
    retVal = parseInt(minutes_elapsed, 10) + "m";
  }
  return retVal;
}

Now the above code injects dynamic data into the Handlebars’ template variables. Allow me to explain:

The ribbits value is replaced by a collection instance with all of the values stored in the database, in the reverse order of creation.
The buddyFullName is returned by searching the users collection for a user with the same id as the current ribbit’s user. Notice that the ribbitUserId is a parameter received by the function from the template.
The buddyUserName is obtained in a similar manner as the buddyFullName.
The elapsedTime is calculated based on the current time and the time of when a ribbit was originally created.

The buddies template now looks like the following:

<!-- client/buddies.html -->
<div class="panel left">
  <h1>Your Ribbit Buddies</h1>
  {{#each ribbits}}
  <div class="ribbitWrapper">
    <img class="avatar" src="gfx/user2.png">
    <span class="name">{{buddyFullName user_id}}</span> @{{buddyUserName user_id}} <span class="time">{{elapsedTime ribbit}}</span>
    <p>
    {{ribbit}}
    </p>
  </div>
  {{/each}}
</div>

What is interesting to notice here, is that the buddyFullName is followed by the user_id variable taken from the ribbits collection. This is the parameter from the function we described above.

Step 6: Creating the Public Ribbits Page

This task shouldn’t be a show stopper since we already did something similar with the buddies page. What needs to be done here is to take the public.html file from the downloaded assets and turn it into a template. We’ll then hook the data that we want displayed on the page, to some JavaScript functions, that will take that data from the database and return it to the template.

First, let’s see the template file:

<!-- client/public.html -->
<template name="public">
  <div id="content">
    <div class="wrapper">
      <div class="panel right">
        <h1>Create a Ribbit</h1>
        <p>
          <textarea name="text" class="ribbitText"></textarea>
          <input type="submit" value="Ribbit!">
        </p>
      </div><!-- panel right -->
      <div id="ribbits" class="panel left">
        <h1>Public Ribbits</h1>
        {{#each ribbits}}
          <div class="ribbitWrapper">
            <img class="avatar" src="gfx/user2.png">
            <span class="name">{{publicUserFull user_id}}</span> @{{publicUserName user_id}} <span class="time">{{elapsedTime ribbit}}</span>
            <p>
              {{ribbit}}
            </p>
          </div><!-- ribbitWrapper -->
        {{/each}}
      </div><!-- panel left -->
    </div><!-- wrapper -->
  </div><!-- content -->
</template>

Nothing spectacular here: in the Public Ribbits section there is the publicUserFull and publicUserName displayed as well as the ribbit user_id is sent to the function connected to these.

The JavaScript code looks like following:

// client/client.js
...
Template.public.helpers({
  ribbits: function () {
    return Ribbits.find({}, {sort: {created_at: -1}});
  },

  publicUserFull: function (currentRibbitId) {
    var theUser = Meteor.users.findOne({_id: currentRibbitId});

    return theUser.profile.name;
  },

  publicUserName: function (currentRibbitId) {
    var theUser = Meteor.users.findOne({_id: currentRibbitId});

    return theUser.username;
  },

  elapsedTime: function (text) {
    var currentDate = new Date(),
      ribbitDate,
      minutes_elapsed,
      hours_elapsed,
      days_elapsed,
      retVal,
      record = Ribbits.findOne({ribbit: text});

    ribbitDate = new Date(record.created_at);
    minutes_elapsed = (currentDate - ribbitDate) / 60000;
    if (minutes_elapsed > 60) {
      hours_elapsed = minutes_elapsed / 60;
      if (hours_elapsed > 24) {
        days_elapsed = hours_elapsed / 24;
        retVal = parseInt(days_elapsed, 10) + "d";
      } else {
        retVal = parseInt(hours_elapsed, 10) + "h";
      }
    } else {
      retVal = parseInt(minutes_elapsed, 10) + "m";
    }
    return retVal;
  }
});
...

Notice that all of the functions in this code fragment are similar to the ones defined in the buddies part of the file. You should see that we have massive code repetition going on here, which is not so good. But for the moment, let’s leave it as is, in order to see that it works and we’ll figure out how we can refactor this in a later step.

Step 7: Display the Public Ribbits Page

So we have a functional public Ribbits page, but there’s no way to display it. This is what we’ll fix in this step.

In order to perform this task, we will use Meteor’s reactive data concept. We already saw this in action on the buddies page; when a new ribbit is added, it is automatically displayed on the page.

What we want to achieve is that whenever a user clicks on the Public Ribbits link in the header, to change from the current page that is being displayed, to the Public Ribbits page. So we need a currentPage variable, that we can change periodically, so it points to the desired page.

We will use the Meteor’s Session global object, which contains an arbitrary set of key-value pairs. We will store a string in a currentPage variable, which points to the name of the page that we want to display. Then when clicking on a link, we change the currentPage variable as appropriate to display the new page.

Let’s get started on the code. First, the main template file, index.html:

<!-- client/index.html -->
<head>

</head>

<body>
  <header>
    {{> header}}
  </header>
  {{> content}}
  <footer>
    {{> footer}}
  </footer>
</body>

<template name="content">
  {{#if currentUser}}
    {{#if currentPage "buddies"}}
      {{> buddiescontent}}
    {{/if}}

    {{#if currentPage "public"}}
      {{> public}}
    {{/if}}

    {{#if currentPage "profile"}}
      {{> profile}}
    {{/if}}
  {{else}}
    {{> homecontent}}
  {{/if}}
</template>

The changes in this file are small. A new template named content was added and in this template the value of the currentPage variable is checked and the appropriate sub-template is rendered.

Let’s see how this reflects in the JavaScript code:

// client/main.js
Template.content.helpers({
  currentPage: function (type) {
    var thePage = Session.get("currentPage");
    return thePage === type;
  }
});

A helper for the content template was added, containing a currentPage function, which returns a boolean by checking the parameter sent to this function against the value of the currentPage, stored in the Meteor’s Session. The function returns true if they are the same or false if they are not. This way its value can be checked in the template to decide which sub-template is rendered.

// client/client.js
Session.set("currentPage", "buddies");

Now for the client.js file, the value of currentPage is initialized at the beginning of the application by storing it in buddies, so by default, if a user is logged in, the buddies page is displayed.

// client/client.js
...
'click #public': function (event, template) {
  Session.set("currentPage", "public");
},

'click #buddies': function (event, template) {
  Session.set("currentPage", "buddies");
},
...

Then, the header events object is updated by adding two click handlers, one for the Public Ribbits link and one for the Your Buddies link, so that the user can navigate to these pages:

As you can see in this function, the only thing we do is set the Session value of currentPage to the desired value.

And that’s it. Now it’s possible to switch between buddies and the public page using the links in the page’s header.

Step 8: Creating the Profiles Page

As you might have noticed, there is a pattern we’ve followed throughout this tutorial while creating each new page: we started with the implementation of the page template, then we added the necessary JavaScript code for making the page dynamic. Lastly, we add in some event handlers for the actions that the user may take on the page. Let’s follow this same pattern now for creating the profiles page.

The Profile Template

The profile template code should look like this:

<template name="profile">
  <div id="content">
    <div class="wrapper">
      <div class="panel right">
        <h1>Search for profiles</h1>
        <p>
          <input name="query" type="text">
          <input type="submit" value="Search!">
        </p>
      </div>
      <div id="ribbits" class="panel left">
        <h1>Public Profiles</h1>
        {{#each users}}
          <div class="ribbitWrapper">
            <img class="avatar" src="{{gravatarLink _id}}">
            <span class="name">{{profile.name}}</span> @{{username}} <span class="time">{{noOfFollowers _id}}<a href="#" id='{{_id}}' class='follow'>{{followText _id}}</a></span>
            <p id='last-ribbit'>
              {{lastRibbit _id}}
            </p>
          </div>
        {{/each}}
      </div>
    </div>
  </div>
</template>

For this template, our page is split into two panels: the right panel allows searching for a specific user profile and on the left panel we have a list of registered users. Also on the left panel, the logged in user has the possibility to follow or unfollow another user by clicking the follow/unfollow link.

Since the left panel’s code looks more interesting, let’s start analyzing it first: after the header part, we iterate over the Meteor users collection using a Handlebars #each construction.

For each user we display the following information:

The user’s gravatar (retrieved just as we did on the buddies page).
The user’s name, taken from the mongoDb database.
The user’s username.
The user’s number of follows.
The follow/unfollow link.
The user’s last ribbit.

As for the right panel, this code contains a form that allows the user to search for a specific profile.

Now, let’s move on to the second step of our pattern by adding in the JavaScript.

Getting the Profile Data

The first thing the template needs is the users collection. Let’s take a look at the code that returns this collection:

users: function () {
  if (Session.get('searchedName') !== undefined) {
    return Meteor.users.find({
      $and: [
        {_id: {$ne: Meteor.userId()}},
        {username: Session.get('searchedName')}
      ]
    });
  } else {
    return Meteor.users.find({_id: {$ne: Meteor.userId()}});
  }
},

The above code takes care of the profile search functionality for our app. In short, the collection returned to the template is filtered to contain only the searched user and if no search was performed, all of the users are returned, except the current logged in user.

Let me explain this in more detail: When a search for a specific user profile is performed, as we will see later, we set a profile variable named searchedName, which hold the name that the user is searching for. We then filter the users collection using the $and Mongo construction to meet the following criteria: It should only return users that are not the current logged in user and inside of username we’ll store the searchedName. I have to admit, that I am simplifying things a bit here, since I could implement a like clause instead to return all users with a username similar to the searchedName. But since this is not a Mongo tutorial, I’m going to leave this as an additional exercise for you to try out on your own.

By default, this should display all of the users except the current
logged in user, which is done in the else clause.

As for obtaining the gravatar image, the code is similar to what we used in the buddies template, so I won’t describe it again here.

The same also applies for the name, username, and noOfFollowers.

Now for the followText part, we are going to do things a little differently. The behavior we intend to have here is to display the ‘follow’ text if the current logged in user is not already following the displayed user. Otherwise, we want to allow the user to stop following the user by displaying the text ‘unfollow’.

The code looks like following:

followText: function (userId) {
  var followee = Follows.findOne({
    $and: [
      {followee_id: Meteor.userId()},
      {user_id: userId}
    ]
  });
  if (followee) {
    return 'unfollow';
  } else {
    return 'follow';
  }
},

This function receives the userId parameter from the template and performs a search in the Follows table for a user that has a followee_id which matches the current user’s id and which also has a user_id, which matches the id of the user who was clicked on. If this criteria is met, that means the user is being followed by the current logged in user, so return the ‘unfollow’ string, otherwise return the ‘follow’ string.

Actions on the Profile Page

There are really only two actions users can make on the Profile page: clicking on the Search! button, to search for other user profiles. And the second action is to follow or unfollow another user by clicking the appropriate link.

Let’s tackle them one by one:

Searching a User

'click input[type="submit"]': function(event, template) {
  var searchedUser = template.find('input[type="text"]').value;
  if (searchedUser !== "") {
    Session.set('searchedName', searchedUser);
  } else {
    Session.set('searchedName', undefined);
  }
  Template.profile();
}

When the user clicks on our Search! button (the input field with a type of submit), we read the name entered into the text field and set a session variable named searchedName to hold this value. We’ve already saw from the previous step how this variable is used to perform a search in the database and return a subset of the users collection containing only users with that username.

Following a User

'click .follow': function(event, template) {
  var isFollowed, theClickedUserId = event.currentTarget.id,
    theFollowees = Follows.find({user_id: theClickedUserId});

  theFollowees.forEach(function (theFollowee) {
    if (theFollowee.followee_id === Meteor.userId()) {
      isFollowed = true;
    } else {
      isFollowed = false;
    }
  });

  if (!isFollowed) {
    Follows.insert({
      user_id: theClickedUserId,
      followee_id: Meteor.userId()
    });
  } else {
    Follows.remove({
     $and: [
       {user_id: theClickedUserId},
       {followee_id: Meteor.user()._id}
      ]
    });
  }
},

We start off with the theClickedUserId variable, which will store the id of the user who was clicked on. Then we search in the Follows collection for all of the users with that id and store the result in the theFollowees variable.

The next step is to loop through the theFollowees collection and check if the current followee has the same followee_id as the current logged in user’s id. If so, the logged in user is following the clicked on user.

Finally, if the user is not followed, just add it to the Follows collection, which will result in following that user, otherwise, remove it from the Follows collection, to unfollow that user.

Conclusion

This concludes the Profile page implementation and also this tutorial. I hope you will find it interesting and discover that working with Meteor indeed changes the way JavaScript applications are being developed. If you have any questions feel free to ask away in the comments below!

Tuts+ Jobs is now free!

Joel Bankhead — Fri, 14 Jun 2013 15:12:08 +0000

Our awesome new job board is now free and full of enticing opportunities!

Tuts+ Jobs is a job board for full time, part time and casual employment opportunities for web and creative professionals. A brand new site to go alongside the Tuts+ Educational Network and the Envato Marketplaces, all run by Envato.

There’s no need to sign up to apply for jobs, and it’s now free to post a job – try it now!

Earn and Learn

Envato is committed to helping people earn and learn online. We provide a wealth of educational material through Tuts+ and Tuts+ Premium, alongside the Envato Marketplaces to help you benefit from the creative skills you learn.

Tuts+ Jobs furthers this vision perfectly. With Tuts+ you can learn the skills you need to become a creative professional, and with Tuts+ Jobs you can find an amazing job that uses those skills. Our goal here is to take the hassle out of finding a job, so we’ve made the process as simple as can be!

Finding a Job

With Tuts+ Jobs we’re committed to making every step of the job-finding process more intuitive, simpler and more efficient. We don’t want to stand between you and a great opportunity.

It’s totally free to sign up and start applying for jobs, and if nothing matches your initial search you can receive notifications when jobs come up that match your criteria. We will have full time, part time and casual listings from all around the world. Check the listing to find out where the job is based, as some jobs may offer remote opportunities.

Posting a Job

Posting a job is now free!

If you have a position that you need to fill, or a great part/full-time opportunity, then Tuts+ Jobs is the best way to find a talented and creative individual to do that job. Each 30-day listing is completely free, regardless of the type of job you’re posting.

Your job listing will be promoted across the entire Tuts+ network. With exposure to our ten million visitors over the course of 30 days, it’s a brilliant way to reach exactly the type of audience you’re looking for.

Sign Up, Explore, and Subscribe

There are lots of great opportunities already on Tuts+ Jobs, and we’ll be posting even more exciting job positions over the next few weeks, so now is the best time to head over to the site and subscribe to a job search that interests you. That way you’ll be the first to know when any new listings show up that match your skill set and requirements.

Check out Tuts+ Jobs

Combining Laravel 4 and Backbone

Conar Welsh — Fri, 14 Jun 2013 01:33:16 +0000

For this tutorial, we’re going to be building a single page app using Laravel 4 and Backbone.js. Both frameworks make it very easy to use a different templating engine other than their respective default, so we’re going to use Mustache, which is an engine that is common to both. By using the same templating language on both sides of our application, we’ll be able to share our views betweem them, saving us from having to repeat our work multiple times.

Our Backbone app will be powered by a Laravel 4 JSON API which we’ll develop together. Laravel 4 comes with some new features that make the development of this API very easy. I’ll show you a few tricks along the way to allow you to stay a bit more organized.

All of our dependencies will be managed by Package Managers, there will be no manual downloading or updating of libraries for this application! In addition, I’ll be showing you how to leverage a little extra power from some of our dependencies.

For this project we’ll be using:

Laravel 4: A great PHP framework.
Mustache.php: The PHP rendering engine for Mustache.
Mustache.js: The JavaScript rendering engine for Mustache.
Jeffrey Way's Generators for Laravel 4: We can improve our workflow by generating some boilerplate code for us using these generators.
Twitter Bootstrap: A front-end library to aid in our styling.
PHPUnit: A PHP testing suite.
Mockery: Used for mocking PHP Objects while testing.
Backbone.js: A Javascript MVC for our single page app.
Underscore.js: A dependency of Backbone, and a great little toolkit of functions.

To complete this tutorial, you’ll need the following items installed:

Composer: You can download this from the homepage, I recommend the global install instructions located here.
Node + NPM: the installer on the homepage will install both items.
LESS Compiler: If you’re on a Mac, I recommend CodeKit. However, regardless of your operating system, or if you do not feel like paying for CodeKit, you can just install the LESS Compiler for Node.js by typing npm install -g less at the command prompt.

Part 1: The Base Architecture

First things first, we need to get our application setup before we can begin adding our business logic to it. We’ll do a basic setup of Laravel 4 and get all of our dependencies installed using our Package Managers.

Git

Let's start by creating a git repository to work in. For your reference, this entire repo will be made publicly available at https://github.com/conarwelsh/nettuts-laravel4-and-backbone.

mkdir project && cd project
git init

Laravel 4 Installation

Laravel 4 uses Composer to install all of its dependencies, but first we’ll need an application structure to install into. The “develop” branch on Laravel's Github repository is the home for this application structure. However, at the time of writing this article, Laravel 4 was still in beta, so I needed to be prepared for this structure to change at any time. By adding Laravel as a remote repository, we can pull in these changes whenever we need to. In fact, while something is in beta-mode, it’s a good practice to run these commands after each composer update. However, Laravel 4 is now the newest, stable version.

git remote add laravel https://github.com/laravel/laravel
git fetch laravel
git merge laravel/develop
git add . && git commit -am "commit the laravel application structure"

So we have the application structure, but all of the library files that Laravel needs are not yet installed. You’ll notice at the root of our application there’s a file called composer.json. This is the file that will keep track of all the dependencies that our Laravel application requires. Before we tell Composer to download and install them, let's first add a few more dependencies that we’re going to need. We’ll be adding:

Jeffrey Way's Generators: Some very useful commands to greatly improve our workflow by automatically generating file stubs for us.
Laravel 4 Mustache: This will allow us to seamlessly use Mustache.php in our Laravel project, just as we would Blade.
Twitter Bootstrap: We’ll use the LESS files from this project to speed up our front-end development.
PHPUnit: We’ll be doing some TDD for our JSON API, PHPUnit will be our testing engine.
Mockery: Mockery will help us "mock" objects during our testing.

PHPUnit and Mockery are only required in our development environment, so we’ll specify that in our composer.json file.

composer.json

{
  "require": {
    "laravel/framework": "4.0.*",
    "way/generators": "dev-master",
    "twitter/bootstrap": "dev-master",
    "conarwelsh/mustache-l4": "dev-master"
  },
  "require-dev": {
    "phpunit/phpunit": "3.7.*",
    "mockery/mockery": "0.7.*"
  },
  "autoload": {
    "classmap": [
      "app/commands",
      "app/controllers",
      "app/models",
      "app/database/migrations",
      "app/database/seeds",
      "app/tests/TestCase.php"
    ]
  },
  "scripts": {
    "post-update-cmd": "php artisan optimize"
  },
  "minimum-stability": "dev"
}

Now we just need to tell Composer to do all of our leg work! Below, notice the --dev switch, we’re telling composer that we’re in our development environment and that it should also install all of our dependencies listed in "require-dev".

composer install --dev

After that finishes installing, we’ll need to inform Laravel of a few of our dependencies. Laravel uses "service providers" for this purpose. These service providers basically just tell Laravel how their code is going to interact with the application and to run any necessary setup procedures. Open up app/config/app.php and add the following two items to the "providers" array. Not all packages require this, only those that will enhance or change the functionality of Laravel.

app/config/app.php

...

'Way\Generators\GeneratorsServiceProvider',
'Conarwelsh\MustacheL4\MustacheL4ServiceProvider',

...

Lastly, we just need to do some generic application tweaks to complete our Laravel installation. Let's open up bootstrap/start.php and tell Laravel our machine name so that it can determine what environment it’s in.

bootstrap/start.php

/*
|--------------------------------------------------------------------------
| Detect The Application Environment
|--------------------------------------------------------------------------
|
| Laravel takes a dead simple approach to your application environments
| so you can just specify a machine name or HTTP host that matches a
| given environment, then we will automatically detect it for you.
|
*/

$env = $app->detectEnvironment(array(

  'local' => array('your-machine-name'),

));

Replace "your-machine-name" with whatever the hostname for your machine is. If you are unsure of what your exact machine name is, you can just type hostname at the command prompt (on Mac or Linux), whatever it prints out is the value that belongs in this setting.

We want our views to be able to be served to our client from a web request. Currently, our views are stored outside of our public folder, which would mean that they are not publicly accessible. Luckily, Laravel makes it very easy to move or add other view folders. Open up app/config/view.php and change the paths setting to point to our public folder. This setting works like the PHP native include path, it will check in each folder until it finds a matching view file, so feel free to add several here:

app/config/view.php

'paths' => array(__DIR__.'/../../public/views'),

Next you will need to configure your database. Open up app/config/database.php and add in your database settings.

Note: It is recommended to use 127.0.0.1 instead of localhost. You get a bit of a performance boost on most systems, and with some system configurations, localhost will not even connect properly.

Finally, you just need to make sure that your storage folder is writable.

chmod -R 755 app/storage

Laravel is now installed, with all of its dependencies, as well as our own dependencies. Now let's setup our Backbone installation!

Just like our composer.json installed all of our server-side dependencies, we’ll create a package.json in our public folder to install all of our client-side dependencies.

For our client-side dependencies we’ll use:

Underscore.js: This is a dependency of Backbone.js, and a handy toolbelt of functions.
Backbone.js: This is our client-side MVC that we’ll use to build out our application.
Mustache.js: The Javascript version of our templating library, by using the same templating language both on the client and the server, we can share views, as opposed to duplicating logic.

public/package.json

{
  "name": "nettuts-laravel4-and-backbone",
  "version": "0.0.1",
  "private": true,
  "dependencies": {
    "underscore": "*",
    "backbone": "*",
    "mustache": "*"
  }
}

Now just switch into your public folder, and run npm install. After that completes, lets switch back to our application root so we’re prepared for the rest of our commands.

cd public
npm install
cd ..

Package managers save us from a ton of work, should you want to update any of these libraries, all you have to do is run npm update or composer update. Also, should you want to lock any of these libraries in at a specific version, all you have to do is specify the version number, and the package manager will handle the rest.

To wrap up our setup process we’ll just add in all of the basic project files and folders that we’ll need, and then test it out to ensure it all works as expected.

We’ll need to add the following folders:

public/views
public/views/layouts
public/js
public/css

And the following files:

public/css/styles.less
public/js/app.js
public/views/app.mustache

To accomplish this, we can use a one-liner:

mkdir public/views public/views/layouts public/js public/css && touch public/css/styles.less public/js/app.js public/views/app.mustache

Twitter Bootstrap also has two JavaScript dependencies that we’ll need, so let's just copy them from the vendor folder into our public folder. They are:

html5shiv.js: allows us to use HTML5 elements without fear of older browsers not supporting them
bootstrap.min.js: the supporting JavaScript libraries for Twitter Bootstrap

cp vendor/twitter/bootstrap/docs/assets/js/html5shiv.js public/js/html5shiv.js
cp vendor/twitter/bootstrap/docs/assets/js/bootstrap.min.js public/js/bootstrap.min.js

For our layout file, the Twitter Bootstrap also provides us with some nice starter templates to work with, so let's copy one into our layouts folder for a head start:

cp vendor/twitter/bootstrap/docs/examples/starter-template.html public/views/layouts/application.blade.php

Notice that I am using a blade extension here, this could just as easily be a mustache template, but I wanted to show you how easy it is to mix the templating engines. Since our layout will be rendered on page load, and will not need to be re-rendered by the client, we are safe to use PHP here exclusively. If for some reason you find yourself needing to render this file on the client-side, you would want to switch this file to use the Mustache templating engine instead.

Now that we have all of our basic files in place, let's add some starter content that we can use to test that everything is working as we would expect. I’m providing you with some basic stubs to get you started.

public/css/styles.less

We’ll just import the Twitter Bootstrap files from the vendor directory as opposed to copying them. This allows us to update Twitter Bootstrap with nothing but a composer update.

We declare our variables at the end of the file, the LESS compiler will figure out the value of all of its variables before parsing the LESS into CSS. This means that by re-defining a Twitter Bootstrap variable at the end of the file, the value will actually change for all of the files included, allowing us to do simple overrides without modifying the Twitter Bootstrap core files.

/**
 * Import Twitter Bootstrap Base File
 ******************************************************************************************
 */
@import "../../vendor/twitter/bootstrap/less/bootstrap";
/**
 * Define App Styles
 * Do this before the responsive include, so that it can override properly as needed.
 ******************************************************************************************
 */
body {
  padding-top: 60px; /* 60px to make the container go all the way to the bottom of the topbar */
}

/* this will set the position of our alerts */
#notifications {
  width: 300px;
  position: fixed;
  top: 50px;
  left: 50%;
  margin-left: -150px;
  text-align: center;
}

/**
 * Import Bootstrap's Responsive Overrides
 * now we allow bootstrap to set the overrides for a responsive layout
 ******************************************************************************************
 */
@import "../../vendor/twitter/bootstrap/less/responsive";
/**
 * Define our variables last, any variable declared here will be used in the includes above
 * which means that we can override any of the variables used in the bootstrap files easily
 * without modifying any of the core bootstrap files
 ******************************************************************************************
 */

// Scaffolding
// -------------------------
@bodyBackground:    #f2f2f2;
@textColor:       #575757;

// Links
// -------------------------
@linkColor:       #41a096;

// Typography
// -------------------------
@sansFontFamily:    Arial, Helvetica, sans-serif;

public/js/app.js

Now we’ll wrap all of our code in an immediately-invoking-anonymous-function that passes in a few global objects. We’ll then alias these global objects to something more useful to us. Also, we’ll cache a few jQuery objects inside the document ready function.

//alias the global object
//alias jQuery so we can potentially use other libraries that utilize $
//alias Backbone to save us on some typing
(function(exports, $, bb){

  //document ready
  $(function(){

    /**
     ***************************************
     * Cached Globals
     ***************************************
     */
    var $window, $body, $document;

    $window  = $(window);
    $document = $(document);
    $body   = $('body');
  });//end document ready

}(this, jQuery, Backbone));

public/views/layouts/application.blade.php

Next is just a simple HTML layout file. We’re however using the asset helper from Laravel to aid us in creating paths to our assets. It is good practice to use this type of helper, because if you ever happen to move your project into a sub-folder, all of your links will still work.

We made sure that we included all of our dependencies in this file, and also added the jQuery dependency. I chose to request jQuery from the Google CDN, because chances are the visiting user of this site will already have a copy from that CDN cached in their browser, saving us from having to complete the HTTP request for it.

One important thing to note here is the way in which we are nesting our view. Mustache does not have Block Sections like Blade does, so instead, the contents of the nested view will be made available under a variable with the name of the section. I will point this out when we render this view from our route.

<!DOCTYPE html>
<html lang="en">
<head>
 <meta charset="utf-8">
 <title>Laravel4 & Backbone | Nettuts</title>
 <meta name="viewport" content="width=device-width, initial-scale=1.0">
 <meta name="description" content="A single page blog built using Backbone.js, Laravel, and Twitter Bootstrap">
 <meta name="author" content="Conar Welsh">

 <link href="{{ asset('css/styles.css') }}" rel="stylesheet">

 <!-- HTML5 shim, for IE6-8 support of HTML5 elements -->
 <!--[if lt IE 9]>
 <script src="{{ asset('js/html5shiv.js') }}"></script>
 <![endif]-->
</head>
<body>

 <div id="notifications">
 </div>

 <div class="navbar navbar-inverse navbar-fixed-top">
  <div class="navbar-inner">
   <div class="container">
    <button type="button" class="btn btn-navbar" data-toggle="collapse" data-target=".nav-collapse">
     <span class="icon-bar"></span>
     <span class="icon-bar"></span>
     <span class="icon-bar"></span>
    </button>
    <a class="brand" href="#">Nettuts Tutorial</a>
    <div class="nav-collapse collapse">
     <ul class="nav">
      <li class="active"><a href="#">Blog</a></li>
     </ul>
    </div><!--/.nav-collapse -->
   </div>
  </div>
 </div>

 <div class="container" data-role="main">
  {{--since we are using mustache as the view, it does not have a concept of sections like blade has, so instead of using @yield here, our nested view will just be a variable that we can echo--}}

  {{ $content }}

 </div> <!-- /container -->

 <!-- Placed at the end of the document so the pages load faster -->
 <script src="//ajax.googleapis.com/ajax/libs/jquery/1.9.1/jquery.min.js"></script> <!-- use Google CDN for jQuery to hopefully get a cached copy -->
 <script src="{{ asset('node_modules/underscore/underscore-min.js') }}"></script>
 <script src="{{ asset('node_modules/backbone/backbone-min.js') }}"></script>
 <script src="{{ asset('node_modules/mustache/mustache.js') }}"></script>
 <script src="{{ asset('js/bootstrap.min.js') }}"></script>
 <script src="{{ asset('js/app.js') }}"></script>
 @yield('scripts')
</body>
</html>

public/views/app.mustache

Next is just a simple view that we’ll nest into our layout.

<dl>
  <dt>Q. What did Biggie say when he watched inception?</dt>
  <dd>A. "It was all a dream!"</dd>
</dl>

app/routes.php

Laravel should have already provided you with a default route, all we’re doing here is changing the name of the view which that route is going to render.

Remember from above, I told you that the nested view was going to be available under a variable named whatever the parent section was? Well, when you nest a view, the first parameter to the function is the section name:

View::make('view.path')->nest($sectionName, $nestedViewPath, $viewVariables);

In our nest command we called the section "content", that means if we echo $content from our layout, we’ll get the rendered contents of that view. If we were to do return View::make('layouts.application')->nest('foobar', 'app'); then our nested view would be available under a variable named $foobar.

<?php

//backbone app route
Route::get('/', function()
{
  //change our view name to the view we created in a previous step
  //notice that we do not need to provide the .mustache extension
  return View::make('layouts.application')->nest('content', 'app');
});

With all of our basic files in place, we can test to ensure everything went OK. Laravel 4 utilizes the new PHP web server to provide us with a great little development environment. So long to the days of having a million virtual hosts setup on your development machine for every project that you work on!

Note: make sure that you’ve compiled your LESS file first!

php artisan serve

If you followed along correctly, you should be laughing hysterically at my horrible sense of humor, and all of our assets should be properly included into the page.

Part 2: Laravel 4 JSON API

Now we’ll build the API that will power our Backbone application. Laravel 4 makes this process a breeze.

API Guidelines

First let's go over a few general guidelines to keep in mind while we build our API:

Status Codes: Responses should reply with proper status codes, fight the temptation to just place an { error: "this is an error message" } in the body of your response. Use the HTTP protocol to its fullest!
- 200: success
- 201: resource created
- 204: success, but no content to return
- 400: request not fulfilled //validation error
- 401: not authenticated
- 403: refusal to respond //wrong credentials, do not have permission (un-owned resource)
- 404: not found
- 500: other error
Resource Methods: Even though controllers will be serving different resources, they should still have very similar behavior. The more predictable your API is, the easier it is to implement and adopt.
- index: Return a collection of resources.
- show: Return a single resource.
- create: Return a form. This form should detail out the required fields, validation, and labels as best as possible. As well as anything else needed to properly create a resource. Even though this is a JSON API, it is very useful to return a form here. Both a computer and a person can parse through this form, and very easily decipher which items are needed to fill out this form successsfully. This is a very easy way to “document” the needs of your API.
- store: Store a new resource and return with the proper status code: 201.
- edit: Return a form filled with the current state of a resource. This form should detail out the required fields, validation, and labels as best as possible. As well as anything else needed to properly edit a resource.
- update: Update an existing resource and return with the proper status code.
- delete: Delete an existing resource and return with the proper status code: 204.

Routing & Versioning

API's are designed to be around for a while. This is not like your website where you can just change its functionality at the drop of a dime. If you have programs that use your API, they are not going to be happy with you if you change things around and their program breaks. For this reason, it’s important that you use versioning.

We can always create a "version two" with additional, or altered functionality, and allow our subscribing programs to opt-in to these changes, rather than be forced.

Laravel provides us with route groups that are perfect for this, place the following code ABOVE our first route:

<?php

//create a group of routes that will belong to APIv1
Route::group(array('prefix' => 'v1'), function()
{
  //... insert API routes here...
});

Generating Resources

We’re going to use Jeffrey Way's generators to generate our resources. When we generate a resource, it will create the following items for us:

Controller
Model
Views (index.blade.php, show.blade.php, create.blade.php, edit.blade.php)
Migration
Seeds

We’re only going to need two resources for this app: a Post resource and a Comment resource.

Note: in a recent update to the generators, I have been receiving a permissions error due to the way my web servers are setup. To remedy this problem, you must allow write permissions to the folder that the generators write the temp file to.

sudo chmod -R 755 vendor/way/generators/src/Way/

Run the generate:resource command

php artisan generate:resource post --fields="title:string, content:text, author_name:string"

php artisan generate:resource comment --fields="content:text, author_name:string, post_id:integer"

You should now pause for a second to investigate all of the files that the generator created for us.

Adjust the Generated Resources

The generate:resource command saved us a lot of work, but due to our unique configuration, we’re still going to need to make some modifications.

First of all, the generator placed the views it created in the app/views folder, so we need to move them to the public/views folder

mv app/views/posts public/views/posts
mv app/views/comments public/views/comments

app/routes.php

We decided that we wanted our API to be versioned, so we’ll need to move the routes that the generator created for us into the version group. We’ll also want to namespace our controllers with the corresponding version, so that we can have a different set of controllers for each version we build. Also the comments resource needs to be nested under the posts resource.

<?php

//create a group of routes that will belong to APIv1
Route::group(array('prefix' => 'v1'), function()
{
  //... insert API routes here...
  Route::resource('posts', 'V1\PostsController'); //notice the namespace
  Route::resource('posts.comments', 'V1\PostsCommentsController'); //notice the namespace, and the nesting
});

//backbone app route
Route::get('/', function()
{
  //change our view name to the view we created in a previous step
  //notice that we do not need to provide the .mustache extension
  return View::make('layouts.application')->nest('content', 'app');
});

Since we namespaced our controllers, we should move them into their own folder for organization, let's create a folder named V1 and move our generated controllers into it. Also, since we nested our comments controller under the posts controller, let's change the name of that controller to reflect the relationship.

mkdir app/controllers/V1
mv app/controllers/PostsController.php app/controllers/V1/
mv app/controllers/CommentsController.php app/controllers/V1/PostsCommentsController.php

We’ll need to update the controller files to reflect our changes as well. First of all, we need to namespace them, and since they are namespaced, any classes outside of that namespace will need to be manually imported with the use statement.

app/controllers/PostsController.php

<?php
//use our new namespace
namespace V1;

//import classes that are not in this new namespace
use BaseController;

class PostsController extends BaseController {

app/controllers/PostsCommentsController.php

We also need to update our CommentsController with our new name: PostsCommentsController

<?php
//use our new namespace
namespace V1;

//import classes that are not in this new namespace
use BaseController;

//rename our controller class
class PostsCommentsController extends BaseController {

Adding in Repositories

By default, repositories are not part of Laravel. Laravel is extremely flexible though, and makes it very easy to add them in. We’re going to use repositories to help us separate our logic for code re-usability, as well as for testing. For now we’ll just get setup to use repositories, we’ll add in the proper logic later.

Let’s make a folder to store our repositories in:

mkdir app/repositories

To let our auto-loader know about this new folder, we need to add it to our composer.json file. Take a look at the updated "autoload" section of our file, and you’ll see that we added in the repositories folder.

composer.json

{
  "require": {
    "laravel/framework": "4.0.*",
    "way/generators": "dev-master",
    "twitter/bootstrap": "dev-master",
    "conarwelsh/mustache-l4": "dev-master"
  },
  "require-dev": {
    "phpunit/phpunit": "3.7.*",
    "mockery/mockery": "0.7.*"
  },
  "autoload": {
    "classmap": [
      "app/commands",
      "app/controllers",
      "app/models",
      "app/database/migrations",
      "app/database/seeds",
      "app/tests/TestCase.php",
      "app/repositories"
    ]
  },
  "scripts": {
    "post-update-cmd": "php artisan optimize"
  },
  "minimum-stability": "dev"
}

Seeding Our Database

Database seeds are a useful tool, they provide us with an easy way to fill our database with some content. The generators provided us with base files for seeding, we merely need to add in some actual seeds.

app/database/seeds/PostsTableSeeder.php

<?php

class PostsTableSeeder extends Seeder {

  public function run()
  {
    $posts = array(
      array(
        'title'    => 'Test Post',
        'content'   => 'Lorem ipsum Reprehenderit velit est irure in enim in magna aute occaecat qui velit ad.',
        'author_name' => 'Conar Welsh',
        'created_at' => date('Y-m-d H:i:s'),
        'updated_at' => date('Y-m-d H:i:s'),
      ),
      array(
        'title'    => 'Another Test Post',
        'content'   => 'Lorem ipsum Reprehenderit velit est irure in enim in magna aute occaecat qui velit ad.',
        'author_name' => 'Conar Welsh',
        'created_at' => date('Y-m-d H:i:s'),
        'updated_at' => date('Y-m-d H:i:s'),
      ),
    );

    // Uncomment the below to run the seeder
    DB::table('posts')->insert($posts);
  }

}

app/database/seeds/CommentsTableSeeder.php

<?php

class CommentsTableSeeder extends Seeder {

  public function run()
  {
    $comments = array(
      array(
        'content'   => 'Lorem ipsum Nisi dolore ut incididunt mollit tempor proident eu velit cillum dolore sed',
        'author_name' => 'Testy McTesterson',
        'post_id'   => 1,
        'created_at' => date('Y-m-d H:i:s'),
        'updated_at' => date('Y-m-d H:i:s'),
      ),
      array(
        'content'   => 'Lorem ipsum Nisi dolore ut incididunt mollit tempor proident eu velit cillum dolore sed',
        'author_name' => 'Testy McTesterson',
        'post_id'   => 1,
        'created_at' => date('Y-m-d H:i:s'),
        'updated_at' => date('Y-m-d H:i:s'),
      ),
      array(
        'content'   => 'Lorem ipsum Nisi dolore ut incididunt mollit tempor proident eu velit cillum dolore sed',
        'author_name' => 'Testy McTesterson',
        'post_id'   => 2,
        'created_at' => date('Y-m-d H:i:s'),
        'updated_at' => date('Y-m-d H:i:s'),
      ),
    );

    // Uncomment the below to run the seeder
    DB::table('comments')->insert($comments);
  }

}

Don’t forget to run composer dump-autoload to let the Composer auto loader know about the new migration files!

composer dump-autoload

Now we can run our migrations and seed the database. Laravel provides us with a single command to do both:

php artisan migrate --seed

Tests

Testing is one of those topics in development that no one can argue the importance of, however most people tend to ignore it due to the learning curve. Testing is really not that difficult and it can dramatically improve your application. For this tutorial, we’ll setup some basic tests to help us ensure that our API is functioning properly. We’ll build this API TDD style. The rules of TDD state that we are not allowed to write any production code until we have failing tests that warrants it. However, if I were to walk you through each test individually, this would prove to be a very long tutorial, so in the interest of brevity, I will just provide you with some tests to work from, and then the correct code to make those tests pass afterwards.

Before we write any tests though, we should first check the current test status of our application. Since we installed PHPUnit via composer, we have the binaries available to us to use. All you need to do is run:

vendor/phpunit/phpunit/phpunit.php

Whoops! We already have a failure! The test that is failing is actually an example test that comes pre-installed in our Laravel application structure, this tests against the default route that was also installed with the Laravel application structure. Since we modified this route, we cannot be surprised that the test failed. We can however, just delete this test altogether as it does not apply to our application.

rm app/tests/ExampleTest.php

If you run the PHPUnit command again, you will see that no tests were executed, and we have a clean slate for testing.

Note: it is possible that if you have an older version of Jeffrey Way's generators that you’ll actually have a few tests in there that were created by those generators, and those tests are probably failing. Just delete or overwrite those tests with the ones found below to proceed.

For this tutorial we’ll be testing our controllers and our repositories. Let's create a few folders to store these tests in:

mkdir app/tests/controllers app/tests/repositories

Now for the test files. We’re going to use Mockery to mock our repositories for our controller tests. Mockery objects do as their name implies, they "mock" objects and report back to us on how those objects were interacted with.

In the case of the controller tests, we do not actually want the repositories to be called, after all, these are the controller tests, not the repository tests. So Mockery will set us up objects to use instead of our repositories, and let us know whether or not those objects were called as we expected them to.

In order to pull this off, we’ll have to tell the controllers to use our "mocked" objects as opposed to the real things. We’ll just tell our Application to use a mocked instance next time a certain class is requested. The command looks like this:

App::instance($classToReplace, $instanceOfClassToReplaceWith);

The overall mocking process will go something like this:

Create a new Mockery object, providing it the name of the class which it is to mock.
Tell the Mockery object which methods it should expect to receive, how many times it should receive that method, and what that method should return.
Use the command shown above to tell our Application to use this new Mockery object instead of the default.
Run the controller method like usual.
Assert the response.

app/tests/controllers/CommentsControllerTest.php

<?php

class CommentsControllerTest extends TestCase {

  /**
   ************************************************************************
   * Basic Route Tests
   * notice that we can use our route() helper here!
   ************************************************************************
   */

  //test that GET /v1/posts/1/comments returns HTTP 200
  public function testIndex()
  {
    $response = $this->call('GET', route('v1.posts.comments.index', array(1)) );
    $this->assertTrue($response->isOk());
  }

  //test that GET /v1/posts/1/comments/1 returns HTTP 200
  public function testShow()
  {
    $response = $this->call('GET', route('v1.posts.comments.show', array(1,1)) );
    $this->assertTrue($response->isOk());
  }

  //test that GET /v1/posts/1/comments/create returns HTTP 200
  public function testCreate()
  {
    $response = $this->call('GET', route('v1.posts.comments.create', array(1)) );
    $this->assertTrue($response->isOk());
  }

  //test that GET /v1/posts/1/comments/1/edit returns HTTP 200
  public function testEdit()
  {
    $response = $this->call('GET', route('v1.posts.comments.edit', array(1,1)) );
    $this->assertTrue($response->isOk());
  }

  /**
   *************************************************************************
   * Tests to ensure that the controller calls the repo as we expect
   * notice we are "Mocking" our repository
   *
   * also notice that we do not really care about the data or interactions
   * we merely care that the controller is doing what we are going to want
   * it to do, which is reach out to our repository for more information
   *************************************************************************
   */

  //ensure that the index function calls our repository's "findAll" method
  public function testIndexShouldCallFindAllMethod()
  {
    //create our new Mockery object with a name of CommentRepositoryInterface
    $mock = Mockery::mock('CommentRepositoryInterface');

    //inform the Mockery object that the "findAll" method should be called on it once
    //and return a string value of "foo"
    $mock->shouldReceive('findAll')->once()->andReturn('foo');

    //inform our application that we have an instance that it should use
    //whenever the CommentRepositoryInterface is requested
    App::instance('CommentRepositoryInterface', $mock);

    //call our controller route
    $response = $this->call('GET', route('v1.posts.comments.index', array(1)));

    //assert that the response is a boolean value of true
    $this->assertTrue(!! $response->original);
  }

  //ensure that the show method calls our repository's "findById" method
  public function testShowShouldCallFindById()
  {
    $mock = Mockery::mock('CommentRepositoryInterface');
    $mock->shouldReceive('findById')->once()->andReturn('foo');
    App::instance('CommentRepositoryInterface', $mock);

    $response = $this->call('GET', route('v1.posts.comments.show', array(1,1)));
    $this->assertTrue(!! $response->original);
  }

  //ensure that our create method calls the "instance" method on the repository
  public function testCreateShouldCallInstanceMethod()
  {
    $mock = Mockery::mock('CommentRepositoryInterface');
    $mock->shouldReceive('instance')->once()->andReturn(array());
    App::instance('CommentRepositoryInterface', $mock);

    $response = $this->call('GET', route('v1.posts.comments.create', array(1)));
    $this->assertViewHas('comment');
  }

  //ensure that the edit method calls our repository's "findById" method
  public function testEditShouldCallFindByIdMethod()
  {
    $mock = Mockery::mock('CommentRepositoryInterface');
    $mock->shouldReceive('findById')->once()->andReturn(array());
    App::instance('CommentRepositoryInterface', $mock);

    $response = $this->call('GET', route('v1.posts.comments.edit', array(1,1)));
    $this->assertViewHas('comment');
  }

  //ensure that the store method should call the repository's "store" method
  public function testStoreShouldCallStoreMethod()
  {
    $mock = Mockery::mock('CommentRepositoryInterface');
    $mock->shouldReceive('store')->once()->andReturn('foo');
    App::instance('CommentRepositoryInterface', $mock);

    $response = $this->call('POST', route('v1.posts.comments.store', array(1)));
    $this->assertTrue(!! $response->original);
  }

  //ensure that the update method should call the repository's "update" method
  public function testUpdateShouldCallUpdateMethod()
  {
    $mock = Mockery::mock('CommentRepositoryInterface');
    $mock->shouldReceive('update')->once()->andReturn('foo');
    App::instance('CommentRepositoryInterface', $mock);

    $response = $this->call('PUT', route('v1.posts.comments.update', array(1,1)));
    $this->assertTrue(!! $response->original);
  }

  //ensure that the destroy method should call the repositories "destroy" method
  public function testDestroyShouldCallDestroyMethod()
  {
    $mock = Mockery::mock('CommentRepositoryInterface');
    $mock->shouldReceive('destroy')->once()->andReturn(true);
    App::instance('CommentRepositoryInterface', $mock);

    $response = $this->call('DELETE', route('v1.posts.comments.destroy', array(1,1)));
    $this->assertTrue( empty($response->original) );
  }
}

app/tests/controllers/PostsControllerTest.php

Next, we’ll follow the exact same procedure for the PostsController tests

<?php

class PostsControllerTest extends TestCase {

  /**
   * Test Basic Route Responses
   */
  public function testIndex()
  {
    $response = $this->call('GET', route('v1.posts.index'));
    $this->assertTrue($response->isOk());
  }

  public function testShow()
  {
    $response = $this->call('GET', route('v1.posts.show', array(1)));
    $this->assertTrue($response->isOk());
  }

  public function testCreate()
  {
    $response = $this->call('GET', route('v1.posts.create'));
    $this->assertTrue($response->isOk());
  }

  public function testEdit()
  {
    $response = $this->call('GET', route('v1.posts.edit', array(1)));
    $this->assertTrue($response->isOk());
  }

  /**
   * Test that controller calls repo as we expect
   */
  public function testIndexShouldCallFindAllMethod()
  {
    $mock = Mockery::mock('PostRepositoryInterface');
    $mock->shouldReceive('findAll')->once()->andReturn('foo');
    App::instance('PostRepositoryInterface', $mock);

    $response = $this->call('GET', route('v1.posts.index'));
    $this->assertTrue(!! $response->original);
  }

  public function testShowShouldCallFindById()
  {
    $mock = Mockery::mock('PostRepositoryInterface');
    $mock->shouldReceive('findById')->once()->andReturn('foo');
    App::instance('PostRepositoryInterface', $mock);

    $response = $this->call('GET', route('v1.posts.show', array(1)));
    $this->assertTrue(!! $response->original);
  }

  public function testCreateShouldCallInstanceMethod()
  {
    $mock = Mockery::mock('PostRepositoryInterface');
    $mock->shouldReceive('instance')->once()->andReturn(array());
    App::instance('PostRepositoryInterface', $mock);

    $response = $this->call('GET', route('v1.posts.create'));
    $this->assertViewHas('post');
  }

  public function testEditShouldCallFindByIdMethod()
  {
    $mock = Mockery::mock('PostRepositoryInterface');
    $mock->shouldReceive('findById')->once()->andReturn(array());
    App::instance('PostRepositoryInterface', $mock);

    $response = $this->call('GET', route('v1.posts.edit', array(1)));
    $this->assertViewHas('post');
  }

  public function testStoreShouldCallStoreMethod()
  {
    $mock = Mockery::mock('PostRepositoryInterface');
    $mock->shouldReceive('store')->once()->andReturn('foo');
    App::instance('PostRepositoryInterface', $mock);

    $response = $this->call('POST', route('v1.posts.store'));
    $this->assertTrue(!! $response->original);
  }

  public function testUpdateShouldCallUpdateMethod()
  {
    $mock = Mockery::mock('PostRepositoryInterface');
    $mock->shouldReceive('update')->once()->andReturn('foo');
    App::instance('PostRepositoryInterface', $mock);

    $response = $this->call('PUT', route('v1.posts.update', array(1)));
    $this->assertTrue(!! $response->original);
  }

  public function testDestroyShouldCallDestroyMethod()
  {
    $mock = Mockery::mock('PostRepositoryInterface');
    $mock->shouldReceive('destroy')->once()->andReturn(true);
    App::instance('PostRepositoryInterface', $mock);

    $response = $this->call('DELETE', route('v1.posts.destroy', array(1)));
    $this->assertTrue( empty($response->original) );
  }

}

app/tests/repositories/EloquentCommentRepositoryTest.php

Now for the repository tests. In writing our controller tests, we pretty much already decided what most of the interface should look like for the repositories. Our controllers needed the following methods:

findById($id)
findAll()
instance($data)
store($data)
update($id, $data)
destroy($id)

The only other method that we’ll want to add here is a validate method. This will mainly be a private method for the repository to ensure that the data is safe to store or update.

For these tests, we’re also going to add a setUp method, which will allow us to run some code on our class, prior to the execution of each test. Our setUp method will be a very simple one, we’ll just make sure that any setUp methods defined in parent classes are also called using parent::setUp() and then simply add a class variable that stores an instance of our repository.

We’ll use the power of Laravel's IoC container again to get an instance of our repository. The App::make() command will return an instance of the requested class, now it may seem strange that we do not just do $this->repo = new EloquentCommentRepository(), but hold that thought, we’ll come back to it momentarily. You probably noticed that we’re asking for a class called EloquentCommentRepository, but in our controller tests above, we were calling our repository CommentRepositoryInterface… put this thought on the back-burner as well… explainations for both are coming, I promise!

<?php

class EloquentCommentRepositoryTest extends TestCase {

  public function setUp()
  {
    parent::setUp();
    $this->repo = App::make('EloquentCommentRepository');
  }

  public function testFindByIdReturnsModel()
  {
    $comment = $this->repo->findById(1,1);
    $this->assertTrue($comment instanceof Illuminate\Database\Eloquent\Model);
  }

  public function testFindAllReturnsCollection()
  {
    $comments = $this->repo->findAll(1);
    $this->assertTrue($comments instanceof Illuminate\Database\Eloquent\Collection);
  }

  public function testValidatePasses()
  {
    $reply = $this->repo->validate(array(
      'post_id'   => 1,
      'content'   => 'Lorem ipsum Fugiat consectetur laborum Ut consequat aliqua.',
      'author_name' => 'Testy McTesterson'
    ));

    $this->assertTrue($reply);
  }

  public function testValidateFailsWithoutContent()
  {
    try {
      $reply = $this->repo->validate(array(
        'post_id'   => 1,
        'author_name' => 'Testy McTesterson'
      ));
    }
    catch(ValidationException $expected)
    {
      return;
    }

    $this->fail('ValidationException was not raised');
  }

  public function testValidateFailsWithoutAuthorName()
  {
    try {
      $reply = $this->repo->validate(array(
        'post_id'   => 1,
        'content'   => 'Lorem ipsum Fugiat consectetur laborum Ut consequat aliqua.'
      ));
    }
    catch(ValidationException $expected)
    {
      return;
    }

    $this->fail('ValidationException was not raised');
  }

  public function testValidateFailsWithoutPostId()
  {
    try {
      $reply = $this->repo->validate(array(
        'author_name' => 'Testy McTesterson',
        'content'   => 'Lorem ipsum Fugiat consectetur laborum Ut consequat aliqua.'
      ));
    }
    catch(ValidationException $expected)
    {
      return;
    }

    $this->fail('ValidationException was not raised');
  }

  public function testStoreReturnsModel()
  {
    $comment_data = array(
      'content'   => 'Lorem ipsum Fugiat consectetur laborum Ut consequat aliqua.',
      'author_name' => 'Testy McTesterson'
    );

    $comment = $this->repo->store(1, $comment_data);

    $this->assertTrue($comment instanceof Illuminate\Database\Eloquent\Model);
    $this->assertTrue($comment->content === $comment_data['content']);
    $this->assertTrue($comment->author_name === $comment_data['author_name']);
  }

  public function testUpdateSaves()
  {
    $comment_data = array(
      'content' => 'The Content Has Been Updated'
    );

    $comment = $this->repo->update(1, 1, $comment_data);

    $this->assertTrue($comment instanceof Illuminate\Database\Eloquent\Model);
    $this->assertTrue($comment->content === $comment_data['content']);
  }

  public function testDestroySaves()
  {
    $reply = $this->repo->destroy(1,1);
    $this->assertTrue($reply);

    try {
      $this->repo->findById(1,1);
    }
    catch(NotFoundException $expected)
    {
      return;
    }

    $this->fail('NotFoundException was not raised');
  }

  public function testInstanceReturnsModel()
  {
    $comment = $this->repo->instance();
    $this->assertTrue($comment instanceof Illuminate\Database\Eloquent\Model);
  }

  public function testInstanceReturnsModelWithData()
  {
    $comment_data = array(
      'title' => 'Un-validated title'
    );

    $comment = $this->repo->instance($comment_data);
    $this->assertTrue($comment instanceof Illuminate\Database\Eloquent\Model);
    $this->assertTrue($comment->title === $comment_data['title']);
  }

}

app/tests/repositories/EloquentPostRepositoryTest.php

<?php

class EloquentPostRepositoryTest extends TestCase {

  public function setUp()
  {
    parent::setUp();
    $this->repo = App::make('EloquentPostRepository');
  }

  public function testFindByIdReturnsModel()
  {
    $post = $this->repo->findById(1);
    $this->assertTrue($post instanceof Illuminate\Database\Eloquent\Model);
  }

  public function testFindAllReturnsCollection()
  {
    $posts = $this->repo->findAll();
    $this->assertTrue($posts instanceof Illuminate\Database\Eloquent\Collection);
  }

  public function testValidatePasses()
  {
    $reply = $this->repo->validate(array(
      'title'    => 'This Should Pass',
      'content'   => 'Lorem ipsum Fugiat consectetur laborum Ut consequat aliqua.',
      'author_name' => 'Testy McTesterson'
    ));

    $this->assertTrue($reply);
  }

  public function testValidateFailsWithoutTitle()
  {
    try {
      $reply = $this->repo->validate(array(
        'content'   => 'Lorem ipsum Fugiat consectetur laborum Ut consequat aliqua.',
        'author_name' => 'Testy McTesterson'
      ));
    }
    catch(ValidationException $expected)
    {
      return;
    }

    $this->fail('ValidationException was not raised');
  }

  public function testValidateFailsWithoutAuthorName()
  {
    try {
      $reply = $this->repo->validate(array(
        'title'    => 'This Should Pass',
        'content'   => 'Lorem ipsum Fugiat consectetur laborum Ut consequat aliqua.'
      ));
    }
    catch(ValidationException $expected)
    {
      return;
    }

    $this->fail('ValidationException was not raised');
  }

  public function testStoreReturnsModel()
  {
    $post_data = array(
      'title'    => 'This Should Pass',
      'content'   => 'Lorem ipsum Fugiat consectetur laborum Ut consequat aliqua.',
      'author_name' => 'Testy McTesterson'
    );

    $post = $this->repo->store($post_data);

    $this->assertTrue($post instanceof Illuminate\Database\Eloquent\Model);
    $this->assertTrue($post->title === $post_data['title']);
    $this->assertTrue($post->content === $post_data['content']);
    $this->assertTrue($post->author_name === $post_data['author_name']);
  }

  public function testUpdateSaves()
  {
    $post_data = array(
      'title' => 'The Title Has Been Updated'
    );

    $post = $this->repo->update(1, $post_data);

    $this->assertTrue($post instanceof Illuminate\Database\Eloquent\Model);
    $this->assertTrue($post->title === $post_data['title']);
  }

  public function testDestroySaves()
  {
    $reply = $this->repo->destroy(1);
    $this->assertTrue($reply);

    try {
      $this->repo->findById(1);
    }
    catch(NotFoundException $expected)
    {
      return;
    }

    $this->fail('NotFoundException was not raised');
  }

  public function testInstanceReturnsModel()
  {
    $post = $this->repo->instance();
    $this->assertTrue($post instanceof Illuminate\Database\Eloquent\Model);
  }

  public function testInstanceReturnsModelWithData()
  {
    $post_data = array(
      'title' => 'Un-validated title'
    );

    $post = $this->repo->instance($post_data);
    $this->assertTrue($post instanceof Illuminate\Database\Eloquent\Model);
    $this->assertTrue($post->title === $post_data['title']);
  }

}

Now that we have all of our tests in place, let's run PHPUnit again to watch them fail!

vendor/phpunit/phpunit/phpunit.php

You should have a whole ton of failures, and in fact, the test suite probably did not even finish testing before it crashed. This is OK, that means we have followed the rules of TDD and wrote failing tests before production code. Although, typically these tests would be written one at a time and you would not move on to the next test until you had code that allowed the previous test to pass. Your terminal should probably look something like mine at the moment:

What’s actually failing is the assertViewHas method in our controller tests. It’s kind of intimidating to deal with this kind of an error when we have lumped together all of our tests without any production code at all. This is why you should always write the tests one at a time, as you’ll find these errors in stride, as opposed to just a huge mess of errors at once. For now, just follow my lead into the implementation of our code.

Sidebar Discussion

Before we proceed with the implementations, let's break for a quick sidebar discussion on the responsibilities of the MVC pattern.

From The Gang of Four:

The Model is the application object, the View is its screen presentation, and the Controller defines the way the user interface reacts to user input.

The point of using a structure like this is to remain encapsulated and flexible, allowing us to exchange and reuse components. Let's go through each part of the MVC pattern and talk about its reusability and flexibility:

View

I think most people would agree that a View is supposed to be a simple visual representation of data and should not contain much logic. In our case, as developers for the web, our View tends to be HTML or XML.

reusable: always, almost anything can create a view
flexible: not having any real logic in these layers makes this very flexible

Controller

If a Controller "defines the way the user interface reacts to user input", then its responsibility should be to listen to user input (GET, POST, Headers, etc), and build out the current state of the application. In my opinion, a Controller should be very light and should not contain more code than is required to accomplish the above.

reusable: We have to remember that our Controllers return an opinionated View, so we cannot ever call that Controller method in a practical way to use any of the logic inside it. Therefore any logic placed in Controller methods, must be specific to that Controller method, if the logic is reusable, it should be placed elsewhere.
flexible: In most PHP MVCs, the Controller is tied directly to the route, which does not leave us very much flexibility. Laravel fixes this issue by allowing us to declare routes that use a controller, so we can now swap out our controllers with different implementations if need be:

Route::get('/', array(
  'uses' => 'SomeController@action'
));

Model

The Model is the "application object" in our definition from the Gang of Four. This is a very generic definition. In addition, we just decided to offload any logic that needs to be reusable from our Controller, and since the Model is the only component left in our defined structure, it’s logical to assume that this is the new home for that logic. However, I think the Model should not contain any logic like this. In my opinion, we should think of our "application object", in this case as an object that represents its place in the data-layer, whether that be a table, row, or collection entirely depends on state. The model should contain not much more than getters and setters for data (including relationships).

reusable: If we follow the above practice and make our Models be an object that represents its place in the database, this object remains very reusable. Any part of our system can use this model and by doing so gain complete and unopinionated access to the database.
flexible: Following the above practice, our Model is basically an implementation of an ORM, this allows us to be flexible, because we now have the power to change ORM's whenever we’d like to just by adding a new Model. We should probably have a pre-defined interface that our Model's must abide by, such as: all, find, create, update, delete. Implementation of a new ORM would be as simple as ensuring that the previously mentioned interface was accomodated.

Repository

Just by carefully defining our MVC components, we orphaned all kinds of logic into no-man's land. This is where Repositories come in to fill the void. Repositories become the intermediary of the Controllers and Models. A typical request would be something like this:

The Controller receives all user input and passes it to the repository.
The Repository does any "pre-gathering" actions such as validation of data, authorization, authentication, etc. If these "pre-gathering" actions are successful, then the request is passed to the Model for processing.
The Model will process all of the data into the data-layer, and return the current state.
The Repository will handle any "post-gathering" routines and return the current state to the controller.
The Controller will then create the appropriate view using the information provided by the repository.

Our Repository ends up as flexible and organized as we have made our Controllers and Models, allowing us to reuse this in most parts of our system, as well as being able to swap it out for another implementation if needed.

We have already seen an example of swapping out a repository for another implementation in the Controller tests above. Instead of using our default Repository, we asked the IoC container to provide the controller with an instance of a Mockery object. We have this same power for all of our components.

What we have accomplised here by adding another layer to our MVC, is a very organized, scalable, and testable system. Let's start putting the pieces in place and getting our tests to pass.

Controller Implementation

If you take a read through the controller tests, you’ll see that all we really care about is how the controller is interacting with the repository. So let's see how light and simple that makes our controllers.

Note: in TDD, the objective is to do no more work than is required to make your tests pass. So we want to do the absolute bare minimum here.

app/controllers/V1/PostsController.php

<?php
namespace V1;

use BaseController; 
use PostRepositoryInterface; 
use Input;
use View;

class PostsController extends BaseController {

  /**
   * We will use Laravel's dependency injection to auto-magically
   * "inject" our repository instance into our controller
   */
  public function __construct(PostRepositoryInterface $posts)
  {
    $this->posts = $posts;
  }

  /**
   * Display a listing of the resource.
   *
   * @return Response
   */
  public function index()
  {
    return $this->posts->findAll();
  }

  /**
   * Show the form for creating a new resource.
   *
   * @return Response
   */
  public function create()
  {
    $post = $this->posts->instance();
    return View::make('posts._form', compact('post'));
  }

  /**
   * Store a newly created resource in storage.
   *
   * @return Response
   */
  public function store()
  {
    return $this->posts->store( Input::all() );
  }

  /**
   * Display the specified resource.
   *
   * @param int $id
   * @return Response
   */
  public function show($id)
  {
    return $this->posts->findById($id);
  }

  /**
   * Show the form for editing the specified resource.
   *
   * @param int $id
   * @return Response
   */
  public function edit($id)
  {
    $post = $this->posts->findById($id);
    return View::make('posts._form', compact('post'));
  }

  /**
   * Update the specified resource in storage.
   *
   * @param int $id
   * @return Response
   */
  public function update($id)
  {
    return $this->posts->update($id, Input::all());
  }

  /**
   * Remove the specified resource from storage.
   *
   * @param int $id
   * @return Response
   */
  public function destroy($id)
  {
    $this->posts->destroy($id);
    return '';
  }

}

app/controllers/PostsCommentsController.php

<?php
namespace V1;

use BaseController; 
use CommentRepositoryInterface; 
use Input;
use View;

class PostsCommentsController extends BaseController {

  /**
   * We will use Laravel's dependency injection to auto-magically
   * "inject" our repository instance into our controller
   */
  public function __construct(CommentRepositoryInterface $comments)
  {
    $this->comments = $comments;
  }

  /**
   * Display a listing of the resource.
   *
   * @return Response
   */
  public function index($post_id)
  {
    return $this->comments->findAll($post_id);
  }

  /**
   * Show the form for creating a new resource.
   *
   * @return Response
   */
  public function create($post_id)
  {
    $comment = $this->comments->instance(array(
      'post_id' => $post_id
    ));

    return View::make('comments._form', compact('comment'));
  }

  /**
   * Store a newly created resource in storage.
   *
   * @return Response
   */
  public function store($post_id)
  {
    return $this->comments->store( $post_id, Input::all() );
  }

  /**
   * Display the specified resource.
   *
   * @param int $id
   * @return Response
   */
  public function show($post_id, $id)
  {
    return $this->comments->findById($post_id, $id);
  }

  /**
   * Show the form for editing the specified resource.
   *
   * @param int $id
   * @return Response
   */
  public function edit($post_id, $id)
  {
    $comment = $this->comments->findById($post_id, $id);

    return View::make('comments._form', compact('comment'));
  }

  /**
   * Update the specified resource in storage.
   *
   * @param int $id
   * @return Response
   */
  public function update($post_id, $id)
  {
    return $this->comments->update($post_id, $id, Input::all());
  }

  /**
   * Remove the specified resource from storage.
   *
   * @param int $id
   * @return Response
   */
  public function destroy($post_id, $id)
  {
    $this->comments->destroy($post_id, $id);
    return '';
  }

}

It doesn’t get much simpler than that, all the Controllers are doing is handing the input data to the repository, taking the response from that, and handing it to the View, the View in our case is merely JSON for most of our methods. When we return an Eloquent Collection, or Eloquent Model from a Controller in Laravel 4, the object is parsed into JSON auto-magically, which makes our job very easy.

Note: notice that we added a few more "use" statements to the top of the file to support the other classes that we’re using. Do not forget this when you’re working within a namespace.

The only thing that is a bit tricky in this controller is the constructor. Notice we’re passing in a typed variable as a dependency for this Controller, yet there is no point that we have access to the instantiation of this controller to actually insert that class… welcome to dependency injection! What we’re actually doing here is hinting to our controller that we have a dependency needed to run this class and what its class name is (or its IoC binding name). Laravel uses App::make() to create its Controllers before calling them. App::make() will try to resolve an item by looking for any bindings that we may have declared, and/or using the auto-loader to provide an instance. In addition, it will also resolve any dependencies needed to instantiate that class for us, by more-or-less recursively calling App::make() on each of the dependencies.

The observant, will notice that what we’re trying to pass in as a dependency is an interface, and as you know, an interface cannot be instantiated. This is where it gets cool and we actually already did the same thing in our tests. In our tests however, we used App::instance() to provide an already created instance instead of the interface. For our Controllers, we’re actually going to tell Laravel that whenever an instance of PostRepositoryInterface is requested, to actually return an instance of EloquentPostRepository.

Open up your app/routes.php file and add the following to the top of the file

App::bind('PostRepositoryInterface', 'EloquentPostRepository');
App::bind('CommentRepositoryInterface', 'EloquentCommentRepository');

After adding those lines, anytime App::make() asks for an instance of PostRepositoryInterface, it will create an instance of EloquentPostRepository, which is assumed to implement PostRepositoryInterface. If you were to ever change your repository to instead use a different ORM than Eloquent, or maybe a file-based driver, all you have to do is change these two lines and you’re good to go, your Controllers will still work as normal. The Controllers actual dependency is any object that implements that interface and we can determine at run-time what that implementation actually is.

The PostRepositoryInterface and CommentRepositoryInterface must actually exist and the bindings must actually implement them. So let's create them now:

app/repositories/PostRepositoryInterface.php

<?php

interface PostRepositoryInterface {
  public function findById($id);
  public function findAll();
  public function paginate($limit = null);
  public function store($data);
  public function update($id, $data);
  public function destroy($id);
  public function validate($data);
  public function instance();
}

app/repositories/CommentRepositoryInterface.php

<?php

interface CommentRepositoryInterface {
  public function findById($post_id, $id);
  public function findAll($post_id);
  public function store($post_id, $data);
  public function update($post_id, $id, $data);
  public function destroy($post_id, $id);
  public function validate($data);
  public function instance();
}

Now that we have our two interfaces built, we must provide implementations of these interfaces. Let's build them now.

app/repositories/EloquentPostRepository.php

As the name of this implementation implies, we’re relying on Eloquent, which we can call directly. If you had other dependencies, remember that App::make() is being used to resolve this repository, so you can feel free to use the same constructor method we used with our Controllers to inject your dependencies.

<?php

class EloquentPostRepository implements PostRepositoryInterface {

  public function findById($id)
  {
    $post = Post::with(array(
        'comments' => function($q)
        {
          $q->orderBy('created_at', 'desc');
        }
      ))
      ->where('id', $id)
      ->first();

    if(!$post) throw new NotFoundException('Post Not Found');
    return $post;
  }

  public function findAll()
  {
    return Post::with(array(
        'comments' => function($q)
        {
          $q->orderBy('created_at', 'desc');
        }
      ))
      ->orderBy('created_at', 'desc')
      ->get();
  }

  public function paginate($limit = null)
  {
    return Post::paginate($limit);
  }

  public function store($data)
  {
    $this->validate($data);
    return Post::create($data);
  }

  public function update($id, $data)
  {
    $post = $this->findById($id);
    $post->fill($data);
    $this->validate($post->toArray());
    $post->save();
    return $post;
  }

  public function destroy($id)
  {
    $post = $this->findById($id);
    $post->delete();
    return true;
  }

  public function validate($data)
  {
    $validator = Validator::make($data, Post::$rules);
    if($validator->fails()) throw new ValidationException($validator);
    return true;
  }

  public function instance($data = array())
  {
    return new Post($data);
  }

}

app/repositories/EloquentCommentRepository.php

<?php

class EloquentCommentRepository implements CommentRepositoryInterface {

  public function findById($post_id, $id)
  {
    $comment = Comment::find($id);
    if(!$comment || $comment->post_id != $post_id) throw new NotFoundException('Comment Not Found');
    return $comment;
  }

  public function findAll($post_id)
  {
    return Comment::where('post_id', $post_id)
      ->orderBy('created_at', 'desc')
      ->get();
  }

  public function store($post_id, $data)
  {
    $data['post_id'] = $post_id;
    $this->validate($data);
    return Comment::create($data);
  }

  public function update($post_id, $id, $data)
  {
    $comment = $this->findById($post_id, $id);
    $comment->fill($data);
    $this->validate($comment->toArray());
    $comment->save();
    return $comment;
  }

  public function destroy($post_id, $id)
  {
    $comment = $this->findById($post_id, $id);
    $comment->delete();
    return true;
  }

  public function validate($data)
  {
    $validator = Validator::make($data, Comment::$rules);
    if($validator->fails()) throw new ValidationException($validator);
    return true;
  }

  public function instance($data = array())
  {
    return new Comment($data);
  }

}

If you take a look in our repositories, there are a few Exceptions that we are throwing, which are not native, nor do they belong to Laravel. Those are custom Exceptions that we’re using to simplify our code. By using custom Exceptions, we’re able to easily halt the progress of the application if certain conditions are met. For instance, if a post is not found, we can just toss a NotFoundException, and the application will handle it accordingly, but, not by showing a 500 error as usual, instead we’re going to setup custom error handlers. You could alternatively use App::abort(404) or something along those lines, but I find that this method saves me many conditional statements and repeat code, as well as allowing me to adjust the implementation of error reporting in a single place very easily.

First let's define the custom Exceptions. Create a file in your app folder called errors.php

touch app/errors.php

app/errors.php

<?php

class PermissionException extends Exception {

  public function __construct($message = null, $code = 403)
  {
    parent::__construct($message ?: 'Action not allowed', $code);
  }

}

class ValidationException extends Exception {

  protected $messages;

  /**
   * We are adjusting this constructor to receive an instance
   * of the validator as opposed to a string to save us some typing
   * @param Validator $validator failed validator object
   */
  public function __construct($validator)
  {
    $this->messages = $validator->messages();
    parent::__construct($this->messages, 400);
  }

  public function getMessages()
  {
    return $this->messages;
  }

}

class NotFoundException extends Exception {

  public function __construct($message = null, $code = 404)
  {
    parent::__construct($message ?: 'Resource Not Found', $code);
  }

}

These are very simple Exceptions, notice for the ValidationException, we can just pass it the failed validator instance and it will handle the error messages accordingly!

Now we need to define our error handlers that will be called when one of these Exceptions are thrown. These are basically Event listeners, whenever one of these exceptions are thrown, it’s treated as an Event and calls the appropriate function. It’s very simple to add logging or any other error handling procedures here.

app/filters.php

...

/**
 * General HttpException handler
 */
App::error( function(Symfony\Component\HttpKernel\Exception\HttpException $e, $code)
{
  $headers = $e->getHeaders();

  switch($code)
  {
    case 401:
      $default_message = 'Invalid API key';
      $headers['WWW-Authenticate'] = 'Basic realm="CRM REST API"';
    break;

    case 403:
      $default_message = 'Insufficient privileges to perform this action';
    break;

    case 404:
      $default_message = 'The requested resource was not found';
    break;

    default:
      $default_message = 'An error was encountered';
  }

  return Response::json(array(
    'error' => $e->getMessage() ?: $default_message
  ), $code, $headers);
});

/**
 * Permission Exception Handler
 */
App::error(function(PermissionException $e, $code)
{
  return Response::json($e->getMessage(), $e->getCode());
});

/**
 * Validation Exception Handler
 */
App::error(function(ValidationException $e, $code)
{
  return Response::json($e->getMessages(), $code);
});

/**
 * Not Found Exception Handler
 */
App::error(function(NotFoundException $e)
{
  return Response::json($e->getMessage(), $e->getCode());
});

We now need to let our auto-loader know about these new files. So we must tell Composer where to check for them:

composer.json

Notice that we added the "app/errors.php" line.

{
  "require": {
    "laravel/framework": "4.0.*",
    "way/generators": "dev-master",
    "twitter/bootstrap": "dev-master",
    "conarwelsh/mustache-l4": "dev-master"
  },
  "require-dev": {
    "phpunit/phpunit": "3.7.*",
    "mockery/mockery": "0.7.*"
  },
  "autoload": {
    "classmap": [
      "app/commands",
      "app/controllers",
      "app/models",
      "app/database/migrations",
      "app/database/seeds",
      "app/tests/TestCase.php",
      "app/repositories",
      "app/errors.php"
    ]
  },
  "scripts": {
    "post-update-cmd": "php artisan optimize"
  },
  "minimum-stability": "dev"
}

We must now tell Composer to actually check for these files and include them in the auto-load registry.

composer dump-autoload

Great, so we have completed our controllers and our repositories, the last two items in our MVRC that we have to take care of is the models and views, both of which are pretty straight forward.

app/models/Post.php

<?php
/**
 * Represent a Post Item, or Collection
 */
class Post extends Eloquent {

  /**
   * Items that are "fillable"
   * meaning we can mass-assign them from the constructor
   * or $post->fill()
   * @var array
   */
  protected $fillable = array(
    'title', 'content', 'author_name'
  );

  /**
   * Validation Rules
   * this is just a place for us to store these, you could
   * alternatively place them in your repository
   * @var array
   */
  public static $rules = array(
    'title'    => 'required',
    'author_name' => 'required'
  );

  /**
   * Define the relationship with the comments table
   * @return Collection collection of Comment Models
   */
  public function comments()
  {
    return $this->hasMany('Comment');
  }

}

app/models/Comment.php

<?php
/**
 * Represent a Comment Item, or Collection
 */
class Comment extends Eloquent {

  /**
   * Items that are "fillable"
   * meaning we can mass-assign them from the constructor
   * or $comment->fill()
   * @var array
   */
  protected $fillable = array(
    'post_id', 'content', 'author_name'
  );

  /**
   * Validation Rules
   * this is just a place for us to store these, you could
   * alternatively place them in your repository
   * @var array
   */
  public static $rules = array(
    'post_id'   => 'required|numeric',
    'content'   => 'required',
    'author_name' => 'required'
  );

  /**
   * Define the relationship with the posts table
   * @return Model parent Post model
   */
  public function post()
  {
    return $this->belongsTo('Post');
  }

}

As far as views are concerned, I’m just going to mark up some simple bootstrap-friendly pages. Remember to change each files extension to .mustache though, since our generator thought that we would be using .blade.php. We’re also going to create a few "partial" views using the Rails convention of prefixing them with an _ to signify a partial.

Note: I skipped a few views, as we will not be using them in this tutorial.

public/views/posts/index.mustache

For the index page view we’ll just loop over all of our posts, showing the post partial for each.

{{#posts}}
  {{> posts._post}}
{{/posts}}

public/views/posts/show.mustache

For the show view we’ll show an entire post and its comments:

<article>
  <h3>
    {{ post.title }} {{ post.id }}
    <small>{{ post.author_name }}</small>
  </h3>
  <div>
    {{ post.content }}
  </div>
</article>

<div>
  <h2>Add A Comment</h2>
  {{> comments._form }}

  <section data-role="comments">
    {{#post.comments}}
      <div>
        {{> comments._comment }}
      </div>
    {{/post.comments}}
  </section>
</div>

public/views/posts/_post.mustache

Here’s the partial that we’ll use to show a post in a list. This is used on our index view.

<article data-toggle="view" data-target="posts/{{ id }}">
  <h3>{{ title }} {{ id }}</h3>
  <cite>{{ author_name }} on {{ created_at }}</cite>
</article>

public/views/posts/_form.mustache

Here’s the form partial needed to create a post, we’ll use this from our API, but this could also be a useful view in an admin panel and other places, which is why we choose to make it a partial.

{{#exists}}
  <form action="/v1/posts/{{ post.id }}" method="post">
    <input type="hidden" name="_method" value="PUT" />
{{/exists}}
{{^exists}}
  <form action="/v1/posts" method="post">
{{/exists}}

  <fieldset>

    <div class="control-group">
      <label class="control-label"></label>
      <div class="controls">
        <input type="text" name="title" value="{{ post.title }}" />
      </div>
    </div>

    <div class="control-group">
      <label class="control-label"></label>
      <div class="controls">
        <input type="text" name="author_name" value="{{ post.author_name }}" />
      </div>
    </div>

    <div class="control-group">
      <label class="control-label"></label>
      <div class="controls">
        <textarea name="content">{{ post.content }}"</textarea>
      </div>
    </div>

    <div class="form-actions">
      <input type="submit" class="btn btn-primary" value="Save" />
    </div>

  </fieldset>
</form>

public/views/comments/_comment.mustache

Here’s the comment partial which is used to represent a single comment in a list of comments:

<h5>
  {{ author_name }}
  <small>{{ created_at }}</small>
</h5>
<div>
  {{ content }}
</div>

public/views/comments/_form.mustache

The form needed to create a comment – both used in the API and the Show Post view:

{{#exists}}
  <form class="form-horizontal" action="/v1/posts/{{ comment.post_id }}/{{ id }}" method="post">
    <input type="hidden" name="_method" value="PUT" />
{{/exists}}
{{^exists}}
  <form class="form-horizontal" action="/v1/posts/{{ comment.post_id }}" method="post">
{{/exists}}

  <fieldset>

    <div class="control-group">
      <label class="control-label">Author Name</label>
      <div class="controls">
        <input type="text" name="author_name" value="{{ comment.author_name }}" />
      </div>
    </div>

    <div class="control-group">
      <label class="control-label">Comment</label>
      <div class="controls">
        <textarea name="content">{{ comment.content }}</textarea>
      </div>
    </div>

    <div class="form-actions">
      <input type="submit" class="btn btn-primary" value="Save" />
    </div>

  </fieldset>
</form>

public/views/layouts/_notification.mustache

And here’s the helper view partial to allow us to show a notification:

<div class="alert alert-{{type}}">
  {{message}}
</div>

Great, we have all of our API components in place. Let's run our unit tests to see where we’re at!

vendor/phpunit/phpunit/phpunit.php

Your first run of this test should pass with flying (green) colors. However, if you were to run this test again, you’ll notice that it fails now with a handful of errors, and that is because our repository tests actually tested the database, and in doing so deleted some of the records our previous tests used to assert values. This is an easy fix, all we have to do is tell our tests that they need to re-seed the database after each test. In addition, we did not receive a noticable error for this, but we did not close Mockery after each test either, this is a requirement of Mockery that you can find in their docs. So let's add both missing methods.

Open up app/tests/TestCase.php and add the following two methods:

/**
 * setUp is called prior to each test
 */
public function setUp()
{
  parent::setUp();
  $this->seed();
}

/**
 * tearDown is called after each test
 * @return [type] [description]
 */
public function tearDown()
{
  Mockery::close();
}

This is great, we now said that at every "setUp", which is run before each test, to re-seed the database. However we still have one problem, everytime you re-seed, it’s only going to append new rows to the tables. Our tests are looking for items with a row ID of one, so we still have a few changes to make. We just need to tell the database to truncate our tables when seeding:

app/database/seeds/CommentsTableSeeder.php

Before we insert the new rows, we’ll truncate the table, deleting all rows and resetting the auto-increment counter.

<?php

class CommentsTableSeeder extends Seeder {

  public function run()
  {
    $comments = array(
      array(
        'content'   => 'Lorem ipsum Nisi dolore ut incididunt mollit tempor proident eu velit cillum dolore sed',
        'author_name' => 'Testy McTesterson',
        'post_id'   => 1,
        'created_at' => date('Y-m-d H:i:s'),
        'updated_at' => date('Y-m-d H:i:s'),
      ),
      array(
        'content'   => 'Lorem ipsum Nisi dolore ut incididunt mollit tempor proident eu velit cillum dolore sed',
        'author_name' => 'Testy McTesterson',
        'post_id'   => 1,
        'created_at' => date('Y-m-d H:i:s'),
        'updated_at' => date('Y-m-d H:i:s'),
      ),
      array(
        'content'   => 'Lorem ipsum Nisi dolore ut incididunt mollit tempor proident eu velit cillum dolore sed',
        'author_name' => 'Testy McTesterson',
        'post_id'   => 2,
        'created_at' => date('Y-m-d H:i:s'),
        'updated_at' => date('Y-m-d H:i:s'),
      ),
    );

    //truncate the comments table when we seed
    DB::table('comments')->truncate();
    DB::table('comments')->insert($comments);
  }

}

app/database/seeds/PostsTableSeeder.php

<?php

class PostsTableSeeder extends Seeder {

  public function run()
  {
    $posts = array(
      array(
        'title'    => 'Test Post',
        'content'   => 'Lorem ipsum Reprehenderit velit est irure in enim in magna aute occaecat qui velit ad.',
        'author_name' => 'Conar Welsh',
        'created_at' => date('Y-m-d H:i:s'),
        'updated_at' => date('Y-m-d H:i:s'),
      ),
      array(
        'title'    => 'Another Test Post',
        'content'   => 'Lorem ipsum Reprehenderit velit est irure in enim in magna aute occaecat qui velit ad.',
        'author_name' => 'Conar Welsh',
        'created_at' => date('Y-m-d H:i:s'),
        'updated_at' => date('Y-m-d H:i:s'),
      )
    );

    //truncate the posts table each time we seed
    DB::table('posts')->truncate();
    DB::table('posts')->insert($posts);
  }

}

Now you should be able to run the tests any number of times and get passing tests each time! That means we have fulfilled our TDD cycle and we’re not allowed to write anymore production code for our API!! Let's just commit our changes to our repo and move onto the Backbone application!

git add . && git commit -am "built out the API and corresponding tests"

Backbone App

Now that we have completed all of the back-end work, we can move forward to creating a nice user interface to access all of that data. We’ll keep this part of the project a little bit on the simpler side, and I warn you that my approach can be considered an opinionated one. I have seen many people with so many different methods for structuring a Backbone application. My trials and errors have led me to my current method, if you do not agree with it, my hope is that it may inspire you to find your own!

We’re going to use the Mustache templating engine instead of Underscore, this will allow us to share our views between the client and server! The trick is in how you load the views, we’re going to use AJAX in this tutorial, but it’s just as easy to load them all into the main template, or precompile them.

Router

First we’ll get our router going. There are two parts to this, the Laravel router, and the Backbone router.

Laravel Router

There are two main approaches we can take here:

Approach #1: The catch-all

Remember I told you when you were adding the resource routes that it was important that you placed them ABOVE the app route?? The catch-all method is the reason for that statement. The overall goal of this method is to have any routes that have not found a match in Laravel, be caught and sent to Backbone. Implementing this method is easy:

app/routes.php

// change your existing app route to this:
// we are basically just giving it an optional parameter of "anything"
Route::get('/{path?}', function($path = null)
{
  return View::make('app');
})
->where('path', '.*'); //regex to match anything (dots, slashes, letters, numbers, etc)

Now, every route other than our API routes will render our app view.

In addition, if you have a multi-page app (several single page apps), you can define several of these catch-alls:

Route::get('someApp1{path?}', function($path = null)
{
  return View::make('app');
})
->where('path', '.*');

Route::get('anotherApp/{path?}', function($path = null)
{
  return View::make('app');
})
->where('path', '.*');

Route::get('athirdapp{path?}', function($path = null)
{
  return View::make('app');
})
->where('path', '.*');

Note: Keep in mind the '/' before {path?}. If that slash is there, it’ll be required in the URL (with the exception of the index route), sometimes this is desired and sometimes not.

Approach #2:

Since our front and back end share views… wouldn't it be extremely easy to just define routes in both places? You can even do this in addition to the catch-all approach if you want.

The routes that we’re going to end up defining for the app are simply:

GET /
GET /posts/:id

app/routes.php

<?php

App::bind('PostRepositoryInterface', 'EloquentPostRepository'); 
App::bind('CommentRepositoryInterface', 'EloquentCommentRepository');
//create a group of routes that will belong to APIv1
Route::group(array('prefix' => 'v1'), function()
{
  Route::resource('posts', 'V1\PostsController');
  Route::resource('posts.comments', 'V1\PostsCommentsController');
});
/**
 * Method #1: use catch-all
 * optionally commented out while we use Method 2
 */
// change your existing app route to this:
// we are basically just giving it an optional parameter of "anything"
// Route::get('/{path?}', function($path = null)
// {
//   return View::make('layouts.application')->nest('content', 'app');
// })
// ->where('path', '.*'); //regex to match anything (dots, slashes, letters, numbers, etc)
/**
 * Method #2: define each route
 */
Route::get('/', function()
{
  $posts = App::make('PostRepositoryInterface')->paginate();
  return View::make('layouts.application')->nest('content', 'posts.index', array(
    'posts' => $posts
  ));
});

Route::get('posts/{id}', function($id)
{
  $post = App::make('PostRepositoryInterface')->findById($id);
  return View::make('layouts.application')->nest('content', 'posts.show', array(
    'post' => $post
  ));
});

Pretty cool huh?! Regardless of which method we use, or the combination of both, your Backbone router will end up mostly the same.

Notice that we’re using our Repository again, this is yet another reason why Repositories are a useful addition to our framework. We can now run almost all of the logic that the controller does, but without repeating hardly any of the code!

Keep in mind a few things while choosing which method to use, if you use the catch-all, it will do just like the name implies… catch-ALL. This means there is no such thing as a 404 on your site anymore. No matter the request, its landing on the app page (unless you manually toss an exception somewhere such as your repository). The inverse is, with defining each route, now you have two sets of routes to manage. Both methods have their ups and downs, but both are equally easy to deal with.

Base View

One view to rule them all! This BaseView is the view that all of our other Views will inherit from. For our purposes, this view has but one job… templating! In a larger app this view is a good place to put other shared logic.

We’ll simply extend Backbone.View and add a template function that will return our view from the cache if it exists, or get it via AJAX and place it in the cache. We have to use synchronous AJAX due to the way that Mustache.js fetches partials, but since we’re only retrieving these views if they are not cached, we shouldn’t receive much of a performance hit here.

/**
 ***************************************
 * Array Storage Driver
 * used to store our views
 ***************************************
 */
var ArrayStorage = function(){
  this.storage = {};
};
ArrayStorage.prototype.get = function(key)
{
  return this.storage[key];
};
ArrayStorage.prototype.set = function(key, val)
{
  return this.storage[key] = val;
};
/**
 ***************************************
 * Base View
 ***************************************
 */
var BaseView = bb.View.extend({

  /**
   * Set our storage driver
   */
  templateDriver: new ArrayStorage,

  /**
   * Set the base path for where our views are located
   */
  viewPath: '/views/',

  /**
   * Get the template, and apply the variables
   */
  template: function()
  {
    var view, data, template, self;

    switch(arguments.length)
    {
      case 1:
        view = this.view;
        data = arguments[0];
        break;
      case 2:
        view = arguments[0];
        data = arguments[1];
        break;
    }

    template = this.getTemplate(view, false);
    self = this;

    return template(data, function(partial)
    {
      return self.getTemplate(partial, true);
    });
  },

  /**
   * Facade that will help us abstract our storage engine,
   * should we ever want to swap to something like LocalStorage
   */
  getTemplate: function(view, isPartial)
  {
    return this.templateDriver.get(view) || this.fetch(view, isPartial);
  },

  /**
   * Facade that will help us abstract our storage engine,
   * should we ever want to swap to something like LocalStorage
   */
  setTemplate: function(name, template)
  {
    return this.templateDriver.set(name, template);
  },

  /**
   * Function to retrieve the template via ajax
   */
  fetch: function(view, isPartial)
  {
    var markup = $.ajax({
      async: false,

      //the URL of our template, we can optionally use dot notation
      url: this.viewPath + view.split('.').join('/') + '.mustache'
    }).responseText;

    return isPartial
      ? markup
      : this.setTemplate(view, Mustache.compile(markup));
  }
});

PostView

The PostView renders a single blog post:

// this view will show an entire post
// comment form, and comments
var PostView = BaseView.extend({

  //the location of the template this view will use, we can use dot notation
  view: 'posts.show',

  //events this view should subscribe to
  events: {
    'submit form': function(e)
    {
      e.preventDefault();
      e.stopPropagation();

      return this.addComment( $(e.target).serialize() );
    }
  },

  //render our view into the defined `el`
  render: function()
  {
    var self = this;

    self.$el.html( this.template({
      post: this.model.attributes
    }) );
  },

  //add a comment for this post
  addComment: function(formData)
  {
    var
      self = this,

      //build our url
      action = this.model.url() + '/comments'
    ;

    //submit a post to our api
    $.post(action, formData, function(comment, status, xhr)
    {
      //create a new comment partial
      var view = new CommentViewPartial({
        //we are using a blank backbone model, since we done have any specific logic needed
        model: new bb.Model(comment)
      });

      //prepend the comment partial to the comments list
      view.render().$el.prependTo(self.$('[data-role="comments"]'));

      //reset the form
      self.$('input[type="text"], textarea').val('');

      //prepend our new comment to the collection
      self.model.attributes.comments.unshift(comment);

      //send a notification that we successfully added the comment
      notifications.add({
        type: 'success',
        message: 'Comment Added!'
      });
    });

  }
});

Partial Views

We’ll need a few views to render partials. We mainly just need to tell the view which template to use and that it should extend our view that provides the method to fetch our template.

// this will be used for rendering a single comment in a list
var CommentViewPartial = BaseView.extend({
  //define our template location
  view: 'comments._comment',
  render: function()
  {
    this.$el.html( this.template(this.model.attributes) );
    return this;
  }
});

//this view will be used for rendering a single post in a list
var PostViewPartial = BaseView.extend({
  //define our template location
  view: 'posts._post',
  render: function()
  {
    this.$el.html( this.template(this.model.attributes) );
    return this;
  }
});

Blog View

This is our overall application view. It contains our configuration logic, as well as handling the fetching of our PostCollection. We also setup a cool little infinite scroll feature. Notice how we’re using jQuery promises to ensure that the fetching of our collection has completed prior to rendering the view.

var Blog = BaseView.extend({
  //define our template location
  view: 'posts.index',

  //setup our app configuration
  initialize: function()
  {
    this.perPage = this.options.perPage || 15;
    this.page   = this.options.page || 0;
    this.fetching = this.collection.fetch();

    if(this.options.infiniteScroll) this.enableInfiniteScroll();
  },

  //wait til the collection has been fetched, and render the view
  render: function()
  {
    var self = this;
    this.fetching.done(function()
    {
      self.$el.html('');
      self.addPosts();

      // var posts = this.paginate()

      // for(var i=0; i<posts.length; i++)
      // {
      //   posts[i] = posts[i].toJSON();
      // }

      // self.$el.html( self.template({
      //   posts: posts
      // }) );

      if(self.options.infiniteScroll) self.enableInfiniteScroll();
    });
  },

  //helper function to limit the amount of posts we show at a time
  paginate: function()
  {
    var posts;
    posts = this.collection.rest(this.perPage * this.page);
    posts = _.first(posts, this.perPage);
    this.page++;

    return posts;
  },

  //add the next set of posts to the view
  addPosts: function()
  {
    var posts = this.paginate();

    for(var i=0; i<posts.length; i++)
    {
      this.addOnePost( posts[i] );
    }
  },

  //helper function to add a single post to the view
  addOnePost: function(model)
  {
    var view = new PostViewPartial({
      model: model
    });
    this.$el.append( view.render().el );
  },

  //this function will show an entire post, we could alternatively make this its own View
  //however I personally like having it available in the overall application view, as it
  //makes it easier to manage the state
  showPost: function(id)
  {
    var self = this;

    this.disableInifiniteScroll();

    this.fetching.done(function()
    {
      var model = self.collection.get(id);

      if(!self.postView)
      {
        self.postView = new self.options.postView({
          el: self.el
        });
      }
      self.postView.model = model;
      self.postView.render();
    });
  },

  //function to run during the onScroll event
  infiniteScroll: function()
  {
    if($window.scrollTop() >= $document.height() - $window.height() - 50)
    {
      this.addPosts();
    }
  },

  //listen for the onScoll event
  enableInfiniteScroll: function()
  {
    var self = this;

    $window.on('scroll', function()
    {
      self.infiniteScroll();
    });
  },

  //stop listening to the onScroll event
  disableInifiniteScroll: function()
  {
    $window.off('scroll');
  }
});

PostCollection

Setup our PostCollection – we just need to tell the Collection the URL it should use to fetch its contents.

// the posts collection is configured to fetch
// from our API, as well as use our PostModel
var PostCollection = bb.Collection.extend({
  url: '/v1/posts'
});

Blog Router

Notice that we’re not instantiating new instances of our views, we’re merely telling them to render. Our initialize functions are designed to only be ran once, as we don’t want them to run but once, on page load.

var BlogRouter = bb.Router.extend({
  routes: {
    "": "index",
    "posts/:id": "show"
  },
  initialize: function(options)
  {
    // i do this to avoid having to hardcode an instance of a view
    // when we instantiate the router we will pass in the view instance
    this.blog = options.blog;
  },
  index: function()
  {
    //reset the paginator
    this.blog.page = 0;

    //render the post list
    this.blog.render();
  },
  show: function(id)
  {
    //render the full-post view
    this.blog.showPost(id);
  }
});

Notifications Collection

We’re just going to setup a simple Collection to store user notifications:

var notifications = new bb.Collection();

NotificationsView

This view will handle the displaying and hiding of user notifications:

var NotificationView = BaseView.extend({
  el: $('#notifications'),
  view: 'layouts._notification',
  initialize: function()
  {
    this.listenTo(notifications, 'add', this.render);
  },
  render: function(notification)
  {
    var $message = $( this.template(notification.toJSON()) );
    this.$el.append($message);
    this.delayedHide($message);
  },
  delayedHide: function($message)
  {
    var timeout = setTimeout(function()
    {
      $message.fadeOut(function()
      {
        $message.remove();
      });
    }, 5*1000);

    var self = this;
    $message.hover(
      function()
      {
        timeout = clearTimeout(timeout);
      },
      function()
      {
        self.delayedHide($message);
      }
    );
  }
});
var notificationView = new NotificationView();

Error Handling

Since we used the custom exception handlers for our API, it makes it very easy to handle any error our API may throw. Very similar to the way we defined our event listeners for our API in the app/filters.php file, we’ll define event listeners for our app here. Each code that could be thrown can just show a notification very easily!

$.ajaxSetup({
  statusCode: {
    401: function()
    {
      notification.add({
        type: null, //error, success, info, null
        message: 'You do not have permission to do that'
      });
    },
    403: function()
    {
      notification.add({
        type: null, //error, success, info, null
        message: 'You do not have permission to do that'
      });
    },
    404: function()
    {
      notification.add({
        type: 'error', //error, success, info, null
        message: '404: Page Not Found'
      });
    },
    500: function()
    {
      notification.add({
        type: 'error', //error, success, info, null
        message: 'The server encountered an error'
      });
    }
  }
});

Event Listeners

We’ll need a few global event listeners to help us navigate through our app without refreshing the page. We mainly just hijack the default behavior and call Backbone.history.navigate(). Notice how on our first listener, we’re specifying the selector to only match those that don’t have a data attribute of bypass. This will allow us to create links such as <a href="/some/non-ajax/page" data-bypass="true">link</a> that will force the page to refresh. We could also go a step further here and check whether the link is a local one, as opposed to a link to another site.

$document.on("click", "a[href]:not([data-bypass])", function(e){
  e.preventDefault();
  e.stopPropagation();

  var href = $(this).attr("href");
  bb.history.navigate(href, true);
});

$document.on("click", "[data-toggle='view']", function(e)
{
  e.preventDefault();
  e.stopPropagation();

  var
    self = $(this),
    href = self.attr('data-target') || self.attr('href')
  ;

  bb.history.navigate(href, true);
});

Start The App

Now we just need to boot the app, passing in any config values that we need. Notice the line that checks for the silentRouter global variable, this is kind of a hacky way to be able to use both back-end routing methods at the same time. This allows us to define a variable in the view called silentRouter and set it to true, meaning that the router should not actually engage the backbone route, allowing our back-end to handle the initial rendering of the page, and just wait for any needed updates or AJAX.

var BlogApp = new Blog({
  el       : $('[data-role="main"]'),
  collection   : new PostCollection(),
  postView    : PostView,
  perPage    : 15,
  page      : 0,
  infiniteScroll : true
});

var router = new BlogRouter({
  blog: BlogApp
});

if (typeof window.silentRouter === 'undefined') window.silentRouter = true;

bb.history.start({ pushState: true, root: '/', silent: window.silentRouter });

Conclusion

Notice that for the Backbone portion of our app, all we had to do was write some Javascript that knew how to interact with the pre-existing portions of our application? That’s what I love about this method! It may seem like we had a lot of steps to take to get to that portion of things, but really, most of that work was just a foundation build-up. Once we got that initial foundation in place, the actual application logic falls together very simply.

Try adding another feature to this blog, such as User listings and info. The basic steps you would take would be something like this:

Use the generator tool to create a new "User" resource.
Make the necessary modifications to ensure that the UserController is in the V1 API group.
Create your Repository and setup the proper IoC bindings in app/routes.php.
Write your Controller tests one at a time using Mockery for the repository, following each test up with the proper implementation to make sure that test passes.
Write your Repository tests one at a time, again, following each test up with the implementation.
Add in the new functionality to your Backbone App. I suggest trying two different approaches to the location of the User views. Decide for yourself which is the better implementation.
- First place them in their own routes and Main view.
- Then try incorporating them into the overall BlogView.

I hope this gave you some insight into creating a scalable single page app and API using Laravel 4 and Backbone.js. If you have any questions, please ask them in the comment section below!

New from Envato: The Easiest Way for Web Devs to Get Design Work Done

Skellie — Thu, 13 Jun 2013 20:33:03 +0000

As we’ve talked about before on Nettuts+, sometimes it feels impossible to keep up with all the skills, languages and tools you need to learn to be a great web developer.

In an ideal world we’d be masters of back-end, front-end and graphic design, but being truly great at any one of those three things is, on its own, a full-time challenge.

Many of us eventually realize that we should focus only on the skills we love to use, and let others help us with the rest.

For web developers, the skill we often decide we’ll never be able to master is design.

And yet, we struggle on with websites and apps we know look less than great because we don’t have the time to get truly good at design, or we lack the time, energy and money to collaborate with a talented designer.

Envato (the people behind Nettuts+) have recently launched a new service, currently in beta, that aims to solve this problem: Microlancer.

What is it?

Microlancer makes it easy for web developers to access design services that are:

Affordable – prices are fixed and stated upfront for different types of jobs.
Discoverable – browse service providers and find someone with a style and approach you like.
Predictable – service providers state upfront how fast their turnaround time will be and how many revisions are included in the price.
Good! – service providers are reviewed for quality and must provide several work examples.

Rather than committing to a big project upfront, services are broken down into smaller pieces and priced individually. For example, rebranding your web development business might involve the following services:

Detailed, Illustrated Logo Design by DesignSyndicate ($475)
Single Page Responsive Web Design by kamleshyadav ($200)
Creative Professional Business Card Design by ShermanJackson ($60)
Twitter Background, Header Design & Customization by Alex_Yves ($70)

These are actual services and prices currently on Microlancer, and with these, a complete rebranding with logo design, website, business cards and Twitter graphics would cost $805.

Though Microlancer is a great place to get work done for your own business, it’s also an excellent place to outsource any design work that clients need done. Prices are fixed, so it’s easy to calculate your profit margin on any given job.

On Microlancer, services are sold in a shop-like format, where they can be browsed and purchased individually. Though the vast majority of jobs proceed without a hitch, any issues that occasionally do arise can be resolved with Microlancer’s robust and fair dispute resolution process. Trained members of the team are on-hand to ensure that all users who play by the rules have a great Microlancer experience.

Here’s a quick preview of the kinds of things you can get done on Microlancer:

Logo Design
PSD Customization
Image Editing
Marketing Design (including Infographics)
Web Banner Design
Print Design (including Business Card Design)
Presentation Design
Social Media Graphics
Web Design
UI Element Design

If you haven’t yet, why not take 2 minutes to explore Microlancer?

Become a Service Provider

On Microlancer, users who provide services for buyers are called — you guessed it — service providers. Microlancer in its beta form currently offers graphic design services exclusively. If you have some design chops as well as code clout, you may be eligible to sell design services on Microlancer and unlock a new source of income.

Learn more about becoming a service provider on Microlancer.

Intro to Flask: Signing In and Out

Lalith Polepeddi — Wed, 12 Jun 2013 23:53:13 +0000

Many web applications require users to sign in and out in order to perform important tasks (like administration duties). In this article, we’ll create an authentication system for our application.

In the previous article, we built a contact page using the Flask-WTF and Flask-Mail extensions. We’ll use Flask-WTF, once again, this time to validate a user’s username and password. We’ll save these credentials into a database using yet another extension called Flask-SQLAlchemy.

You can find the source code for this tutorial on GitHub. While following along with this tutorial, when you see a caption, such as Checkpoint: 13_packaged_app, it means that you can switch to the GIT branch named “13_packaged_app” and review the code at that point in the article.

Growing the Application

So far, our Flask app is a fairly simple application. It consists of mostly static pages; so, we’ve been able to organize it as a Python module. But now, we need to reorganize our application to make it easier to maintain and grow. The Flask documentation recommends that we reorganize the app as a Python package, so let’s start there.

Our app is currently organized like this:

flaskapp/
└── app/
        ├── static/
        ├── templates/
        ├── forms.py
        ├── routes.py
        └── README.md

To restructure it as a package, let’s first create a new folder inside app/ named intro_to_flask/. Then move static/, templates/, forms.py and routes.py into intro_to_flask/. Also, delete any .pyc files that are hanging around.

flaskapp/
└── app/
        ├── intro_to_flask/
        │      ├── static/
        │      ├── templates/
        │      ├── forms.py
        │      ├── routes.py
        └── README.md

Next, create a new file named __init__.py and place it inside intro_to_flask/. This file is required to make Python treat the intro_to_flask/ folder as a package.

flaskapp/
└── app/
        ├── intro_to_flask/
        │      ├── __init__.py
        │      ├── static/
        │      ├── templates/
        │      ├── forms.py
        │      ├── routes.py
        └── README.md

When our app was a Python module, the application-wide imports and configurations were specified in routes.py. Now that the app is a Python package, we’ll move these settings from routes.py into __init__.py.

app/intro_to_flask/__init__.py

from flask import Flask

app = Flask(__name__)

app.secret_key = 'development key'

app.config["MAIL_SERVER"] = "smtp.gmail.com"
app.config["MAIL_PORT"] = 465
app.config["MAIL_USE_SSL"] = True
app.config["MAIL_USERNAME"] = 'contact@example.com'
app.config["MAIL_PASSWORD"] = 'your-password'

from routes import mail
mail.init_app(app)

import intro_to_flask.routes

The top of routes.py now looks like this:

app/intro_to_flask/routes.py

from intro_to_flask import app
from flask import render_template, request, flash
from forms import ContactForm
from flask.ext.mail import Message, Mail

mail = Mail()
.
.
.
# @app.route() mappings start here

We previously had app.run() inside of routes.py, which allowed us to type $ python routes.py to run the application. Since the app is now organized as a package, we need to employ a different strategy. The Flask docs recommend adding a new file named runserver.py and placing it inside app/. Let’s do that now:

flaskapp/
└── app/
        ├── intro_to_flask/
        │      ├── __init__.py
        │      ├── static/
        │      ├── templates/
        │      ├── forms.py
        │      ├── routes.py
        ├── runserver.py        
        └── README.md

Now take the app.run() call from routes.py and place it inside of runserver.py.

app/runserver.py

from intro_to_flask import app

app.run(debug=True)

Now you can type $ python runserver.py and view the app in the browser. From the top, here’s how you’ll enter your development environment and run the app:

$ cd flaskapp/
$ . bin/activate
$ cd app/
$ python runserver.py

The app is now organized as a package, we’re ready to move on and install a database to manage user credentials.

— Checkpoint: 13_packaged_app —

Flask-SQLAlchemy

We’ll use MySQL for our database engine and the Flask-SQLAlchemy extension to manage all of the database interaction.

Flask-SQLAlchemy uses Python objects instead of SQL statements to query the database. For example, instead of writing SELECT * FROM users WHERE firstname = "lalith", you would write User.query.filter_by(username="lalith").first().

The moral of this aside is to not completely rely on, or abandon a database abstraction layer like Flask-SQLAlchemy, but to be aware of it, so that you can determine when it’s useful for your needs.

But why can’t we just write raw SQL statements? What’s the point of using this weird syntax? As with most things, using Flask-SQLAlchemy, or any database abstraction layer, depends on your needs and preferences. Using Flask-SQLAlchemy allows you to work with your database by writing Python code instead of SQL. This way you don’t have SQL statements scattered amidst your Python code, and that’s a good thing, from a code quality perspective.

Also, if implemented correctly, using Flask-SQLAlchemy will help make your application to be database-agnostic. If you start building your app on top of MySQL and then decide to switch to another database engine, you shouldn’t have to rewrite massive chunks of sensitive database code. You could simply switch out Flask-SQLAlchemy with your new database abstraction layer without much of an issue. Being able to easily replace components is called modularity, and it’s a sign of a well designed application.

On the other hand, it might be more intuitive and readable if you write raw SQL statements instead of learning how to translate it into Flask-SQLAlchemy’s Expression Language. Fortunately, it’s possible to write raw SQL statements in Flask-SQLAlchemy too, if that’s what you need.

The moral of this aside is to not completely rely on, or abandon a database abstraction layer like Flask-SQLAlchemy, but to be aware of it, so that you can determine when it’s useful for your needs. For the database queries in this article, I’ll show you both the Expression Language version and the equivalent SQL statement.

Installing MySQL

Check to see if your system already has MySQL by running the following command in your terminal:

$ mysql --version

If you see a version number, you can skip to the “Creating a Database” section. If the command was not found, you’ll need to install MySQL. With the large variety of different operating systems out there, I’ll defer to Google to provide installation instructions that work for your OS. The installation usually consists of running a command or an executable. For example, the Linux command is:

$ sudo apt-get install mysql-server mysql-client

Creating a Database

Once MySQL is installed, create a database for your app called ‘development‘. You can do this from a web interface like phpMyAdmin or from the command line, as shown below:

$ mysql -u username -p
Enter password:

mysql> CREATE DATABASE development;

Installing Flask-SQLAlchemy

Inside the isolated development environment, install Flask-SQLAlchemy.

$ pip install flask-sqlalchemy

When I tried to install Flask-SQLAlchemy, I received an error stating that the installation had failed. I searched the error and found that others had resolved the problem by installing libmysqlclient15-dev, which installs MySQL’s development files. If your Flask-SQLAlchemy installation fails, Google the error for solutions or leave a comment and we’ll try to help you figure it out.

Configuring Flask-SQLAlchemy

Just as we did with Flask-Mail, we need to configure Flask-SQLAlchemy so that it knows where the development database lives. First, create a new file named models.py, along with adding in the following code

app/intro_to_flask/models.py

from flask.ext.sqlalchemy import SQLAlchemy

db = SQLAlchemy()

Here we import the SQLAlchemy class from Flask-SQLAlchemy (line one) and create a variable named db, containing a usable instance of the SQLAlchemy class (line three).

Next, open __init__.py and add the following lines after mail.init_app(app) and before import intro_to_flask.routes.

app/intro_to_flask/__init__.py

app.config['SQLALCHEMY_DATABASE_URI'] = 'mysql://your-username:your-password@localhost/development'

from models import db
db.init_app(app)

Let’s go over this:

Line one tells the Flask app to use the ‘development‘ database. We specify this through a data URI which follows the pattern of: mysql://username:password@server/database. The server is ‘localhost’ because we’re developing locally. Make sure to fill in your MySQL username and password.
db, the usable instance of the SQLAlchemy class we created in models.py, still doesn’t know what database to use. So we import it from models.py (line three) and bind it to our app (line four), so that it also knows to use the ‘development‘ database. We can now query the ‘development‘ database through our db object.

Now that our configuration is complete, let’s ensure that everything works. Open routes.py and create a temporary URL mapping so that we can perform a test query.

app/intro-to-flask/routes.py

from intro_to_flask import app
from flask import Flask, render_template, request, flash, session, redirect, url_for
from forms import ContactForm, SignupForm, SigninForm
from flask.ext.mail import Message, Mail
from models import db
.
.
.
@app.route('/testdb')
def testdb():
  if db.session.query("1").from_statement("SELECT 1").all():
    return 'It works.'
  else:
    return 'Something is broken.'

First we import the database object (db) from models.py (line five). We then create a temporary URL mapping (lines 9-14) wherein we issue a test query to ensure that the Flask app is connected to the ‘development‘ database. Now when we visit the URL /testdb, a test query will be issued (line 11); this is equivalent to the SQL statement SELECT 1;. If all goes well, we’ll see “It works” in the browser. Otherwise, we’ll see an error message stating what went wrong.

I received an error when I visited the /testdb URL: ImportError: No module named MySQLdb. This meant that I didn’t have the mysql-python library installed, so I tried to install it by typing the following:

$ pip install mysql-python

That installation failed, too. The new error message suggested that I first run easy_install -U distribute and then try the mysql-python installation again. So I did, just like below:

$ easy_install -U distribute
$ pip install mysql-python

This time the mysql-python installation succeeded, and then I received the “It works” success message in the browser. Now the reason I’m recounting the errors I’ve received and what I did to solve them is because installing and connecting to databases can be a tricky process. If you get an error message, please don’t get discouraged. Google the error message or leave a comment, and we’ll figure it out.

Once the test query works, delete the temporary URL mapping from routes.py. Make sure to retain the “from models import db“” part, because we’ll need it next.

— Checkpoint: 14_db_config —

Create a User Model

It’s not a good idea to store passwords in plain text, for security reasons.

Inside the ‘development‘ database, we need to create a users table where we can store each user’s information. The information we want to collect and store are the user’s first name, last name, email, and password.

It’s not a good idea to store passwords in plain text, for security reasons. If an attacker gains access to your database, they would be able to see each user’s login credentials. One way to defend against such an attack is to encrypt passwords with a hash function and a salt (some random data), and store that encrypted value in the database instead of the plain text password. When a user signs in again, we’ll collect the password that was submitted, hash it, and check if it matches the hash in the database. Werkzeug, the utility library on which Flask is built, provides the functions generate_password_hash and check_password_hash for these two tasks, respectively.

With this in mind, here are the columns we’ll need for the users table:

Column	Type	Constraints
uid	int	Primary Key, Auto Increment
firstname	varchar(100)
lastname	varchar(100)
email	varchar(120)	Unique
password	varchar(54)

Just like before, you can create this table from a web interface such as phpMyAdmin or from the command line, as shown below:

mysql> CREATE TABLE users (
uid INT NOT NULL PRIMARY KEY AUTO_INCREMENT,
firstname VARCHAR(100) NOT NULL,
lastname VARCHAR(100) NOT NULL,
email VARCHAR(120) NOT NULL UNIQUE,
pwdhash VARCHAR(100) NOT NULL
);

Next, in models.py, let’s create a class to model a user with attributes for a user’s first name, last name, email, and password.

app/intro_to_flask/models.py

from flask.ext.sqlalchemy import SQLAlchemy
from werkzeug import generate_password_hash, check_password_hash

db = SQLAlchemy()

class User(db.Model):
  __tablename__ = 'users'
  uid = db.Column(db.Integer, primary_key = True)
  firstname = db.Column(db.String(100))
  lastname = db.Column(db.String(100))
  email = db.Column(db.String(120), unique=True)
  pwdhash = db.Column(db.String(54))
  
  def __init__(self, firstname, lastname, email, password):
    self.firstname = firstname.title()
    self.lastname = lastname.title()
    self.email = email.lower()
    self.set_password(password)
    
  def set_password(self, password):
    self.pwdhash = generate_password_hash(password)
  
  def check_password(self, password):
    return check_password_hash(self.pwdhash, password)

We use the set_password() function to set a salted hash of the password, instead of using the plain text password itself.

Lines one and four already existed in models.py, so we’ll start on line two by importing the generate_password_hash and check_password_hash security functions from Werkzeug. Next, we create a new class named User, inheriting from the database object db‘s Model class (line six.)

Inside of our User class, we create attributes for the table’s name, primary key, and the user’s first name, last name, email, and password (lines 10-14). We then write a constructor which sets the class attributes (lines 17-20). We save names in title case and email addresses in lowercase to ensure a match regardless of how a user types in his credentials on subsequent sign ins.

We use the set_password function (lines 22-23) to set a salted hash of the password, instead of using the plain text password itself. Lastly, we have a function named check_password that uses check_password_hash, to check a user’s credentials on any subsequent sign ins (lines 25-26).

— Checkpoint: 15_user_model —

Sweet! We’ve created a users table and a user model, thereby laying down the foundation of our authentication system. Now let’s build the first user-facing component of the authentication system: the signup page.

Building a Signup Page

Planning

Take a look at Fig. 1 below, to see how everything will fit together.

Fig. 1

Implement SSL site-wide so that passwords and session tokens cannot be intercepted.

Let’s go over the figure from above:

A user visits the URL /signup to create a new account. The page is retrieved through an HTTP GET request and loads in the browser.
The user fills in the form fields with his first name, last name, email, and password.
The user clicks the “Create account” button, and the form submits to the server with an HTTP POST request.
On the server, a function validates the form data.
If one or more fields do not pass validation, the signup page reloads with a helpful error message, prompting the the user to try again.
If all fields are valid, a new User object will be created and saved into the database. The user will then be signed in and redirected to a profile page.

This sequence of steps should look familiar, as it’s identical to the sequence of steps we took to create a contact form. Here, instead of sending an email at the end, we save a user’s credentials to the database. The previous article already explained creating a form in detail, I’ll move more quickly in this section so that we can get to the more exciting parts, faster.

Creating a Signup Form

We installed Flask-WTF in the previous article, so let’s proceed with creating a new form inside forms.py.

app/intro_to_flask/forms.py

from flask.ext.wtf import Form, TextField, TextAreaField, SubmitField, validators, ValidationError, PasswordField
from models import db, User
.
.
.
class SignupForm(Form):
  firstname = TextField("First name",  [validators.Required("Please enter your first name.")])
  lastname = TextField("Last name",  [validators.Required("Please enter your last name.")])
  email = TextField("Email",  [validators.Required("Please enter your email address."), validators.Email("Please enter your email address.")])
  password = PasswordField('Password', [validators.Required("Please enter a password.")])
  submit = SubmitField("Create account")

  def __init__(self, *args, **kwargs):
    Form.__init__(self, *args, **kwargs)

  def validate(self):
    if not Form.validate(self):
      return False
    
    user = User.query.filter_by(email = self.email.data.lower()).first()
    if user:
      self.email.errors.append("That email is already taken")
      return False
    else:
      return True

We start by importing one more Flask-WTF class named PasswordField (line one), which is like TextField except that it generates a password textbox. We’ll need the db database object and the User model to handle some custom validation logic inside the SignupForm class; so we import them too (line two).

Then we create a new class named SignupForm containing a field for each piece of user information we wish to collect (lines 7-11). There’s a presence validator on each field to ensure it’s filled in, and a format validator which requires that email addresses match the pattern: user@example.com.

Next, we write a simple constructor for the class that just calls the base class’ constructor (lines 13-14).

So we’ve added some presence and format validators to our form fields, but we need an additional validator that ensures an account does not already exist with the user’s email address. To do this we hook into Flask-WTF’s validation process (lines 16-25).

Now inside of the validate() function, we first ensure the presence and format validators run by calling the base class’ validate() method; if the form is not filled in properly, validate() returns False (lines 16-17).

Next we define the custom validator. We start by querying the database with the email that the user submitted (line 18). If you remember from our models.py file, the email address is converted to lowercase to ensure a match regardless of how it was typed in. This Flask-SQLAlchemy expression corresponds to the following SQL statement:

SELECT * FROM users 
  WHERE email = self.email.data.lower() 
  LIMIT 1

If a user record already exists with the submitted email, validation fails giving the following error message: “That email is already taken” (lines 21-22).

Using the Signup Form

Let’s now create a new URL mapping and a new web template for the signup form. Open routes.py and import the newly created signup form so that we can use it.

app/intro_to_flask/routes.py

from intro_to_flask import app
from flask import render_template, request, flash
from forms import ContactForm, SignupForm

Next, create a new URL mapping.

app/intro_to_flask/routes.py

@app.route('/signup', methods=['GET', 'POST'])
def signup():
  form = SignupForm()
  
  if request.method == 'POST':
    if form.validate() == False:
      return render_template('signup.html', form=form)
    else:   
      return "[1] Create a new user [2] sign in the user [3] redirect to the user's profile"
  
  elif request.method == 'GET':
    return render_template('signup.html', form=form)

Inside the signup() function, we create a variable named form that contains a usable instance of the SignupForm class. If a GET request has been issued, we’ll return the signup.html web template containing the signup form for the user to fill out.

Otherwise, we’ll see just a temporary placeholder string. For now, the temp string lists the three actions that should take place when the form has been successfully submitted. We’ll come back and replace this string with real code in “The First Signup” section below.

Now that we’ve created a URL mapping, the next step is to create the web template signup.html and place it inside the templates/ folder.

app/intro_to_flask/templates/signup.html

{% extends "layout.html" %}

{% block content %}
  <h2>Sign up</h2>

  {% for message in form.firstname.errors %}
    <div class="flash">{{ message }}</div>
  {% endfor %}
  
  {% for message in form.lastname.errors %}
    <div class="flash">{{ message }}</div>
  {% endfor %}
  
  {% for message in form.email.errors %}
    <div class="flash">{{ message }}</div>
  {% endfor %}
  
  {% for message in form.password.errors %}
    <div class="flash">{{ message }}</div>
  {% endfor %}
  
  <form action="{{ url_for('signup') }}" method=post>
    {{ form.hidden_tag() }}
    
    {{ form.firstname.label }}
    {{ form.firstname }}
    
    {{ form.lastname.label }}
    {{ form.lastname }}
    
    {{ form.email.label }}
    {{ form.email }}
    
    {{ form.password.label }}
    {{ form.password }}
    
    {{ form.submit }}
  </form>
    
{% endblock %}

This template looks just like contact.html. We first loop through and display any error messages if necessary. We then let Jinja2 generate most of the HTML form for us. Remember how in the Signup form class we appended the error message “That email is already taken” to self.email.errors? That’s the same object that Jinja2 loops through in this template.

The one difference from the contact.html template is the omission of the if...else logic.

In this template, we want to register and sign in the user on a successful form submission. This takes place on the back-end, so the if...else statement is not needed here.

Finally, add in these CSS rules to your main.css file so that the signup form looks nice and pretty.

app/intro_to_flask/static/css/main.css

/* Signup form */
form input#firstname,
form input#lastname,
form input#password {
  width: 400px;
  background-color: #fafafa;
  -webkit-border-radius: 3px;
     -moz-border-radius: 3px;
          border-radius: 3px;
  border: 1px solid #cccccc;
  padding: 5px;
  font-size: 1.1em;
}

form input#password {
  margin-bottom: 10px;
}

Let’s check out the newly created signup page by typing:

$ cd app/
$ python runserver.py

And browse to http://localhost:5000/signup in your favorite web browser.

Excellent! We just created a signup form from scratch, handled complex validation, and created a good looking signup page with helpful error messages.

— Checkpoint: 16_signup_form —

If any of these steps were unclear, please take a moment to review the previous article. It covers each step in greater detail, and I followed the same steps from that article, to create this signup form.

The First Signup

Let’s start by replacing the temporary placeholder string in routes.py‘s signup() function with some real code. Upon a successful form submission, we need to create a new User object, save it to the database, sign the user in, and redirect to the user’s profile page. Let’s take this step by step, starting with creating a new User object and saving it to the database.

Saving a New User Object

Add in lines five and 17-19 to

routes.py.

app/intro_to_flask/routes.py

from intro_to_flask import app
from flask import render_template, request, flash, session, url_for, redirect
from forms import ContactForm, SignupForm
from flask.ext.mail import Message, Mail
from models import db, User
.
.
.
@app.route('/signup', methods=['GET', 'POST'])
def signup():
  form = SignupForm()
  
  if request.method == 'POST':
    if form.validate() == False:
      return render_template('signup.html', form=form)
    else:
      newuser = User(form.firstname.data, form.lastname.data, form.email.data, form.password.data)
      db.session.add(newuser)
      db.session.commit()
      
      return "[1] Create a new user [2] sign in the user [3] redirect to the user's profile"
  
  elif request.method == 'GET':
    return render_template('signup.html', form=form)

First, we import the User class from models.py so that we can use it in the signup() function (line five). Then we create a new User object called newuser and populate it with the signup form's field data (line 17).

Next, we add newuser to the database object's session (line 18), which is Flask-SQLAlchemy's version of a regular database transaction. The add() function generates an INSERT statement using the User object's attributes. The equivalent SQL for this Flask-SQLAlchemy expression is:

  INSERT INTO users (firstname, lastname, email, pwdhash)
  VALUES (form.firstname.data, form.lastname.data, form.email.data, form.password.data)

Lastly, we update the database with the new user record by committing the transaction (line 19).

Signing in the User

Next, we need to sign in the user. The Flask app needs to know that subsequent page requests are coming from the browser of the user who has successfully signed up. We can accomplish this by setting a cookie in the user's browser containing some sort of ID and associating that key with the user's credentials in the Flask app.

This way, the ID in the browser's cookie will be passed to the app on each subsequent page request, and the app will look up the ID to determine whether it maps to valid user credentials.

If it does, the app allows access to the parts of the website that you need to be signed in for. This combination of having a key stored on the client and a value stored on the server is called a session.

Flask has a session object that accomplishes this functionality. It stores the session key in a secure cookie on the client and the session value in the app. Let's use it in our signup() function.

app/intro_to_flask/routes.py

from flask import render_template, request, flash, session
.
.
.
@app.route('/signup', methods=['GET', 'POST'])
def signup():
  form = SignupForm()
  
  if request.method == 'POST':
    if form.validate() == False:
      return render_template('signup.html', form=form)
    else:
      newuser = User(form.firstname.data, form.lastname.data, form.email.data, form.password.data)
      db.session.add(newuser)
      db.session.commit()
      
      session['email'] = newuser.email
      
      return "[1] Create a new user [2] sign in the user [3] redirect to the user's profile"
  
  elif request.method == 'GET':
    return render_template('signup.html', form=form)

We start by importing Flask's session object on line one. Next, we associate the key 'email' with the value of the newly registered user's email (line 17). The session object will take care of hashing 'email' into an excrypted ID and storing it in a cookie on the user's browser. At this point, the user is signed in to our app.

Redirecting to a Profile page

The last step is to redirect the user to a Profile page after signing in. We'll use the url_for function (which we've seen in layout.html and contact.html) in conjunction with Flask's redirect() function.

app/intro_to_flask/routes.py

from intro_to_flask import app
from flask import render_template, request, flash, session, url_for, redirect
.
.
.
@app.route('/signup', methods=['GET', 'POST'])
def signup():
  form = SignupForm()
  
  if request.method == 'POST':
    if form.validate() == False:
      return render_template('signup.html', form=form)
    else:
      newuser = User(form.firstname.data, form.lastname.data, form.email.data, form.password.data)
      db.session.add(newuser)
      db.session.commit()
      
      session['email'] = newuser.email
      return redirect(url_for('profile'))
  
  elif request.method == 'GET':
    return render_template('signup.html', form=form)

on line two, we import Flask's url_for() and redirect() functions. Then on line 19, we replace our temporary placeholder string with a redirect to the URL /profile. We don't have a URL mapping for /profile yet, so let's create that next.

app/intro_to_flask/routes.py

@app.route('/profile')
def profile():

  if 'email' not in session:
    return redirect(url_for('signin'))

  user = User.query.filter_by(email = session['email']).first()

  if user is None:
    return redirect(url_for('signin'))
  else:
    return render_template('profile.html')

Here we can finally see sessions in action. We start on line four by fetching the browser's cookie and checking if it contains a key named 'email'. If it doesn't exist, that means the user is not authenticated, so we redirect the user to a signin page (we'll create this in the next section).

If the 'email' key does exist, we look up the server-side user email value associated with the key using session['email'], and then query the database for a registered user with this same email address (line seven). The equivalent SQL for this Flask-SQLAlchemy expression is:

SELECT * FROM users WHERE email = session['email'];

If no registered user exists, we'll redirect to the signup page. Otherwise, we render the profile.html template. Let's create profile.html now.

app/intro_to_flask/templates/profile.html

{% extends "layout.html" %}
{% block content %}
  <div class="jumbo">
    <h2>Profile<h2>
    <h3>This is {{ session['email'] }}'s profile page<h3>
  </div>
{% endblock %}

I've kept this profile template simple. If we focus in on line five — you'll see that we can use Flask's session object inside Jinja2 templates. Here, I've used it to create a user-specific string, but you could use this ability to pull other types of user-specific information instead.

We're finally ready to see the result of all our hard work. Type the following into your terminal:

$ python runserver.py

Go to http://localhost:5000/ in your favorite web browser, and complete the sign up process. You should be greeted with a profile page that looks like the following screenshot:

Signing up users is a huge milestone for our app. We can adapt the code in our /signup() function and round out our authentication system by allowing users to sign in and out of the app.

— Checkpoint: 17_profile_page —

Building a Signin Page

Creating a signin page is similar to creating a signup page — we'll need to create a signin form, a URL mapping, and a web template. Let's start by creating the SigninForm class in forms.py.

app/intro_to_flask/forms.py

class SigninForm(Form):
  email = TextField("Email",  [validators.Required("Please enter your email address."), validators.Email("Please enter your email address.")])
  password = PasswordField('Password', [validators.Required("Please enter a password.")])
  submit = SubmitField("Sign In")
  
  def __init__(self, *args, **kwargs):
    Form.__init__(self, *args, **kwargs)

  def validate(self):
    if not Form.validate(self):
      return False
    
    user = User.query.filter_by(email = self.email.data.lower()).first()
    if user and user.check_password(self.password.data):
      return True
    else:
      self.email.errors.append("Invalid e-mail or password")
      return False

The SigninForm class is similar to the SignupForm class. To sign a user in, we need to capture their email and password, so we create those two fields with presence and format validators (lines 2-3). Then we define our custom validator inside the validate() function (lines 10-15). This time the validator needs to make sure the user exists in the database and has the correct password. If a record does exist with the supplied information, we check to see if the password matches (line 14). If it does, the validation check passes (line 15), otherwise they get an error message.

Next, let's create a URL mapping in routes.py.

app/intro_to_flask/routes.py

...
from forms import ContactForm, SignupForm, SigninForm
.
.
.
@app.route('/signin', methods=['GET', 'POST'])
def signin():
  form = SigninForm()
  
  if request.method == 'POST':
    if form.validate() == False:
      return render_template('signin.html', form=form)
    else:
      session['email'] = form.email.data
      return redirect(url_for('profile'))
                
  elif request.method == 'GET':
    return render_template('signin.html', form=form)

Once again, the signin() function is similar to the signup() function. We import SigninForm (line two), so that we can use it in the signin() function. Then in signin(), we return the signin.html template for GET requests (lines 17-18).

If the form has been POSTed and any validation check fails, the signin form reloads with a helpful error message (lines 11-12). Otherwise, we sign in the user by creating a new session and redirecting to their profile page (lines 14-15).

Lastly, let's create the web template signin.html.

app/intro_to_flask/templates/signin.html

{% extends "layout.html" %}

{% block content %}
  <h2>Sign In</h2>

  {% for message in form.email.errors %}
    <div class="flash">{{ message }}</div>
  {% endfor %}
  
  {% for message in form.password.errors %}
    <div class="flash">{{ message }}</div>
  {% endfor %}
  
  <form action="{{ url_for('signin') }}" method=post>
    {{ form.hidden_tag() }}
    
    {{ form.email.label }}
    {{ form.email }}
    
    {{ form.password.label }}
    {{ form.password }}
    
    {{ form.submit }}
  </form>
    
{% endblock %}

Similar to signup.html template, we first loop through and display any error messages, then we let Jinja2 generate the form for us.

And that does it for the signin page. Visit http://localhost:5000/signin to check it out. Go ahead and sign in, you should get redirected to your profile page.

— Checkpoint: 18_signin_form —

Signing Out

In "The First Signup" section above, we saw that "signing in" meant setting a cookie in the user's browser containing an ID and associating that ID with the user's data in the Flask app. Therefore, "signing out" means clearing the cookie in the browser and dissociating the user data.

This can be accomplished in one line: session.pop('email', None).

We don't need a form or even a web template to sign out. All we need is a URL mapping in routes.py, which terminates the session and redirects to the Home page. The mapping, therefore, is short and sweet:

app/intro_to_flask/routes.py

@app.route('/signout')
def signout():

  if 'email' not in session:
    return redirect(url_for('signin'))
    
  session.pop('email', None)
  return redirect(url_for('home'))

The user is not authenticated if the browser's cookie does not contain a key named 'email', in that case, we just redirect to the signin page (lines 4-5). Otherwise, we terminate the session (line seven) and redirect back to the home page (line eight).

You can test the sign out functionality by visiting http://localhost:5000/signout. If you're signed in, the app will sign you out and redirect to http://localhost:5000/. Once you've signed out, try visiting the profile page http://localhost:5000/profile. You shouldn't be allowed to see the profile page if you're not signed in, and the app should redirect you back to the Signin page.

— Checkpoint: 19_signout —

Tidying Up

Now we need to update the site header with navigation links for "Sign Up", "Sign In", "Profile", and "Sign Out". The links should change based on whether the user is signed in or not. If the user is signed out, links for "Sign Up" and "Sign In" should be visible. When the user is signed in, we want links for "Profile" and "Sign Out" to appear, while hiding the "Sign Up" and "Sign In" links.

So how can we do this? Think back to the profile.html template where we used Flask's session object. We can use the session object to show navigation links based on the user's authentication status. Let's open layout.html and make the following changes:

app/intro_to_flask/templates/layout.html

<!DOCTYPE html>
<html>
  <head>
    <title>Flask App</title>
    <link rel="stylesheet" href="{{ url_for('static', filename='css/main.css') }}">
  </head>
  <body>

  <header>
    <div class="container">
      <h1 class="logo">Flask App</h1>
      <nav>
        <ul class="menu">
          <li><a href="{{ url_for('home') }}">Home</a></li>
          <li><a href="{{ url_for('about') }}">About</a></li>
          <li><a href="{{ url_for('contact') }}">Contact</a></li>
          {% if 'email' in session %}
          <li><a href="{{ url_for('profile') }}">Profile</a></li>
          <li><a href="{{ url_for('signout') }}">Sign Out</a></li>
          {% else %}
          <li><a href="{{ url_for('signup') }}">Sign Up</a></li>
          <li><a href="{{ url_for('signin') }}">Sign In</a></li>
          {% endif %}
        </ul>
      </nav>
    </div>
  </header>

    <div class="container">
      {% block content %}
      {% endblock %}
    </div>

  </body>
</html>

Starting on line 17, we use Jinja2's if...else syntax and the session() object to check if the browser's cookie contains the 'email' key. If it does, then the user is signed in and therefore should see the "Profile" and "Sign Out" navigation links. Otherwise, the user is signed out and should see links to "Sign Up" and "Sign In".

Now give it a try! Check out how the navigation links appear and disappear by signing in and out of the app.

The last task remaining is a similar issue: when a user is signed in, we don't want him to be able to visit the signup and signin pages. It makes no sense for a signed in user to authenticate themselves again. If signed in users try to visit these pages, they should instead be redirected to their profile page. Open routes.py and add the following piece of code to the beginning of the signup() and signin() functions:

if 'email' in session:
  return redirect(url_for('profile'))

Here's what your routes.py file will look like after adding in that snippet of code:

app/intro_to_flask/routes.py

@app.route('/signup', methods=['GET', 'POST'])
def signup():
  form = SignupForm()

  if 'email' in session:
    return redirect(url_for('profile')) 
.
.
.
@app.route('/signin', methods=['GET', 'POST'])
def signin():
  form = SigninForm()

  if 'email' in session:
    return redirect(url_for('profile'))  
  ...

And with that we're finished! Try visiting the the "signup" or "signin" pages while you are currently signed in, to test it out.

— Checkpoint: 20_visibility_control —

Conclusion

We've accomplished a lot in this article. We've taken our Flask app from being a simple Python module and turned it into a well organized application, capable of handling user authentication.

There are several directions in which you can take this app from here. Here are some ideas:

Let users sign in with an existing account, such as their Google account, by adding support for OpenID.
Give users the ability to update their account information, as well as delete their account.
Let users reset their password if they forget it.
Implement an authorization system.
Deploy to a production server. Note that when you deploy this app to production, you will need to implement SSL site-wide so that passwords and session tokens cannot be intercepted. If you deploy to Heroku, you can use their SSL certificate.

So go forth and continue to explore Flask, and build your next killer app! Thanks for reading.

Working With Data in Sails.js

Gabriel Manricks — Wed, 12 Jun 2013 01:49:56 +0000

Sails.js is an up and coming Node.js framework, with a focus on freedom and smart defaults. In this article we’ll take a look at some of the data features Sails provides out-of-the-box, for easily making complex apps.

Why Sails Is Different Than Other Frameworks

The reason to choose Sails is best put by the Sails creator, Mike McNeil, "Sails was created out of necessity". Many frameworks you see around, are built almost for the academic side of things, these frameworks usually foster best practices, and create a platform for developers to create things faster, or better.

Sails on the other hand, was created for production, it’s not trying to feed you a new syntax or platform, it’s a solid base, meant for creating 'client-work' with speed. The contrast may be subtle, but there are a few distinct differences.

To illustrate what I’m referring to, let's take a look at Meteor. Meteor is probably the leading JS platform around today, but it’s a prime example of a framework, for the sake of a framework. Now this is not a bad thing, I am a big supporter of Meteor, what I mean is, they set out to build a framework and they did a good job at it, Mike on the other hand set out to make client-work faster. Sails is just a means to reach an end.

In Meteor, pretty much everything is abstracted away and you use JavaScript plus the Meteor API to code everything. Whereas Sails is not meant to be a new platform, so nothing is hidden.

It rests on top of Socket.io and the popular Express framework, and you have access to them in their entirety, natively. Are you beginning to see the difference?

Furthermore, since Sails is geared for production first, it’s built with multiple options for scaling and security.

There is alot to talk about, but in this article I would like to focus on how Sails handles data, and how you can leverage some of Sails' more advanced features to perform some really cool actions.

Installation

Just in case you do not have Sails intalled yet, you can do so via NPM by running:

sudo npm install -g sails

Socket.io & Express

Now let's talk a little bit about Socket.io and Express before we get into Sails. There’s a good premium series on Express by Andrew Burgess if you’re interested, but I'll run through the relevant basics of both of these libraries here:

Socket.io

Socket.io is a pub/sub library, which is run on both the server and on the client, and it allows them to speak via web sockets.

A brief example could look something like this:

//Code For Server
var io = require("socket.io");
io.sockets.on("connection", function (sock) {
    sock.emit("welcomeMessage", { hello: "world" });
}
io.listen(80);

This code starts out by requiring the socket.io library, listening for a connection, and then when another socket connects, it will send it a message, addressed to the welcomeMessage event, and finally passing along some JSON.

Next, on the client you would write something like:

//Code For Client
var sock = io.connect('http://localhost');
sock.on('welcomeMessage', function (json) {
    //Handle Event Received
});

Here we’re connecting to the server and listening for that welcomeMessage event we just created. As you can see it's a fairly simple publish / subscribe server, which is bidirectional (the client could emit messages for the server as well).

Now let's take a look at Express:

Express

The simplest form of an Express route could be something like:

app.get('/users', function(req, res) {
    res.send("Hello from '/users' !");
});

This defines a simple route, so that when a user goes to your site's address and tries to access the /users page, they will be presented with the message "Hello from '/users' !".

So Express is a framework for handling HTTP requests and Socket.io is a websocket communications library. What the Sails team have done though, is map all Express routes to Socket.io internally. What this means is, you can call any of the HTTP routes through web sockets.

Now that’s pretty cool! But, there is still one piece of the puzzle missing and that is the Sails Blueprints.

Sails allows you to generate models just like in other frameworks, the difference is, Sails can also generate a production ready RESTfull API to go with them. This means if you generate a model named 'users' you can immediately run RESTfull queries on the '/users' resource without any coding necessary.

If you are new to RESTful APIs, it’s just a way of accessing data, to where CRUD operations are mapped to various HTTP methods.

So a GET request to '/users' will get all of the users, a POST request will create a new user, etc.

So what does all of this mean?

It means we have a full RESTfull API, mapped to Socket.io via Sails, without writing a single line of code!

But why are sockets better at retrieving data then an Ajax request? Well, besides being a leaner protocol, sockets stay open for bidirectional communication, and Sails has taken advantage of this. Not only will Sails pass you the data, but it will automatically subscribe you to updates on that database, and whenever something gets added, removed, or updated, your client will receive a notification via the web socket, letting you know about it.

This is why Sails is so awesome!

Sails + Backbone

The next topic I’d like to cover is Backbone integration, because if you aren't using a JavaScript framework, you are doing it wrong.

With this in mind, Sails and Backbone are the perfect pair. Backbone, like Sails, is extremely unobtrusive, all of its features are available, capable of being overridden, and optional.

If you have used Backbone before then you may know it connects natively with REST APIs, so out of the box, you can sync the data on the front-end with your Sails application.

But enough talk for now, let's take a look at all of this in action by creating a basic chat application. To get started, open up a terminal window and type:

sails new ChatApp
cd ChatApp
sails generate model users
sails generate model messages
sails generate controller messages
sails generate controller main

This will create a new app and generate some files for us. You can see from above, there are two different resources that you can generate; models and controllers. If you are familiar with the MVC design pattern, then you should know what these are, but in short, models are your data and controllers hold your logic code. So we are going to need two collections, one to hold the users, and one for the messages.

Next, for the controllers, we need one to handle the page routes, I called it 'main', then we have a second controller named 'messages'. Now you might wonder why I created a controller with the same name as our messages model? Well, if you remember, I said that Sails can create a REST API for you. What happens is, by creating a blank controller with the same name as a model, Sails will know to fall back and build a REST API for the corresponding resource.

So, we’ve created a controller for our messages model, but there’s no need to create one for the users model, so I just left it out. And that’s all there is to creating models and controllers.

Next, let’s setup some routes.

Routes

Routes are always a safe place to begin, because you usually have a good idea of which pages are going to be made.

So open up the routes.js file which is in the config folder, it may look a little overwhelming at first, but if you remove all of the comments and add the in the following routes, you will be left with something like this:

module.exports.routes = {
    '/' : {
         controller: 'main',
         action: 'index'
    },
    '/signup' : {
         controller: 'main',
         action: 'signup'
    },
    '/login' : {
         controller: 'main',
         action: 'login'
    },
    '/chat' : {
         controller: 'main',
         action: 'chat'
    }
};

We have a home page, a chat page, and then two pages for handling both the login and signup pages. I put them all in the same controller, but in Sails, you can create as many controllers as you’d like.

Models

Next, let's take a look at the generated messages model which can be located at "api > models > Messages.js". We need to add the necessary columns to our model. Now this is not absolutely necessary, but it will create some helper functions for us that we can use:

//Messages Model  
module.exports = {
    attributes  : {
        userId: 'INT',
        username: 'STRING',
        message: 'STRING'      
    }
};

For the messages model, we start with the id of the user that this message belongs to, a username so we won't have to query this separately, and then the actual message.

Now let's fill in the user's model:

//Users Model 
module.exports = {
    attributes  : {
         username: 'STRING',
         password: 'STRING'      
    }
};

And that’s it, we have just the username and password attributes. The next step is to create our route functions inside of the MainController.

Controllers

So open up the MainController, which can be found at "api > controllers > MainController.js". Let's begin by creating a function for each of the routes we defined above:

var MainController = {
    index: function (req, res) {
        
    },
    signup: function (req, res) {
        
    },
    login: function (req, res) {
        
    },
    chat: function (req, res) {
        
    }
};
module.exports = MainController;

If you’re familiar with Express, then you’ll be happy to see that these functions are standard Express route functions. They receive two variables, req for the HTTP request and res to create the response.

Following the MVC pattern, Sails offers a function for rendering views. The home page doesn't need anything special, so let's just render the view.

index: function (req, res) {
    res.view();
},

Sails leans more toward convention over configuration, so when you call res.view(); Sails will look for a view file (with a .ejs extension by default) using the following pattern: 'views > controllerName > methodName.ejs'. So for this call, it will search for 'views > main > index.ejs'. It's also worth noting, these views only contain the view specific parts of the page. If you take a look at 'views > layout.ejs', you will see a call in the middle for <%- body %>, this is where your view file will be inserted. By default it uses this 'layout.ejs' file, but you can use other layout files just by passing the layout name into the res.view() function, under the property named 'layout'. For example: 'res.view( { layout: "other.ejs" } );'.

I’m going to use the default layout file with a small adjustment, I’m going to add jQuery, Backbone, and Underscore. So in the 'layout.ejs' file right before the closing </head> tag, add the following lines:

<script src="//cdnjs.cloudflare.com/ajax/libs/jquery/1.9.1/jquery.min.js"></script>
<script src="//cdnjs.cloudflare.com/ajax/libs/underscore.js/1.4.4/underscore-min.js"></script>
<script src="//cdnjs.cloudflare.com/ajax/libs/backbone.js/1.0.0/backbone-min.js"></script>

With that in place we’re now ready to create the home page.

The Home Page

Let’s create a new folder inside of the views folder named main, and inside of our new main folder, we’ll create a new file named 'index.ejs'.

Inside the file let's just create a login and signup form:

<h1>Code Chat</h1>
<div>
    <h3>Login</h3>
    <input type="text" id="loginName" placeholder="name" />
    <input type="password" id="loginPassword" placeholder="password" />
    <button id="loginButton">Login</button>
</div>
<div>
    <h3>Signup</h3>
    <input type="text" id="signupName" placeholder="name" />
    <input type="password" id="signupPassword" placeholder="password" />
    <input type="password" id="signupConfirmPassword" placeholder="confirm password" />
    <button id="signupButton">Signup</button>
</div>

Pretty simple, just the essentials.

The Login and Signup Areas

Next we need to add a little JS to get this communicating with the server. Now this won’t be Sails specific, we are just going to send an AJAX request via jQuery to the Sails server.

This code can either be included on the page itself or loaded in via a separate JS file. For the sake of convenience, I’m just going to put it at the bottom of the same page:

<script>
    $("#loginButton").click(function(){
        var username = $("#loginName").val();
        var password = $("#loginPassword").val();
        if (username && password) {
            $.post(
                '/login',
                {username: username, password:password},
                function () {
                    window.location = "/chat";
                }
            ).fail(function(res){
                alert("Error: " + res.getResponseHeader("error"));
            });
        } else {
            alert("A username and password is required");
        }
    });
</script>

This is all just standard JS and jQuery, we’re listening for the click event on the login button, making sure the username and password fields are filled in, and posting the data to the '/login' route. If the login is successful, we redirect the user to the chat page, otherwise we will display the error returned by the server.

Next, let's create the same thing for the signup area:

$("#signupButton").click(function(){
    var username = $("#signupName").val();
    var password = $("#signupPassword").val();
    var confirmPassword = $("#signupConfirmPassword").val();
    if (username && password) {
        if (password === confirmPassword) {
            $.post(
                '/signup',
                {username: username, password:password},
                function () {
                    window.location = "/chat";
                }
            ).fail(function(res){
                alert("Error: " + res.getResponseHeader("error"));
            });
        } else {
            alert("Passwords don't match");
        }   
    } else {
        alert("A username and password is required");
    }
});

This code is almost identical, so much so, that you can probably just abstract the whole Ajax part out into its own function, but for this tutorial it's fine.

Now we need to go back to our 'MainController' and handle these two routes, but before we do that, I want to install a Node module. We’re going to need to hash the password, as plain text passwords are not a good thing, not even for demonstration! I found a nice module named 'password-hash' by David Wood which will work out nicely.

To install it, just go to the root of your Sails app, from within your terminal and type: npm install password-hash.

Once that installs, let's open the MainController and implement the two needed routes. Let’s start with signup:

signup: function (req, res) {
        var username = req.param("username");
        var password = req.param("password");
        
        Users.findByUsername(username).done(function(err, usr){
            if (err) {
                res.send(500, { error: "DB Error" });
            } else if (usr) {
                res.send(400, {error: "Username already Taken"});
            } else {
                var hasher = require("password-hash");
                password = hasher.generate(password);
                
                Users.create({username: username, password: password}).done(function(error, user) {
                if (error) {
                    res.send(500, {error: "DB Error"});
                } else {
                    req.session.user = user;
                    res.send(user);
                }
            });
        }
    });
}

It's a bit verbose, but all we’re doing here is reading the username and password from the POST request and making sure the username is not already taken. You can see I’m also using the password-hasher we just installed, it's super simple to use, just pass the password into the generate method and it will hash it using a random salt.

It's also worth mentioning, that at every possible location where we may run into an error or issue, we’re sending back an HTTP error code and returning a message via a custom header named 'error' which, if you remember, we’re displaying in an alert message on the index page.

Another noteworthy point, is the fact that we’re using a 'magic' function named 'findByUsername', this is made possible because we have a username column inside of our Users model.

Finally, at the bottom you can see if everything went well, we’re storing the user in a session variable and returning it with a default status code of 200, which will tell jQuery that the AJAX request was successful.

Next, let's write the login function:

login: function (req, res) {
    var username = req.param("username");
    var password = req.param("password");
    
    Users.findByUsername(username).done(function(err, usr) {
        if (err) {
            res.send(500, { error: "DB Error" });
        } else {
            if (usr) {
                var hasher = require("password-hash");
                if (hasher.verify(password, usr.password)) {
                    req.session.user = usr;
                    res.send(usr);
                } else {
                    res.send(400, { error: "Wrong Password" });
                }
            } else {
                res.send(404, { error: "User not Found" });
            }
        }
    });
}

Again, this is very similar to the previous signup function, we’re searching for a user with the same username that was posted from the form and if it finds one, we check if the password matches using the hasher’s verify method. The reason we can't just hash the password again and pass it into the models find function is because the hasher uses a random salt, so if we hashed the password again it would be equal to something else.

The rest of the code is the same; if everything checks out, we store the user in a session and return it, otherwise we send back an error message.

The login system is now complete and we’re finally able to move on to building the chat feature.

Building the Chat Feature

Since we’ll be using Backbone for getting the messages, the actual route function will be very simple. Here’s the complete chat function:

chat: function (req, res) {
    if (req.session.user) {
        res.view({username: req.session.user.username});
    } else {
        res.redirect('/');
    }
}

We start off by checking if the user is logged in or not, if this checks out, then it will load the view, passing it the username that was in the session, otherwise we just redirect to the home page.

Now let’s create a new view named 'chat.ejs' inside of the main folder. Open it up and let's create a simple form to post new messages and a div container for displaying all of them.

<h2>Welcome <%= username %></h2>
<div id="newMessageForm">
    <textarea id="message" placeholder="Enter your message here:"></textarea>
    <button id="postMessageButton">Add Message</button>
</div>
<div id="messagesContainer">
</div>

So for this view, we just used some pretty standard HTML. The only thing that may require some explanation is the <%= username %> code, this style of coding is not specific to Sails, it’s actually the syntax for EJS. This syntax is very similar to PHP’s short tags. <% is the equivalent of <? in PHP and <%= is the same as <?=. The first snippet of EJS allows you to integrate standard JS code on the page, whereas the second prints out the code within. Here we’re just printing out the username that we passed in from the controller.

The rest of our chat feature will be all JavaScript. To get started, let's take a look at how you would write the chat functionality using standard Backbone, and then we’ll see how to take advantage of web sockets.

At the bottom of the page, add the following JS:

<script>
    var MessageModel = Backbone.Model.extend({
        urlRoot: '/messages',
    });

    var MessageCollection = Backbone.Collection.extend({
        url: '/messages',
        model: MessageModel,
    });

    var messages = new MessageCollection();
    messages.fetch();

    $("#postMessageButton").click(function(){
        var messageText = $("#message").val();
        messages.create({message: messageText}, {wait: true});
        $("#message").val("");
    });
</script>

Since Sails automatically creates an API that Backbone understands natively, no extra server code has to be written, it doesn’t get much easier than that. This is what I was talking about when I said that Sails was not made to be a 'framework'. It doesn't try to make you use its own syntax, it was made to get things done and as you can see, it delivers.

To test it out, open up a terminal window and navigate to your Sails app folder, then type 'sails lift' to start it up. By default it will launch to http://localhost:1337. Now just signup and post a few messages.

To see your posted messages you can console.log the messages variable, or look at it inside of the browser’s console. Now the next thing we should implement is a view so we can see the posted messages in the browser.

_.templateSettings = {
    interpolate : /\{\{(.+?)\}\}/g
};
var MessagesView = Backbone.View.extend({
    el: '#messagesContainer',
    initialize: function () {
        this.collection.on('add', this.render, this);
        this.render();
    },
    template: _.template("<div><p>{{ message }}</p></div>"),
        render: function () {
            this.$el.html("");
            this.collection.each(function(msg){
                this.$el.append(this.template(msg.toJSON()));
            }, this)
    }
});

var mView = new MessagesView({collection: messages});

We start off by defining a view, attaching it to the div that we created earlier, then we add an event handler on the collection to re-render the div every time a new model gets added to the collection.

You can see at the top, I had to change the default Underscore settings from using the EJS syntax inside of the templates, to instead, use Mustache’s syntax. This is because the page is already an EJS document, so it would be processed out on the server and not in Underscore.

Note: I didn't come up with the Regex for this, that credit goes to the Underscore docs itself.

Lastly, at the bottom you can see we created a new instance of this view, passing it the collection variable.

If all went well, you should now see your messages in the browser, and it should update whenever you create a new post.

Sails Policies

Now you may have noticed that we’re not setting the userId or the username when we submit the posts, and this is for security purposes.

You don't want to put this kind of control on the client side. If all someone has to do is modify a JavaScript variable to control another user’s account, you’ll have a major problem.

So, how should you handle this? Well, with policies of course.

Policies are basically middleware, which run before the actual web request, where you can stop, modify, or even redirect the request, as needed.

For this app, let's create a policy for our messages. Policies are applied to controllers, so they can even be run on normal pages, but for this tutorial let's just stick with one for our messages Model.

Create a file named 'MessagesPolicy.js' inside the 'api > policies' folder, and enter the following:

module.exports = function (req, res, next) {
    if (req.session.user) {
        var action = req.param('action');
        if (action == "create") {
            req.body.userId = req.session.user.id;
            req.body.username = req.session.user.username;
        }
        next();
    } else {
        res.send("You Must Be Logged In", 403);
    }
};

So, what’s going on here? You can see this function resembles a normal route function, the difference though is the third parameter, which will call the next middleware in the stack. If you’re new to the idea of middleware, you can think of it like a Russian nesting doll. Each layer gets the request, along with the response variables and they can modify them as they see fit. If they pass all of the requirements, the layer can pass it further in, until it reaches the center, which is the route function.

So here we are, checking if the user is logged in, if the user isn't, we display a 403 error and the request ends here. Otherwise, (i.e. the user is logged in) we call next(); to pass it on. In the middle of the above code, is where we inject some post variables. We’re applying this to all calls on the 'messages' controller (basically the API), so we get the action and check if this request is trying to create a new message, in which case we add the post fields for the user's id and username.

Next, open up the policies.js file which is in the config folder, and add in the policy that we just created. So your file should look like this:

module.exports.policies = { 
    '*': true,
    'messages': 'MessagesPolicy'
};

With this put in to place, we’ll need to delete all of the old records, as they do not have these new pieces of information. So, close the Sails server (ctrl-c) and in the same terminal window type: rm -r .tmp to remove the temporary database giving us a clean slate.

Next, let's add the username to the actual posts, so in the 'chat.ejs' change the template to:

template: _.template("<div><p><b>{{ username }}: </b>{{ message }}</p></div>"),

Restart the Sails server (again using sails lift) and signup another new user to test it out. If everything is working correctly, you should be able to add messages and see your name in the post.

At this point we have a pretty good setup, we fetch the post automatically using Backbone and the API, plus we have some basic security in place. The problem is, it won't update when other people post messages. Now you could solve this by creating a JavaScript interval and poll for updates, but we can do better.

Leveraging Websockets

I mentioned earlier that Sails leverages websockets’ bidirectional abilities to post updates on the subscribed data. Using these updates, we can listen for new additions to the messages table and update the collection accordingly.

So in the chat.ejs file, let's create a new kind of collection; a SailsCollection:

var SailsCollection = Backbone.Collection.extend({
    sailsCollection: "",
    socket: null,
    sync: function(method, model, options){
        var where = {};
        if (options.where) {
            where = {
                where: options.where
            }
        }       
        if(typeof this.sailsCollection === "string" && this.sailsCollection !== "") {
            this.socket = io.connect();
            this.socket.on("connect", _.bind(function(){
                this.socket.request("/" + this.sailsCollection, where, _.bind(function(users){
                    this.set(users);
                }, this));
    
                this.socket.on("message", _.bind(function(msg){
                    var m = msg.uri.split("/").pop();
                    if (m === "create") {
                        this.add(msg.data);
                    } else if (m === "update") {
                        this.get(msg.data.id).set(msg.data);
                    } else if (m === "destroy") {
                        this.remove(this.get(msg.data.id));
                    }
                }, this));
            }, this));
        } else {
            console.log("Error: Cannot retrieve models because property 'sailsCollection' not set on the collection");
        }
    }
});

Now it may be long, but it's actually very simple, let's walk through it. We start off by adding two new properties to the Collection object, one to hold the name of the Sails 'model' and one to hold the web socket. Next, we modify the sync function, if you’re familiar with Backbone, then you’ll know that this is the function which interfaces with the server when you call things such as fetch. Usually, it fires off Ajax requests, but we’re going to customize it for socket communication.

Now, we’re not using most of the functionality that the sync function offers, mainly because we haven't added the ability for users to update or delete messages, but just to be complete, I will include them within the function definition.

Let’s take a look at the first part of the sync function:

var where = {};
if (options.where) {
    where = {
        where: options.where
    }
}

This code first checks if any 'where' clauses were sent through, this would let you do things like: messages.fetch({ where : { id: 4 } }); to only fetch rows where the id equals four.

After that, we then have some code that makes sure the 'sailsCollection' property has been set, otherwise we log an error message. Afterwards, we create a new socket and connect to the server, listening for the connection with the on('connect') event.

Once connected, we request the index of the 'sailsCollection' specified to pull in the current list of models. When it receives the data, we use the collection's set function to initially set the models.

Alright, now so far, we have the equivalent of the standard fetch command. The next block of code is where the push notifications happen:

this.socket.on("message", _.bind(function(msg){
    var m = msg.uri.split("/").pop();
    if (m === "create") {
        this.add(msg.data);
    } else if (m === "update") {
        this.get(msg.data.id).set(msg.data);
    } else if (m === "destroy") {
        this.remove(this.get(msg.data.id));
    }
}, this));

Now the action that’s being performed (whether we’re creating, updating, or destroying a message) can be found inside of the actual msg, which is then inside of the uri. To get the action, we split the URI on forward slashes ('/') and grab just the last segment using the pop function. We then try to match it up with the three possible actions of create, update, or destroy.

The rest is standard Backbone, we either add, edit, or remove the specified model. With our new class almost complete, all that’s left to do is change the current MessageCollection. Instead of extending the Backbone collection, it needs to extend our new collection, like so:

var MessageCollection = SailsCollection.extend({
    sailsCollection: 'messages',
    model: MessageModel
});

In addition to extending our new collection, we’ll make another change so that instead of setting the URL property, we now set the sailsCollection property. And that's all there is to it. Open up the application into two different browsers (e.g. Chrome and Safari) and signup two separate users. You should see that posted messages from either of the browsers get immediately shown on the other, no polling, no trouble.

Conclusion

Sails is a breath of fresh air, within a clutter of frameworks. It checks its ego at the door, and does what it can to help the developer instead of the brand. I have been chatting with the Sails devs and I can tell you that there’s even more awesomeness in the works, and it will be interesting to see where this framework goes.

So in conclusion, you’ve learned how to setup, use, and secure your data from within Sails, as well as how to interface it with the popular Backbone library.

Like always, if you have any comments, feel free to leave them below, or join us on the Nettuts+ IRC channel ("#nettuts" on freenode). Thank you for reading.

What Are You Using?

Rey Bango — Mon, 10 Jun 2013 22:30:10 +0000

We spend a lot of time following the thought leaders in web development, in many cases using the tools and libraries they’ve built, reading the posts they’ve written, articulating cool techniques they’ve learned, and in some cases, attending the defining conference for a specific language. But wouldn’t it be great to learn what they focus on and what they use to build such awesomeness?

I reached out to a group of some of the best and brightest developers in web development to answer those very questions. These are developers that have made strong contributions to the web development community, are highly regarded by their peers for their technical abilities, and continue to help push web development forward via content, code, and leadership. You can check out their bios, below, for more details about them.

These folks are incredibly busy, so I narrowed my questions down to four simple ones:

What’s your primary development focus?
What hardware are you using for development?
Which editor or IDE do you use?
What software can you not live without on a daily basis?

Below, you’ll find the answers they gave and hopefully discover some tools that could make your development much easier. You’ll definitely find common themes (Sublime) and a few nuggets that are new, at least to me.

Scott Gonzalez

Bio: I'm a full-stack web application developer, with a focus on JavaScript. I've been contributing to jQuery since 2007 and I’m currently the Project Lead for jQuery UI. I'm active in the Node.js, WHATWG, and W3C communities as well.

Connect with Scott on his Blog and Twitter.

Q What’s your primary development focus?

Web development, mostly client-side JS and Node.js.

Q What hardware are you using for development?

MacBook Pro.

Q Which editor or IDE do you use?

Sublime Text with TrailingSpaces, Pretty JSON, GitGutter, and Markdown Preview.

Q What software can you not live without on a daily basis?

Git, GitHub, Apache, Chrome, Linkinus, Skype, Node.

Raymond Camden

Bio: Raymond is a senior developer evangelist for Adobe. His work focuses on web standards, mobile development, and ColdFusion. He's a published author and presents at conferences and user groups on a variety of topics.

Connect with Raymond on his Blog and Twitter.

Q What’s your primary development focus?

My primary development focus, in general, is on web standards, Creative Cloud (with a focus on the HTML tools), and typically the "non-sexy" part of client-side dev. I can appreciate CSS, but I get more excited by things like storage, JavaScript, and forms.

Q What hardware are you using for development?

My hardware is a MacBook Pro. I do a lot of mobile development though, so I test with an iPhone, an Android phone, and various tablets. Oddly, the only Microsoft hardware I use daily is my keyboard. I've been using "Natural" keyboards for almost 15 years. I'm addicted to them.

Q Which editor or IDE do you use?

My editor is Brackets, an open source editor we launched about a year or so ago. It's got great ties to Chrome and a cool extensibility layer.

Q What software can you not live without on a daily basis?

Based on what I see in my OS X dock right now, the tools I use every day are my browser, my editor, Tweetdeck, and Evernote. (I keep everything in Evernote, from project stuff, to random tips.)

John-David Dalton

Bio: JDD is the co-maintainer of jsPerf/Benchmark.js, an ES5 compliance evangelist, a JavaScript library enthusiast, a two time Microsoft MVP recipient for IE testing, and a Chakra performance program manager at Microsoft.

Connect with JDD on Github and Twitter.

Q What’s your primary development focus?

Low level utility libs and benchmarking. I'm a libs fan, and dev’ing around them. JS development isn’t my day job at Microsoft, so no client work or anything like that.

Q What hardware are you using for development?

My day job dev’ing is done on Windows 8 and IE10. My personal dev machine is a 13″ MacBook Pro running OS X 10.7.5, with a 2.3GHz processor and 8GBs of RAM.

Q Which editor or IDE do you use?

I use Komodo Edit on OS X and Windows. I love its advanced search. I search by regexp and nested files all the time.

Q What software can you not live without on a daily basis?

I use just about every browser..

Besides my text editor, I use total finder for OS X (this makes its windows manageable). I also have just about every browser; 22 different Chrome versions, five versions of IE, 23 versions of Firefox, 12 versions of Opera, and six versions of Safari. Before I release a version bump, I ensure the given project runs in the environments/browsers that I state it should run in, which is why I have all the browsers. I also have Node, Ringo, Rhino, and Narwhal.

Stephanie Sullivan Rewis

Bio: Stephanie is Director, Web Strategy and Marketing Technologies at Contatta. As a front-end developer, she’s presented sessions at conferences worldwide, including HOW Design, UI16, An Event Apart, Microsoft’s MIX, Macworld, SXSW, Adobe Max, and numerous others. She’s a published author, and while principal at W3Conversions, worked with a wide variety of organizations from Newsweek, MLB, New York Magazine, Adobe, and SalesForce.com to Disney’s “TRON” movie site.

Connect with Stephanie on her Blog and Twitter.

Q What’s your primary development focus?

I'm a front-end dev, but I'm wearing a few extra hats due to being in startup mode at Contatta. That means I have the honor of choosing and implementing our new corporate CMS—so I'm digging into PHP a bit, in addition to my usual development.

Q What hardware are you using for development?

I don't use any special hardware. I work on a 17" MacBook Pro (that I haven't upgraded because I don't want to downsize to a 15"). If I'm at my desk, I'm also plugged into a second monitor. I choose to buy my monitors somewhere like Costco, rather than using the high-end models, so that I can view my site like "an average user" does.

Q Which editor or IDE do you use?

Currently, I'm using Sublime Text 2 for daily development. However, if Adobe's open source code editor, Brackets, keeps going in the direction they showed at MAX, I'll probably give it a whirl at some point. I also use Dreamweaver to build my email campaigns in.

Q What software can you not live without on a daily basis?

I give Chrome a daily workout, sometimes with over 150 tabs.

Outside of my editors, I always have CodeKit to compile my Sass and minify my CSS and JS. I use Git for Source control and push my code to the repository via Tower.

I use HipChat to communicate with the rest of the company throughout the day. But the best thing about HipChat is Hu Bot, which handles the deployment from my Git repository to either staging or production. I adore my bot!

My designer uses Fireworks, so I sometimes have that open, slicing and dicing. (Yea, I know Adobe isn't updating it, but it does what he needs, so we'll have to pry it from his cold, dead fingers someday.) I always have Spotify open—usually playing an Artist Radio I make based on my mood or the kind of dev I'm doing (today, it's a Hellsongs station).

Finally, I give Chrome a daily work out, sometimes with over 150 tabs (yes, I have a problem). I use it both for the Inspector and the plugins that I can't live without (1Password, OneTab, ColorZilla, Cache Killer, and Dragdis, to name a few) as well as to house tabs of all the other things I have to keep track of, like Google Analytics, MailChimp, SproutSocial, Google Docs, and anything else I'm currently researching.

Christian Heilmann

Bio: Chris Heilmann has dedicated a lot of his time to making the web better. Originally coming from a radio journalism background, he built his first web site from scratch around 1997 and spent the following years working on lots of large, international web sites. He then spent a few years in Yahoo building products, explaining and training people, and is now at Mozilla. Chris wrote and contributed to four books on web development and wrote many articles and hundreds of blog posts for Ajaxian, Smashing Magazine, Yahoo, Mozilla, ScriptJunkie, and many more.

Connect with Christian on his Blog and Twitter.

Q What’s your primary development focus?

My primary focus is making sure that the next person taking over after me, gets code that is easy to understand and simple to extend. Sadly enough, this is going out of fashion and a lot of focus is placed on "getting it out the door". I really think that we learn the most from each other's work and where better to write clean and understandable code than in our deliveries?

My focus is the web, not just one browser or one closed environment. This means you need to be very flexible in your code. Being very flexible can mean making it work right here, right now, or spending time on making it easily extendable, for new features that may be added in the future. I think after 16 years of web development, we should be at a stage where we stop hacking things together and replacing them continuously.

Q What hardware are you using for development?

MacBook Air, I am always on the go and have little space or time to set up large hardware components.

Q Which editor or IDE do you use?

Sublime Text 2, it’s incredible.

Q What software can you not live without on a daily basis?

Browsers and their developer tools, the command line for Git, an IRC client, Dropbox and Spotify. I have found more and more, that I’ve become independent of fat client software and I use a lot of online services instead.

Ryan Grove

Bio: Ryan Grove is a Sorcerer at SmugMug, a YUI reviewer, and was once an underage model for a Japanese clothing catalog. He likes pie, movies, pie, old style sailing ships, and pie.

Connect with Ryan on his Blog and Twitter.

Q What’s your primary development focus?

I spend most of my time these days building client-side JS components and features for SmugMug. In my free time I write lots of Node.js code and occasionally some Ruby (although JS has really taken over my focus in the last few years). At SmugMug, our framework of choice is YUI 3, and I'm also a core reviewer for YUI, so I contribute a lot of code to it and also review changes from other committers and contributors.

On a typical day, I might write some non-public SmugMug code, some open source SmugMug code, and some open source YUI code. We try to open source as much of our JS as possible, and a lot of that goes back into YUI these days.

Q What hardware are you using for development?

When I’m traveling I use a 13″ Macbook Air.

I work from home on a 2.8GHz Quad-Core Mac Pro with 16GB of RAM, an SSD, and two 30″ HP ZR30w monitors that are fantastic for viewing big gorgeous photos and videos. When I'm traveling I use a 13″ MacBook Air (mid-2011, Core i7), which I love to death. Although, the screen isn't quite as good as my desktop setup for viewing big gorgeous photos.

I also have a ridiculous collection of mobile devices that I use for testing. At least one of every model of iPhone and iPad, several Android devices, a Windows phone, and a Chromebook.

Q Which editor or IDE do you use?

I use Sublime Text 3 for everything. I want to marry it and have its kittens.

Q What software can you not live without on a daily basis?

In addition to Sublime Text, my must-have apps include JSHint (for keeping me honest), iTerm 2, Adium (for IRC), OS X Messages (for iMessage and Jabber), Tower (a GUI Git client), Dropbox and Rsync (for syncing source and data across various machines), Arq (for backup), Gmail, GitHub, and of course Google Chrome as my browser of choice.

SmugMug has lots of remote workers, so we use Google Hangouts for meetings and quick face-to-face chats. Additionally, IRC is an important communication medium for us. SmugMug has its own IRC server with a znc bouncer, and I use the Colloquy iOS app to get push notifications if someone mentions my name or sends me a message while I'm not at my desk.

Charles, cURL, and the REST Console Chrome extension are indispensable for debugging HTTP requests and working with APIs. Oh, and I've gotten so used to using Alfred (an app launcher and search tool) that I can't function when I use a machine that doesn't have it installed. I hit Control+Space and type "mdn [something or other]" about a thousand times a day to look up docs, among many other things.

Cody Lindley

Bio: Front-end/UI/JS engineer and author. Lover of Christ, people, logic, and the dying art of debate, conversation, and rational thinking. Husband and father of three boys.

Connect with Cody on his Blog and Twitter.

Q What’s your primary development focus?

My focus is on front-end code (i.e. HTML, CSS, and JavaScript) used to build thick client web applications and websites for desktop users. I cross over into tablet and mobile web development when necessary.

Q What hardware are you using for development?

My setup for years now has been a MacBook Pro, 27″ Apple Display, Apple Magic Mouse, and an Apple wireless keyboard.

Q Which editor or IDE do you use?

I use the Sublime Text 2 editor.

Q What software can you not live without on a daily basis?

That would be SmartGit, Sublime Text, Divvy, JumpCut, Google (search, gmail, drive, calendar), Skype, Terminal, Chrome, Tweetdeck, Parallels, Textual, Dropbox, Github, Assembla.

Luke Smith

Bio: Locally sourced, (indirectly) grass fed, all organic web developer out of Portland, Oregon. I'm fortunate to work with the amazing and inspiring team at SmugMug. Speaking of amazing and inspiring teams, previously I was a YUI core developer for five years, and am still contributing to the project today. Did I say team? I meant community. Seriously, you all are awesome.

Connect with Luke on Twitter.

Q What’s your primary development focus?

More than anything, really, I like helping people out when I can.

I'm pretty much all JavaScript, all day every day. Specifically, I prefer to work on lower level API stuff, writing abstractions and tools to build other stuff on top of, such as event systems, promises, and XHR/data IO frameworks. I try very hard to make APIs that are intuitive, flexible, and fast. Principle of, least surprise-friendly stuff. But at the same time, I like solving little UI implementation challenges with higher level components. More than anything, really, I like helping people out when I can. I want to contribute more to JS, DOM spec, and standards work.

Q What hardware are you using for development?

13" MacBook Air (one for work, one for personal), 2GHz Core i7 with 8GBs of RAM and a 500GB SSD (personal is 4GB/256GB). My work machine is hooked up to a 22" POS display that I bought years ago for extra real estate, an Apple keyboard, and a magic track pad. I hate the non-split keyboard layout, but love the key shape and action more, so I deal with it. I also hate the use of the word "magic" in the trackpad's name.

As to the Air, I've gone through a number of laptops and desktop units and until the Air, I never had any affection (good or bad) for the tech. The Air is awesome. It's the best laptop I've ever had, the best computer I've ever had. I genuinely love the thing. Fortunately, I haven't pushed its processing limits (yet) to the extent where beefier hardware is necessary. And I suppose I should mention the Yapster headset, since I work remotely. It's good enough for the money vs. how much I use it.

Q Which editor or IDE do you use?

Vim FTW. That is all.

Q What software can you not live without on a daily basis?

I live in Terminal, Vim, Git, and Chrome (dev tools \m/). Supporting staff are Shifter (for YUI stuff), Github's pull request and issues systems, VirtualBox, and 1Password. The distractionary cast and crew are: Mail.app (for lack of a better client), Itsy (minimalist twitter client), Reeder, iOS Reminders app, and Adium. I'm sure I'm missing several that I take for granted.

Chris Williams

Bio: Chris is the VP of Product Engineering at SaferAging and the organizer of JSConf US and RobotsConf. He is the author of the node-serialport, originator of the JS Community Logo, and assists in the creation of amazing tech events like NodeBots and NodeCopter, among others. He is constantly inspired by his wife and two amazing children.

Connect with Chris on JSconf and RobotsConf.

Q What’s your primary development focus?

My focus actually spreads across the entire bow of development these days, from programming and developing hardware sensors, to high availability server infrastructure and development, to frontend information presentation. In a given week, I will be soldering hardware, writing squirrel firmware, monitoring and upgrading servers, and developing new user interfaces and interactions (not necessarily web or visual based). To pick a single "primary" focus is tough because they are all supportive and necessary for creating the products I am working towards or supporting.

Q What hardware are you using for development?

I mainly use Apple products (2010 MacBook Air when on the road, 2008 Mac Pro when at work, 2012 Mac Mini at home) due to their incredible construction quality and visual appeal. I split my time almost down the middle between raw terminal (Fish Shell yay!) use and GUI interface applications, so the fluidity between the two afforded by Mac OS X nicely supports my standard workflow.

Q Which editor or IDE do you use?

I’ll opt for different editors based on the contextual environment I’m working in.

I’ll opt for different editors based on the contextual environment I’m working in. On servers, I’ll use VIM simply because it’s everywhere. On my local machine, I have opted for Sublime Text 3 at this point, though I am always dabbling with others, mainly out of curiosity rather than need.

I use a simple set of plugins (EJS, Emmet, Go, GitGutter, JSFormat, Squirrel) that basically provide syntax highlighting and formatting for me. I find standard IDEs to be too cumbersome and heavy. I get lost in all it provides to be honest. Even with Sublime, I often get lost in the key combos as I just want to get done, what I need to get done, no fluff, no pomp, just optimize on time to complete the task.

Q What software can you not live without on a daily basis?

Sublime Text 3 (as mentioned above), but really that boils down to a raw text editor more than anything. iTerm2 and Fish shell have saved my life on a near daily basis. Wunderlist is something I basically cannot live without because I am just juggling too much stuff in my head at any given point. Outside of software, I couldn't live without my Das Keyboard — typing on anything else these days feels unnatural and discomforting.

Aaron Newton

Bio: Aaron Newton is a jack of all trades and, arguably, king of none. A veteran of numerous startups, most recently Cloudera, he is now Head of Product at Thanx, a mobile loyalty application. As a contributor to the MooTools framework from its first release, he has authored numerous tutorials, a book, and more code than he can ever hope to maintain, most of it being JavaScript.

Connect with Aaron on MooTools and Twitter.

Q What’s your primary development focus?

These days I'm a product manager at Thanx. I'm very technical, so I have a lot of discussions about specific implementations while still doing code reviews and chipping in on development when I can. Our product is primarily a mobile application with a native iOS implementation and an HTML5 version for mobile browsers. We have an Android app which is basically a browser that uses this HTML5 implementation. I tend to contribute on the HTML5 implementation and also to our more traditional web appliations – a dashboard for our customers and internal tools for our own use.

Q What hardware are you using for development?

My trusty MacBook Pro. Everything else runs on AWS / Heroku.

Q Which editor or IDE do you use?

Lately Sublime. I was pretty hardcore about TextMate for years, but finally got tired of not having any updates. I like all the community support for Sublime (and that it can run TextMate plugins), but the two features that finally, really made me switch were the split views and, more than anything, the lightning fast search. Searching on TextMate is the new "my code is compiling so I'm going to go get a coffee."

Q What software can you not live without on a daily basis?

Aside from obvious stuff like web browsers (I use Chrome primarily, but I have Fluid apps for Gmail and JIRA, etc), other tools that come to mind are:

Cloudapp – I share files with this all the time. I pay for the pro model and don't give it a second thought when I want to send a screenshot or something somewhere.
Jing – most of the time I use the built in screenshot functionality in OSX, but sometimes I want to capture video or annotate the screenshot and that’s when Jing comes in handy.
Sequel Pro – aside from being a great, free SQL tool for Mac, I'm in love with the app icon, which is perhaps the best icon for any app, I've ever seen.
Jumpcut – this little clipboard saver has removed that nagging sensation that I used to have whenever I hit "copy", that feeling that there's something on your clipboard that you need to put somewhere before you replace the buffer.
Total Terminal – I switch to my terminal all the time. Visor locks it to the top of the screen (think of the console in Quake) so it's always only one keystroke away. See also: Total Finder, from the same publisher, which adds chrome-style tabs to Finder (and a bunch of other features).
GitX – I use the command line for nearly everything git related… except for staging commits. I still use the command line if I'm staging EVERYTHING, but with GitX it's super easy to take several changes and break it up into small commits, even at the file level.

Ben Cherry

Bio: Ben lives in San Francisco, where he is an Engineer at Pushd, focusing on Ruby and iOS. Previously he built a failed startup on iOS and Node.js, and before that he wrote JavaScript at Twitter and Slide. He maintains a programming blog, mostly about JavaScript, at http://www.adequatelygood.com

Connect with Ben on his Blog and Twitter.

Q What’s your primary development focus?

I spent three years doing heavy JavaScript for Slide and Twitter, but more recently I'm working on iOS and Ruby, while only occasionally working with JavaScript.

Q What hardware are you using for development?

A 13" Retina MacBook Pro with a 27" Thunderbolt display.

Q Which editor or IDE do you use?

SublimeText 2. I Love it.

Q What software can you not live without on a daily basis?

That would be SublimeText 2, iTerm 2, Google Chrome, 1Password, Gmail, Flipboard, Twitter, Facebook, Reddit.

Jacob Thornton

Bio: I write code @medium and I've opensourced a few things (like Bootstrap and Bower). I used to give talks and occasionally, I write about the web.

Connect with Jacob on his Blog and Twitter.

Q What’s your primary development focus?

I suppose I spend most of my time on general front-end (JS/CSS/HTML) development, though I do a good amount of tool/build development and a fair amount of simple server work. I've also been trying to up my accessibility chops – meeting a lot of really rad/super helpful people in this space (like Victor Tsaran from Paypal and Joshua Miele).

Q What hardware are you using for development?

I use a 15" Retina MacBook Pro at work and a 13" MacBook Air at home. I do a ton of testing on different devices though: iPads, iPhones, Android tablets/phones, Blackberry, and even Windows slate.

Q Which editor or IDE do you use?

I use Sublime Text 2.

I use Sublime Text 2. I used Vim (poorly) for a while, but I found most other front-end developers didn't really use it either, and it was weird mentally, always switching back and forth for me. So, I just point and click like a noob now. Honestly, I’m terrible with computers. I know a lot of engineers who are so dope with their computers, and make all of this custom stuff happen, but I literally have no idea what I’m doing most of the time.

Q What software can you not live without on a daily basis?

Rdio – I am useless without music.

Lea Verou

Bio: Lea works as a Developer Advocate for W3C. She has a long-standing passion for open web standards, which she fulfills by researching new ways to use them, blogging, speaking, writing, and coding popular open source projects to help fellow developers. She’s also a member of the CSS Working Group, which architects the language itself.

Connect with Lea on her Blog and Twitter.

Q What’s your primary development focus?

Anything about the client-side: I do HTML, CSS, JavaScript, or even design and UX! However, I have a penchant for CSS, which became even stronger after I joined the CSS Working Group.

Q What hardware are you using for development?

A 13″ MacBook Air Ultimate that I bought last June. I love it. Its touchpad had some issues for the past few months, but I recently got it replaced so I fell in love with my lil’ MacBaby (yes, I have a pet name for it, is there a problem? :) all over again. <3

Q Which editor or IDE do you use?

Espresso. I know there are better text editors around, but its FTP integration is addictively good.

Q What software can you not live without on a daily basis?

Native apps: Espresso, Transmit, CodeKit, Adobe Illustrator, Chrome, Safari, Firefox, Opera, WebKit nightlies, Chrome Canary.

Web apps: Browserstack, Dabblet, copypastecharacter.com, css.coloratum.com, Workflowy, and Github.

Jonathan Snook

Bio: Jonathan Snook is a Web Designer/Developer who works at Shopify. He can speak, he can write, he can develop websites with all his might.

Connect with Jonathan on his Blog and Twitter.

Q What’s your primary development focus?

These days, I mostly focus on front-end development. HTML, CSS and JavaScript fill up my time.

Q What hardware are you using for development?

I have an 11″ MacBook Air that is my primary machine. When it comes to testing, though, I prefer testing on an actual device as much as I can. I have a Samsung tablet with Windows 8, a Nexus 7, an iPad Mini, an iPhone, and even an older phone with WebOS. Okay, I might not use that last one very much. No BlackBerry device yet. For IE6 through IE9 testing, I use VMWare with Windows XP, Windows 7 and Windows 8 VMs, too.

Q Which editor or IDE do you use?

Vim. It’s so handy and I like having something familiar to use when I’m logged into a remote Linux box.

Q What software can you not live without on a daily basis?

Vim, of course, and my browser of choice these days: Chrome. I’ve been using Google Docs more and more, which has replaced Microsoft Office. Google has really been doing a good job at improving their toolset. I’m starting to buy into the whole Google ecosystem.

Peter Wagenet

Bio: Peter has been developing web applications for nearly fifteen years. He is currently an Ember Core Team member, previously serving on the SproutCore team. He also has extensive experience with Ruby on Rails and has dabbled in iOS development. These days you can find him working for Tilde on awesome projects like Skylight.

Connect with Peter on Twitter.

Q What’s your primary development focus?

I spend most of my time doing Ember development with a bit of Rails thrown in. Right now, I’m consulting two days a week on Ember and the rest is spent working on Tilde projects, mainly Skylight, which is an Ember client with a Rails backend. It also has a Java component, though I don’t work on that.

Q What hardware are you using for development?

I use a 2012 MacBook Air with a 2GHz Core i7 and 8GBs of RAM. I also have a 27″ Cinema Display. I used to have a 20″ LCD hooked up as well with a USB to DVI adapter, but decided to make my desk a little less crowded.

Q Which editor or IDE do you use?

Recently I’ve been using Sublime Text 3. When I first started, I used TextMate, which I loved. However, I got tired of waiting for TextMate 2 and ended up switching to VIM. I fell in love with the VIM keybindings, but was still frustrated by the plugin experience and wished I could have something a bit more polished. For me, Sublime’s Vintage mode gets 95% of what I did with VIM’s keybindings, it has a better plugin architecture, and it’s more polished. I’ve never felt a big pull towards using a full-blown IDE, autocomplete is good enough for me. The only full-blown IDE I’ve used is Xcode and I find that I’m always missing keybindings when I use it.

Q What software can you not live without on a daily basis?

I browse in Chrome.

I browse in Chrome, though I’ve heard some cool things are in works for the Firefox dev tools, so I might be convinced to change. For chat and communication, I use Adium for AIM, GTalk, IRC and Flint for Campfire. I still use plain old Mail.app and the basic Twitter app as well. I’m not a huge power tools sort of guy, but I’ve been enjoying Divvy a lot recently for positioning windows. 1Password is also indispensable. Since I have a lot of menu bar items, I’ve also found Bartender to be super useful.

In Closing

So, there you go. You now have a nice reference to what many of the leaders in web development are using day in, and day out. We’d like to send out a big “Thank You” to all of the developers who participated!

Feel free to share the tools, hardware, editor/IDE, and software that you use everyday, in the comments.

Parallel Testing for PHPUnit with ParaTest

Brian Scaturro — Thu, 06 Jun 2013 21:27:38 +0000

PHPUnit has hinted at parallelism since 2007, but, in the meantime, our tests continue to run slowly. Time is money, right? ParaTest is a tool that sits on top of PHPUnit and allows you to run tests in parallel without the use of extensions. This is an ideal candidate for functional (i.e Selenium) tests and other long-running processes.

ParaTest at your Service

ParaTest is a robust command line tool for running PHPUnit tests in parallel. Inspired by the fine folks at Sauce Labs, it was originally developed to be a more complete solution for improving the speed of functional tests.

Since its inception – and thanks to some brilliant contributors (including Giorgio Sironi, the maintainer of the PHPUnit Selenium extension) – ParaTest has become a valuable tool for speeding up functional tests, as well as integration tests involving databases, web services, and file systems.

ParaTest also has the honor of being bundled with Sauce Labs’ testing framework Sausage, and has been used in nearly 7000 projects, at the time of this writing.

Installing ParaTest

Currently, the only official way to install ParaTest is through Composer. For those of you who are new to Composer, we have a great article on the subject. To fetch the latest development version, include the following within your composer.json file:

"require": {
    "brianium/paratest": "dev-master"
}

Alternatively, for the latest stable version:

"require": {
    "brianium/paratest": "0.4.4"
}

Next, run composer install from the command line. The ParaTest binary will be created in the vendor/bin directory.

The ParaTest Command Line Interface

ParaTest includes a command line interface that should be familiar to most PHPUnit users – with some added bonuses for parallel testing.

Your First Parallel Test

Using ParaTest is just as simple as PHPUnit. To quickly demonstrate this in action, create a directory, paratest-sample, with the following structure:

Let’s install ParaTest as mentioned above. Assuming that you have a Bash shell and a globally installed Composer binary, you can accomplish this in one line from the paratest-sample directory:

echo '{"require": { "brianium/paratest": "0.4.4" }}' > composer.json && composer install

For each of the files in the directory, create a test case class with the same name, like so:

class SlowOneTest extends PHPUnit_Framework_TestCase
{
    public function test_long_running_condition()
    {
        sleep(5);
        $this->assertTrue(true);
    }
}

Take note of the use of sleep(5) to simulate a test that will take five seconds to execute. So we should have five test cases that each take five seconds to run. Using vanilla PHPUnit, these tests will run serially and take twenty-five seconds, total. ParaTest will run these tests concurrently in five separate processes and should only take five seconds, not twenty-five!

Now that we have an understanding of what ParaTest is, let’s dig a little deeper into the problems associated with running PHPUnit tests in parallel.

The Problem at Hand

Testing can be a slow process, especially when we start talking about hitting a database or automating a browser. In order to test more quickly and efficiently, we need to be able to run our tests concurrently (at the same time), instead of serially (one after the other).

The general method for accomplishing this is not a new idea: run different test groups in multiple PHPUnit processes. This can easily be accomplished using the native PHP function proc_open. The following would be an example of this in action:

/**
 * $runningTests - currently open processes
 * $loadedTests - an array of test paths
 * $maxProcs - the total number of processes we want running
 */
while(sizeof($runningTests) || sizeof($loadedTests)) {
    while(sizeof($loadedTests) && sizeof($runningTests) < $maxProcs)
        $runningTests[] = proc_open("phpunit " . array_shift($loadedTests), $descriptorspec, $pipes);
    //log results and remove any processes that have finished ....
}

Because PHP lacks native threads, this is a typical method for acheiving some level of concurrency. The particular challenges of testing tools that use this method can be boiled down to three core problems:

How do we load tests?
How do we aggregate and report results from the different PHPUnit processes?
How can we provide consistency with the original tool (i.e PHPUnit)?

Let’s look at a few techniques that have been employed in the past, and then review ParaTest and how it differs from the rest of the crowd.

Those Who Came Before

As noted earlier, the idea of running PHPUnit in multiple processes is not a new one. The typical procedure employed is something along the following lines:

Grep for test methods or load a directory of files containing test suites.
Open a process for each test method or suite.
Parse output from STDOUT pipe.

Let’s take a look at a tool that employs this method.

Hello, Paraunit

Paraunit was the original parallel runner bundled with Sauce Labs’ Sausage tool, and it served as the starting point for ParaTest. Let’s look at how it tackles the three main problems mentioned above.

Test Loading

Paraunit was designed to ease functional testing. It executes each test method rather than an entire test suite in a PHPUnit process of its very own. Given the path to a collection of tests, Paraunit searches for individual test methods, via pattern matching against file contents.

preg_match_all("/function (test[^\(]+)\(/", $fileContents, $matches);

Loaded test methods can then be run like so:

proc_open("phpunit --filter=$testName $testFile", $descriptorspec, $pipes);

In a test where each method is setting up and tearing down a browser, this can make things quite a bit faster, if each of those methods is run in a separate process. However, there are a couple of problems with this method.

While methods that begin with the word, “test,” is a strong convention among PHPUnit users, annotations are another option. The loading method used by Paraunit would skip this perfectly valid test:

/**
 * @test
 */
public function twoTodosCheckedShowsCorrectClearButtonText()
{
    $this->todos->addTodos(array('one', 'two'));
    $this->todos->getToggleAll()->click();
    $this->assertEquals('Clear 2 completed items', $this->todos->getClearButton()->text());
}

In addition to not supporting test annotations, inheritance is also limited. We might argue the merits of doing something like this, but let’s consider the following setup:

abstract class TodoTest extends PHPUnit_Extensions_Selenium2TestCase
{
    protected $browser = null;

    public function setUp()
    {
       //configure browser
    }

    public function testTypingIntoFieldAndHittingEnterAddsTodo()
    {
        //selenium magic
    }
}

/**
 * ChromeTodoTest.php
 * No test methods to read!
 */
class ChromeTodoTest extends TodoTest
{
    protected $browser = 'chrome';
}

/**
 * FirefoxTodoTest.php
 * No test methods to read!
 */
class FirefoxTodoTest extends TodoTest
{
    protected $browser = 'firefox';
}

The inherited methods aren’t in the file, so they will never be loaded.

Displaying Results

Paraunit aggregates the results of each process by parsing the output generated by each process. This method allows Paraunit to capture the full gamut of short codes and feedback presented by PHPUnit.

The downside to aggregating results this way is that it is pretty unwieldy and easy to break. There are a lot of different outcomes to account for, and a lot of regular expressions at work to display meaningful results in this way.

Consistency with PHPUnit

Due to the file grepping, Paraunit is fairly limited in what PHPUnit features it can support. It’s an excellent tool for running a simple structure of functional tests, but, in addition to some of the difficulties mentioned already, it lacks support for some useful PHPUnit features. Some such examples include test suites, specifying configuration and bootstrap files, logging results, and running specific test groups.

Many of the existing tools follow this pattern. Grep a directory of test files and either run the entire file in a new process or each method – never both.

ParaTest At Bat

The goal of ParaTest is to support parallel testing for a variety of scenarios. Originally created to fill the gaps in Paraunit, it has become a robust command line tool for running both test suites and test methods in parallel. This makes ParaTest an ideal candidate for long running tests of different shapes and sizes.

How ParaTest Handles Parallel Testing

ParaTest deviates from the established norm in order to support more of PHPUnit, and acts as a truly viable candidate for parallel testing.

Test Loading

ParaTest loads tests in a similar manner to PHPUnit. It loads all tests in a specified directory that end with the *Test.php suffix, or will load tests based on the standard PHPUnit XML configuration file. Loading is accomplished, via reflection, so it is easy to support @test methods, inheritance, test suites and individual test methods. Reflection makes adding support for other annotations a snap.

Because reflection allows ParaTest to grab classes and methods, it can run both test suites and test methods in parallel, making it a more versatile tool.

ParaTest does impose some constraints, but well-founded ones in the PHP community. Tests do need to follow the PSR-0 standard, and the default file suffix of *Test.php is not configurable, as it is in PHPUnit. There is a current branch in progress for supporting the same suffix configuration allowed in PHPUnit.

Displaying Results

ParaTest also deviates from the path of parsing STDOUT pipes. Instead of parsing output streams, ParaTest logs the results of each PHPUnit process in the JUnit format and aggregates results from these logs. It’s much easier to read test results from an established format than an output stream.

<?xml version="1.0" encoding="UTF-8"?>
<testsuites>
  <testsuite name="AnotherUnitTestInSubLevelTest" file="/home/brian/Projects/parallel-phpunit/test/fixtures/tests/level1/AnotherUnitTestInSubLevelTest.php" tests="3" assertions="3" failures="0" errors="0" time="0.005295">
    <testcase name="testTruth" class="AnotherUnitTestInSubLevelTest" file="/home/brian/Projects/parallel-phpunit/test/fixtures/tests/level1/AnotherUnitTestInSubLevelTest.php" line="7" assertions="1" time="0.001739"/>
    <testcase name="testFalsehood" class="AnotherUnitTestInSubLevelTest" file="/home/brian/Projects/parallel-phpunit/test/fixtures/tests/level1/AnotherUnitTestInSubLevelTest.php" line="15" assertions="1" time="0.000477"/>
    <testcase name="testArrayLength" class="AnotherUnitTestInSubLevelTest" file="/home/brian/Projects/parallel-phpunit/test/fixtures/tests/level1/AnotherUnitTestInSubLevelTest.php" line="23" assertions="1" time="0.003079"/>
  </testsuite>
</testsuites>

Parsing JUnit logs does have some minor drawbacks. Skipped and ignored tests are not reported in the immediate feedback, but they will be reflected in the totaled values displayed after a test run.

Consistency with PHPUnit

Reflection allows ParaTest to support more PHPUnit conventions. The ParaTest console supports more PHPUnit features out of the box than any other similar tool, such as the ability to run groups, supply configuration and bootstrap files, and log results in the JUnit format.

ParaTest Examples

ParaTest can be used to gain speed in several testing scenarios.

Functional Testing With Selenium

ParaTest excels at functional testing. It supports a -f switch in its console to enable functional mode. Functional mode instructs ParaTest to run each test method in a separate process, instead of the default, which is to run each test suite in a separate process.

It’s often the case that each functional test method does a lot of work, such as opening a browser, navigating around the page, and then closing the browser.

The example project, paratest-selenium, demonstrates testing a Backbone.js todo application with Selenium and ParaTest. Each test method opens a browser and tests a specific feature:

public function setUp()
{
    $this->setBrowserUrl('http://backbonejs.org/examples/todos/');
    $this->todos = new Todos($this->prepareSession());
}

public function testTypingIntoFieldAndHittingEnterAddsTodo()
{
    $this->todos->addTodo("parallelize phpunit tests\n");
    $this->assertEquals(1, sizeof($this->todos->getItems()));
}

public function testClickingTodoCheckboxMarksTodoDone()
{
    $this->todos->addTodo("make sure you can complete todos");
    $items = $this->todos->getItems();
    $item = array_shift($items);
    $this->todos->getItemCheckbox($item)->click();
    $this->assertEquals('done', $item->attribute('class'));
}

//....more tests

This test case could take a hot second if it were to run serially, via vanilla PHPUnit. Why not run multiple methods at once?

Handling Race Conditions

As with any parallel testing, we have to be mindful of scenarios that will present race conditions – such as multiple processes trying to access a database. The dev-master branch of ParaTest sports a really handy test token feature, written by collaborator Dimitris Baltas (dbaltas on Github), that makes integration testing databases much easier.

Dimitris has included a helpful example that demonstrates this feature on Github. In Dimitris’ own words:

TEST_TOKEN attempts to deal with the common resources issue in a very simple way: clone the resources to ensure that no concurrent processes will access the same resource.

A TEST_TOKEN environment variable is provided for tests to consume ,and is recycled when the process has finished. It can be used to conditionally alter your tests, like so:

public function setUp()
{
    parent::setUp();
    $this->_filename = sprintf('out%s.txt', getenv('TEST_TOKEN'));
}

ParaTest and Sauce Labs

Sauce Labs is the Excalibur of functional testing. Sauce Labs provides a service that allows you to easily test your applications in a variety of browsers and platforms. If you haven’t checked them out before, I strongly encourage you to do so.

Testing with Sauce could be a tutorial in itself, but those wizards have already done a great job of providing tutorials for using PHP and ParaTest to write functional tests using their service.

The Future of ParaTest

ParaTest is a great tool for filling in some of the gaps of PHPUnit, but, ultimately, it’s just a plug in the dam. A much better scenario would be native support in PHPUnit!

In the meantime, ParaTest will continue increasing support for more of PHPUnit’s native behavior. It will continue to offer features that are helpful to parallel testing – particularly in the functional and integration realms.

ParaTest has many great things in the works to beef up the transparency between PHPUnit and itself, primarily in what configuration options are supported.

The latest stable version of ParaTest (v0.4.4) comfortably supports Mac, Linux, and Windows, but there are some valuable pull requests and features in dev-master that definitely cater to the Mac and Linux crowds. So that will be an interesting conversation moving forward.

Additional Reading and Resources

There are a handful of articles and resources around the web that feature ParaTest. Give them a read, if you’re interested:

ParaTest on Github
Parallel PHPUnit by ParaTest contributor and PHPUnit Selenium extension maintainer Giorgio Sironi
Contributing to Paratest. An excellent article on Giorgio’s experimental WrapperRunner for ParaTest
Giorgio’s WrapperRunner Source Code
tripsta/paratest-sample. An example of the TEST_TOKEN feature by it’s creator Dimitris Baltas
brianium/paratest-selenium. An example of using ParaTest to write functional tests

Whoops! PHP Errors for Cool Kids

Filipe Dobreira — Wed, 05 Jun 2013 14:56:14 +0000

Whoops is a small library, available as a Composer package, that helps you handle errors and exceptions across your PHP projects.

Out of the box, you get a sleek, intuitive and informative error page each time something goes pants-up in your application. Even better, under all that is a very straight-forward, but flexible, toolset for dealing with errors in a way that makes sense for whatever it is that you’re doing.

The library’s main features are:

Detailed and intuitive page for errors and exceptions
Code view for all frames
Focus on error/exception analysis through the use of custom, simple middle-ware/handlers
Support for JSON and AJAX requests
Included providers for Silex and Zend projects through the bundled providers, and included as part of the Laravel 4 core
Clean, compact, and tested code-base, with no extra dependencies

Whoops achieves this through a system of stacked handlers. You tell Whoops which handlers you want to use (you can choose from the included handlers or make your own), and if something happens, all the handlers are given, in order, a chance do to something – this can be anything from analyzing the error (Whoops makes it easier to extract meaningful information from an error or exception), to displaying helpful error screens (like the built-in PrettyPageHandler, which gives you the cool looking page pictured above).

Let’s give it a try, first, by looking at the basics, and then by having a go at building our own handler with Whoops and the Laravel framework. For this short guide, I’ll assume that you’re moderately comfortable with PHP, and that you’ve heard of Composer. If this is not the case, read-up on it here at Nettuts+.

Installing Whoops

Create a directory for your project, change into it, create a composer.json file with the Whoops requirement, and install it. Whoops (as of version 1.0.5) has no dependencies, so this will only take a second.

$ cd /path/to/your/project
$ composer require filp/whoops 1.*
$ composer install

Using Whoops: The Basics

To see that sleek error page in action, let’s setup Whoops and ensure that something breaks by throwing an exception within our code. Create a file within your project’s directory; for this guide, let’s say that it’s called, index.php.

$ cd /path/to/your/project
$ your-favorite-editor index.php

Because we installed Whoops with Composer, and it is PSR-0 compliant, all we need to do is require the Composer autoloader and we’re ready to start using the library within our own code!

<?php
# index.php
require __DIR__ . "/vendor/autoload.php";

$whoops = new Whoops\Run();
$whoops->pushHandler(new Whoops\Handler\PrettyPageHandler());

// Set Whoops as the default error and exception handler used by PHP:
$whoops->register();  

throw new RuntimeException("Oopsie!");
?>

If you already have a web-server running, go ahead and access the file that you just created. Don’t forget: if you’re using PHP 5.4, you can leverage the built-in development server, like so:

$ cd /path/to/your/project
$ php -S localhost:8080

This is what you’ll get:

Pretty neat, right? Handlers, themselves, can expose options to modify or augment their behavior. For example, among other things, you can set the title of the default error page, and even insert extra information:

<?php
# index.php
$whoops = new Whoops\Run();

// Configure the PrettyPageHandler:
$errorPage = new Whoops\Handler\PrettyPageHandler();

$errorPage->setPageTitle("It's broken!"); // Set the page's title
$errorPage->setEditor("sublime");         // Set the editor used for the "Open" link
$errorPage->addDataTable("Extra Info", array(
      "stuff"     => 123,
      "foo"       => "bar",
      "useful-id" => "baloney"
));

$whoops->pushHandler($errorPage);
$whoops->register();

throw new RuntimeException("Oopsie!");
?>

Also, since this is simply a regular Whoops handler, we can mix-and-match with other handlers to achieve more dynamic results. Let’s imagine that you’re working on an AJAX+JSON-driven website. Right now, if your application were to fail somehow, you would get a bunch of nasty HTML coming down the pipe, when you were expecting JSON. No big deal:

<?php
# index.php
$whoops->pushHandler(new Whoops\Handler\PrettyPageHandler());
$whoops->pushHandler(new Whoops\Handler\JsonResponseHandler());

$whoops->register();

throw new RuntimeException("Oopsie!");

That’s it. Now, if something fails during an AJAX request, Whoops will respond with a JSON response detailing the error. If it’s NOT an AJAX request, you will continue to see the regular error page. If an error ocurrs, Whoops will filter through each of the registered handlers (starting at the last handler to be registered), and give them a chance to analyze, modify and respond to the request.

Now that you have a general idea of how Whoops works, let’s have a go at building our own handler with Whoops and the Laravel 4 framework.

Whoops and Laravel 4

Laravel 4 bundles Whoops as a core exception handler, enabled by default in development mode, including a custom color-scheme by Dayle Rees:

If you haven’t installed Laravel yet, head on over and follow the installation steps. Laravel is covered extensively on Nettuts+ and Tuts+ Premium, so you’ll find plenty of training here, if you’d like to dig in further.

For the next steps, I’ll assume that you are somewhat comfortable with the basics of Laravel 4. However, even if you aren’t, it should still be easy to follow.

If you’re in development (debug) mode, Whoops is available through the IoC container as whoops, and pre-set with one of two handlers: PrettyPageHandler or JsonResponseHandler, as whoops.handler (the same two we just talked about). Both of these handlers expose useful additional methods, as you’ve seen above with the PrettyPageHandler. By accessing these services, we can start customizing our Whoops experience within the framework.

For simplicity’s sake, in your app/routes.php file, let’s hook into the Whoops service and set a custom page title for our error pages:

<?php
#app/routes.php

/*
|--------------------------------------------------------------------------
| Application Routes
|--------------------------------------------------------------------------
*/

use Whoops\Handler\PrettyPageHandler;

// Use the Laravel IoC container to get the Whoops\Run instance, if whoops
// is available (which will be the case, by default, in the dev
// environment)
if(App::bound("whoops")) {
    // Retrieve the whoops handler in charge of displaying exceptions:
    $whoopsDisplayHandler = App::make("whoops.handler");

    // Laravel will use the PrettyPageHandler by default, unless this
    // is an AJAX request, in which case it'll use the JsonResponseHandler:
    if($whoopsDisplayHandler instanceof PrettyPageHandler) {

        // Set a custom page title for our error page:
        $whoopsDisplayHandler->setPageTitle("Houston, we've got a problem!");

        // Set the "open:" link for files to our editor of choice:
        $whoopsDisplayHandler->setEditor("sublime");
    }
}

Route::get('/', function()
{
    // Force the execution to fail by throwing an exception:
    throw new RuntimeException("Oopsie!");
});
?>

Tip: Whoops supports a few editors by default, and allows you to implement support for your own as you wish. Read more about it here.

If you now access your Laravel application, you’ll be greeted with an error message with your custom page title. If you click the file path above the code box, it should open the referenced file right in your editor or IDE of choice. How cool is that? Also, since we can reach the handler already setup by the Laravel core, we can put to use the other features we’ve learned about above. For example, we can add custom tables (with PrettyPageHandler::addDataTable) with useful information about our application.

Let’s have a go at one more example. This will be our first attempt at writing our own custom handler. We want to get all the stack frames for an exception, and remove anything that’s not part of our application code. Sounds simple enough, right?

<?php
#app/routes.php

/*
|--------------------------------------------------------------------------
| Application Routes
|--------------------------------------------------------------------------
*/

use Whoops\Handler\Handler;

// Use the Laravel IoC to get the Whoops\Run instance, if whoops
// is available (which will be the case, by default, in the dev
// environment)
if(App::bound("whoops")) {
    $whoops = App::make("whoops");

    $whoops->pushHandler(function($exception, $exceptionInspector, $runInstance) {
        // Get the collection of stack frames for the current exception:
        $frames = $exceptionInspector->getFrames();

        // Filter existing frames so we only keep the ones inside the app/ folder
        $frames->filter(function($frame) {
            $filePath = $frame->getFile();

            // Match any file path containing /app/...
            return preg_match("/\/app\/.+/i", $filePath);
        });

        return Handler::DONE;
    });
}

Route::get('/', function()
{
    // Force the execution to fail by throwing an exception:
    throw new RuntimeException("Oopsie!");
});
?>

Tip: You don’t actually have to return Handler::DONE – this serves only a semantic purpose. If you want Whoops to stop running any extra handlers after yours, write return Handler::LAST_HANDLER. If you want Whoops to exit the script execution after your handler, return Handler::QUIT.

You can see that it’s remarkably concise. Whoops\Run‘s pushHandler method accepts a closure that receives up to three arguments: the exception object, an exception inspector, which exposes some utility methods to, you guessed it, inspect exceptions, and the Whoops\Run instance that captured the exception. Through this handler, we use the exception inspector to extract the stack frames, all within a neat FrameCollection object:

<?php
$frames = $exceptionInspector->getFrames(); // #=> Whoops\Exception\FrameCollection;
count($frames); #=> int

foreach($frames as $frame) {
    get_class($frame); // #=> Whoops\Exception\Frame

    print $frame->getFile() . ":" . $frame->getLine() . "\n";
        #=> "/path/to/file.php:123"
}
?>

Tip: Whoops internally converts closures to a special handler: Whoops\Handler\CallbackHandler.

You can count, iterate, map and filter the contents of this class, with the interesting but important aspect that the map and filter operations mutate the object in-place. This means that both of these operations modify the original instance directly, instead of creating a new collection. How is this important? It means handlers can more easily perform changes that propagate downwards to all the other handlers in the stack. This is exactly what we did with our simple handler above. If you now run the script again, you’ll see that we get a shorter list of stack frames, only concerning the code living within our application directory.

As for the Frame object, itself (Whoops\Exception\Frame), it exposes a set of methods to gather information about the frame contents (the file path, line number, the method or function call, class name, etc,), and methods that allow you to attach comments to individual stack frames. A Frame comment is a useful feature in Whoops that allows handlers to provide additional information that they gather from an exception by attaching notes directly to individual frames in the stack. Handlers like the PrettyPageHandler, for example, can then gather those comments and display them along with the frame’s file path and line number.

<?php
$whoops->pushHandler(function($exception, $exceptionInspector, $runInstance) {
    foreach($exceptionInspector->getFrames() as $i => $frame) {
        $frame->addComment("This is frame number {$i}");
    }

    return Handler::DONE;
});
?>

Frame comments may also receive a second scope argument. If you have multiple custom handlers, you can, for example, filter frame comments by this argument to gather only the information that you care about.

<?php
$frames = $exceptionInspector->getFrames();
foreach($frames as $frame) {
    // Was this frame within a controller class? (ends in Controller)
    $className = $frame->getClass();
    if(substr($className, -10) == "Controller") {
        $frame->addComment("This frame is inside a controller: $className", "controller-error");
    }

    // Later, in another handler, get all comments within the 'controller-errors' scope:
    $controllerErrors = $frame->getComments("controller-errors"); // #=> array
}
?>

Also of interest, the PrettyPageHandler naturally HTML-escapes the frame comments before displaying them, but will intelligently capture URIs in the comment’s body and convert them to clickable anchor elements. Want to link frames to documentation, or to GitHub repositories? It’s easy enough; let’s create our own handler class for this example.

Tip: Using your own class instead of a closure provides you with some extra control over your handler – not to mention making it easier to cover with automated tests. Your custom handler classes must implement the Whoops\Handler\HandlerInterface interface, but you may instead simply extend the Whoops\Handler\Handler class, and implement the missing handle method, as shown in the example below.

<?php
# LaravelGithubLinkHandler.php

use Whoops\Handler\Handler as BaseHandler;

/**
 * When possible, link laravel source files to their location
 * on the laravel/framework repo @ github.com
 */
class LaravelGithubLinkHandler extends BaseHandler
{
    private $repoBase = "https://github.com/laravel/framework/blob/master";

    /**
     * @return int
     */
    public function handle()
    {
        $frames = $this->getInspector()->getFrames();

        foreach($frames as $frame) {
            $file = $frame->getFile();
            $line = $frame->getLine();

            // Some frames may not have a file path, for example, if it ocurred within
            // a Closure, so we'll need to check for that:
            if(!$file) continue;

            // Check if the file path for this frame was within the laravel/framework
            // directory, inside the Composer vendor/ directory, and use a regex capture
            // to extract just the parts we want:
            if(preg_match("/\/vendor\/laravel\/framework\/(.+)$/", $file, $matches)) {
                $path = $matches[1]; // First match is whole path, second is our capture
                $url  = "{$this->repoBase}/$path";

                // We can also link directly to a line number, if we have it. Github
                // supports this by appending #L<line number> to the end of the URL:
                if($line !== null) {
                    $url .= "#L{$line}";
                }

                $frame->addComment($url, "github-linker");
            }
        }

        return Handler::DONE;
    }
}
?>

That’s it, as far as our handler goes. Put that class somewhere in your project, and all that’s left to do is enable and try it out:

<?php
# app/routes.php
// ...
$whoops->pushHandler(new LaravelGithubLinkHandler());
// ...
?>

With only a handful of lines of code, we’ve added an extra layer of (possibly useless, but hey, it’s an example) functionality to our error pages. Here are a few extra ideas, if you’re looking for a challenge:

Package your custom error handler as a Laravel service provider.
Are you using Git to manage your project? Build a custom handler that hooks into git-blame to determine who the last person to touch that file that keeps throwing an exception (and yell at them) was, directly from the error page.
If you’re feeling brave, use nikic’s PHP-Parser to analyze the troublesome code, and provide suggestions for fixes (I promise it’s not as complicated as it sounds).

Final Thoughts

I hope that this short guide has helped you gain an understanding of the sort of possibilities that this library enables in your every-day projects. For more information, refer to the complete API documentation.

Whoops is framework-agnostic, light-weight and, I believe, quite powerful in its simplicity and focus on mix-and-matching small tools. It’s also open-source and open to suggestions and improvements. If you’d like to contribute or report a bug, head to the official repository!