Testing, for web developers, is usually a fairly disorganised affair. Most of the time, we test “on the job”, i.e., we code something and then we run it; if it breaks, we fix it. In fact, often, the thought of actually structuring our testing process can seem somewhat of an over complication. This is certainly not the case.

The benefits of a structured testing process are legion, as any software engineer will attest. At the very least, it provides confidence in our code — something which can only be of incredible benefit when code is shared across a wider team. At its best, the tests we write can influence the way we develop the code being tested; and save effort, time, and money.

In this post I’m going to look at “unit testing”, and how it can be of benefit to web developers in general.

What is unit testing?

Unit testing is the process of validating and verifying the individual “units” of code in a system. A unit is the smallest testable part of a system or application. In the majority of cases within web development, these units are usually individual functions or methods of our object-oriented back-end and front-end code.

At this point, it’s probably worth noting that unit testing is really only applicable to functional code, such as PHP, JavaScript, Python, Perl, Ruby etc., because it breaks down into units nicely. Testing mark-up and CSS is a slightly different process and one that I may look at in a future blog post.

A note on language

The concept of unit testing is entirely language agnostic, therefore the examples in the following tutorials can be ported to PHP, Perl, Ruby, JavaScript, or any other functional or object-oriented language you are using.

I’ll use Python as the language for all examples in this post as it’s my current weapon of choice (and the interactive interpreter is very handy for trying out ideas); if you’ve never seen Python syntax, don’t worry, it’s easily understandable (if you want more information, take a look at the Python tutorial).

Writing a simple test

To begin with, let’s imagine we have a function for calculating the area of a rectangle:

def get_area_of_rect( width, height ):
    return width * height

Let’s paste this function into the interactive interpreter, so that we can play with it:

>>> def get_area_of_rect( width, height ):
...     return width * height
...
>>> 

Now if we call it, we’ll receive the area of our rectangle like so:

>>> get_area_of_rect( 2, 3 )
6
>>>

Now if we wanted to test this function, we could call it with a some defined parameters and test the output against our expectations:

def test_get_area_of_rect():
    width = 2
    height = 3
    expected_result = 6

    result = get_area_of_rect( width, height )

    if result == expected_result:
        print "Test passed: Received expected result!"

Now if we run our test, we should see the following:

>>> test_get_area_of_rect()
Test passed: Received expected result!
>>>

Brilliant; our code passed the test because the expected result was the same as the actual result — however, that’s actually not very useful to us. It’s far more important that our tests can trap failing code than code that works.

Imagine our get_area_of_rect function had been updated as follows:

def get_area_of_rect( width, height ):
    return width / height

Anyone with rudimentary maths skills can tell that this will no longer return the correct area of a rectangle. In fact, if we rerun our test, we’ll get no output:

>>> test_get_area_of_rect()
>>>

A fat lot of good that is. If our code is failing for some reason, we want to know about it. After all, that’s the whole reason for testing it.

Let’s update the if statement in our test_get_area_of_rect function:

if not result == expected_result:
    raise AssertionError
else:
    print "Test passed: Received expected result!"

In fact, we could probably dump the else clause since it’s of little real importance except for logging purposes:

if not result == expected_result:
    raise AssertionError

That’s nicer, but Python provides us a shortcut to this if not … raise AssertionError syntax with the assert statement:

assert result == expected_result

With that in mind, the test_get_area_of_rect function should now look like this:

def test_get_area_of_rect():
    width           = 2
    height          = 3
    expected_result = 6

    result = get_area_of_rect( width, height )

    assert result == expected_result

If we run our test again, we should receive more constructive output, considering our get_area_of_rect function is still dividing:

>>> test_get_area_of_rect()
Traceback (most recent call last):
  File "", line 1, in 
  File "", line 8, in test_get_area_of_rect
AssertionError
>>>

We now have a decent automated test, but we’re only checking a single expected result. If, for instance, we passed in a width of 1 and a height of 1, both multiplication and division would return the same result: 1. This means that we should pass multiple sets of width and height parameters and test against their expected results. Since these parameters will be constant, let’s define them as tuples and loop through them, testing each in turn:

def test_get_area_of_rect():
    known_values = ( ( 0, 0, 0 ),
                     ( 1, 1, 1 ),
                     ( 2, 2, 4 ),
                     ( 2, 3, 6 ),
                     ( 3, 0, 0 ) )

    for width, height, expected_result in known_values:
        result = get_area_of_rect( width, height )
        assert result == expected_result

As you can see, we’re now passing several sets of parameters. Each set of parameters is valid, and as such, we should expect a successful outcome from our get_area_of_rect() call.

Now if we run our test, we should get something like the following:

>>> test_get_area_of_rect()
Traceback (most recent call last):
  File "", line 1, in 
  File "", line 8, in test_get_area_of_rect
  File "", line 2, in get_area_of_rect
ZeroDivisionError: integer division or modulo by zero
>>>

Ah ha! By passing multiple sets of parameters we’ve uncovered more significant information regarding the AssertionError that was raised earlier; we have a division by zero error on line 2 of get_area_of_rect. This has supplied us with all the information we need to fix our error, so let’s take a look at that function again:

def get_area_of_rect( width, height ):  # Line 1
    return width / height               # Line 2

Change the division back to multiplication, like so:

def get_area_of_rect( width, height ):  # Line 1
    return width * height               # Line 2

And then rerun our test:

>>> test_get_area_of_rect()
>>> 

We get no output at all because the test passed successfully. As the old proverb goes, “no news is good news”.

Now if we add to our original code in the future, we’ll still be able to run this testing code to see what has been affected. What’s more, we can add to it by writing more tests, should the complexity grow. In fact, creating an extensive suite of tests is generally the aim of unit testing, but to do so, it’s a good idea to utilise a testing framework, which I’ll look at in the next post of this series.

Summary

Writing tests for code enables more efficient debugging, faster bug fixing, and an overall confidence in the quality of your code. Furthermore, the rest of your team (or whomsoever you’re sharing said code with) will equally have confidence thanks to the tests included.

In brief, here’s what we’ve learnt about unit testing:

• It’s most important that your tests catch failure, not success.
• Test the smallest amount of code, or “unit”, possible each time.
• Test as many different parameters, inputs, and outputs as possible.
• Run the tests often.

The next post

In the next post, I’ll be expanding on unit testing by looking at test frameworks, test architecture, code coverage, and other more advanced techniques that can improve your testing environment.

Due to an ever more stressful workplace and the arrival of another child in the household, I took a somewhat forced break from blogging. This was entirely unintended and I’ve decided to get back on the wagon again by finishing the aforementioned post.

So here it is; the finished article on setting up Postfix on a shared mail and web server. Please remember that I’m technically a mail server numpty and probably won’t have all the answers to questions should you have them. However, feel free to leave them in the comments and hopefully someone out there (or someone I can poke for comment elsewhere) can help you out.

A bit of background

I made the jump from supported, pre-built hosting to entirely self-managed hosting last year. As a result, I soon discovered the need to handle email received by the various hosted domains on my server.

One solution to this problem would be to use Google Apps for Business. Too easy, drill sergeant; and also $50 more than I was willing to spend a year. Another solution would be to install my own email server and manage it myself. Despite this idea giving me The Fear™, it also meant I’d be learning something new — a buzz I find difficult to ignore at the best of times.

Given those options, I’m sure it’s not too hard to figure out which route I took, and the following blog post is a small amount of what I learnt in the process.

Hostname

Often, when building a web server, the server hostname is overlooked as unimportant. This is generally because the DNS system provides all the functionality needed when serving files over HTTP. However, when the server is required to handle email, it is important to set a hostname because it is used by the various email protocols as a source and destination identifier; ergo a key component in the identification of spam.

Postfix will let you send and receive mail just fine between accounts on a machine with a fantasy name. However, you cannot use a fantasy hostname in your email address when sending email via the Internet, because no-one would be able to reply to your mail. In fact, more and more sites refuse mail addresses with non-existent domain names.

Further to this, you may want to specify a fully-qualified domain name hostname. A hostname is considered to be a fully qualified domain name (FQDN) if all the labels up to and including the top-level domain name (TLD) are specified. The hostname “mail.nerfabulousdesigns.co.uk” terminates with the top-level domain “co.uk” and is thus fully-qualified. Depending on the system, an unqualified hostname such as “server1″ or “test-server” may be combined with default domain names in order to determine the fully qualified domain name. However, in most cases, a web server or mail server won’t be running in that sort of environment.

With all that in mind, it’s best to use a domain name you can access the DNS records for, as you’ll want to forward email to it using the MX record. An accepted standard, when working with mail, is to use “mail.mydomain.com”, however, in this case, we’re sharing email and web serving duties on the same box. A better solution would be to come up with any name that you can use to uniquely identify the server in your DNS settings. For the purpose of my examples, I’m going to work with the hostname “spartacus.nerfabulousdesigns.co.uk”.

DNS for your hostname

Before I start wading into the installing and configuring the server, it’s important to get the DNS set up in your DNS manager. So, if the IP address of our server is 123.456.789.123 we’ll want the following DNS records:

A Records:
spartacus.nerfabulousdesigns.co.uk -> 123.456.789.123
mail.nerfabulousdesigns.co.uk -> 123.456.789.123

MX Record:
mail.nerfabulousdesigns.co.uk

Any other domains you want to set up for email only need the following rules:

A Records:
mail.yourdomain.com -> 123.456.789.123

MX Record:
mail.yourdomain.com

Once you’ve saved those changes, and you’re awaiting DNS propagation, it’s time to get back to our server.

Server settings for your hostname

To find out the current hostname of your server (which is defined during installation), type the following:

$ hostname

The hostname command will output the current hostname if you run it without any parameters. If you want to see the fully qualified hostname, type the following:

$ hostname -f

To change your hostname temporarily, you can simply enter the new hostname as the first parameter of the hostname command. So, assuming I want to identify my server as “spartacus”, I’d enter the following:

$ hostname spartacus.nerfabulousdesigns.co.uk

Now we can check our change:

$ hostname
spartacus.nerfabulousdesigns.co.uk

Huzzah!

Unfortunately, that change will be reset should the machine be rebooted for any reason.

Not-huzzah!

To make the change permanent, you’ll need to edit the single-line /etc/hostname file:

$ sudo vim /etc/hostname

Update it to contain the following:

spartacus.nerfabulousdesigns.co.uk

Now you can trigger your change by restarting the hostname daemon:

$ /etc/init.d/hostname.sh start

However, you don’t need to do that just yet since you’ll probably want to restart your server later on in the process.

The hosts file

Next we’ll need to update the hosts file on the server so that it recognises the new hostname as a referer to the loopback IP 127.0.0.1. To do that, edit the hosts file with the following command:

$ sudo vi /etc/hosts

And then update it so that it contains a line similar to this:

127.0.0.1        spartacus.nerfabulousdesigns.co.uk

Once this file has been saved, we can do that reboot we were going to do earlier:

$ sudo reboot

Once your server is back up, the hostname change is complete. Your server will now use the loopback address when hostname references are used, instead of wasting valuable time checking DNS records.

Install Postfix

Now that the server is primed for email, it’s time to install postfix — and while we’re at it, let’s install mailx, an improved version of the mail email user agent program, and let’s also confirm we have telnet installed as we’re going to use it for testing later on:

$ sudo aptitude install postfix mailx telnet

As Postfix installs, you’ll be asked a number of configuration questions:

For “general type of mail configuration”, choose:

Internet Site

You will then be asked the fully qualified domain name of your server, which in our case is:

spartacus.nerfabulousdesigns.co.uk

Once you’ve done that, the installation is complete. Your web server is now capable of sending email; and to prove it, let’s just test it from the command line.

Testing

Let’s send an email to a real email address (remember to replace somebody@somewhere.com with the real email address of your choice):

$ mail somebody@somewhere.com

The mail program will now ask you for a subject. Once you press return, mail will now be accepting the body of your email (albeit without a prompt). To tell the program that you’ve reached the end of your email body, you’ll need to place a single full-stop character, “.”, on a line by itself (since the return character is perfectly valid in the body of your mail). Finally, the mail program will then ask you for any CC addresses.

Here’s an example of the complete screen output:

$ mail somebody@somewhere.com
Subject: Testing my all new email server!
Red leader, this is mail server. Are you receiving, over?
.
Cc:
$

You won’t receive any confirmation that the email has been sent — you should just return to the command prompt — however, as long as everything was set up correctly, you should have received the email.

The next step is to configure our mail server so that it knows what to do with received email.

Configuration

Configuration of a mail server is where all the pain and torment begins; but in this case, I only want to set up mail forwarding — which takes all the pain and torment away and leaves us with a fairly straight forward, unburdened mail server that’s using very little disk space. I could usually do this at the DNS level, but I chose not to in case I decided to work with real mail boxes in the future — and also because I was still relishing the learning experience. If you want to set up something different, I’m afraid you’re going to have to look elsewhere for a tutorial because I haven’t the time (or the patience) to go through the multitude of different set-ups that are possible.

The first step in setting up mail forwarding in Postfix is to create a virtual hash. To do that, we need to create the following file:

$ sudo vim /etc/postfix/virtual

And inside that file, we need to define some values. Here’s how we’d set up nerfabulousdesigns.co.uk as a mail forwarding domain:

john@nerfabulousdesigns.co.uk    john.smith@yahoo.com
joe@nerfabulousdesigns.co.uk     joe.bloggs@yahoo.com

# For a catch-all, do the following
@nerfabulousdesigns.co.uk        example@yahoo.com

Once you’ve saved that file, run the following command to create the postmap:

$ postmap /etc/postfix/virtual

Next you’ll also need to update this file:

$ sudo vim /etc/postfix/main.cf

Add your domain to virtual_alias_domains and your virtual hash to virtual_alias_maps like so:

virtual_alias_domains = nerfabulousdesigns.co.uk ...other hosted domains...
virtual_alias_maps = hash:/etc/postfix/virtual

Finally, you’ll need to do the following to restart Postfix:

$ postfix reload

Any emails to the domains or addresses you added to the virtual hash should now be forwarded as expected.

Telnet

Finally, let’s test the email server is working using Telnet. This is a handy little technique for checking everything is as expected, since it will also display responses at every stage of the mail sending process.

You can telnet from any machine that has it installed — and since we installed it on our server earlier, we can even telnet the server to itself, which might be useful when trying to track down problems.

To begin with, let’s Telnet to our server over the default SMTP port 25:

$ telnet mail.nerfabulousdesigns.co.uk 25

Which should trigger a response similar to this:

Trying 209.20.86.249...
Connected to mail.nerfabulousdesigns.co.uk.
Escape character is '^]'.
220 spartacus.nerfabulousdesigns.co.uk ESMTP Postfix (Ubuntu)

If the connection was a success, you should receive a 220 SMTP reply code. For more information, here’s a handy reference on SMTP reply codes.

The next thing we need to do, in the essence of being polite, is greet the mail server. We can do that one of two ways. We can either use the SMTP command, HELO:

HELO my.local.domain
250 spartacus.nerfabulousdesigns.co.uk

Or we can use the more recent EHLO command, which will return more detailed information about the server:

EHLO my.local.domain
250-spartacus.nerfabulousdesigns.co.uk
… More detailed info. here…

Great, so now that we’ve identified ourselves, we’re reading to start defining the envelope for our message. First, we need to tell the mail server who the sender is:

MAIL FROM:
250 2.1.0 Ok

Second, we need to tell the mail server who we’d like the email delivered to:

RCPT TO:joe@nerfabulousdesigns.co.uk
250 2.1.5 Ok

That’s it; the envelope is now complete. We can now move onto filling our message with data. To do that, we simply use the command DATA, which will then allow us to enter the subject and main content for our message:

DATA
354 End data with .
Subject: Test Email
This is a test email.
.
250 2.0.0 Ok: queued as 40F438D023

Our message is complete, and has been queued by the mail server for redistribution to the recipient. We’re done here, so let’s quite out of the telnet session:

QUIT
221 2.0.0 Bye
Connection closed by foreign host.
$ 

And we’re now back at our original command prompt. If you check the mailbox of your recipient, you should have successfully received the email.

Summary

Despite the fact that I could probably have handled email in more efficient ways, I thoroughly enjoyed learning about Postfix and the black magic therein. I’m sure that some of my peers will have plenty to correct me on, and I sincerely hope they do; as previously stated, this was intended as a learning experience, and learning is what I have done.

Hopefully, some — if not all — the techniques used in this article will be of some use to you. Certainly, they have aided me in my understanding of core internet technologies.

Please feel free to comment, question, or to point out flaws with my methods. I hope the discussion is fruitful.