Basic User Login

Like most sites, we used the PHP session to maintain a logged in user for our site.   We started a session, kept track of some data indicating if the user is logged in or not, and that was about it.

I never looked at sessions and cookies in-depth before.  I knew generally how sessions worked.  PHP sets a cookie in the client’s browser.  The cookie contains a session ID.  When a request comes in, PHP reads the session ID, looks for a file corresponding to the ID on disk (or in a database, memcached, etc.), reads in the file containing the session data, and loads the session into the request.  When the request finishes, the session data is saved to the file again.

Implementing The “Remember Me” Checkbox

First, naively, I thought all I had to do was find the right php.ini directive to make sessions last forever.  Browsing the PHP manual and googling, I came across the session.cookie_lifetime directive, configured in either php.ini or by session_set_cookie_params().

session.cookie_lifetime specifies the lifetime of the cookie in seconds which is sent to the browser. The value 0 means “until the browser is closed.” Defaults to 0.

I set this to 24 hours.  Well, that was easy, I thought.

Except it didn’t work.  Users reported logging in, going out to lunch, coming back, and getting logged out on the first link clicked.  I dug deeper and found another directive.

session.gc_maxlifetime specifies the number of seconds after which data will be seen as ‘garbage’ and cleaned up. Garbage collection occurs during session start.

It defaults to 1440 seconds, or 24 mins.

It’s important to know that session.cookie_lifetime starts when the cookie is set, regardless of last user activity, so it is an absolute expiration time.  session.gc_maxlifetime starts from when the user was last active (clicked), so it’s more like a maximum idle time.

Starting To Understand

Now I could see that both of these directives must cooperate to get the desired effect. Specifically, the shorter of these two values determines my session duration.

For example, let’s say I have session.cookie_lifetime set to its default of 0, and session.gc_maxlifetime is set to its default of 24 mins.  A user who logs in can stay logged in forever, provided he never closes his browser, and he never stops clicking for more than 24 mins.

Now, let’s say the same user takes a 30 min. lunch break, and leaves his browser open.  When, he gets back, he’ll most likely have been logged out because his session data was garbage collected on the server, even though his browser cookie was still there.

Now, let’s change session.cookie_lifetime to 1 hour.  A user who logs in can stay logged in for up to an hour if he clicks away for the whole time.  This is regardless of whether or not he closes/reopens his browser.  If he takes his 30 min. lunch break after working for 15 mins. he will most likely be logged out when he returns, even though his browser cookie had 15 more mins. of life.

Now, keeping session.cookie_lifetime at 1 hour, let’s set session.gc_maxlifetime to 2 hours.  A user who logs in can stay logged in for up to an hour, period.  He does not have to click at all in that time, but he’ll be logged out after an hour.

The Real “Remember Me” Solution

Back to my problem.  At this point, I could’ve just set both directives to something  like 1 year.  But since session.gc_maxlifetime controls garbage collection of session data, I’d have session data up to a year old left on the server!  I did a quick check on the PHP session directory.  There were already several thousand sessions, and that was only for a 24-minute lifetime!

Clearly, this was not how Twitter did it.  A little more digging, and I realized that sites like those do not keep your specific session around for long periods of time.  What they do is set a long-lasting cookie that contains some sort of security token.  From that token, they can authenticate you, and re-create your session, even if your session data has already been removed from the server.  (The cookie name for Twitter is auth_token and looks to have a lifetime of 20 years.)

With the session recreation method, I could control when and how to log out users, if at all.  So this enabled us to give users indefinite sessions, while keeping all session directives at their default values.

Beyond Session Cookies

This only scratches the surface of authentication topics of course.  We didn’t talk about security implications of the session re-creation method, though I will say that the best security practice against session-based attacks seems to prompt for a password if the user attempts to change or view sensitive account information.  LinkedIn is the first example that comes to mind.

Shortly after implementing this, a request came down from high above to centralize the authentication for our multiple products.  I began to investigate single sign-on (like Google accounts) and federated identity (like OpenID), but those are topics of another post.

Here are a couple blogs that got me on my way to the final solution. Be sure to read the comments:

• PHP session cookie refresh
• Auto-login
• Simple Single Sign-On for PHP

1

$sql = "select event_id from events where event_dt >= curdate()";

Because MySQL knows that this query run today is not the same query when it is run tomorrow. There are other SQL functions such as rand() and unix_timestamp() that will bypass the query cache. These are listed here.

So I avoid these functions when possible by calculating the value in PHP. For example, I’d rewrite the above query as:

1
2

$date = date('Y-m-d');
$sql = "select event_id from events where event_dt >= '$date'";

public function sum($x, $y)
{
    return $x + $y;
}

And the test.

public function testSum()
{
    $summer = new Summer();
 
    $this->assertEquals(7, $summer->sum(3, 4));
 
    // and after awhile, trying to break the 
    // function turned out to be part of the fun.
    $this->assertEquals(4, $summer->sum(7, -3));
    $this->assertEquals(4, $summer->sum(4, null));
}

In any case, this was nice and all for arithmetic, but like in any typical web app, I soon ran into the big problem: how do I test the database?

Now, there is a chapter on database testing in the PHPUnit manual with DBUnit for PHP.  But as I read about those, I saw a lot of documentation about syntax and handling of null values and empty tables. I wondered why I couldn’t just create a real database and test it with real queries?

Enter SQLite

SQLite is a self-contained, serverless, zero-configuration, ACID-compliant RDBMS.  Since SQLite requires no setup, and runs completely in memory, it makes the perfect database to run self-contained, serverless, automated unit tests against.

Database Access Abstraction To The Rescue

Being able to test seamlessly with a SQLite database in my unit tests and run against a MySQL database in live code required some careful thought about how I went about setting up my classes.  First of all, I created a database access interface with some standard functions like query(), getRows(), startTransaction(), commit(), etc.  (You are using a database access class, aren’t you?)  Then, I implemented the interface with a MySQL implementation, and an SQLite implementation.

Since there are places in the code that takes advantage of MySQL specific features, I had to emulate these features in my SQLite driver.  For example, there was no lpad() function in SQLite.  Fortunately, thanks to the magic of createFunction(), I could create a SQLite User-Defined Function.

public function lpad($value, $numchars, $char)
{
    return str_pad($value, $numchars, $char, STR_PAD_LEFT);
}
 
$sqlite->createFunction('lpad', 'lpad', 3);

Some of the MySQL bells and whistles are of course missing, such as ON DUPLICATE KEY UPDATE. With some fancy query re-writing, I got those MySQL queries working with SQLite.  Here is my current SQLite database access class in its entirety.

Dependency Injection FTW!

Now, that I had a MySQL database driver, and an equivalent SQLite database driver, I needed a way to get my business objects to use the SQLite version in a test, and the MySQL version during real execution.  If only there was a pattern that let me “inject” the proper database dependency into the object.  Well, this is exactly what Dependency Injection is!

Instead of instantiating a database driver within the class, I passed the database object in the constructor.  There are other ways to pass in the object, such as with a setter method, or with a global registry, but I like constructor the best.  So the class might look like this:

class Customer
{
    private $_db;
 
    public function __construct($db)
    {
        $this->_db = $db;
    }
}

Writing The Test

With the database access classes in place set to be injected into my classes, I could test functions that persisted data to the database, and then actually inspect the database.  There was no XML data files.  There was no database testing DSL.  There was only plain ol’ SQL.

Here is a simplified function that saves fictional customer data, and a test for it.

public function save($firstName, $lastName)
{
    $this->_db->query("INSERT INTO customers (first_name, last_name) VALUES (?, ?)", 
        array($firstName, $lastName));
 
    return $this->_db->lastInsertId();
}

And the test.

class CustomerTest extends PHPUnit_Framework_TestCase
{
    private $_db;
 
    public function setUp()
    {
        $this->_db = new Test_Database;
 
        $this->_db->query("CREATE TABLE customers (first_name, last_name_);");
    }
 
    public function tearDown()
    {
        $this->_db->query("DROP TABLE customers");
    }
 
    public function testSave()
    {
        $customer = new Customer($this->_db);
 
        $customer->save("Mickey", "Mouse");
 
        // test that customer is saved to DB
        $results = $this->_db->getRows("SELECT * FROM customers");
 
        $this->assertEquals("Mickey", $results[0]['first_name']);
        $this->assertEquals("Mouse", $results[0]['last_name']);
        $this->assertEquals(1, count($results));
    }
}

One Last Thing

This method could get to be overkill in certain situations as you write tons of SQL to set up your tests.  But even then I find it preferable to setting up XML or CSV data.  As far as mocking out a database with some sort of Mock Object, I still maintain, why use a mock and a DSL when you can use the real thing?  This has worked well for me so far.

In which I give a short introduction to unit testing, and then outline the story that finally turned on the light bulb about real test-driven development and test first for me. The third part of the talk gives an introduction to Dependency Injection (DI), an integral part to unit testing.

One of the most successful and respected coworking space (from what I have gathered on the interweb anyway) is Independents Hall in Philadelphia. Fortunately, in true coworking spirit, they have done a great job of blogging about their progress and I have been leaning heavily on it as a resource. 

But enough of that for now.  I want to save some of this for the Cincinnati Coworks blog itself, whenever I get around to setting it up.

Some other coworking resources.

• Coworking on Wikipedia
• The coworking wiki
• The coworking buzz

I’ll spare you the problems I ran into and overcame, and just list the steps to get this done.  These instructions are based on a 64-bit, RHEL 5 server running the pre-packaged Apache server.  So if you follow these instructions on a different setup, of course, filenames, directories, versions, etc. may differ.

Install mod_vhost_dbd

Download dbd-modules from Google Code.  This is a great piece of code in the form of an Apache module that uses mod_dbd and a DBD Mysql (or other database) driver to fetch the DocumentRoot for a given domain from a database.

% wget http://dbd-modules.googlecode.com/files/dbd-modules-1.0.5.zip

Unzip the archive in a directory. As indicated on the website, build and install the module.

% apxs -c mod_vhost_dbd.c
% apxs -i mod_vhost_dbd.la

This places mod_vhost_dbd.so in /usr/lib64/httpd/modules.  Enable both this module and mod_dbd by adding two lines to httpd.conf, or equivalently creating a new include file in /etc/httpd/conf.d containing these lines.

LoadModule dbd_module modules/mod_dbd.so
LoadModule vhost_dbd_module modules/mod_vhost_dbd.so

In true unit fashion, now might be a good time to restart Apache, just so you can be sure everything is working up to this point.

% service httpd restart

Install Mysql DBD Driver to APR

Unfortunately, on my system, the Mysql DBD driver was nowhere to be found.  I had to rebuild Apache Portable Runtime (APR) utils with the Mysql driver enabled.

Download apr and apr-util from Apache.  Note these are not the latest versions, but the versions that matched the packages in worked for RHEL 5.

% wget http://archive.apache.org/dist/apr-1.2.8.tar.bz2
% wget http://archive.apache.org/dist/apr-util-1.2.8.tar.bz2

Unpack and untar these archives in the same parent directory.

Build and install APR.  Now, I do not think this is absolutely necessary, but it seems like a good idea to keep the versions in sync.

% ./configure --prefix=/usr
% make
% make install

Build and install apr-util.  Due to licensing issues, apr-util does not actually contain the Mysql DBD driver until apr-util-1.2.12.  Prior to that version, it must be downloaded separately, and the configure script rebuilt.

% wget http://apache.webthing.com/svn/apache/apr/apr_dbd_mysql.c
% ./buildconf --with-apr=../apr-1.2.7

Now for the three commands every Linux admin loves.

% ./configure --prefix=/usr --with-apr=/usr --libdir=/usr/lib64 --with-expat=builtin --with-ldap-include=/usr/include --with-ldap-lib=/usr/lib64 --with-ldap=ldap --with-mysql
% make
% make install

The first time I tried this, Apache could not find any LDAP-related modules.  Adding those configure switches seemed to do the trick.  Restart Apache.

% service httpd restart

Apache should now be able to query a Mysql database to get the DocumentRoot for a domain.  My VirtualHost block looked something like this.

<VirtualHost *:80>
    ServerName *.example.com
    DocumentRoot "/path/to/default/document/root"
 
    DBDriver mysql
    DBDParams host=localhost,user=root,pass=secret,dbname=vhosts
 
    DBDocRoot "SELECT path FROM vhosts WHERE host = %s"  HOSTNAME
</VirtualHost>

For more details and instructions on mod_vhost_dbd configuration directives, read the project wiki.

The solution I found starts with this thread about upgrading PHP on the CentOS forums.  To summarize, run these commands:

% cd /etc/yum.repos.d
% wget http://dev.cnetos.org/centos/5/CentOS-Testing.repo

Then, edit the .repo file and set the "enabled" value to "1".

% yum list available php
Available Packages
php.i386                                 5.2.6-2.el5s2          c5-testing

But turning this option on and allowing .htaccess files may be a performance hit. More specifically, enabling the AllowOverride option in Apache is a performance hit. The Apache docs sums up the problem best:

“Wherever in your URL-space you allow overrides (typically .htaccess files) Apache will attempt to open .htaccess for each filename component. For example,

1 2 3 4

DocumentRoot /www/htdocs <Directory /> AllowOverride all </Directory>

and a request is made for the URI /index.html. Then Apache will attempt to open /.htaccess, /www/.htaccess, and /www/htdocs/.htaccess.”

So I disabled all .htaccess files in production, and inserted each file’s individual mod_rewrite rules into the main Apache config file. After a quick Apache Bench run, one project looked around 3% faster. Note that there are a few other useful optimizations on that page.

Let’s Start From The Beginning

Let’s back up a minute.  What is software?  Is it art?  Science?  Engineering?  Craft?  When in doubt, the dictionary is a good place to start.

engineering -noun the art or science of making practical application of the knowledge of pure sciences, as physics or chemistry, as in the construction of engines, bridges, buildings, mines, ships, and chemical plants.

Software has never felt much like engineering to me, maybe because there’s no calculus involved (usually).  There are of course parallels in resource trade-offs, input/output, etc.  But to me there is something inherently different between building a bridge, or even a stereo, and writing a program. 

Software development is still more a craft than an engineering discipline. This is primarily because of a lack of rigor in the critical processes of validating and improving a design.

- Jack Reeves, What Is Software Design?

According to Reeves, if we adjust our definition of things slightly, and see that coding is really equivalent to the design that takes place before engineering, it can start to look like other engineering disciplines.  But tools (i.e. languages) and coding and testing processes need to evolve much more before that becomes true.

art -noun the quality, production, expression, or realm, according to aesthetic principles, of what is beautiful, appealing, or of more than ordinary significance.

Software, which is so technical, as art has always appealed in an esoteric sort of way.  They say that artists will destroy the first 100 pieces that they do. In the same way, many programmers fight the urge to rewrite something they’ve written, almost immediately after it’s completed.  I know I do.  But fundamentally speaking, the problem with this analog is that while art can exist for art’s sake, software is usually more practical.  That is, even if I write something for fun, I still want it to at least do something.

science -noun a branch of knowledge or study dealing with a body of facts or truths systematically arranged and showing the operation of general laws.

If I’m not convinced of software as an applied science like engineering, I don’t see how software as science can be so.  In that sense, it falls to the same problem as art: science may exists for science’s sake – to discover new knowledge – but for software, at the end of the day, there always seems to be a customer waiting.

craft -noun an art, trade, or occupation requiring special skill, esp. manual skill.

Oh, but I like this one.  It allows us to impose the special skills that is the craft.  It leaves room for the creativity that is so essential for success – and rewarding.  It speaks to the practical requirements that usually pay the bills at the end of the day and (hopefully) make life easier for someone.

If we consider software as craft, we cannot avoid talking about masters and apprentices.  People often forget that there are apprentices around today.  It used to be that if you wanted to become an electrician, plumber, or home inspector, you must have first trained as an apprentice.  Only after many (usually unpaid) months of apprenticeship under a master could you get a license to work.  These days, there are ways around that requirements, and apprenticeship is often found only in union crews.  (And in unions it seems that politics has overtaken the original goal: to further the craft.)  But I digress.

The main problem with this is the reason there is a market for this type of qualifying: it is the law.  You technically cannot pull a permit on a home’s electrical (except your own) unless you are licensed.  You cannot get your license until you apprentice or take a test.  So would it be better or worse if there was a law requiring all software developers to get licensed before practicing their craft?  I’m certain quality would go up.  But the profession suddenly becomes less accessible.

A Rose By Any Other Name…

While we’re on the subject, what do software professionals like to be called?  Software engineers?  Developers?  Programmers?  Artisans?  Architects?  I hate using the title “software engineer” (see above).  Besides, “software artisan” is closer, but does not sound any less pretentious.  Personally, I prefer “developer” if I’m in the company of those who know what that means, and for everyone else, “programmer” will do.

I am reminded of “The human programmer” that compares programmers’ quest for a title to musicians’ and writers’.

Maybe programmers are just like the 1950s musicians that lacked … confidence, snatching desperately for public nobility. Thus far our attempts at title theft have been less successful than theirs, though not for lack of trying (e.g. software developer / architect / engineer). Like musicians, our work requires not only talent but years of practice, and we see ourselves as “different.” The average person cannot walk up to a piano, or a computer keyboard, and produce anything of value. Writers? Throw them in here too, certainly. Are reporters not a little too serious about being called “journalists”? Of course, this line of thought would annoy both of those professions, them being old, established, and respected compared to programming.

Software As…?

Of course, none of this really matters.  We still go about our jobs every day doing what we do and enjoying it.  But we forget sometimes that software as a profession is only decades old!  Compared to other professions in art, science, or engineering, software is in its infancy!  Someday I think a classification will be agreed upon.  In the meantime, I’ll throw one more definition out there.

alchemy -noun any magical power or process of transmuting a common substance, usually of little value, into a substance of great value.

I like the idea of turning nothing but keystrokes and time into something of great value to someone.  Software alchemist.  I’m ordering new business cards.

While there are a few Virtual PC hard disk images (.vhd) for Windows XP around, VMware cannot directly import .vhd files.  It needs the actual Virtual PC virtual machine file (.vmc).  After again losing my Windows XP virtual machine that I use for IE6 testing, I thought I’d document the process of running Windows XP in VMware so I don’t have to figure it out again the next time it happens. 

Note: though these instructions are for VMware Workstation, some of this may apply to the free VMware Player.

1. Download the IE6 Virtual PC Virtual Hard Disk (.vhd) image from Microsoft.
2. Download and install Virtual PC from Microsoft, if you don’t have it already.
3. Start Virtual PC.  If you have no virtual machines, you will get the New Virtual Machine Wizard.  Click Next.
   
4. Select “Use default settings to create a new virtual machine”. Click Next.
    
5. Pick a location to save your Virtual PC virtual machine.  This should be the location you will create the VMware virtual machine.  I keep all my VM’s in the same directory with meaningful names.
   
6. Click Finish to create the new virtual machine. 
   
7. If you selected “When I click Finish, open Settings,” in the previous step, you will see the settings dialog.  If you did not, select the new VM and click Settings.  Select “Virtual hard disk file:” and find the .vhd file you downloaded in step 1.  After finding it, click OK.
   
8. You should see your VM in the Virtual PC Console. 
   
9. Select your VM and click Start.  Your Windows XP virtual machine should boot in its own window.
   
10. Shut down the virtual machine using the Start button.  Then exit out of Virtual PC.  Start VMware Workstation.  Once it’s started, select “Import or Export…” from the “File” menu.  You should see the Conversion Wizard.  Click Next.
      
11. You are at Step 1 of the conversion. Click Next to select a Source Type. Under “Select the type of source you want to use:”, select Other.  Click Next.
    
12. Under “Source VM or image:”, find the Virtual PC (.vmc) file you created earlier.  Click Next.
    
13. Select “Convert all disks and maintain size.” Click Next.
    
14. You are at Step 2 of the conversion. Click Next to select a destination type.  Under “Select the destination type,” select “Other Virtual Machine.”  Click Next.
    
15. Under “Virtual machine name,” fill in a meaningful name.  Under “Location:”, find the place you want to store your virtual machine.  Click Next.
    
16. The wizard tells you that the source files are in Microsoft virtual disk (.vhd) format.  Under “How do you want to convert them?”, select “Import and convert (full-clone).”  Under “Disk Allocation,” Select “Allow virtual disk files to expand.”  Click Next.
    
17. The next step allows you to configure your VM networking.  You should probably stick to the default of 1 NIC, bridged, that connects at power on.  Click Next.
    
18. Step 3 allows for some VMware customisation.  You definitely want to install the VMware Tools.  Click Next.
    
19. You’re Virtual PC image is ready to be converted to VMware.  Click Finish to begin the conversion!
    
20. Get up from your desk and take a walk around.  Go get a cup of coffee. 
21. After the conversion is completed, you should see your new Windows XP virtual machine in VMware Workstation.
    
22. Click on “Power on this virtual machine” and your Windows XP VM should boot inside of VMware Workstation.  You can uninstall Virtual PC at this point, if you want (which is likely, since you’re running VMware).