Find your Path…

The first thing that we need to configure is the path settings for your project. Vim uses the path setting to know where to look for files in projects and setting it correctly can save you from having to remember long file names. I use the localvimrc plugin to configure settings for different projects that I’m working on and I add the path info to my .lvimrc file. Whether you use this plugin or just a global .vimrc file you simply need to add a setting line for each directory path that you would like Vim to include in searches. For example, I’m currently doing a lot of work with Symfony2. To add my src and vendor directories to my Vim path I simply need to add the following two lines:

set path+=/path/to/project/src/**
set path+=/path/to/project/vendor/**

You can of course add as many directories as you like. With that setup to search for a file you can use the :find command or any of it’s siblings. For example to find the file fooBar.php simply type “:find foo” and hit tab. Vim will offer you a choice of all files in your path that match what you’ve typed. For more info type “:h find”.

Getting gf to work with PHP

Vim’s gf command opens the file name that is under the cursor to edit. This works well ‘out of the box’ with files included or required in PHP but if you’re doing a lot of work with PHP frameworks you probably don’t have too many calls to include or require in your code. What you probably will have though is a lot of namespace declarations and use statements in your code. With a little tweaking you can make the gf command work with the namespaced classes though. Add the following two lines to your .vimrc file:

The first line replaces ‘\’ with ‘/’ whenever the gf command is using, assuming that your namespaces are PSR-0 compliant. The second makes sure that .php files are added to the list of file suffixes. With these two lines in your .vimrc, and assuming you’ve setup your path correctly, placing the cursor in a classname of a fully namespaced class and typing gf should result in the file with that class definition being opened. If you place the cursor in any other part of the namespace name and type gf you should find the directory that maps to that part of the namespace being opened in the netrw plugin. Two quick and easy adjustments to Vim config but that make day to day work much easier.

The Plan

This project is relatively new and we’d setup the following process for it:

1. We’re using Git and GitHub for the project. Every developer working on it uses a fork of the main repository.
2. When some work is ready to be merged we send a pull request. Another developer reviews the code before it is merged.
3. Once code is merged into the main repository GitHub notifies our Jenkins server, which then runs a build on the project. This performs many tasks including lint checking changed files, running our unit and functional test suite, checking that our database is in sync with our entities and performing various metrics against our code base.

What we needed is for code to be automatically deployed at the end of a build only if the build succeeds, i.e. all tests are passing, there are no syntax errors and the entities are in sync with the database. Deployments should also be able to be quickly rolled back should anything go wrong with code in production.

Enter Capifony

To automate the deployment we chose to work with Capifony. This is a deployment solution for Symfony 1 and 2 apps that is a series of deployment recipes that is built on top of Capistrano. This does entail installing Ruby and Ruby Gems on the system from which you want to run the deployment but the advantages it offers outweigh any hassle this may entail. Some of the advantages of using Capifony are:

1. Once your deployment script is defined running a deployment is as simple as changing into the root directory of your project and typing the command ‘cap deploy’.
2. Capifony stores multiple releases (you get to choose how many, the default is 3) of your project on your server. A symlink called current is created that links to the most recent release. This makes rolling back the project in the event of problems a cinch: to go back to the previous deployment simply type ‘cap deploy:rollback’ from the root directory of your project.
3. Capifony performs deployments by logging into the production server over SSH. It uses SCM such as Git to pull code down to the server.
4. Since Capifony connects to the server using SSH it can run any bash commands or symfony console commands that you wish during the deployment.
5. The deployment is run as a ‘transaction’: if any part of it fails the entire deployment is rolled back and aborted.

Capifony adds a number of default tasks that it performs with every deployment. These can easily be extended or overridden in your own deploy.rb script. Capifony can also run any symfony console command on the live server as part of the deployment by using the keyword ‘symfony’. For example, to update the Symfony2 translation file you would simply need to add ‘run “symfony:translation:update”‘ to your deploy.rb file.

Once you have installed capifony you need to cd to the root directory of your project and to run ‘capifony .’ in the terminal. This will automatically detect if you’re working wth a Symfony 1 or 2 application and will create a Capfile in in the root directory as well as creating a  deploy.rb file in the config directory. These need to be added to source control. Once you’ve setup a basic deployment script run ‘cap deploy:setup’ to setup the deployment on the remote server.

A sample deployment script

Below is a sample deploy.rb script that I’m using to deploy a Symfony2 app:

A few notes on this:

• Since Capifony performs actions on the live server it needs a user to log in as to perform actions. I created a user on the server called deploy with the primary group of www-pub and the secondary group of www-data. I then added www-pub as a secondary group to the user www-data that apache runs as. This solved any permission issues that I had.
• Setting the option “set :deploy_via, :remote_cache” tells capifony to keep a local, cached copy of the repository on the server. This speeds up deployments since only changes to the code base need to be fetched. We needed to install Git on the server to allow this to work and to add a deployment key to our GitHub repo for the deploy user to use when fetching code from Git.
• Capifony allows you to share files between deployments. This deployment script shares the parameters.yml file, the log, vendors and uploads directories.
• The line “ssh_options[:keys] = [File.join(ENV["HOME"], “.ssh”, “KEY FILE NAME”)]” tells Capifony to use an SSH keyfile with the supplied name. This allows us to register a key on the server for the deploy user.
• Since the deploy user needs to access the GitHub repository we needed to add a deployment key for the deploy user to use on GitHub.
• The line “before “symfony:cache:warmup”, “symfony:doctrine:migrations:migrate”" automatically runs any new doctrine migrations with every deployment. The line “set :interactive_mode, false” ensures that Symfony won’t ask for confirmation when running this command.
• The final part of the script overrides the restart method. This is left blank in Capifony (although I think it’s a standard task in Capistrano Ruby deployments). I needed to restart Apache with every deployment to make sure that the APC cache was cleared since a lot of cached data is stored in APC. The deploy user was given sudo permission only to restart the apache process.

Getting it working with Jenkins

We elected to have the Capifony deployment set up as a separate job on our Jenkins server rather than having it run as a post build task in our main Jenkins job for the project. The deployment job simply pulls in the latest version of the code from our GitHub repository and performs ‘cap deploy’ as a shell command for its single build action. This job is triggered as a downstream job once the main build for our project successfully completes. We chose this configuration for a couple of reasons:

• This setup gives us a little more freedom to run the deployment job on it’s own if we ever need to.
• If a build or a deployment fails it’s a little easier for is to see instantly where things have gone wrong.

We needed to add an ssh key on the Jenkins server for Jenkins to use when connecting to the live server and to add this to authorized_keys for the deploy user on the server. This setup did give us one other problem though. By default Capifony will deploy the latest commit to a Git repository. If a new commit is made while Jenkins is running a build then Capifony will deploy that, meaning that we could end up with code that has not been through our build process being deployed. Our solution was to have our main Jenkins build tag the repository and to push this tag back to GitHub using  the Git Publisher post build action. This tag is given the same number as the build number in Jenkins for easy identification. As you can see in the deployment script above our Capifony deployment looks for the latest tag in GitHub and deploys this to the live environment. To make sure there are no problems with this process we made sure that Jenkins will only build this project sequentially, not in parallel. This should ensure that the the latest tag in Git is the most recent tested version of the code. Naturally, for the deployment to work Capifony needs to be installed on the Jenkins server.

After setting all of this up I found that it wasn’t working. I could run ‘cap deploy’ from my development environment with it working perfectly, deploying the latest tag from GitHub. When the Jenkins server ran this it failed with a cryptic error about not being able to find the specified tag. I could see the tags being created in GitHub though and spent a couple of frustrating hours trying to work this out. Eventually, I found the problem. It seems that the Jenkins Git plugin creates an internal tag every time a build is run. This is only created on the Jenkins server and not pushed to GitHub. What was happening is that when the Capifony deployment was run on the Jenkins server it connected to the live server and then tried to checkout the tag that was only created on the Jenkins server, resulting in the deployment failing. The solution was to go into the advanced Git config and to make sure that the skip internal tag option is checked. The image below shows the option to check for this.

Conclusion

Setting all of this up did take me quite a lot of time, the majority of which was simply due to my needing to learn how to configure and use setup Capifony. As a result of it we have a great continuous deployment setup that works seamlessly. I’d definitely use Capifony again and am already looking at how we can use it to deploy a couple of legacy Symfony1 apps we have at work. If you have a Symfony app I’d strongly recommend using Capifony for deployments. If you have a Jenkins server, take it one step further and setup a continuous deployment.

What is Graylog2?

Graylog2 is a logging server that can aggregate log messages from multiple sources. It’s an open source project and you can configure your servers and applications to send all log messages to Graylog for later analysis. Graylog stores them in an Elastic Search database and allows you to perform custom searches on them. You can also see in real time how many log messages are being received and processed. The real killer feature in Graylog is streams. A stream in Graylog is a predefined custom search. The server will display to you log messages that only match the stream criteria selected, graphing the results for you. You can also attach alarms to streams so that if the number of messages recorded for a stream exceeds a certain level over a period of time an email alert can be sent out.

Messages can be sent to Graylog via UDP or AMQP and there are libraries for many languages allowing you to send messages. UDP has the advantage of being quick in that the application does not need to wait for confirmation in the way that it would with TCP. However AMQP allows for log messages to be saved in the event that the Graylog server ever goes down. You can configure your servers to send messages to Graylog via Rsyslog so that even system logs can be aggregated in one place. Getting the server installed and running can be a little time consuming as it uses MongoDb and ElasticSearch for storage, Java for the Graylog server that accepts and stores incoming messages and Ruby on Rails for the web frontend. The best guide to getting all of this going in my opinion can be found at Getting Started with Graylog2 for Logging. I would add two ‘pro tips’ to this from my experience:

1. If you’re using an Amazon EC2 instance to run your Graylog server don’t forget to open ports 514 and 12201 for UDP in the security configuration. Opening 514 allows your Graylog server to receive Rsyslog messages and 12201 is the default port that Graylog receives messages in Gelf (Graylog Extended Log Format) on. If you don’t make sure these ports are open you’ll spend hours like me wondering why your server isn’t receiving messages…
2. Double check that the disk that ElasticSearch is saving data to has plenty of space. Graylog uses ElasticSearch to store log messages and depending on the volume of messages you’re handling this can mount up pretty quickly. You can configure how long messages are kept for (the default is 30 days) however.

Obviously, setting up and maintaining a Graylog server only makes sense if you have multiple servers to manage. Assuming that you do it can be a real godsend. There are libraries in many common languages for sending messages to a Graylog server and the Graylog project maintains a test server for you to try sending messages to (link on their website).

So, you now have a working Graylog server: how do you send messages to it from PHP?

Introducing Monolog

Monolog is a logging library for PHP 5.3 + is well worth checking out. It supports the 8 official logging levels defined in RFC 5424 and allows you to log messages to multiple formats. A Monolog logger has at least one handler attached to it, and handlers are used to process log messages. A handler takes care of recording log messages to different backends such as streams (files), emails or Graylog. When attaching a handler you also specify the log level that handler records, allowing you to specify multiple handlers, each recording a different log level. Handlers come with appropriate message formatters but this can be overridden, allowing you to send emails with JSON embedded data if you wish. Logging a message is as simple as calling a method with a string as the first argument and an optional associative array of extra ‘context’ data as a second argument. Monolog will take care of finding a handler that can handle the message and error level being logged.

Thanks to Matt Lehner Monolog has a handler for Graylog. To log messages to a Graylog server you’ll also need the Gelf-PHP package available from Packagist. Assuming a composer installation of Monolog and Gelf-PHP the following code will write test warning, error, info and debug log messages to your Graylog server on the default Gelf port of 12201:

Sending Logging Messages from Applications

So, how would you go about sending log messages from your applications to Graylog? Symfony2 uses Monolog by default for logging and it’s relatively simple to add the Graylog handler to your application config. This Gist gives information on how to do that. Once it’s all configured sending a message to Graylog is as simple as retrieving the logging service from the service container and adding a message.

It’s a little (but not much) harder with applications where a DIC is not available. We have an older application at work that uses Symfony 1.4 that I wanted to add logging for. To do that I ended up adding a class that returns a singleton (evil, I know) instance of Monolog, fully configured to send all messages to our Gelf server. The code for that looks something like this:

This code also attaches a processor to the Monolog instance that automatically collects all data from $_GET, $_POST, $_COOKIE and $_SERVER and adds it to the context of a log message, helping us to debug problems more easily.

Logging all PHP Errors to Graylog

Sending error messages from your application code is likely to only be half of the story though: what about errors from the PHP interpreter itself? We have some legacy code that I work with that produces a lot of errors and warnings and I wanted these recorded in Graylog too to give us a better chance of tracking them down. Luckily PHP allows you to register error handlers, exception handlers and shutdown functions. Using a combination of these I was able to capture all PHP errors and log them to Graylog. To do this I used the logging helper class from above and the following code:

This code is contained in a ‘logging.php’ file that is simply included in our index.php file for our application.

Don’t let the Gorillas put you off…

I really like Graylog2, despite the abundance of pink and Gorillas in the UI design. It’s a very useful application and is something that we’re going to be making increasing use of at work over the next year or two.

The Problem

In collecting data for the application we are getting a lot of date time information that can be entered in multiple timezones. When the information is retrieved we allow the user to select the timezone that they want to view the data in. We quickly decided that the only way to go was to store all date time information in the UTC timezone but we still needed a way to seamlessly convert date time data into that timezone when it is saved. The project is being built using a combination of Symfony2 and Doctrine2. We discovered that Doctrine will always populate DateTime objects for entities with a timezone that matches the default timezone set on the server when entities are hydrated. Solving half of the problem just involved making sure that UTC was set as the default server timezone. The advice of the Doctrine project is to convert instances of DateTime to use UTC before they are persisted in the database and to store the timezone information separately and we wanted to find a way to do this that was as simple and maintainable as possible.

Potential Solutions

The first solution we considered was to perform the timezone conversions at the entity level but we quickly abandoned this idea for a number of reasons. Firstly, this would involve a huge amount of code duplication. Every setter in every entity that dealt with DateTime objects would need to perform a conversion on DateTime instances that are saved. This would either involve lot of duplicated code or some complicated inheritance structure. Worse, if a developer created an entity that didn’t implement this behavior we could quickly find that we had corrupted data in the database.

The second solution we considered was an extension of the first. Doctrine allows you to define a series of lifecycle callbacks on entities that can be fired when certain events take place. We thought about adding PrePersist and PreUpdate callbacks that would perform the timezone conversion automatically whenever an entity is saved or updated by the entity manager. However this solution suffers from the same problems as the one above in that it requires code duplication and/or inheritance. It would also mean that developers would need to remember to add this behavior to all entities that need to work with DateTimes, leading to problems if this is forgotten.

The next solution we considered was possibly the worst. We thought about enforcing the timezone changes at the application level, before the entities were ever given to the entity manager. This was very quickly discarded due to the obvious code bloat and possibilities for duplication that exist.

The Solution

The solution we hit on was to leverage Doctrine’s system of event listeners to help us do the work. Doctrine allows you to register listeners with the entity manager that are called whenever certain events occur. We created an event listener that is triggered on the onFlush event. This collects all entities that are scheduled for insertion or update and uses Reflection to iterate over their private and protected properties looking for DateTime objects. Any that are found have their timezones set to UTC. The code for the event listener class is below:

<?php
namespace Events\Doctrine;

use ReflectionObject;
use ReflectionProperty;
use DateTime;
use DateTimeZone;
use Doctrine\ORM\Event\OnFlushEventArgs;
/**
* Class to provide an event listener that processes entities when they are persisted or updated
*/
class DateTimeListener
{
/**
* Method to listen to the onFlush event
* When this event is triggered the method will look at all entities
* scheduled for update or insertion.
* It will retrieve all DateTime instance from these and ensure they are
* set to UTC timezone
* @param Doctrine\ORM\EventListener\OnFlushEventArgs $args
*
* @return void
*/
public function onFlush(OnFlushEventArgs $args)
{
    $zone = new DateTimeZone('UTC');
    //Get hold of the unit of work.
    $entityManager = $args->getEntityManager();
    $unitOfWork = $entityManager->getUnitOfWork();
    //Get hold of the entities that are scheduled for insertion or update
    $entities = array_merge($unitOfWork->getScheduledEntityInsertions(), $unitOfWork->getScheduledEntityUpdates());
    foreach ($entities as $entity) {
        $reflect = new ReflectionObject($entity);
        foreach ($reflect->getProperties(ReflectionProperty::IS_PRIVATE | ReflectionProperty::IS_PROTECTED) as $prop) {
            $prop->setAccessible(true);
            $value = $prop->getValue($entity);
            if (! $value instanceof DateTime || $value->getTimezone()->getName() === 'UTC') {
                $prop->setAccessible(false);
                continue;
            }
            $value->setTimezone($zone);
            $prop->setValue($entity, $value);
            $prop->setAccessible(false);
            $unitOfWork->recomputeSingleEntityChangeSet($entityManager->getClassMetadata(get_class($entity)), $entity);
        }
    }
}
}

What I like about this solution is that it doesn’t require any intervention on the part of the developer higher up in the application. Entities are simply created or edited and no special attention needs to be paid to time zones saved in DateTime instances. However, this solution isn’t without it’s own issues. Firstly, the fact that this is done by the entity manager at the event level gives the whole thing a touch of ‘magic’. We’ll need to make sure this is well documented so that developers working on this code in the future don’t end up spending time trying to work out how this behavior is happening. Secondly, using Reflection and iterating over every property of every entity that is being saved or updated isn’t the fastest thing in the world to do. At this stage I’m not too concerned as, like most systems, there will be many more read than write operations in this application. If this does become a problem we could possibly introduce some sort of class map that we could use to decide which entities have DateTime instances in them that we might need to modify. It’s possible that the class meta data, held by the entity manager, would be able to give us this information. For now I’m quite happy with this solution and with the event system in Doctrine in general. We’ve also used Doctrine events to automatically calculate some statistics data for us when entities are flushed but that’s a topic for another post.

Iterating over Objects

PHP natively allows us to iterate over objects in a foreach loop. However, this only works with public properties of the object. If your object has public properties these will be the values provided in each iteration. If you’re following the concept of encapsulation in your objects though you’ll probably be using private or protected properties with getters and setters. Obviously this presents a problem if you want to iterate through the properties of the object. This is where IteratorAggregate and Iterator come in. If you implement one of these interfaces in your class you can define your own logic which governs the values to return when an object is iterated over.

On Being Traversable

Before we have a look at implementing IteratorAggregate or Iterator I should briefly mention the Traversable interface. Traversable is a bit of an oddity. The PHP manual defines it as an ‘abstract base interface that cannot be implemented alone. Instead it must be implemented by either IteratorAggregate or Iterator’. The manual goes on to explain that internal classes can implement Traversable and these can be used in a foreach construct. If Traversable is an interface that you can’t implement yourself then what good is it? You can use it in type hinting or with the instanceof operator to detect if an object can be used in a foreach construct. If you simply need to find out if you can use an object in a foreach loop and are not too worries about the exact type of the object checking if Traversable is implemented is a quick and easy solution.

Implementing IteratorAggregate

Implementing IteratorAggreagate is extremely easy. You only need to implement one method, getIterator. This method has to return a data structure that can be iterated over and the PHP engine automatically calls this method when it detects an object implementing this interface being used in a foreach loop. Of course it’s up to you whether this is an array or an object that implements the Traversable interface. A simple, contrived example of implementing this interface is as follows:

class Foo implements IteratorAggregate

{

protected $values = array();

//various methods here to work with the object and set values in $values

public function getIterator()

{

return $this->values;

}

}

IteratorAggregate is perfect for occasions when your object holds an array or Traversable object of values and all you want to do is to iterate over these. If you need to do anything more complicated, such as altering or examining values before they’re returned, you need to use it’s big brother, Iterator.

Implementing Iterator

The iterator interface is a bit more complicated that IteratorAggregate and requires that you implement five methods: current, key, next, rewind and valid. Current should return whatever the current value should be in the iteration while key should return a key value for it. Next is used to advance the Iterator forwards by one step while rewind should reset whatever pointer is being used to determine the iteration. Finally, valid should return a boolean value to indicate if a value exists for the current iteration. This seems like a lot but the PHP engine takes care of calling these methods for you at the appropriate times in a foreach loop. In most cases you’ll also find that next, rewind and valid can be pretty minimal methods. A (once again contrived) example of using Iterator is below.

class Bar implements Iterator

{

protected $values;

protected $counter = 0;

//Various methods to set data and to work with the object

public function rewind()

{

$this->counter = 0;

}

public function next()

{

$this->counter++;

}

public function valid()

{

return array_key_exists($this->counter, $this->values);

}

public function current()

{

return $this->values[$this->counter];

}

public function key()

{

return $this->counter;

}

}

This example is obviously very simple and holds no real advantages over IteratorAggregate. The true advantage of Iterator over IteratorAggregate is that it allows you to manipulate the data in key and current before it is returned. One place that I’ve done this a lot recently is in working with the SalesNet CRM API. The API returns results as XML which I process using XPath. The result of this is an array of SimpleXML elements that I can iterate over. By using Iterator for this instead of IteratorAggregate I can transform the SimpleXMLElement objects into scalar values in the current method before they’re returned. I can also have the key method return account or deal id’s for the current result. This would not be possible with IteratorAggregate. For me this was an ideal way of working with the results of a web service call in a view layer. I would usually implement the countable interface too which would be used to return the number of results held by the object. A simple call to count() would check this, followed by iteration over the object and display of the results.

I’ll post another article in this series in the next few weeks looking at using Countable, ArrayAccess and Iterator/IteratorAggregate all together in one object. I’ll also take a brief spin through the ArrayObject class and when you might want to use it.

Solving this problem in PHPUnit

This problem isn’t exclusive to unit testing with Doctrine. Mike Lively, the author of PHPUnit’s DbUnit extension, has provided a solution in a Gist. Basically, this extends the PHPUnit_Extensions_Database_Operation_Truncate class to suspend MySQL foreign key checks for the length of the truncate operation. The getSetUpOperation() method then needs to be overridden in the test class to use the custom MySQL truncate class. All in all a neat and elegant solution.

Porting this Solution to Doctrine

Porting this solution turned out to be fairly simple, with the exception of one issue which needed to be addressed. I started with the custom MySQL truncate class, which is almost an exact copy of the one Mike Lively wrote:

<?php

namespace Tests\Models\Db;

use PHPUnit_Extensions_Database_Operation_Truncate,
 PHPUnit_Extensions_Database_DB_IDatabaseConnection,
 PHPUnit_Extensions_Database_DataSet_IDataSet;
/**
 * MySQL truncate class that allows foreign key checks to be suspended for the duration of the truncate operation.
 * @author Mike Lively
 * @author Jeremy Cook
 * @version 1.0
 * @package Veridis
 * @see https://gist.github.com/1319731
 */
class MySQLTruncate extends PHPUnit_Extensions_Database_Operation_Truncate
{
 /**
 * Executes the truncate.
 * Note: the $connection variable here is an instance of DoctrineExtensions\PHPUnit\TestConnection
 * (non-PHPdoc)
 * @see PHPUnit_Extensions_Database_Operation_Truncate::execute()
 */
 public function execute(PHPUnit_Extensions_Database_DB_IDatabaseConnection $connection, PHPUnit_Extensions_Database_DataSet_IDataSet $dataSet)
 {
 $connection->getConnection()->exec("SET @PREVIOUS_foreign_key_checks = @@foreign_key_checks");
 $connection->getConnection()->exec("SET @@foreign_key_checks = 0");
 parent::execute($connection, $dataSet);
 $connection->getConnection()->exec("SET @@foreign_key_checks = @PREVIOUS_foreign_key_checks");
 }

}

This is pretty self explanatory and simply disables MySQL’s foreign key checks while the database tables are truncated and populated with data. I then created an abstract base class which extends the OrmTestCase class to set up the entity manager. This can then be extended by any class that needs to test against the database with an entity manager. To this class I added the code to register the custom MySQL truncate class, and a simplified version of the class looks like this:

<?php

use DoctrineExtensions\PHPUnit\OrmTestCase,
 Doctrine\ORM\Configuration,
 Doctrine\Common\Cache\ArrayCache,
 Doctrine\ORM\EntityManager,
 PHPUnit_Extensions_Database_Operation_Composite,
 PHPUnit_Extensions_Database_Operation_Factory;

/**
 * Base class for test cases that need an entity manager to work with.
 * Defines an entity manager for the tests to work with.
 * @author Jeremy Cook
 * @version 1.0
 */
 abstract class Base extends OrmTestCase
 {
 /**
 * Method to create the entity manager, used by parent methods.
 * @return \Doctrine\ORM\EntityManager
 * @see OrmTestCase::createEntityManager()
 */
 protected function createEntityManager ()
 {
 $config = new Configuration;
 $cache = new ArrayCache();
 //Set up config here.
 $conn = array();
 return EntityManager::create($conn, $config);
 }
 /**
 * Overrides the parent method to add a custom MySQL truncate operation.
 * This suspends foreign key checks for the duration of the truncate command.
 * (non-PHPdoc)
 * @see DoctrineExtensions\PHPUnit\DatabaseTestCase::getSetUpOperation()
 */
 public function getSetUpOperation()
 {
 $truncate = new MySQLTruncate();
 return new PHPUnit_Extensions_Database_Operation_Composite(array(
 $truncate,
 PHPUnit_Extensions_Database_Operation_Factory::INSERT()
 ));
 }
 }

The createEntityManager method is defined as abstract in OrmTestCase and needs to return an instance of Doctrine\ORM\EntityManager. The getSetUpOperation method is defined in PHPUnit_Extension_Database_TestCase and is overriden here to add the custom MySQL truncate class. There’s one final abstract method that needs to be implemented in a test class and that’s getDataSet. This is defined in PHPUnit_Extension_Database_TestCase and needs to return a PHPUnit_Extensions_Database_DataSet_IDataSet instance to populate the database with data. With this I created a test case extending this class, added data to populate the database, ran it and got an exception. What was going on?

The Last Problem

I traced the exception down to DoctrineExtensions\PHPUnit\TestConnection::getTruncateCommand(). This method was hard coded to throw an exception when called. I spoke to Benjamin Beberlei on IRC to ask him about this and he couldn’t remember why this code was added here. I made an edit so that the getTruncateCommand method looks as follows:

<?php

/**

* Returns the command used to truncate a table.
*
* @return string
*/
public function getTruncateCommand()
{
 return 'TRUNCATE';
}

With this everything seems to work. Doctrine’s PHPUnit test extension is able to truncate tables in a MySQL database that have foreign key constraints.

What is the ArrayAccess Interface?

ArrayAccess allows you to treat an object that implements it as if it is an array for the purposes of setting, unsetting and retrieving data from it. Please note the emphasis in the last sentence! ArrayAccess does not make an object behave like an array in any other way. If you pass an object that implements ArrayAccess to a PHP array function such as in_array() you’ll still get an error. This will become a little clearer with some of the examples below.

To implement ArrayAccess you need to implement four methods: offsetExists, offsetGet, offsetSet and offsetUnset. ArrayAccess::offsetExists must return a boolean, offsetGet can return any valid PHP type while offsetSet and offsetUnset should not return any value. Once you implement these methods you can treat an object as if it is an array for the purposes of saving and retrieving properties. Here’s a brief example.

  1: class Foo implements ArrayAccess {

  2:

  3: 	protected $_values = array();

  4:

  5: 	public function offsetExists($offset)

  6: 	{

  7: 		return array_key_exists($offset, $this->_values);

  8: 	}

  9:

 10: 	public function offsetGet($offset)

 11: 	{

 12: 		return $this->offsetExists($offset) ? $this->_values[$offset]:NULL;

 13: 	}

 14:

 15: 	public function offsetSet($offset, $value)

 16: 	{

 17: 		$this->_values[$offset] = $value;

 18: 	}

 19:

 20: 	public function offsetUnset($offset)

 21: 	{

 22: 		if ($this->offsetExists($offset)

 23: 			unset($this->_values[$offset]);

 24: 	}

 25: }

 26:

 27: $bar = new Foo();

 28: $baz = isset($bar['baz']); //Calls offsetExists and returns false

 29: $bar['baz'] = 1; //Calls offsetSet and adds a new member to the _values array with the key of 'baz' and the value of 1

 30: $baz = isset($bar['baz']); //true

 31: echo $bar['baz']; //Calls offsetGet and prints 1.

 32: unset($bar['baz']);//Calls offsetUnset and removes the value from the internal _values array

Hopefully this should all be fairly self-explanatory and you can see how ArrayAccess can allow you to treat an object like an array. Once again, note that this only allows you to use the array notation to set, unset, retrieve and find out if values are set in the internal _values property. If you want to count the number of values set you’d need to implement Countable and if you wanted to be able to iterate over the object you’d need to implement Iterator or IteratorAggregate. Even then the object could not be passed to PHP array functions.

The above example is also incredibly simplistic. There’s nothing to say that the internal storage for the properties being set has to be an array. Likewise, the logic in your implementation of the ArrayAccess methods can be as simple or as complicated as you need.

Hang on a minute…

In reality the only thing that ArrayAccess gives you that can’t be achieved using PHP’s magic methods is the ability to use array notation. Apart from that there’s nothing here that can’t be achieved with the magic __get(), __set(), __isset() and __unset() methods. So, the question here has to be why bother with ArrayAccess at all when it only provides syntactic sugar? This is a hard one to answer and really comes down to a matter of personal preference. I personally quite like the syntax ArrayAccess gives, especially when coupled with the Countable and Iterator interfaces. For me as well it also gives me a different mindset. I’m used to my IDE providing me with code completion, which of course doesn’t work with magic methods. I always find it a little unnerving when the code completion doesn’t appear, even when I know that it’s not working because of PHP’s ‘magic’. This isn’t a problem with the array syntax that ArrayAccess provides. Of course this is a very subjective viewpoint and those of you reading this will I’m sure have or develop your own opinions on this.

A practical example

One place where ArrayAccess is very useful is in an object where you have a large number of arbitrary values that you need to store and retrieve. I’ve recently been doing a lot of work with the SalesNet CRM system for a client through their SOAP API. The SalesNet API exposes a massive amount of functionality and a lot of the methods can accept a large number of fields to set data for or retrieve data from. SalesNet also allows it’s customers to create custom defined fields and these can both be set and retrieved through the API.

Since there are so many possible values that can be set and retrieved this is a perfect case for using ArrayAccess. I simple pass the name of the SalesNet field that I’m setting or retrieving data from as the ‘array key’. If I’m storing the data in the object to be sent to SalesNet I can retrieve the key and value pairs and process them as needed before making the SOAP call.

Things can get a little more interesting when retrieving data from SalesNet. The response payload is a large lump of XML in most cases. I’ve often resorted to processing this with XPath to get to the nodes I interested in and storing these as an internal property of the object. I can then use offsetGet and offsetExists to work with the data, usually transforming it in some way before returning it. I also often implement the Countable interface to return the number of results. Consider the following (simplified) example:

  1: class SalesNetDemo implements ArrayAccess, Countable

  2: {

  3: 	//Array of SimpleXML objects following processing of the response.

  4: 	protected $_data = array();

  5:

  6: 	protected $_soap;

  7:

  8: 	public function __construct($soap)

  9: 	{

 10: 		$this->_soap = $soap;

 11: 	}

 12:

 13: 	public function doSoapCall()

 14: 	{

 15: 		//Code to do a Soap call and process the results here using XPath.

 16: 	}

 17:

 18: 	public function count()

 19: 	{

 20: 		return count($this->_data);

 21: 	}

 22:

 23: 	public function offsetExists($offset)

 24: 	{

 25: 		return (isset($this->_data->$offset) &&

                 '' !== (string) $this->_data->$offset);

 26: 	}

 27:

 28: 	public function offsetGet($offset)

 29: 	{

 30: 		return (isset($this->_data->$offset) &&

                 '' !== (string) $this->_data->$offset) ? (string) $this->_data->$offset : NULL;

 31: 	}

 32:

 33:

 34:     	public function offsetSet ($offset, $value)

 35:     	{

 36:         	throw new BadMethodCallException(sprintf(

 37:             		'Values cannot be changed in this class, thrown in %s',

 38:             		__METHOD__

 39:         	));

 40:     	}

 41:

 42:     	public function offsetUnset ($offset)

 43:     	{

 44:         	throw new BadMethodCallException(sprintf(

 45:             		'Values cannot be changed in this class, thrown in %s',

 46:             		__METHOD__

 47:         	));

 48:     	}

 49: }

 50:

 51: $foo = new SalesNetDemo(new SalesNetSoapObject);

 52: $foo->doSoapCall();

 53: if(count($foo)) {

 54: 	//Results retrieved from the API.

 55: 	//Try to get a valueand echo it.

 56: 	echo $foo['CDF12345'];

 57: }

Again, this is a pretty simple example. The object is only designed to allow data to be retrieved and read, which is why offsetSet and offsetUnset throw exceptions. A real world example might allow the same object to amend the data through the API. The real point of the example though is to show you how the data can be transformed before returning it from the object. Again, the example is deliberately simplistic but you can get the idea of the kinds of things that can be achieved. This is also an example as to how multiple predefined interfaces can be combined to achieve some pretty powerful results.

I hope you’ve enjoyed this brief, and simple, look at ArrayAccess. Let me know in your comments how you you’ve used ArrayAccess in your own code. I’ll add another post as soon as I have time looking at the Iterator and IteratorAggregate interfaces and I’ll also try to add an example that implements Countable, ArrayAccess and Iterator to show how they can work together.

The Countable Interface

The Countable interface at it’s simplest level allows you to pass objects that implement it to PHP’s native count function. The interface is, on the surface, extremely easy to implement. You only need to add the Countable::count() method to implement the interface in your class and this method only needs to return an integer. If you pass an object that implements Countable to the PHP count function the PHP interpreter will automatically call the count method in the object, returning the value from it to you to use.

Why is Countable so Useful?

Like all of the predefined interfaces and SPL functionality the real power of countable comes from the ability to effectively override and re-implement core PHP functionality with your own logic. I’ll give three brief examples here of ways that I’ve personally used Countable recently.

Accessing the State of a Private or Protected Property

One of the simplest uses of countable is to return the count of an array that’s held as a protected or private property of an object. Take the following example:

class SomeClass implements Countable {

 protected $_data = array();

 public function doSomething()
 {
//Code to change $_data here
 }

 public function count()
 {
return count($this->_data);
 }
}

In this example some data is stored in the object as an array and the Countable:Count method simply returns the count of that array. While this is obviously a very simple example it can be very powerful. I’ve been doing a lot of work with a web service in the last few weeks and have used code very much like this. After a web service call the resulting XML is processed using XPath and the resulting array is stored in a protected property. The count method would then return the number of results of the web services call. Using count in this way obviously makes a lot of sense when coupled with implementing Iterator or ArrayAccess to allow access to the data held in the protected property.

Using Countable to Signal the State of an Object

Countable can also be used to signal the state of an object. Imagine the above code, amended slightly to this:

class SomeOtherClass implements Countable {

protected $_data;

public function doSomething()
{
//Code to change $_data here
}

public function count()
{
return is_null($this->_data) ? 0 : 1;
}
}

In this case count will only ever return 0 or 1, depending on whether the protected property $_data has a value set in it or not. Using this code could look something like this:

$obj = new SomeOtherClass();

$obj->doSomething();

if (count($obj)) {

//$_data is not null, proceed accordingly.

}

I’ve also used this recently with a web service call. In this case the web service call was expected to return a single value, which is stored in $_data. Implementing countable like this simply signifies to the calling code if a value has been successfully retrieved or not.

Using Countable with a Database Table

Another interesting use of Countable is with a class that maps to a database table in some way. Imagine a case where you have a database table where values are only considered ‘live’ if some condition is met. Implementing countable on an object that encapsulates the db table can then be used to flag if the table contains any ‘live’ values. The following example uses Zend_Db_Table:

class AnotherClass extends Zend_Db_Table implements Countable {

 public function count()
 {
$select = $this->select(Zend_Db_Table::SELECT_WITH_FROM_PART)
->from('some_table', array('Count' => 'COUNT(*)'))
->where('SomeCondition = SomeValue');
$row = $this->fetchRow($select);
return (int) $row->Count;
 }
}

In your calling code you would then simply do this:

$obj = new AnotherClass();

if (count($obj)) {

//Table has 'live' values. Do something here

}

This can be a very useful way of working with a database table.

A Word of Caution…

The only limit to the way Countable can be used is the imagination of the developer, and this is also a danger. All developers are familiar with using the count() function to get hold of the number of values in an array. When implementing Countable the further you move the count() method away from this functionality the less intuitive the code becomes and the greater the chance that the developer using it might make a mistake. This is true of all of the predefined interfaces however. While they provide great power the further away from core PHP functionality you get while implementing them in your own code the less intuitive they become. However, don’t let this stop you! The next time you go to write a method called something like ‘getItemCount()’ in a class don’t do it! Implement the Countable interface instead and start hooking into the power of the core PHP engine.

API Format

The SalesNet API is SOAP based and consists of 8 separate endpoints, each of which allows you to operate on different parts of a companies data such as deals, accounts, contacts, etc. Before you can use any of these services it’s necessary to authenticate against a separate security endpoint. This returns a token which is valid for between 1-12 hours (depending on the account setup in SalesNet). This token must then be set in a SOAP header for use in requests to any of the main SalesNet endpoints. Each of the API methods accepts many arguments to control precisely the data that is returned and the only thing I can suggest to anyone looking to use the API is to refer to the documentation. One of the quirks of the API is that data is returned as XML, embedded in a stdClass object. For example, a call to the GetDeals method of the deals endpoint returns a stdClass object (say $obj), and to access the response data the following would be needed:

$response = $obj->GetDealsResult->any;

In the above example $response would now include the XML data ‘payload’, which can then be parsed. It’s often not that easy to parse the XML returned as it seems to be an XML representation of a Microsoft ADODB recordset, which in some cases is not even properly formed XML. I’ve had luck using SimpleXML and Xpath to extract the nodes I’m after however.

The Code

To work with the various SalesNet API’s I created two classes, one of which handles authentication and the other of which acts as a proxy to catch and pass SOAP calls through to the API. Both of the classes make use of ‘lazy loading’ and will only connect to SalesNet as and when a method is called that requires a SOAP call.

The Authentication Class

My goal in the authentication class was to simplify authenticating against the SalesNet security endpoint. The object accepts a company name, username and password and will retrieve a security token from SalesNet. This is then set in a PHP SoapHeader object and is returned for use whenever the doLogin() method is called. Internally the class will keep track of when the token expires and will automatically fetch a new token from SalesNet as and when required. My thought is that an object of this class can be serialized and cached to be stored and shared between many requests as a security token can be valid for up to 12 hours.

The SoapProxy Class

The SoapProxy class provides a proxy to trap calls to the many SalesNet API methods. Since there are so many methods cross so many endpoints it would be impossible to map these out to individual methods so I chose to use __call() instead to pass calls through to a PHP SoapClient object. The class stores the URL endpoints of the various services as constants and an API endpoint can be set when an object is instantiated or before any method is called. The object also accepts an instance of the Authentication class and uses this internally to set the security SoapHeader for API calls.

A Brief Example

So, enough talk! As I said earlier full code can be found on GitHub but here’s a brief example of how to use the classes to call the GetDeals method of the Deals endpoint.

<?php
use WebServices\Soap\SalesNet\Authentication, WebServices\Soap\SalesNet\SoapProxy;

try {
    $auth = new Authentication();
    $auth->setCompanyLogin('COMPANY LOGIN HERE')
         ->setUserName('USERNAME HERE')
         ->setPassword('PASSWORD HERE');

    $soap = new SoapProxy(SoapProxy::DEALS, $auth);
    $args = array(
        //Set up method arguments here
    );
    $response = $soap->GetDeals($args);
} catch (BadMethodCallException $e) {
    /**
     * Errors will be caught here.
     * The BadMethodCallException can have a previous exception of a SoapFault if there is one.
     */
}

That’s all there is to it. To use other endpoints simply change the constant passed to the SoapProxy constructor and call another Soap method, with the correct arguments of course. Let me know of any comments on this code or improvements people can see. Enhancements and fixes on GitHub are welcome.

Zend_Validate includes two classes to work with database columns: Zend_Validate_Db_NoRecordExists and  Zend_Validate_Db_RecordExists. As their names suggest these check that a value either does or doesn’t exist in a specified column in a database table. The classes work really well in a form when specified as a validator for a value but things can get a little trickier if you need to check two columns in a table. The problem I had was in creating a simple FAQ page for a site. The site owner gets to add FAQs to the site and each FAQ is contained in a category. As a little sanity check I wanted to make sure that the combination of category and question was unique. The database also has a multiple column unique key on these to enforce the integrity. My first attempt looked something like this in a class that extends Zend_Form:

public function init() {
//Category options are populated elsewhere.
$categories = new Zend_Form_Element_Select('category');
$categories->setAllowEmpty(FALSE)
	   ->setAttrib('required', 'required')
	   ->setLabel('FAQ Category:')
	   ->addErrorMessage('Please choose a valid category from the list.')
	   ->addMultiOption('', '--Please Choose--');
$question = new Zend_Form_Element_Text('question');
$question->setRequired(TRUE)
	 ->setLabel('Question:')
	 ->setAttrib('required', 'required')
	 ->setAttrib('max', '100')
	 ->addValidator('StringLength', FALSE, array(0,100))
	 ->addErrorMessage('Enter a question of up to 100 characters in length.');
$validate = new Zend_Validate_Db_NoRecordExists(array('table' => 'faqs', 'field' => 'title'));
$validate->getSelect()->where('FaqCategoryId = ?', $this->getValue('category'), Zend_Db::PARAM_INT);
$question->addValidator($validate);
}

I thought that this would add the extra where clause to the select statement for the validation. This failed miserably though. The problem is that the value referenced in ‘$this->getValue(‘category’)’  doesn’t exist when the init() method is called. Values are only added to the form when the isValid() method is called. The solution was to extend the isValid() method and to perform the extra check there.

public function isValid($data) {
	$result = parent::isValid($data);
	if ($result) {
		$validate = new Zend_Validate_Db_NoRecordExists(array('table' => 'faqs', 'field' => 'question'));
		//Add a where clause to check the FaqCategoryId
	        $validate->getSelect()->where('FaqCategoryId = ?', $this->getValue('category'), Zend_Db::PARAM_INT);
	       	$result = $validate->isValid($this->getValue('question'));
	       	if (! $result)
	       		$this->getElement('question')->setErrors(array('The selected category already has an FAQ with that question. Please edit the question.'));
		}
		return $result;
	}

By extending the isValid() method I can call the parent method to do the majority of the work. If the form looks valid I then perform the extra check to make sure the combination of question and category is unique. If it isn’t an error message is set and the validation fails. I’ve also used this in another part of the site where the client is adding news articles to the site. In that instance the combination of published date and title needs to be unique as these are used in the url to uniquely reference the article. The same method in form validation checks that when the form is submitted. Using this method I’ve been able to clean up my controller and move all validation checks into the form, where they should be. I’d love to hear in the comments how other people approach this problem though.