Tim Fountain: Blog

Deploying Zend Framework apps with Capistrano

2009-05-11T21:03:54+01:00

One of the good things the Ruby community has brought us is Capistrano, a command line tool for automated deployment. Although it was written for Rails apps, it can be used with other languages, as proven by a blog post from a few years ago detailing how to use it to deploy PHP code. I thought I'd update this to show how it can be used to deploy Zend Framework applications.

If you're not familiar with Capistrano, once it's all setup you'll be able to run:

cap production deploy

from your ZF project folder. This will automatically login to your remote server, checkout a copy of your code, run any custom tasks (e.g. setting permissions), and then switch the live site to point at the new release. If anything goes wrong with any of these steps, the deployment will stop, the new checkout will be removed, and your site will remain exactly as it is. This allows you to very easily deploy code updates and new versions with basically no downtime.

I'm going to assume you already have Capistrano installed (if not, there are plenty of guides around for this). You'll also need the capistrano-ext gem for the multistage stuff I'm going to mention later. For reference, at the time of writing I have versions capistrano 2.5.5 and capistrano-ext 1.2.1 of these gems.

Capistrano has a command capify which creates the files it requires, however this doesn't work with ZF applications because the file structure is slightly different. So we're going to create these files manually. If you're using the recommended ZF application structure (i.e. you have an application/configs folder), I've packaged up my examples below into an archive you can just extract into the root folder of your ZF project, but read on anyway so you know what the files do.

The first of these is Capfile, which needs to live in the root directory of your ZF project. This should look something like this:

load 'deploy' if respond_to?(:namespace) # cap2 differentiator
load 'application/configs/deploy'

this file simply loads in your main deployment configuration which in the above example will be application/configs/deploy.rb. If you aren't using the standard ZF app structure, adjust this path to point at wherever you want your deployment configuration to live.

The next file is deploy.rb, which is where most of the deployment settings live and is the file referenced above. Create this file in your application/configs folder. Here's an example:

# general settings
default_run_options[:pty] = true
set :use_sudo, false

# source control settings
set :scm, :git
set :deploy_via, :remote_cache
set :repository, "ssh://git@example.com/yourapp.git"

# stages
set :stages, %w(staging production)
set :stage_dir, "application/configs/deploy"
require 'capistrano/ext/multistage' 


namespace :deploy do

  task :migrate do
    # overrides the standard Rails database migrations task
  end

  task :start, :roles => :app do

  end

  task :stop, :roles => :app do

  end

  task :restart, :roles => :app do
    # no restart required for Apache/mod_php
  end 

end

Adjust the source control section to contain details for the repository your application lives in. The example above shows typical settings for a git repo (replace git@example.com/yourapp.git with your clone URL). If you're using Subversion, try something like this:

set :scm, :subversion
set :repository, "svn+ssh://example.com/yourapp/trunk"
set :scm_username, "your_svn_username"
set :scm_password, "your_svn_password"
set :deploy_via, :export

The stages section allows you to define a number of different environments for your application. In my example I'm defining two stages - production and staging. You then need a file for each which contains the appropriate login details. :stages_dir defines where these files live, in my example it's application/configs/deploy/. These files need to be named the same as your stages (with a .rb extension), e.g. production.rb. Here's an example:

role :app, "example.com"
role :web, "example.com"
role :db, "example.com", :primary => true

set :deploy_to, "/path/to/your/app/"

set :user, "your_ssh_user"
set :password, "your_ssh_password"

Set :app to be the hostname of your remote server. :web and :db will usually be the same, unless your have separate dedicated servers for each of these and need to run any custom tasks on them later. :deploy_to should be the full path to the location on your remote server where you want your application to live. Change :user and :password to be the SSH login details for your remote server.

To add a new stage, simply add it to the :stages variable in deploy.rb (%w(staging production) is the Ruby equivalent of array('staging', 'production')), and then create a file in application/configs/deploy/ with the appropriate login details.

The deploy.rb example above also contains a section that overrides the default Capistrano tasks :migrate, :start, :stop and :restart. These are pretty Rails-specific, so we override them because we don't need them. However if you have any custom restart requirements you'd define them here. E.g. if you're using APC with stat set to 0, you might want to add some code to the :restart task to restart Apache (this will run after each deployment).

Assuming you've got all that setup correctly, it's time to give it a try. (Remember these commands will be performing actions on your remote server!) First run:

cap production deploy:setup

this will create default Capistrano folders on your remote server. After running it, login to your remote server and in your :deploy_to location you should see two folders: releases, and shared. Then (from your local machine again), run:

cap production deploy:cold

this will run the first full deployment of your application. If you check your remote server again, you'll find a new timestamped folder in the releases folder that contains a full copy of your code. You'll also see a 'current' symlink at the :deploy_to location that symlinks to the latest release. So just set your vhost to point at (your path)current/public and your site should be up and running!

Then in future, to deploy a new update you just have to run:

cap production deploy

this will checkout a new copy of the code, and assuming no problems, switch the 'current' symlink to point at the new release. And that's it! To perform the same thing on your staging setup (or any other stages you add), simple replace 'production' with the name of the stage in the commands above. E.g. cap staging deploy.

Hopefully that should be enough to get you started. Run cap -T from your ZF project folder to see what other commands you can use, and the Capistrano FAQ is a good place to start if you need to customise anything.

Optimizing server configuration for PHP applications

2009-05-08T18:47:54+01:00

We recently migrated all of our websites to some new servers, and for the first time we have some dedicated servers for our PHP CMS platform. This gave me the opportunity to fine tune configuration specifically for PHP, and I was surprised at how much difference could be made without even touching the application code.

The setup includes two web servers and a dedicated MySQL server, all with 1G of RAM, running CentOS. I was using Apache bench for the tests, with the parameters ab -c 10 -n 500 http://www.example.com/ (10 concurrent requests, 500 in total). The starting point was a standard yum-installed LAMP setup except for the addition of the Remi repositories for more recent PHP packages (PHP 5.2.9, MySQL 5.0.45, Apache 2.2.3).

If you're not familiar with Apache bench, the 'requests per second' stat is the thing to look out for in the results, higher is better. Each test was run after 'warming up' the caches, and there weren't any failed requests.

Firstly the baseline test (fresh after installation):

Requests per second:    13.31 [#/sec] (mean)
Time per request:       751.477 [ms] (mean)
Time per request:       75.148 [ms] (mean, across all concurrent requests)
Transfer rate:          43.27 [Kbytes/sec] received

Then after installing APC:

Requests per second:    18.52 [#/sec] (mean)
Time per request:       539.833 [ms] (mean)
Time per request:       53.983 [ms] (mean, across all concurrent requests)
Transfer rate:          60.24 [Kbytes/sec] received

Then started the tweaking. The Remi php-apc package has apc.shm_size set to 32M by default (slightly higher than the PHP default of 30M). As I understand it, this variable controls how much RAM is reserved by APC. If it is set too low, APC won't be able to cache all of your commonly included PHP files; but as you increase it, once you pass the point at which your whole app can be cached you'll start to see a reduction in performance, since you're reducing the amount of memory that can be used by Apache. After some experimentation I kept this at 32M.

Next I turned off apc.stat. With this option enabled APC checks the last modified time of each included PHP script on each request to see if it needs to be recompiled. With it off you save quite a lot of file system calls, especially if you have a lot of file includes, but it means you need to restart Apache after any code changes. Results:

Requests per second:    18.89 [#/sec] (mean)
Time per request:       529.346 [ms] (mean)
Time per request:       52.935 [ms] (mean, across all concurrent requests)
Transfer rate:          61.43 [Kbytes/sec] received

Not much difference! The application code in this case is stored on a shared NFS drive, so perhaps the local NFS client is caching some of the file information.

Next I enabled apc.include_once_override, which gets around the problem APC used to have caching files included using include_once or require_once:

Requests per second:    23.00 [#/sec] (mean)
Time per request:       434.698 [ms] (mean)
Time per request:       43.470 [ms] (mean, across all concurrent requests)
Transfer rate:          74.81 [Kbytes/sec] received

If your application uses Zend Framework you'll probably see even more benefit here, since ZF uses include_once extensively.

Next came the MySQL tweaking. Although MySQL comes with some suggested configuration files for different setups, the defaults are quite conservative. If you're not familiar with the different MySQL config options, I'd suggest installing MySQLTuner, a free script you can run which will analyse your installation and suggest some changes.

By far the biggest improvements you can make are enabling the query cache and increasing key_buffer_size. If you're running a dedicated server for MySQL you can assign a decent chunk of RAM to these two. The settings I used:

key_buffer_size=128M
query_cache_size=256M
query_cache_limit=16M
thread_cache_size=4
table_cache=128

Results:

Requests per second:    61.60 [#/sec] (mean)
Time per request:       162.339 [ms] (mean)
Time per request:       16.234 [ms] (mean, across all concurrent requests)
Transfer rate:          199.42 [Kbytes/sec] received

Big improvement!

Obviously there are a number of application-level things that can be done to improve performance further (page/fragment caching, caching objects in APC/memcache), but this shows how much improvement you can gain just by tweaking some config files.

Experiences with Zend_Captcha_Image

2009-01-06T23:48:34+00:00

Yesterday I had to add a CAPTCHA to a form on a site already using some ZF components, which gave me a chance to try out the CAPTCHA component in ZF.

As well as the usual obfuscated image, Zend_Captcha supports figlet (ASCII words) and reCAPTCHA (a 3rd party CAPTCHA service to help transcribing old books). Although I love the idea of reCAPTCHA, I've always thought its standard input box looks a bit like a banner advert (and thus is easy to miss), so I went with the familiar squiggly image (Zend_Captcha_Image).

Most of the standalone CAPTCHA scripts I've used in the past work as follows: You include an image tag in your HTML which points at a PHP script. This script generates an image on the fly (and shows it), as well as storing the generated word in a session. You then include a chunk of code in your form processing script that checks the submitted data against the session word.

The ZF component works slightly differently, in that it generates an actual image file (stored on the file system), and then outputs an image tag that points at this. The generated word is stored in the session (using Zend_Session), and the object has a method for comparing the two when the form is submitted.

Although not a requirement, the component was obviously designed to be used in conjunction with Zend_Form, and it took some digging around in the code to work out how to use it without. Certainly the code example in the docs is incomplete, so hopefully this article will help anyone trying to do the same.

Firstly the absolute minimum amount of code required to generate (and show) a CAPTCHA image:

<?php
require_once('Zend/Captcha/Image.php');

$captcha = new Zend_Captcha_Image(array(
        'font' => '/path/to/fonts/Example.ttf'
));

$id = $captcha->generate();

require_once('Zend/View.php');
echo $captcha->render(new Zend_View());

this will create a PNG image in the folder images/captcha (which needs to be writeable). There are garbage collection methods that will automatically remove old images after a configurable amount of time. You will need to replace the font path with the path to an actual TTF font file on your system. You can either download a free font (of which there are many on the Web), or point it at a font already installed on your server (check /usr/share/fonts/truetype/ in Ubuntu).

View this script in a browser and you should see an image like this:

so, now you need to wrap this in a form, provide a box for the user to enter their input, and then check this data after a post. And this is where the documentation starts to let it down, since no example of this is provided.

Checking the POST data is done using the function isValid(), which has two parameters - $value and $context. $context is optional, and even after looking at the code I'm not clear what this is for, so I've ignored it. You would assume $value to be the string the user enters, but it actually needs to be an array with two keys: input and id. $value['input'] should be what the user typed into the form, and $value['id'] should be the MD5 hash returned when the CAPTCHA was generated ($id in my code example above).

If you are using this with Zend_Form, the supplied decorator will output form fields that generate this array for you. But if not, here's a working example:

<?php
require_once('Zend/Captcha/Image.php');

$captcha = new Zend_Captcha_Image(array(
        'font' => '/path/to/fonts/Example.ttf'
));

if (!empty($_POST)) {
        if ($captcha->isValid($_POST['captcha'])) {
                echo 'Passed CAPTCHA check';
        }
}

$id = $captcha->generate();
?>

<form method="post" action="">
<?php
require_once('Zend/View.php');
echo $captcha->render(new Zend_View());
?>

<input type="text" name="captcha[input]" />
<input type="hidden" name="captcha[id]" value="<?=$id?>" />

<input type="submit" name="submit" value="Submit" />
</form>

It's important that the isValid() check happens before the generate() (as in the example above), as otherwise you'll be checking against a new CAPTCHA and the check will always fail. Normally you would only run the generate() on non-POST requests, or if the check fails (to give the user a fresh image to try).

A number of other options can be supplied to customise the image, e.g.:

<?php
$captcha = new Zend_Captcha_Image(array(
        'wordLen' => 5,
        'font' => '/path/to/fonts/Example.ttf',
        'imgDir' => './captcha/',
        'imgUrl' => '/captcha/',
        'width' => 150,
        'height' => 55,
        'dotNoiseLevel' => 40,
        'lineNoiseLevel' => 3
));

wordLen controls the number of characters in the image (default is 8). imgDir and imgUrl control where the image files are stored. width and height affect the dimensions of the generated image. dotNoiseLevel and lineNoiseLevel control the number of dots and lines added.

Maintainable CSS

2008-10-08T22:39:23+01:00

Natalie Down has put up slides for what looked like an interesting presentation on writing maintainable CSS. This is one of those topics that rarely gets much attention, so it's nice to see some suggested good practices, particularly for stylesheet structure and layout. She also has some good tips on CSS implementations in general.

The biggest problem I've found when maintaining CSS is that when you come back to a site some time after the initial implementation (or if you're maintaining some code written by someone else), it's usually easiest to add new styles to the end of the existing stylesheet instead of reminding yourself how it all fits together, and then putting the new styles in the correct place. This gradually leads to bloat, as you end up putting in overrides for rules you've defined elsewhere, and then the next time it becomes overrides for your overrides, and so on. Grouping your rules as Natalie suggests is the only real solution to this, as it forces you to insert your code at the appropriate place, which in turn encourages to you edit what is already there instead of adding to it.

Some other random notes on the slides:

It's interesting seeing other developers' approaches to design implementation. After some basic planning I tend to start chopping straight from an actual-size mockup provided by the designer, the aim being to end up with as close a representation of this graphic as possible.

Perhaps I'm interpreting it incorrectly, but "the grid" is not a phrase I'm particular fond of in this context - elements of a design rarely translate perfectly to a grid-type layout, and even slight changes in the spacing between elements (in order to conform to your grid) will usually result in some pretty negative feedback from the designer.

I tend to think of layouts more as collections of boxes. E.g. if this site had been designer by a proper designer (obviously it hasn't) and they'd sent you this layout as a graphic, by looking at it you'd put mental boxes around distinct elements such as the right sidebar, navigation and main content area. These then translate directly to semantically named divs in your layout. For more complex layouts, it can help to use the draggable line guides most graphic programs provide in order to work out exactly what lines up with what.

Regarding CSS frameworks, it's nice to see someone speak out against these. We've been experimenting with Blueprint a bit at work and I haven't been too impressed so far.

On the plus side I can see the attraction of something that resets browser defaults to a common baseline (particularly to people who've been stuck by browser inconsistency issues before), and I think grid-style layouts are a little easier for people to get their head round than floats and clears. But to me the negatives far outweigh these advantages:

You're adding a layer of overhead to your site - every user has to download all that additional CSS code, plus (generally) slightly larger HTML files
These frameworks steer you away from good markup, as your classes end up being presentational ("span-36 last") rather than semantic ("navigation").
You're setting yourself up for maintainability problems (as Natalie mentions), as you're starting off with a whole load of CSS code that was written by someone else, making the site more difficult to debug when you hit problems.

CSS frameworks haven't gained too much adoption as yet, so hopefully as the older, buggiest versions of IE continue to slowly lose market share, the demand for them will gradually disappear.

Module-specific models in Zend Framework

2008-08-15T19:27:03+01:00

A post this week on PHP::Impact looks at the handling of modules in ZF, particularly the issue of module-specific models. Frederico's solution for this involves calling a particular function when you want to load a model. The way I solved this was a little different - I have a simple controller plugin that uses a dispatchLoopStartup method to grab the current module name from the request and add its model directory to the PHP include_path. That way, assuming you're using Zend_Loader, you can instantiate your module classes in the same way that you would any other.

Here's a quick example of how this works (code is untested). Assuming a directory structure based on the proposed standard one in the ZF wiki:

application/
   config/
   controllers/
   layouts/
   models/
   modules/
      news/
         controllers/
         models/
         views/
   views/
data/
library/
scripts/
tests/
www/

In my bootstrap I define two constants - ZF_PATH (full path to the Zend Framework library directory), and APP_PATH (full path to the current project). I then use these to set the PHP include path initially:

set_include_path(
        get_include_path() . PATH_SEPARATOR .
        ZF_PATH . PATH_SEPARATOR .
        APP_PATH.'/application/models'
);

then I have a front controller plugin which looks something like this:

class Myapp_Plugin extends Zend_Controller_Plugin_Abstract
{
        public function dispatchLoopStartup(Zend_Controller_Request_Abstract $request)
        {
                set_include_path(
                        get_include_path() . PATH_SEPARATOR .
                        APP_PATH.'/application/modules/'.$request->getModuleName().'/models'
                );
        }
}

Register this plugin with your front controller and then as long as your module models follow the standard class naming conventions you don't have to do anything special to load them in.

The only disadvantage of this approach is the performance hit of an additional directory in the include path, but I'm hoping this will be negated by an optcache (something I have yet to benchmark).

Frederico's post has spawned a more general discussion on modules in ZF which I have some views on but that's a topic for another post.

Zend Framework in the enterprise

2008-07-24T11:11:11+01:00

I just posted a comment on Federico Cargnelutti's (excellent) PHP blog that I feel I should expand on. The topic was Zend Framework's suitability as an 'enterprise' framework. Although I'm sure ZF is used in a number of enterprises (and I've used it for a few things over at Incutio already), I think it is still a way from being the 'perfect' enterprise framework, which is how it is described in the post.

There are three 'enterprisey' areas in which I believe it is particularly lacking:

ORM
Database migrations
Application deployments

None of these things are 'impossible' to do with ZF-based apps, but the ORM has its limitations; and the other two aren't covered by ZF at all and require integration of third party solutions such as Phing.

Zend's view on component development for ZF has been that they wish to focus on areas where they feel they can offer value to the PHP community as a whole. For this reason they have delibrately avoided developing components for things already well served by existing open source PHP projects. This makes complete sense. But as a developer coming to ZF afresh, unless you already know about these projects, you're going to see some pretty big holes in ZF as a platform on which to build your applications.

Firstly, ORM. I'm not going to say too much about this since I've posted about it before. I guess my main complaint about ORM in ZF is that I've found the Table Data Gateway pattern pretty limited (which isn't ZF's fault of course), and because of this you frequently end up having to either write your own ORM layer or turn to third party libraries such as Doctrine or Propel.

As an example, one of the first projects I tried with ZF was a rewrite of one our legacy applications, in which the data for domain objects frequently spans two or more database tables. The only way to load this kind of data with Zend_Db_Table is to create your own joins with Zend_Db_Table_Select, to the point where you'd basically back to writing raw SQL.

It is also suprisingly difficult to get the Zend_Db family of classes to return instances of your application's classes (by default they always returns data arrays which your application then has to convert into objects). It's not impossible to solve this, e.g. you can have your objects extend Zend_Db_Table_Row, or through some calls on Zend_Db_Statement directly you can specify the class you'd like returned, but both of these approaches are a little messy.

I feel a little bad bashing Zend_Db, as it does what it is supposed to do very well, and the adapter, profiler and select families of classes are all excellent. But these are real world problems that I've not found with other frameworks.

Secondly, database migrations. There is nothing in ZF to do these, the closest is a community-submitted proposal for a Zend_Db_Schema_Manager component which hasn't been updated since November last year. Sure there are some third party solutions, but this is functionality Rails provides out of the box.

Thirdly, deployments. Gone are the days when to make an app live you FTP in and manually upload the files (or at least they should be). Whilst Rails doesn't provide anything like this, Capistrano (which is excellent), was written for RoR, and is very easy to install and integrate with Rails apps.

This has been quite a negative post. I don't mean to suggest that Zend have taken the wrong approach in development of the framework - It's still relatively young, and the upcoming 1.6 release does plug a few of the less-enterprisey holes. But if the framework is to be taken seriously as a platform on which to build enterprise applications, these are the areas which I feel need improving. This could be achieved either through the development of new components providing this functionality, or through components that provide much easier integration with the existing PHP solutions (in the same way that symfony has a component for interacting with Propel).

In an ideal world, I'd love to see Zend-contributed components which implement some alternative patterns for use in ORM layers, such as Zend_Data_Mapper and Zend_Active_Record (post PHP5.3) classes. I also have a close eye on the Zend_Tool project, which I hope will eventually cover the other two areas I mentioned.

There are also quite a few big pluses for ZF as an enterprise framework, in addition to the things Federico mentioned in his post, the license of ZF itself and Zend's other offerings (training, IDE etc.) both definitely work in its favour.