Why spec?

Specifications encapsulate a project's goals, requirements, features and scope. This is very important for several reasons:

Understanding project requirements

Understanding what a project sets out to achieve might seem obvious, but in reality, when at least two people are involved - some difference in perception of the project goals and needs is inevitable. Putting in writing the goals and requirements goes a long way towards resolving such differences.

Writing specifications will often reveal requirements not thought of initially, as a result of detailing and thinking through the initial set of requirements. This is very important for determining project scope.

Developing a project scope

The scope of a project is probably the single most important attribute for determining project success. A well defined scope will determine the time-frame, costs and requirements with good precision.

In essence, a scope is the sum of a project features, requirements and deliverables or in other words - the sum of the work required for project completion.

Scope is the main element against which projects are priced at Lionite.

The more complicated and/or innovative a project is, the more likely it is that features will change during the project lifetime (sometimes even drastically).

We can allow that to happen by adhering to scope instead of individual features. With good specifications it is much easier to avoid scope-creep.

Estimating project time-frame and pricing

Accurate time and cost estimates are important are as important to project success as anything else, and specifications are what it should be based on.

The better the written specifications are, the more accurate those estimates would be.

Creating a workable reference for development and design

Working with a good specifications document can do wonders for productivity and focus in the design and development stages. At the very least it can serve as a task list for project completion.

Well thought through description of features and relevant mockups can solve a lot of potential dillemas for both designers and developers and save a lot of time.

The specifications process itself should be used for ironing out as many issues as possible, helping the implementation phase to be as productive as possible.

Having a standard to test against project delivery

Specifications outline the agreement between parties on what constitutes a successful project delivery. Contracts, if relevant, will point to the specifications as the legally binding document for this purpose.

A good specifications document can resolve many potential disputes simply by stating clearly what should be delivered.

How to create better specifications

Creating good specifications entails some ground work, there's no around it. Keep in mind the effort you put in will pay itself back as the project progresses.

Break everything down

Decompose every requirement and feature to a list of small tasks. Add relevant detail that might affect completion time. The level of detail and scope of each task should be enough so that it could be accurately estimated in work hours (and not days, and certainly not weeks).

Research what you are not sufficiently familiar with. This will greatly help avoid nasty surprises later, and will make your estimates that much more accurate.

Be thorough

Try to make sure there are no gaping holes in the specifications - are all requirements and features accounted for? are there any hidden prerequisites you might have over-looked? read everything twice and then give it to a colleague to read.

Don't forget to add more technical detail such as security concerns, coding standards, post-project technical support and supported browsers.

Compose interactively with clients

It is very important for the specifications to capture correctly the vision the client has for the project. Specifications writing can be used to correctly align client's expectations with your own.

Your interaction with the client can (and probably will) lead to more insights on project requirements and features. It is not at all uncommon for a project to change direction somewhat during specifications writing - and it is much better that it happens at that time than at mid-project.

Be clear and concise

Be clear and and use as much detail as necessary. Be concise, since overly descriptive texts are harder to consume and understand.

Don't skimp on rewording and reorganizing bulky / technical paragraphs and edit out unnecessary detail.

Read more articles like this one

Joel Spolsky on painless functional specifications - part 1, part 2
Allen Smith on functional specifications
Jason Fried of 37Signals on avoiding functional specifications (basically refuting everything I've written here. A good read for counter arguments.)

Too often I've seen such queries ran in long loops one at a time, which is very bad for performance (as I will show here) and sometimes equally bad for integrity (if the process is interrupted). So what are the alternatives?

Inserting multiple rows

Insertion of multiple rows comes about often in batch jobs, database migrations and handling table relationships. A naive approach, via PHP, would be to loop over the data to be inserted, inserting one row at a time. Suppose the data is already properly filtered and quoted:

foreach( $data as $row ) {
      $query = "INSERT INTO `test_table` (user_id,content) VALUES ("
               . $row['user_id'] . "," . $row['content'] . ")";
      mysql_query($query);
}

Depending on the amount of rows to be inserted, this can be a costly process. A better approach would be to concatenate the values into one insert query and then execute it:

$values = array();
foreach( $data as $row ) {
      $values[] =  "(" . $row['user_id'] . "," . $row['content'] . ")";
}
if( !empty($values) ) {
     $query = "INSERT INTO `test_table` (user_id,content) VALUES "
               . implode(',',$values);
     mysql_query($query);
}

What's the difference? lets see some benchmarks:

A single query completes much faster than looping through multiple queries. At 2560 rows inserted, it took the loop ~36 seconds to complete, yet the single query took just 0.14 seconds.

Memory consumption shows a reverse trend however:

Since the values are concatenated to create the single query, it consumes more and more memory with more rows as it needs to hold a larger query string. At 2560 rows inserted, the single query approach consumed ~800kb in memory, while the loop consumed just ~60kb.

Since memory is much cheaper than CPU cycles and database connections, I'd usually opt for the single query approach. It's important though to be aware of the implications.

Inserting / Updating (multiple) values

Another use-case of multiple row operations is when we want to insert several rows that might exist already, and in case they exist we want to update existing values instead. Of course, we could check first which rows exist, update the ones that do and insert the ones that don't - for a total of at least three separate queries.

The update query is the major problem here - there is no way to update multiple rows with different values using one query (and it doesn't make much sense in most cases). However in this case it does make sense to do so, and fortunately MySQL provides a way to do just that, using INSERT ... ON DUPLICATE KEY UPDATE.

The KEY in this statement should be a unique key in the table you are inserting to (otherwise duplicates would be allowed and this operation would be moot). The syntax of this statement allows to pick which columns are updated in the case of matching rows. For example, suppose I'm adding translations for content pages. I have the following table schema:

 
pages
 - id
 - content
 - ...
 
pages_translations
 - page_id
 - lang_id
 - content
 

The pages_translations table has a primary (unique) key on ( page_id , lang_id ). When adding / updating a translation, the query would look something like:

INSERT INTO `pages_translations` (page_id,lang_id,content)
VALUES (5,2,'the brown fox ...')
ON DUPLICATE KEY UPDATE content=VALUES(content)
 

This will create a translation if one does not exist and update the content field if it does exist. We can combine this statement with the previous approach for multiple insertion to insert / update multiple rows with one statement.

Inserting / Updating with multiple tables

There are some cases where we would like to use data in one or several tables to update / insert into another table.

With insertion the syntax is relatively straightforward - replace the VALUES part of the statement with a SELECT statement.

INSERT INTO `my_table` (col1,col2,col3)
SELECT col4,col5,col6
FROM `another_table`

The ON DUPLICATE KEY condition can be used with this statement as well.

Updating rows cross-tables is done with a JOIN statement (of the less declarative type).

UPDATE `my_table`,`other_table`
   SET `my_table`.col1 = `other_table`.col2
WHERE `my_table`.parent_id = `other_table`.id

It's important to try the INSERT / UPDATE select statements separately to determine how many rows they will select and how fast will they complete. INSERT / UPDATE operations are relatively costly and can be a severe bottleneck for the database if not managed correctly.

Despite this, the application of those business models in the web arena is still very much experimental, and it's often hard for web businesses to find and implement a model that works well for them.

Got ads?

The most basic misconception about web business models is that you can never go wrong choosing an advertising-based model. If it was really that easy, then everybody would be making mad cash - however, advertising based revenue is actually hard to come by.

In fact, those who can actually make it stick are few and far between (Mike Speiser shows the numbers don't lie). The amount of traffic you need to generate in order to make substantial revenue from online advertising is out of reach for the vast majority of websites.

So when is an advertising model appropriate?

• When generating traffic is the main purpose of your web service. Such sites include all flavors of traditional content sites, social networks, user-generated content sites and the like.
• When your service has the ability to deliver targeted advertising. Targeting advertising at user activity, search criteria and the like, greatly increase the relevancy of ads and thus conversion rates. A little-known company has made targeted advertising its main business to devastating effect.
• Advertising is incorporated as a part of a hybrid model, to subsidize free parts of the service (see below for more details).

The important attribute here is that advertising must target a large enough audience with sufficient relevancy in order to generate significant revenue.

Subscription is king

Subscription business models are nothing new but have been gaining popularity on the web, especially with the proliferation of on-demand online services in which subscription payments often make the most sense.

A subscription-based model is often appropriate when the service / content offered is unique and hard to find elsewhere. For site visitors to shell out even a few dollars, they need to be convinced that they are getting something they can't get elsewhere (since much of the web is available for free). This is doubly true since the process of handing out payments details online is often a barrier by itself.

Whether it's entertainment or productivity, when the value to users is higher than or equal to the monetary cost subscription becomes a very viable model. Niche sites and web services often fall in this category when they provide something of value.

I'd note here that by subscription model I don't necessarily mean just recurring annual subscriptions - there are also milestone-based subscriptions (pay-as-you-go) and feature-based subscriptions (pay-per-feature). Those three types can be mixed together to create many different hybrids, with the common denominator being the "subscribed user" - a person willing to spend money to use the service.

The freemium model

Freemium is a trendy new name for a familiar model - offer basic or limited services for free, and charge for advanced (premium) features. The freemium model is often a derivative of the subscription model, with the basic subscription plan as the free part of it.

Using a freemium model can be very useful for businesses who are counting on long-term users instead of one-time purchases. By getting limited services for free, a user can have a taste of what the web service has to offer thus lowering the barrier of entry. The balance of free to premium is important in creating the right incentive for paying for the premium services (example - basecamp).

One common application of the freemium model is combining the subscription and advertising models, by offering free services that display ads and premium services that are ad-free - this basically allows the site user to decide how he will support the service.

When we are conducting business development with clients I Personally lean mostly to subscription based freemium model, as

The brokers

Brokers are services that bring together buyers and sellers thus creating a marketplace. Revenue is generated by charging usage of the marketplace in a variety of ways:

• Sellers can be charged for being listed by the broker (one time / per listing)
• Buyers can be charged for gaining access to the listing of sellers
• Fee / commission on successful transactions

The best known example of the broker model is Ebay, a giant marketplace that succeeds on the strength of its reputation system and internal policing. A couple of good examples for creative application of the broker model are threadless and crowdspring (which I described in a previous post on web business models).

The partnership, the exit

Partnerships can be the basis of viable business model in certain circumstances - such as affiliates and sponsorships, or can be another part of the whole for others. It is often used as a hybrid with other models (such as advertising or subscription) being the driving force behind revenue.

An extreme case of this model is the outright takeover of the business by another company, often called "an exit". Though some businesses set out with this strategy in mind, its very unpredictable nature makes it very risky without an adequate backup model. I call this strategy the "no-model" model, with businesses basically saying that they'll figure out as they go along, hoping that some giant company will swoop in and end all their problems.

Picking the right one

Deciding on a web business model can be tough for early stage start-ups and traditional offline businesses transitioning to the web. Each model has its pros and cons and the fear of making a mistake can be paralyzing. Sometimes the perfect business model is very obvious but more often than not it's a journey of self discovery for the business.

When we engage in business development with our clients, we usually ask them the following questions:

• What is your target audience?
• How much would you pay for your service?
• Who are your competitors? What is their business model?
• When (or how fast) do you expect web revenue to cover operating expenses?

Those are mostly introductory questions, aimed at helping the business orient itself on what it needs to know to determine its business model.

Analyze, improve, rinse, repeat

Choosing a business model is only the beginning. Constant collection and analysis of related data - such as conversion and churn rates, usage data and statistics - is essential for tweaking and evolving the business model and increasing its chances of success. The process of testing how much customers are willing to pay for services is sometimes called bounding-box pricing.

There are no strict rules for choosing a business model, so don't be afraid to experiment and look for business opportunities in unexpected places. You never know when or where your big break will happen, but if you believe in your product and your team your business model will reveal itself eventually - be prepared for it when it happens.

We went for an illustration-style design. Those characters are actually us (the Lionite team), building the site I am standing on the ladder to the right, in case you wondered.

The content on the site is still going through some incremental upgrades, but I really liked our designer's rendition of my idea for a development iteration:

And how it integrated nicely into the portfolio pieces (inserting shameless plug for our startup):

Those icons are on or off depending on our roles for each project.

We are very pleased with the final result and we the feedback has been very positive so far. Now begins the real challenge - promoting the brand and getting leads through the site. Time to get back to work!

How much are you worth

As a freelancer and sub-contractor, one of the basic skills I had to develop is conveying value to prospective clients. In order to earn the rates I want to charge, it's important to make the client understand the value he is getting in return.

Clients seeking freelancers are often looking for cheaper rates, as the thinking goes that companies inherently charge more - companies have additional costs that freelancers do not, and those costs are reflected in their rates.

While looking for cheaper rates, clients still expect quality - and those two properties have a very tangible relationship. You can assume higher rates reflect greater quality - however, in software development, quality is an attribute that is hard to explain to non-technical people, and as in every other occupation, higher rates don't necessarily mean higher quality.

My goal when communicating with a prospective client is to convey to him what is the value that I bring to the table. While admittedly my rates are not cheap - I believe they are more than fair, and the value I can give greatly exceeds the monetary compensation I demand for it.

So how can I convince a client that I'm worth what I ask for?

Talking about quality in software development

The following are metrics that are widely accepted as proponents of software quality:

• Maintainability - How easy is it to debug and maintain the source code? How is it to make small changes?
  The cost of maintaining the product over any stretch of time is highly dependent on this. Also, larger development projects will bog down during active development if they are not maintainable.
• Extendability - How easy is it to add functionality and features to the product? This affects the cost of future development, cost that may be larger than the original investment.
• Security - How secure is the product against known and unknown exploits in its domain? This concerns mostly products with online accessibility, which places web products at the highest risk spectrum.
• Domain relevance - How well does the product actually answers the problems it sets to solve? this is a metric that is hard to measure without real users testing the product. However, having plenty of experience in this area can definitely reduce the margin of error.

And another metric that is not software-development specific:

• Professionalism - How much respect does a service provider has for his line of work? how much respect does he have for his clients and how much does he put into his communications with them? how good is he with meeting time schedules and price estimates?

There are many ways in which I try to convey the value I can give in each of those areas -

• By presenting my best work in online profiles with a short summary of the what I reviewed above. (Warning - links are in Hebrew )
• By writing in my blog about issues relevant to those qualities.
• By actively participating in forums and other software development communities.
• By always trying to be transparent and professional in any communication with clients.

The fact that I perform those activities because I enjoy and learn from them is to my advantage.

Those activities are mostly behind the scenes though, cause the next most important step is:

Striving for a face-to-face meeting

My number one goal is to meet clients face-to-face. By putting a face behind the CV and answering any  questions in a professional manner, I increase the chances of a closing a deal tenfold.

I know many freelance developers have a reluctance to meet face-to-face, and are generally shy in such meetings. For the most part I embrace it - I strongly believe that after such a meeting the client will know exactly what value I stand for, and then it will only come down to costs versus value considerations.

Another equally important aspect of the face-to-face meeting is to understand in greater detail the scope and features of the project. Often, a second meeting for specifications only is required. I will write about the specification process in a future article, hopefully.

Creating a transparent and fair cost and scope proposal

When creating the actual price proposal it's important to keep two things in mind:

• Be clear on what's included in the proposal
• Make it obvious how much every part costs separately

Being clear on what's included in the proposal is a no-brainer. It will prevent future misunderstandings and points of contention. To a large degree, this is also influenced by the level of specifications document attached to the proposal (you do have one, don't you?).

Making it obvious how much every part costs is an effective way to translate to a client how much effort will it take to develop specific features. This allows a client to make value considerations - how much a feature is worth to the project versus how much it will cost to develop it, and often allows for compromises that can take a price proposal from the "out of range" zone to the "let's do business" zone.

Allow the client to make his own decisions regarding what features to keep or leave out, but provide your opinion as well. We, as web professionals, have a responsibility of educating our clients with our knowledge and experience in the field. Don't be afraid to influence the client with what you think is best for the project.

Stand behind your pricing

This might be difficult advice in those times of economic strife, however I strongly recommend standing firm beyond your price proposals if you believe them to be fair and thought out. If you represent with conviction that the price proposal stands for your worth, it increases your integrity in the eyes of the client.

Never leave money you believe you are worth on the table. There are other ways to come to terms that all sides can live with, such as trimming down on features and scope or converting some of the cost into shares in the project.

Flashback:

I remember that when I wrote this piece I had just submitted a price proposal for a freelance project at the sum of around 4,000$US. I was pretty sure I had conveyed my value perfectly to the client, though he had argued that the price is unacceptable for him. He wanted to close it at 2,500$, which is a pretty steep cut.

I was caught in a serious dillemma - I was out of the market for a couple of months, living off previous projects income, and it was the height of the financial crisis. Should I take the hit and take the project anyway or stick with my pricing?

I decided to hold firm to my pricing. The client went off to request more price proposals from other developers (with my specifications document - which at the time I was not charging for), but continued to contact me in 2-3 weeks interval, asking whether I'd be willing to come closer to his terms.

In the mean time I started my own web company with two other extremely talented partners, and we went on to close several major projects that make that 4000$ seem a bit disproportional right now. I was recently contacted for a last time by that client, and I was happy to respond that I'm no longer available for his project.

My own moral of this story is that if you follow through with your beliefs and represent your value well, you will eventually get through to the right clients. Clients that refuse to recognise that they need to pay for the quality they want to recieve, are clients that I rather not work with.

Suppose we want to find the 6 closest numbers to the number 2500 (including) from a numbers table:

SELECT number, ABS( number - 2500 ) AS distance
FROM numbers
ORDER BY distance
LIMIT 6

This returns:

(6 total, Query took 2.2792 sec)

This works nicely, and is actually relatively performant if the number column is indexed. Despite having to run a full table scan in order to calculate the distance for every number in the table, running this on a ~2 million row table completes in just over 2 seconds.

This kind of performance would be quite enough for smaller tables, but often in real time applications we would like faster response times than 2 seconds for completing a query.

Since we know exactly what we need, we can help MySQL by limiting the range of numbers it has to calculate the distance for. Since we want the 6 closest numbers, we can be sure they'll be at most in a range of 6 numbers lower and 6 numbers higher than our selected value.

If we can do that, then the calculation would run for only 13 numbers, hopefully leading to much improved performance. Selecting those numbers can be done using a union on a couple of SELECT statements:

(
   SELECT number
   FROM `numbers`
   WHERE number >=2500
   ORDER BY number
   LIMIT 7
) UNION ALL (
   SELECT number
   FROM `numbers`
   WHERE number <2500
   ORDER BY number DESC
   LIMIT 6
)

Notice the first one includes the actual value (in case it exists) and hence has one more value to select in its limit clause.

Combining this with our previous successful attempt produces the following ungainly query:

SELECT number, ABS( number - 2500 ) AS distance FROM (
	(
		SELECT number
		FROM `numbers`
		WHERE number >=2500
		ORDER BY number
		LIMIT 6
	) UNION ALL (
		SELECT number
		FROM `numbers`
		WHERE number <2500
		ORDER BY number DESC
		LIMIT 6
	)
) AS n
ORDER BY distance
LIMIT 6

It does return the same results:

Is much more performant:

(6 total, Query took 0.0011 sec)

And there you have it.

A couple of notes:
- All queries were ran with SQL_NO_CACHE at least 5 times to ensure the timings were indicative of the performance.
- The queries were ran against a ~2 million table filled with randomly generated values.
- An index was created on the number column.

As I found no concrete online information on the test (and the guide link from Zend doesn't work), I might as well elaborate a little for the benefit of future test takers reading this blog -

The test is 1.5 hours long and composed of 75 questions. Most questions are multiple choice with the rest being open-ended (usually requiring to enter what you believe will the be the result input from several manipulations). The scope of the test is pretty encompassing, touching some modules that I wouldn't normally use (or even imagine a possible use scenario for), but if you have enough experience of the core features (MVC, Db, Cache, Filter/Validation, Localization / Internationalization and Security) and coding standards - you should do just fine.

The one thing to watch out for is the relatively high percentage of trick questions - which actually made the test somewhat harder than I'd anticipated (after you hit several trick questions in a row, you start being suspicious of every question). Some questions didn't even have an absolutely right answer, but sort of the answer of least incorrectness.

Thankfully, 1.5 hours is plenty long for delibrating some of those more ambigious questions - I had finished my first run in about 35 minutes and rechecked everything in 10 more minutes - leaving me 45 minutes to spare.

I do have some more respect for people holding the certification now (regarding knowledge and experience), though I'm not sure what regard does it hold in the industry (this is the first time I've encountered someone asking for it - and the client is Zend itself, so it's not surprising). If anyone has had previous experience with ZF certification qualification requirements, I would love to hear it.

He has recently decided to have a go at a t-shirt design himself, and came up with this brilliant concept which I think is pretty awesome. If you like t-shirts and have a threadless account, please take the time to have a look and vote for it (I'd really like to see it printed ).

Search-and-replace Vs. Include

As the title suggests, there are two prevalent forms of PHP templating, each with its own cons and pros:

• Search-and-replace systems (to which Smarty belongs) involve running string replacement filters on a template, replacing placeholders with dynamic content.Search-and-replace systems suffer from parsing overhead which can be significant depending on the amount of content being parsed. More advanced systems in this category attempt to offset the overhead by caching parse results (Smarty for example relies heavily on caching for its performance), but this introduces the concern of maintaining the cache (determining when it becomes stale).

  Search-and-replace systems are also better suited for serving mostly static content, as regenerating content dynamically incurs the parsing overhead. Hence it is more suited for content web-sites than highly dynamic web-applications, in which every user sees a different page than the others, limiting the effectiveness of caching.

• Include systems involve including PHP scripts which serve as templates, and passing variables directly to those. Wordpress for example uses a basic implementation of such a system for its templating mechanism.Include systems suffer much less overhead for parsing templates than search-and-replace schemes, simply due to the fact that no such action takes place.

  On the other hand, include systems are prone to scoping problems. If templates are included in the normal flow of the application, then they share and pollute the scope they are in - meaning they could affect data in other parts of the application which could lead to weird behavior and decrease in reusability of templates.

Templating with Objects

Ideally, we would like to combine the pros and solve the cons of both systems. We need a way to pass an arbitrary number of variables into a template without polluting the general scope of the application, while avoiding the parsing overhead of search-and-replace routines.

Fortunately PHP presents us with such a construct - Classes. Class methods are different from regular functions in that they have a shared local scope always available - the object instance ($this). Recent PHP templating systems use this property of classes to their advantage, creating a separate scope for templates thereby allowing them to process without affecting the rest of the application.

Savant is a good example of such a system, and so is the View component of the Zend Framework. I'll be borrowing concepts from both to show how a simple implementation of such a system can be built.

In Code

In the most basic form, a templating system built with the principles I've mentioned above will look like this:

class View {
     public function render($script) {
        ob_start();
        $this->_include($script);
        return ob_get_clean();
    }
 
    public function __get($key) {
    	return (isset($this -> $key) ? $this -> $key : null);
    }
 
    protected function _include() {
        include func_get_arg(0);
    }
}

Usage is pretty simple - load some vars into a View class instance and pass a template into the render method. For a template that looks like:

 
<h2><?php echo $this -> name; ?></h2>
 
Current position: <?php echo $this -> job; ?>
 

Passing parameters into a view instance:

$view = new View();
 
$view -> name = 'Obama';
$view -> job = 'President of the USA';
 
echo $view -> render('/path/to/template.phtml');

Will result in:

 
<h2>Obama</h2>
 
Current position: President of the USA
 

We can also invoke the render() method inside a template to chain several templates together:

 
<h2><?php echo $this -> name; ?></h2>
 
Current position: <?php echo $this -> position; ?>
 
;
 
<?php echo $this -> render('footer.phtml'); ?>
 

'footer.phtml' being another separate template file.

In Review

Lets review what the class does:

     public function render($script) {
        ob_start();
        $this->_include($script);
        return ob_get_clean();
}

The render() method receives a $script parameter which is the path to a template script. It passes that parameter to separate method called _include() and captures the output using output buffering.

Capturing the output into a string instead of outputting to the screen at once is very useful if we want to perform this as a part of a complete process, such as preparing headers and starting sessions(), which has to happen before any output is sent to the browser. It is also useful in case we want to perform additional transformations on the output before sending it to the browser (such as parsing and filtering).

    protected function _include() {
        include func_get_arg(0);
    }

The _include() method is the core of this templating approach. By including the template inside a class method with no arguments (calling func_get_arg() instead), the included script has only the object instance ($this) as the available scope to get information from and pass information to. Any variables we declare inside this scope have no bearing on the outside and so collisions and scoping issues are avoided.

     public function __get($key) {
    	return (isset($this -> $key) ? $this -> $key : null);
    }

This is really a utility method to prevent PHP from throwing a 'variable is not defined' notice in case the template attempts to access a non existing parameter.

Complete solutions like Zend_View and Savant offer many more features, such as plug-ins, filters, base directory configuration and more, but the principle is the same. While the same structure could be mimicked to some degree using procedural functions only, the simple and convenient interface of passing and retrieving parameter to/from templates is unique to this approach - making it the clear winner in my opinion.

As it might be expected, there are several techniques to optimize the delivery of web pages. The Exceptional Performance guide by Yahoo is a great resource for a multitude of optimizations practices, including specifically two techniques which I will address in this article - script minifcation and concatenation.

Reducing script size and the number of request

Modern web sites and web application serve increasingly heavier javascript and CSS files. The price for the increase in script payload is felt both on the client side, where loading time increase making pages feel slugish, and on the server side, where bandwidth and requests are at a premium.

Javascript files especially block the browser while downloading (no more than 2 components can be downloaded simultaneously as per HTML specifications), and are the type of scripts seeing the steepest increase in size in the last several years.

Minification and concatenation

Minifying is the process of removing all unnecessary characters from the source code without changing its functionality (removing whitespaces, line-breaks and comments, replacing variables names and more). Concatenation is the process of unifying two strings into one. Applying both to our scripts we reduce total script size and the amount of requests.

There are several public algorithms for minifying Javascript, and Dean Edwards gives a nice overview of those on his blog.

For PHP we have the handy Minify library, which handles both the minifcation and concatenation of scripts. Minify is quite robust, and comes with a set of standalone classes that can be used in other contexts as well. It also sets Etags and far-future headers for better client script caching.

Deploying Minify

Downloading and extracting the package available on google code creates several text files and two sub-directories. The directory that is of interest to us is '/min' which contains the actual source code, and we'll place it somewhere on our server about our document root (so it can be accessed by a regular http request).

The basic workflow of using Minify is to replace our <script> and <link> tags which load our javascript and CSS respectively with calls to the minify script. We need to tell the Minify script which scripts to process, and there are two main ways to accomplish that:

- Passing the scripts as parameters in the call to the minify script. We pass the path to the scripts separated by commas, translating the following:

<script type="text/javascript" src="/path/to/script_one.js"></script>
<script type="text/javascript" src="/path/to/script_two.js"></script>

Into:

<script type="text/javascript" src="/path/to/min/?f=/path/to/script_one.js,/path/to/script_two.js"></script>

There is actually a shorter way writing this, by using a common base path for both scripts:

<script type="text/javascript" src="/path/to/min/?b=path/to&f=script_one.js,script_two.js"></script>

Provided that all the paths are correct, we have successful concatenated two scripts into one minified script. This can repeated for as many scripts as we like, though there is an artificial limit (due to performance and memory limitations I assume) of 10 scripts per Minify request. This limit can be adjusted in the configuration file which exists at the base path of the library (config.php).

- Using pre-defined script arrays. This method is less flexible as it requires hardcoding the script names into an array, but is slightly more performant. Minify calls those array 'groups', which look something like:

 return array(
    'js' => array('//path/to/script_one.js', '//path/to/script_two.js')
);

You can find an example of those in the installation folder of Minify (groupsConfig.php). The request to the script is then made by specifying the group name:

<script type="text/javascript" src="/path/to/min/?g=js"></script>

There are several options when using groups, and you can read on those on the library's wiki.

Its important to set up the cache folder for Minify. Minifying the scripts can actually take a couple of seconds, which is a relatively long delay - however once generated once they will be cached until the scripts change (provided the cache folder is set-up properly).

To declare the cache folder, simply define a variable named $min_cachePath in the configuration file or the main app file before the main application start.

$min_cachePath = '/path/to/cache';

Make sure the folder exists and has write permissions by PHP.

Real world use

I've implemented Minify in all of my recent projects to great effect. An extreme case would be script deployment in my current startup, Octabox web platform. Being a heavy-duty web-application, it consumes much more javascript and styles than your average marketing site.

Without Minify

Javascript requests:

CSS requests:

With Minify

Javascript requests:

CSS requests:

51 total requests weighing 540kb reduced to 5 requests weighing 116kb.