@lornajane sory for getting on your nervs with #rest,but are subquerys (like couchDB does) restfull to? is there a rule? like .../?type=bla

— Maximilian 'Berghoff (@ElectricMaxxx) June 12, 2013

The blog seems like a good place, as I can put examples and all kinds other things here, and waffle at length (which is really why I like it!). Because when condensed to tweet form, the answer is really "it depends".

The Problem(s)

REST is all about representations of resources. They might come in different formats, and they might appear at their own URI as well as in one or more collections, but essentially you just get a representation of a thing. This is great, apart from when it isn't.

• What if you want a smaller result set with only a limited number of fields?
• What if you want related data? For every resource in a collection?

There are a couple of tactics that I deploy each time I need to solve one of these problems, but they all revolve around remodelling the resource structure. Just as we sometimes move fields around for database design or normalisation, we can do exactly the same with a RESTful service to make the resulting output make more sense to consumers. I thought I'd share examples of these tactics in this post.

More Verbose, Less Verbose

If a resource can be delivered in multiple formats, say XML or JSON, then why not in multiple "verbosities"? Sometimes you only want the outline of a resource, for example if you're display news items, you might only want the title and posting date. When you come to display that specific resource though, you'll want all the details, so you might end up with data structures like this:

Standard record:

• Title
• Author
• Date
• Excerpt

Verbose record:

• Title
• Author
• Date
• Excerpt
• Full Article
• Collection of Related Articles

If you want an in-the-wild example, try the Joind.in API. All the records there come in a fairly minimal format, but you can request more fields by adding verbose=yes as a GET parameter. Compare the API response (it's HTML, you can just click) for the recent DPC conference http://api.joind.in/v2.1/events/1109 with its verbose version http://api.joind.in/v2.1/events/1109?verbose=yes.

Splitting into Subresources

The alternative to allowing different verbosity of representations is to remove part of the record as it might be stored in the database into a separate resource.

So if your article is stored at http://example.com/articles/42, this would show the abbreviated version of the record from the structure above. Then you could offer the main body of the article at a separate location, such as http://example.com/articles/42/body. This does lead to repeated queries but if you only need the extra detail when you want to display a single record, it may well be worth the tradeoff - of course your mileage will vary between different applications.

Nesting Likely Data

The final tactic I want to share is where the server has a good idea that information will be needed from more than one source. There's a good example in the wild from GitHub - if you grab a list of issues over their API, you'll get the full user object of the user that created each issue. This makes sense because we'll always want to display who created the issue.

As an example, try requesting this URL with curl or your favourite API tool: https://api.github.com/repos/zendframework/zf2/issues and have a look at the structure that's returned (and as a total aside, admire the pretty-printing! I love GitHub's API)

Generic Solutions

There certainly are generic solutions, where you can basically express the select statement in URL form, but I'm not a huge fan of this approach. I'm sure there will be links to excellent resources in the comments of this article (thanks, people!)

Lorna is an independent web development consultant, author and trainer, available for work (interesting projects only). This post was originally published at LornaJane

What's alarming about this is that the left half of this graph represents unsupported versions of PHP. PHP 5.2 has been end of life since January 2011. This doesn't mean that you can't use it any more, but it does mean that in terms of security updates, you are out of luck. Some distributions will try to retro-fit some of the fixes but essentially your PHP applications seem a bit lacklustre because, well, you're using technology from 2006.

Where To Go From Here

Nobody chooses a PHP 5.2 platform, these things just happen, and I absolutely don't mean this post as a rant - more as a way of raising the issue and trying to give some pointers for moving forward. When I work with organisations to upgrade their platforms, usually either their hosting company is living in 2006, or they have a "supported" distro which prevents them from using modern technologies, or there's been no budget to upgrade, or ... really there are lots of (really valid) excuses. However good things await in newer versions of PHP!

PHP 5.3 has actual useful OOP features! We have anonymous functions (this feature will change your life, and for JS developers especially will make your PHP make more sense), the SPL extension is more than just iterators, and the DateTime extension is fabulous and finished in PHP 5.3. Also in PHP 5.3 (released 2009, it's not cutting edge) is the vitally important E_DEPRECATED error reporting flag. This will identify any features you are using in the current version of PHP which won't be available in the next version - if you are on PHP 5.3, or can get there, your upgrade path gets much easier! If you have a living PHP application, I can't recommend this move strongly enough.

PHP 5.4 has some cute new features, but whether you need them or not, PHP 5.4 has faster execution time and reduced memory footprint. Here's the graph from some benchmarks I did between versions of PHP last month - the Y axis is the number of seconds each version took, on average, to run the benchmark script included in the PHP source tree (raw numbers are available in a gist):

Whether you want to be able to use traits or not, PHP 5.4 will improve the performance of your application and reduce your hardware costs ... all you have to do is upgrade between versions of free software. Why isn't the PHP 5.4 segment on the first graph bigger? There are some explanations, such as the lack of APC, for this version, but nothing that seem compelling to me (happy to hear your stories in the comments).

PHP 5.5 isn't actually production-ready, as we're still in the release candidate phase for it, but it's probably weeks away. Look out though, a more regular release cycle for PHP means that there will be smaller incremental changes and we'll all be upgrading PHP the same way we maintain our other software - so pretty regularly. Beyond PHP 5.3, the effort and risk in upgrading is much reduced.

The Future

In truth, the future is already here for those people on PHP 5.4 and beyond. Keeping PHP upgraded is just part of our regular maintenance workflow, and the language is progressing in regular and manageable steps. If you've been left behind then I strongly recommend that you start making plans for upgrading your platform, or moving to a newer one. The effort will be well worth it when your application is still evolving and performing for many years to come! If any other platform gave away such incredible improvements in features and performance for free, it would be big news. Let's make it big news in PHP too.

Lorna is an independent web development consultant, author and trainer, available for work (interesting projects only). This post was originally published at LornaJane

Writing a Generator

The generators use the yield keyword to feed values out as they are iterated over. In code, they really look a lot like a function (or method):

<?php
function getValues() {
    // totally trivial example
    yield "Apple";
    yield "Ball";
    yield "Cat";
}

The main difference is that this "function" will retain its state and yield the next value when it is used again.

Using a Generator

It's not obvious from the calling code that a generator is in use - and that's a feature IMO. Here's an example that uses the generator declared above:

<?php
$stuff = getValues();

foreach($stuff as $thing) {
    echo $thing . "\n";
}

Each time the foreach tries to fetch the next item from $stuff, the getValues() generator gets called, and runs until a yield statement causes a value to be emitted.

When To Use Generators

There's nothing in these examples that makes a generator a better choice than, say, returning an array, so why are generators even useful? For the most part, they will be great for either formulaic or very large datasets. If you wanted to perform some task on all prime numbers up to ten digits long, you could certainly generate a list and iterate over it. However, that list could be quite large, and the generator could calculate and supply the next value on an as-needed basis.

The other use is for data sets which are large in comparison to the size of the memory available to PHP. A generator could fetch data in a loop, or read incrementally from a stream, and only have that one piece of data hanging around in PHP's memory at any one time.

Looking Forward to Generators

Generators are just one more in a long string of understated features coming in to the newer versions of PHP, but it's one that will make your data-heavy applications run light and quick - I can't thank the PHP core team enough for bringing us this feature!

Lorna is an independent web development consultant, author and trainer, available for work (interesting projects only). This post was originally published at LornaJane

In fact, you've been able to pass this as an array since PHP 5.2.10, so to set multiple headers in the stream context, I just used this:

<?php
$options = ["http" => [
    "method" => "POST",
    "header" => ["Authorization: token " . $access_token,
        "Content-Type: application/json"],
    "content" => $data
    ]];
$context = stream_context_create($options);

The $access_token had been set elsewhere (in fact I usually put credentials in a separate file and exclude it from source control in an effort not to spread my access credentials further than I mean to!), and $data is already encoded as JSON. For completeness, you can make the POST request like this:

<?php
// make the request
$response = file_get_contents($url, false, $context);

Hopefully this will help someone else doing the same thing next time (or at least I know I can come back here when I can't remember!), the array approach seems more elegant and maintainable to me.

Lorna is an independent web development consultant, author and trainer, available for work (interesting projects only). This post was originally published at LornaJane

Web Root

The web root of your project is never the same thing as the root of your source control folder. Typically I recommend an src directory for all code, then a subdirectory called something like public which will be your web root. The web root contains only the entry point for your application's endpoints, typically an index.php file plus any assets such as javascript, images and css that you want to serve.

Library Code

This may or may not form part of your repository. If it does, it should go in a source folder such as the src example I gave above. You may chose to bring the code in from its own repository, using source control features like submodules in git or externals in SVN. Alternatively, you may consider that the libraries are a platform dependency, place them on each server where they are needed, and either symlink to them or include them as appropriate. This is particularly useful where you have several sites building on the same shared libraries; just put them in a shared place.

Build Scripts

Using a tool like phing or ant to repeatably perform tasks within your project is an excellent practice as it can really help to make sure things are done quickly and correctly every time. Any project which takes advantage of these types of tools should include the configuration files (e.g. build.xml for phing). These should live separate from any application code, perhaps in a tools directory, or even in the root of the project.

Configuration

Configuration might be different for every platform that an application runs on, but we still need a template to start from when we work with configuration. To achieve this, create files called something like config.php.dist which contain example settings that are needed for the config (for bonus points, make these settings correct for the live platform, so that if you ever get this wrong, this platform is fastest to fix!). Every installation will need to copy the file and call it something like config.php - this should then be ignored by your source control tools so that any changes to it, on any platform, are not shared.

For systems with automated deployments, it can be useful to keep the config file(s) separately on the server, and at deploy time insert a symlink at the point that the application expects the config file to be.

Auxilliary Tools

Many applications have extra tools around them, for example I have an open source project that has a command-line tool for trying out the API, and a tool that generates sample data you can use with the application. Both of these are integral to the project and should be kept in the repository - perhaps each inside their own directory.

Database Patches

Most applications will have some way of keeping track of changes to the structure of their databases, and these are as much part of the project changes as the code is! There are tools to help with database changes, and I wrote a post about database patching strategies myself, but either way they usually result in both patch files, and files to manage the patches. Both of these should be in a database directory or similar, as part of the repo.

Tests

Tests are definitely part of your project, so keep those in the repo! This ties in nicely with the comments about phing files, which can be a great way to make it easy for people to run the various test suites that you have. Whether your project uses traditional PHPUnit testing, has functional or behavioural testing, API testing, or all of the above - check all those tests into the repository so that everyone can keep the versions up to date and run them easily.

Everything and the Kitchen Sink

Anything you need for your project belongs in the repo, it's not unusual to also have documentation as well as everything mentioned above, plus several other things I've probably forgotten - so add a comment to tell me what you store in your repo that I didn't mention?

Lorna is an independent web development consultant, author and trainer, available for work (interesting projects only). This post was originally published at LornaJane

This isn't a rant about salaries, the skills of new graduates, or the trials of dealing with recruiters, although each of those is worth a post in itself. It's about the mathematics of providing your organisation with the talent it needs at the time that it needs it.

Shaping the Perfect Recruit

Sometimes, you just can't get the staff. Or rather, you can, but at >25% more salary than your organisation can really pay. Now you have three choices:

• Get someone not-really-qualified and turn out not-really-excellent work
• Get someone not-really-qualified, and then spend yet more money on getting them more qualified
• Get someone qualified, and absorb the costs

Let's put some numbers on those options, looking around it seems like a junior PHP developer in my area earns something between 18k and 24k (GBP). If you can employ someone not qualified on 18k whom we'll call Charlie, or someone qualified on 24k whom we'll call Alex, and we assume that they stay with the organisation for three years and get a 3% pay rise every year then your costs over those three years become:

Charlie: £55636.20
Alex: £74181.60

Both employees gain much from working in your organisation from three years, but Alex is still worth more than Charlie at the end of the time.

Now consider a third way. You still employ Charlie on 18k, but you set aside an additional annual budget for training and professional development, in this example this is a thousand pounds pear year. It's not a lot, but with a bit of luck this should cover a decent training course, more than a handful books, and maybe a relevant local event too. The total cost over three years:

Charlie++: £58636.20

At the end of three years, for very little (financial) input, you've got a much improved junior employee, ready to take on a much more senior role - although you might want to pay them a bit more in order to stop them jumping ship for another job now they have more skills! As a direct comparison, the three options look something like this:

On a very personal note, the jobs I've stayed in the longest are the ones where I was promoted internally (officially, or where I was trusted with more challenging tasks) and in web development, the churn of employees is much higher than it is in other places. By investing in your employees and persuading them to stay, the business continues to reap the benefits of the investment, rather than continuously investing in lining the pockets of recruiters. Getting the right staff does cost money, but whether you direct that money into your business or outside of it is up to you.

(This isn't the point of the article, I think these ideas apply to businesses everywhere, but if you'd like to check out my training pages, then you may)

Lorna is an independent web development consultant, author and trainer, available for work (interesting projects only). This post was originally published at LornaJane

curl http://api.joind.in | python -mjson.tool

You need python installed, but the JSON extension is probably included, and that's all you need for this tool. The result is something like:

You can also use this approach to present JSON data that has been captured to another file, for example, it's a handy trick that I use often when developing something with JSON as a data format.

Lorna is an independent web development consultant, author and trainer, available for work (interesting projects only). This post was originally published at LornaJane

On a related note, I'm also doing a Sitepoint "Talk with the Experts" session on 11th April (early morning UK time, as a special treat for everyone in Europe and further east, that doesn't happen often!). There are more details here: http://www.sitepoint.com/forums/showthread.php?1012242-Talk-Object-oriented-PHP-with-the-Experts and I hope you can join me then.

Lorna is an independent web development consultant, author and trainer, available for work (interesting projects only). This post was originally published at LornaJane