I thought I would share this small piece of advice. It might be obvious to a lot of people but is not always apparent or sometimes we forget or even run out of time to pay enough attention to it.

The advice is simply this:

When you report a bug or issue and the developer tells you [...]]]>

I thought I would share this small piece of advice. It might be obvious to a lot of people but is not always apparent or sometimes we forget or even run out of time to pay enough attention to it.

The advice is simply this:

When you report a bug or issue and the developer tells you it’s fixed, always check that it is actually fixed.

Sounds obvious doesn’t it?

Think back to times when you have taken the word of the developer that a particular bug has been fixed and decided not to test it or verify that it had actually been completed. Ok, it sounds a bit strong that you knowingly decided not to test the bug fix, you could have been swayed by a couple of factors. Perhaps that developer has always fixed bugs satisfactorily in the past or maybe you didn’t have the time to check the bug had been fixed properly and so gambled that it had been fully dealt with.

Whatever the reasons the result is that the bug fix did not get checked out and verified. For many issues, the fix put in place would simply have worked and that would be the end of the matter. That particular bug would never be heard of again, the gamble paid off.

But what are the chances that the fix did not work, or created a new previously undetected bug or that the developer misinterpreted the bug report and his or her fix is not what you expected. In my experience, the chances of things like this happening are pretty good (website testers are such a pessimistic bunch).

On occasion, bug ‘fixes’ like this actually have a bigger impact on the website than the original bug. They can make the situation worse rather than better. A minor bug that is improperly fixed can have a much bigger effect and without testing and verifying it properly you don’t know if that has happened this time or not.

Fixing A Bug And Making It Worse

To illustrate how it is easy to fix a bug and make it worse, consider the following.

Some work I have been managing recently involved using a fairly simple script. The way the script works is that you send a series of locations to it, in the form of a CSV file, and the script fetches data about each of those locations, which is then uploaded to an email marketing system.

The data that is returned by the script is dropped into personalised emails, which are regularly sent to thousands or even tens of thousands of people at a time.

The script is not the greatest of solutions but it is a fairly quick and straightforward piece of programming, which generally works well.

I reported a bug that for one particular location, no data was being returned by the script. We had to manually add the data for that one location, which was frustrating and time consuming but not the end of the world. A developer investigated the bug and soon after pronounces the issue fixed. The information is relayed to me that the bug is fixed. Everyone rejoices (not really, everyone just gets on with the next job).

When I check the script over, send some location data to it and review the results I find that data is indeed now being returned for the offending location. However, it is the wrong data. There are 2 locations in that particular city and the script is returning data for the other location.

Just to recap, the original bug was that no data was returned by the script for that single location. An inconvenience in that the missing data had to be manually added. The bug ‘fix’ has actually introduced a 2nd bug, which is that the incorrect data is now being returned for that single location, a more significant bug than the previous one if it went unnoticed, as thousands of individuals would have had incorrect personalisation details incorporated into emails they received from the organisation creating a great deal of confusion.

It just goes to show that when someone tells you a bug has been fixed, please always check. Something may indeed have been fixed, just not always what you anticipated. On occasion, the situation may have actually been made worse and not better.

I am sure that there are lots of similar stories to this one where a bug fix did not fix the bug but perhaps made it worse. Feel free to share any in the comments.

Image credit BenSutherland

This ad hoc approach to testing means that it is possible to miss [...]]]>

As a website is being built, the step of writing or putting together a test plan is often skipped entirely and testing just ‘begins’. If a test plan is done then it is often completed immediately prior to testing and completed too quickly.

This ad hoc approach to testing means that it is possible to miss large sections of functionality and not be able to fully consider all permutations or variations of events and actions that need to be tested.

For most websites, there is a fairly long list of items that need to be tested as part of any new website development and in order to know what to test, when you tested it and whether those tests passed or failed, it is best to make a proper plan.

We are going to show you how to write a simple and straightforward test plan, which will provide a more systematic method for testing your website but the process outlined below is still essentially a manual one. We are not yet moving into using software tools to help identify and run tests, nor are we covering automated testing here, although I hope to write a post on these aspects in due course.

Writing A Test Plan

A typical test plan should identify all the areas of the website that you need to test before it launches, which you then follow through to conduct your testing, making notes of when you completed the test and adding any bug or issue reports to your bug tracking software along the way.

Start off with getting all the main sections down in a spreadsheet, divided between front end and admin or CMS if the website has one.

Then, start to break each main section down into the constituent parts that you need to test, concentrate on the important parts of the website first such as the home page, which is likely to be one of the most visited pages and the main areas of functionality, which are most likely to be the most used.

For instance, in the case of an ecommerce site, this would include individual tests for items such as adding products to the shopping basket, removing products from the shopping basket, updating the shopping basket quantities, being able to enter the checkout process and completing the checkout process (depending on your website there may be several steps to test within the checkout process).

You should be asking yourself questions for when you start testing the checkout process from the point of view of the user such as:

• Can I complete my order?
• Can I go back and amend my order?
• Do I understand where I am in the checkout process?
• Can I clearly see what I am ordering and how much it will cost, including delivery and taxes?
• Do I get a receipt or email confirmation?

As the owner of the website there will be different questions such as:

• Does the payment processing work correctly?
• What happens if the incorrect card details are entered?
• Does the order confirmation process work correctly?

And so on, when putting a testing plan together you are testing the website from the point of view of the user and also from the point of view of the owner or operator of the website. You may also have specific usability testing lined up but the primary reason for the test plan as far as this blog post is concerned is to test the functional aspects of the website to ensure they work correctly. Does the website do what it says it does? For instance, if the checkout process displays the order details including taxes and delivery then does it show the correct details for both tax and delivery?

Another important area to test is the site search or product search, as this is likely to be used a lot on the website.

Make sure that you test all the forms within the website, such as enquiry forms and newsletter signup forms. Test that the form can only be submitted after the mandatory fields have been completed and that the form is emailed to the correct email address or saved to the database correctly.

Once you go through and think about each section of the website along the lines of the above then you should have compiled a decent list in your spreadsheet of areas and items to test. Add columns for each browser that you need to test each item on, which will be the web browsers that you have agreed to support as part of the project. This would normally cover the major web browsers such as IE6, IE7, IE8, Firefox 3, Chrome and Safari but may include others too depending on the usage of your target market (or existing audience if you have those statistics). There are a range of browser checking tools that can help you with this aspect.

For the admin system or CMS that powers the website, add a row to your test plan for each function. Going back to our ecommerce example, this means that there would be tests for add new product, edit existing product, remove product, view order, edit tax rate, etc. depending on what the system was capable of.

You could break these tests down further to test individual elements of adding a new product. For instance, is there validation present to check that mandatory fields are filled in when adding a product? Does the product image have to be a certain size or dimensions? Does the product title have to be shorter than a certain number of characters? All these items could occupy separate rows in your test plan.

Of course, if you were to break your test plan down to this extent for a large website or CMS then the result will be a long spreadsheet of items to test, which will take a great deal of time to test effectively. You will need to decide whether you have the time available to test in this level of detail or if some areas can be tested more quickly.

A few other items that you may wish to make sure you have as part of every test plan are the following, in no particular order:

• Ensure your HTML and CSS code validates to W3C guidelines
• Test accessibility of the website
• Test the SEO items that you have planned for to make sure they are in place
• Test against the specification for the project that was signed off to ensure that everything has been completed that was agreed
• Test against designs and/or wireframes or prototype html mockups that were completed to make sure the finished website matches the signed off designs or wireframes
• Take into account any changes requested (should be in the form of change control documents)
• Complete any security tests that are required

You may wish to have a look at our list of 10 different forms of website testing to make sure you have covered everything. The list can go on indefinitely, there is always something that can be tested a bit more, but at some point it has to be halted at a relevant point otherwise the testing will never get done.

List or Flow Chart?

Some aspects of the website lend themselves to having a flow chart in place that describes all the permutations or possibilities that need to be tested. This is especially helpful if one action then presents another set of possibilities. When testing without a flow chart it can become extremely difficult to remember which permutations you have tested and which you have not.

Having a flow chart or mindmap worked out that shows you all the possibilities allows you to think more clearly about what you are testing, what you expect the outcome to be and understand what you have tested and what is remaining to test.

Conclusion

If there is too much testing to do for one person then divide it up amongst other people with instructions on what you want testing and what to do with bugs when they are found. Make sure they follow their bit of the test plan and feed back on what they find, handing bugs to you or inputting them into your bug tracking software themselves.

As you test then fill in the test plan to show what you tested and when. If you keep the document client friendly then it can be shown as proof of what testing was completed prior to launch.

Testing is a vital part of any website development and having a decent plan before you start testing saves a great deal of time and effort whilst making sure that all the required testing gets done.

I go on to suggest some solutions but what I really [...]]]>

Over on my main company website and blog, www.webdepend.co.uk, I’ve put up a post aimed at graphic design agencies but may relate to many types of agencies or organisations that produce websites and experience bottlenecks when it comes to fully testing and launching those sites.

I go on to suggest some solutions but what I really wanted to put across to graphic design agencies are what I see as the benefits of outsourcing their website testing to an external company.

The benefits put forward in that post may be arguments that you agree or disagree with. Either way, I would be interested to gain your feedback – would you consider or do you outsource any testing to another party or do you intend on making sure that all testing remains in-house? What are the pros and cons of either approach in your view?

You can read the full post here:

http://www.webdepend.co.uk/2010/05/08/website-testing-problems-for-graphic-design-agencies/

One of the first posts I wrote, which described a typical Web Testing process, mentioned that a bug [...]]]>

An area that has not been looked at in much detail yet on this blog is what software or tools you can use to record and track bugs, issues or defects that you find when testing a website.

One of the first posts I wrote, which described a typical Web Testing process, mentioned that a bug tracking system ‘can be hugely beneficial, as you can set priorities for each bug and then understand how many bugs you have at each priority level. It gives you a much better understanding of how many bugs you have to fix before you can launch and calculate how long you expect that process to take.’

There are a good range of bug tracking applications out there, a lot of them web-based, which are vital when tracking bugs and associated information about each issue. Here we take a look at 4 of the main players in bug tracking software, which are also available in our Testing Tools Directory.

Why Use Bug Tracking Software?

Once you start testing a website you can come up with all sorts of bugs and issues that need addressing. I’m talking a bit more than the odd typo or broken image. Perhaps you find problems with the site search returning strange results or the checkout process not working correctly or there are specific issues in different browsers that work fine in other browsers. There are several different reasons to use an application that tracks bugs and feature requests.

Complicated Bugs

As bugs start to get a little more complex it is important to hand the developer all of the required details about each bug so that he or she can fix it in a reasonable amount of time, without chasing around trying to reproduce it or get a screenshot or URL showing where that bug was found. Bug tracking software gives you a structured format to report bugs so that you include the URL where you found the bug, provide a full description and a screenshot and what browser or operating system you were using. These details will shorten the amount of time that a developer needs in order to squash the bug and improve their productivity. Get more bugs squashed in less time.

Lots of Bugs

Alongside the complicated bugs will still be the relatively quick and simple items to fix such as the odd layout problem, typo, broken image or link, etc. These bugs, whilst they do need to be fixed, are sometimes not as important as other bugs and, when presented in a list or on a spreadsheet, can often be completed ahead of more important items because the developer naturally follows the list through from start to finish. Bug tracking software allows you to prioritise bugs so that developers work on the most critical issues first and leave the less important bugs until later. Rearranging priorities in a list or on a spreadsheet gets too cumbersome when dealing with a large number of bugs but issue tracking software handles this aspect with ease.

Improved Workflow

With bug tracking software you can assign bugs to different developers so everyone knows what they are working on. Once those bugs are completed then the developer assigns them back to you for checking or verifying. If you are happy you can close the bug, if not then reopen it and give it back to the developer. Do this for each bug and you can understand exactly where each bug is in the process, how many are assigned, how many are to be verified, etc. It allows you to have a much better handle on exactly where your web project is at instead of continually poring over your list or spreadsheet and trying to make an educated guess.

This improved workflow eases pressure on you and your developers and improves productivity because developers can just get on with fixing bugs or working on feature requests. It also means that you are not constantly updating lists and spreadsheets with the latest status of all the bugs reported.

Bug Archives

Once bugs are fixed, tested and closed they can be referred to at any time so that you can see when the bug was reported, when it was fixed, who fixed it and what the developer did to fix it (assuming they fill their comments in correctly). This provides an extremely useful archive if you ever need to refer to a particular bug again instead of trying to go back through emails or out of date spreadsheets to find details on that bug or feature request.

Some bug tracking systems allow time tracking so you can see how much time was spent on fixing a bug. Using bug tracking software ensures that developers become more accountable because their comments and times are recorded and can easily be referred to.

Bug Tracking Software

Now that we’ve established what bug tracking software offers and why web project managers and testers should incorporate it as part of most web projects (definitely on all large web projects and probably on most small to medium projects too) let’s have a look at some of the main bug tracking applications currently being used.

My specific experience is with using Bugzilla so let’s start with that one.

Bugzilla

Bugzilla - classic open source bug tracking application

Bugzilla was originally released by Netscape in 1998 and so has been around for a long while now. This open source software is one of the first bug tracking applications that spring to mind and, although it takes some setting up and the interface is a bit tricky to use, it has everything you need to record and track bugs effectively.

It is possible to sidestep the web interface by using a desktop client that synchronizes with the central bugzilla software. My specific experience with bugzilla was using the Deskzilla client on a Mac, which worked well.

As Bugzilla is open source then it is free to use although it needs to be installed and set up on a server in order to be ready to use.

Unfuddle

Unfuddle - web-based bug tracking software

Unfuddle is a hosted bug tracking application that also includes Git hosting and Subversion hosting allowing you to create your own secure code repositories. There are also some project management tools included such as tracking your project against specific milestones and time tracking is available in the more expensive corporate and enterprise packages.

Bugs, issues and feature requests are raised as tickets in the Unfuddle system and each ticket includes workflow management and an audit trail so you can keep track of who made changes to a ticket and when these changes were made.

If you need the code repositories and project tracking functionality along with a straightforward bug tracking system then Unfuddle could be the answer. There is a free package available that includes 1 project for 2 users. After that prices start at $9 per month.

Trac

Trac - open source issue tracking software with project management tools and a wiki

Trac is also open source software that combines an issue tracking system with a wiki and project management features such as milestones and being able to document the roadmap for a web project or software development. Trac can also serve as a web interface for version control systems giving you a usable front-end for Subversion or Git.

Tracking progress of individual bugs, issues or feature requests is straightforward with each one having its own numbered ticket with the ability to filter tickets by severity, project component, version or owner. Keeping on top of progress is also easy with the ability to set milestones and being able to see how far you are from reaching each milestone.

Trac has a good set of features and is a lightweight and flexible system that should be very useful for most web development projects. It runs on most operating systems but does require some setting up and this is perhaps its biggest downside that I can see if you want to start tracking bugs straightaway.

FogBugz

FogBugz - web-based or installed bug tracking software with large feature set

FogBugz is available as a hosted application, called FogBugz on demand, or as installed software on either Windows, Mac or Unix operating systems.

There are several parts to FogBugz, the issue tracking section makes it very easy to report bugs and you can even create bugs via email or create bugs quickly without having to fill in everything so you can get the bug into the system. Whilst I think this is a good feature you do need to make sure you go back and complete all the details before you assign the bug to a developer to work on. Everything is saved in a complete case history for future reference.

But issue tracking is not all that Fogbugz does, there is a project planning section where you can outline tasks, break larger tasks into subtasks, enter estimates and set milestones. Fogbugz also has something called Evidence-Based Scheduling, which looks at what tasks are being estimated and how long they actually took to complete and then plot a probability curve to work out whether a milestone can be hit or not. The FogBugz Wiki allows you to keep all project documentation in one place and there are also a whole load of plugins available to extend the functionality plus FogBugz can integrate with another piece of software from the same company (Fog Creek Software) called Kiln, which is a version control system.

FogBugz is the most expensive of the bug tracking software applications featured here with prices starting at $25 per user per month for FogBugz on demand or $999 for 5 users for the installed version of the software.

Conclusion

Whichever of the above applications you use, as long as you use one of them and get your workflow organised in terms of how you report, record and assign bugs then you will find that it becomes a breeze to keep track of exactly where you are at in your website testing. You won’t miss any issues (as long as they are reported and entered into the bug tracker), you can prioritise showstoppers or critical bugs easily to fix those first and archive completed bugs for future reference.

If I had to pick one of the applications above then I would go for either Unfuddle or FogBugz (on demand) because I like web-based applications as there is nothing really to set up or install. Both of these have a trial although both cost money whereas Bugzilla and Trac are free but require time installing and setting them up on a server.

FogBugz has the better overall feature set and I am intrigued as to how the Evidence-Based Scheduling works, as I have not come across that before. But it terms of a cheap, easy to use, bug tracking system that is ready to go and will do the job, I would pick Unfuddle with the caveat of going for FogBugz if I had a bigger team (and didn’t mind paying more money).

At the moment that testing subject of choice [...]]]>

When I’m reading blog posts, news articles or Twitter discussions concerning the testing of websites, the article or discussion in question actually relate to a single aspect of testing and gives the impression that there is only one form of website testing that we should be concerned with.

At the moment that testing subject of choice is known as Conversion Rate Testing, or sometimes called CRO (Conversion Rate Optimisation). This would normally mean that the tests being carried out are Split Tests or Multivariate Tests, using software such as Google Weboptimizer.

Don’t get me wrong, conversion rate testing is really important and can greatly increase the number of conversions on a particular page, which can raise sales, enquiries, profits or whatever aspect of conversion you are trying to improve.

But there are other forms of testing websites, quite a few in fact, and some that are quite basic that get overlooked. I wanted to give a brief description for each type of testing to help you understand all the areas that can (and should) be tested when launching a new website, rolling out a set of updates or at least checking on a regular basis.

Different Forms of Testing

Functional testing

This form of testing aims to ensure that the website works correctly or as it was intended to. You click on a link, does the website display the correct page? You complete a search, does the website return the correct search results? You add a product to the basket, does the website add the correct product to the basket? Functional testing is going through the entire website, including content management system or admin area, to make sure that each function within the website is performing as it should be.

Browser compatibility testing

Different web browsers display web pages differently from each other. This is one of the least helpful aspects of the Internet and browser compatibility testing or browser checking involves testing the website on each of the major browsers (or each browser that you have decided to support), which generally includes IE6, IE7, IE8, Firefox 3, Chrome, Safari 3 and Safari 4 on both PCs and Macs. You could possibly include Opera 9 and Opera 10 in that list in addition to testing your website on Linux as well as Windows and OSX. You could also check your analytics to see which browsers are most commonly visiting your site. There are several decent browser checking tools listed in our testing tools directory.

Usability testing

Usability testing assumes that your website works correctly and displays in each major browser correctly. However, a website that functions correctly is not necessarily a usable website for your target audience. Usability testing enables you to show your website to a group of users, ask them to perform certain actions and see how they accomplish those tasks. There are many reasonably priced usability testing tools on the market that have made this type of testing much more straightforward, many of which are listed in our testing tools directory. It is possible to gain great insight into how people actually use your website and a lot of the experts agree that you can pick up most usability issues when testing with only 5 users.

Accessibility testing

Accessibility testing is extremely important to make sure that your website complies with best practice and accessibility guidelines plus legislation surrounding accessible websites, which is in force in both the US and UK. In the US there is Section 508, which is part of the Rehabilitation Act and in the UK we have the Disability Discrimination Act or DDA. A fully accessible website not only benefits those users that would otherwise have difficulty using your website but search engines will also be able to navigate and understand your content more easily, thereby helping your search engine rankings. To help you test your website’s accessibility there are some accessibility testing tools available in our directory although manual testing is still recommended.

Performance testing

Performance testing is an area that has always been important but possibly even more so now that Google is starting to incorporate the performance of a website into its ranking algorithms. There used to be an 8 second rule, which believed that a user will wait no longer than 8 seconds for a page to download before they click back and try another website. That rule is a long way out of date, it is now no more than 1 or 2 seconds. Performance testing can show you when your website is slowing down, which pages are taking time to load and what element of the page is causing the problem. Performance testing tools are covered in our testing tools directory.

Stress or load testing

There is a difference between stress and load testing in that stress testing concerns itself with trying to break the system you are testing whilst load testing aims to find out how much volume you can put through the system. Both of these forms of testing are generally for larger websites and web applications. A range of stress and load testing tools can be found in our testing tools directory.

Site monitoring

A really simple test that you can put in place is to automatically monitor your website to check for downtime or outages. Nobody likes a website to be down including users and search engines. Regular downtime will lower confidence in your website and they could be affecting your traffic levels and therefore sales or enquiries without you knowing. A selection of site monitoring tools can be found in our testing tools directory.

Conversion rate testing

The current favourite form of testing, and an important one too. Conversion rate testing or optimisation can greatly improve the amount of visitors your website converts into enquiries and/or sales. If your SEO brings in 100 visitors a day and your site currently converts 5 of those into becoming customers then, with successful conversion rate testing, you could convert more visitors into customers and make more sales. Conversion rate testing normally takes the form of running split or multivariate tests on your website, many of which are listed in our testing tools directory.

Security testing

Websites can have security holes that could be extremely damaging for your business if they were to be found and exploited. Security testing aims to find where those security holes exist so that you can do something about them.

Spelling and Grammar checking

It is always useful to fully read through all text on the website to ensure there are no typos or grammar mistakes. You will also spot how you can improve the readability of text and other areas that perhaps need some attention. Also make sure you read through all messages that appear when signing up for a newsletter, adding products to a basket, or registering including any error messages. Simple changes to the wording of an error message into a nicely worded error message can make a big difference.

So there you have it, website testing can take on many forms and all of them should be completed to a greater or lesser degree. At least by completing each type of testing you can be aware of any issues or areas for improvement so then properly plan out what action you are going to take and when. Good luck with whatever testing you complete.

I’ve taken some time to update, reorganise and expand the testing tools page so that it become more of a directory rather than just a long list. This makes it easier for me to maintain and add to plus easier (hopefully) for you [...]]]>

Now contains a total of 96 website testing tools spanning 14 different categories.

I’ve taken some time to update, reorganise and expand the testing tools page so that it become more of a directory rather than just a long list. This makes it easier for me to maintain and add to plus easier (hopefully) for you to find the testing tools that you are looking for.

The Testing Tools page now serves as the main entrance into the directory with each category having its own page. Each category will continue to have more software and tools added over time plus there will be additional category or two coming along shortly.

Because I’ve been doing a bit of research into Usability Tools, this category is the main beneficiary of the reorganisation and now contains 29 usability testing applications. This is mostly due to the great blog post at Useful Usability, which also gives you pros, cons and pricing information for the 24 usability tools they feature.

Another category that has grown significantly is Split Or Multivariate Testing Tools, which now shows 16 applications. Some of this increase is down to a great comparison resource over at Which Multivariate? This newly released site has reviews and comparison charts of all the major split and multivariate testing software tools out there.

The full list of categories contained in the updated testing tools directory are:

1. Validation and Accessibility Testing Tools
2. Browser Checking Tools
3. Screen Resolutions Tools
4. Link Checking Tools
5. Site Performance Testing Tools
6. Stress or Load Testing Tools
7. Usability Testing Tools
8. Split or Multivariate Testing Tools
9. SEO Tools
10. Web Analytics
11. Online Surveys
12. Site Monitoring Tools
13. Software Testing Tools
14. Web Development Tools

I will continue to pick up on the huge number of testing tools being released and add to each category as time allows. If you spot a testing tool, which you feel is worth me adding to the directory, then please do contact me. At some point I will get around to adding a dedicated submission form if there are enough people willing to add testing applications.

Feel free to retweet, digg or add to delicious as you like or let me know any feedback (good or bad) in the comments.

I have come to the end of my blog series on how the major UK supermarkets use custom 404 pages to improve the user experience.

The supermarket that started this off was Asda, when I stumbled across the fact that the main Asda web site did not have a custom 404 [...]]]>

Example of www.asda.co.uk's 404 page

I have come to the end of my blog series on how the major UK supermarkets use custom 404 pages to improve the user experience.

The supermarket that started this off was Asda, when I stumbled across the fact that the main Asda web site did not have a custom 404 page and took me to the horrible server default 404 page. At the time I couldn’t believe it, I thought all major retailers would have had this basic principle of web development covered and be doing a good job of 404 pages.

Then I realised that Asda had quite a few other web sites so I started investigating those, found more web sites with no custom 404 pages and the list grew and grew until I had 25 Asda web sites in total, 9 of which do not have a custom 404 page at all.

If Asda are that bad, how do the other supermarkets compare?

The obvious next choice was Tesco. I started investigating their web sites with some trepidation – if Asda have 25 web sites then how many are Tesco going to have? Thankfully, the total is ‘only’ 14, 3 of which have no custom 404 page, including Tesco Direct, which is a huge online catalogue web site that would really benefit from a decent 404 page, and 6 have poor 404 pages, including Tesco Insulation, Tesco Diets, Tesco Energy and Tesco Travel Store, which were all terrible.

To wrap up the series I travelled around the web sites for Sainsbury’s, Waitrose, Morrison’s, Somerfield and Co-Operative and so covered all the major UK supermarkets.

The table below shows each supermarket, how many web sites they have in total and then the state of their 404 pages. I have included a percentage figure at the foot of the table, showing how many web sites of that supermarket’s total have good 404 pages, as a percentage.

Supermarkets 404 page use

You will see that Asda and Tesco are pretty close, with about a fifth of their web sites having what I would term as good 404 pages. Sainsbury’s are worse with only 11% (1 out of 9) whilst Waitrose has a quarter of its sites in the ‘Good’ category (1 out of 4).

There is also a figure for the percentage of web sites from that supermarket’s total without custom 404 pages altogether. Asda is pretty poor with 36%, which is 9 of its 25 web sites and Sainsbury’s is also lacking in this area with 44% of its sites without a custom 404 page, or 4 out of 9. Tesco comes next with 3 of its 14 sites without a custom 404 page, or 21%. Finally, Morrison’s single web site does not have a custom 404 page so they score 100% in this aspect.

Conclusion

The main thing to draw from all this is that the overall quality when it comes to the use of 404 pages in UK supermarkets is very poor. I had expected a great deal more from these major companies that obviously spend a huge amount of time, effort and expense on their multitude of web sites.

Whilst some of the supermarket sites do genuinely have good 404 pages, a lot are terrible or simply non-existent. The subject of 404 pages is possibly on the dull side for many (the novelty is wearing thin for me too after this study) but my belief is that they really do add to the user experience and a bad or non-existent 404 page really can have a negative effect on this experience.

Users can hit a 404 error for many reasons and I have discussed previously why a custom 404 page is a good idea. When a user does come across a 404, the important bit is what that user does next. What the owner of the web site should want them to do is to stay on their site, continue browsing until they reach the final goal for that web site – usually ordering something or making an enquiry. How does the user do that if all the 404 page shows is the server default version or does not help them in any way?

It is like reaching a crossroads with no signpost or map or hitting a dead end and having to go back the way you came, neither of which are good experiences for the user.

Supermarkets, and their in-house teams or external digital agencies, really need to look at the quality of the 404 pages that they are producing, often as part of a large scale redesign, and determine whether the result is good for the user or not. In the majority of cases at present, it is not.

The third part of my series on looking at supermarkets web sites to see how they use 404 pages to direct users if they get lost and so make sure that the overall user experience is a decent one.

The first two parts investigated Asda and Tesco who, between them, have almost 40 [...]]]>

Sainsburys Home Page

The third part of my series on looking at supermarkets web sites to see how they use 404 pages to direct users if they get lost and so make sure that the overall user experience is a decent one.

The first two parts investigated Asda and Tesco who, between them, have almost 40 web sites. Sainsbury’s, Waitrose, Morrison’s, Somerfield and the Co-Op have a lot less and so this post should be a bit shorter as a result.

I’ll just quickly mention that if you are not sure what 404 pages are and why web sites should have them (especially major UK retailers) then you can read my introduction to 404 pages here.

Of the 5 supermarkets that we are going to have a look at today, let’s start with Sainsbury’s.

Sainsbury’s

A total of 9 web sites for Sainsbury’s with a couple of smaller sites crawling out of the woodwork.

Sainsburys – www.sainsburys.co.uk

The Sainsbury's 404 page - not a good example

This is a strange one indeed, the main Sainsburys web site presents different messages or outcomes depending on the URL that you are having a problem with. For instance, if you go to http://www.sainsburys.co.uk/testing in the root of the directory structure then you are redirected back to the home page, which is not always helpful but isn’t as big a deal as if you try http://www.sainsburys.co.uk/sol/testing, which gives you a bit ‘Not Found’ error message, completed unbranded and unhelpful for the user.

Finally, if you try going to an incorrect product page such as http://www.sainsburys.co.uk/sol/shop/home_and_garden/cookware/testing.html, then you get a different type of server default 404 page, which is also completely unbranded, unhelpful and doesn’t present the Sainsbury’s web site in a good light.

J Sainsbury Corporate Site – www.jsainsbury.co.uk

The corporate site leads the way with its 404 page use

The corporate site offers a much better 404 page that presents the user with a search first of all and links underneath to a site index and other main pages within the web site. This is kept within the main navigation and footer and is overall a pretty good 404 page. Why can’t the main Sainsbury’s web site be similar to this?

Sainsburys Energy – www.sainsburysenergy.com

The Sainsburys Energy 404 page is very poor...and EDF branded

Sainsburys Energy is provided by EDF Energy and the 404 page is completely EDF Energy branded and totally unhelpful. The user is presented with a logo, an ‘Our Apologies’ heading and 2 reasons why you may have be shown this page, either the user mistyped or the page doesn’t exist.

There are no links to further pages or even back to the home page, no search, no sitemap, no customer service numbers or email addresses, etc. The logo does link to the Sainsburys Energy home page and users do often try clicking on a logo but this 404 page could be greatly improved.

Sainsburys Bank – www.sainsburysbank.co.uk

Sainsbury's Bank offers a good example of a 404 page

Branded as Sainsburys Finance, this 404 page is straightforward enough with the customary error message (that runs outside of the its container on the page) and then links to the home page, site map page and then links to their main product areas. Tucked away in the top left are Sainsbury’s Quick Links, which upon clicking opens a panel laid out like a sitemap page.

Sainsbury’s Business Direct – www.sainsburysbusinessdirect.co.uk

Not any better than a server default 404 page

To be frank, a horrible 404 page. This one tells you the page is not found and gives you a link back to the home page. No explanation or message plus completely unbranded, which makes it look like a server default page although I don’t think it is. This one needs a lot more effort to become a decent 404 page.

Sainsbury’s Gifts – www.sainsburysgifts.co.uk

Sainsbury's Gifts - same terribleness as with Business Direct

The Sainsbury’s Gifts web site looks pretty similar to the Business Direct site and, surprise surprise, the 404 page is similarly rubbish. Unfortunately, it is exactly the same as the poor Business Direct 404 page.

Sainsbury’s Calais – www.sainsburyscalais.co.uk

If I can't find what I'm looking for, I won't be going to Sainsbury's in Calais

A pre-order web site for getting your wine and beer sorted before you get to Calais. I noticed a couple of problems with the web site when I had a quick look around and was not surprise to find that there is no custom 404 page, just the server default kind.

Sainsbury’s Compare and Save – www.sainsburyscompareandsave.com

Sainsbury's Compare and Save's poor attempt at a 404 page

Sainsbury’s Compare and Save compares broadband, digital and phone packages and is powered by Simplify Digital. I was worried about this web site, as it doesn’t even have a title tag specified for the home page but there is a custom 404 page. However, all the 404 page consists of a large, ‘Whoops! We can’t find the page you are looking for.’ with nothing further. Whoops indeed.

If I try going to an incorrectly typed URL such as http://www.sainsburyscompareandsave.com/testing.aspx, I get a server error page, Whoops again.

Sainsbury’s Little Ones – www3.sainsburys.co.uk/littleones

Sainsbury's Little Ones generic 404 page not good enough

The Little Ones web site is a standalone web site even though its URL is on the back of the main Sainsburys domain name. The 404 page is unbranded and gives no less that 6 reasons why a user may have found it. There is a link to the Little Ones home page and I am sure that the final message of, ‘If difficulties persist, please contact the System Administrator of this site.’ is useful to somebody but not me. How would I go about doing that then?

Waitrose

Waitrose operate a total of 4 web sites that I could find without heading off into John Lewis Land.

Waitrose – www.waitrose.com

Waitrose has an odd 404 page

A slightly odd 404 page, you get a nicely short and apologetic message with a request that you use the search facility (which you have to find yourself) or the site map. The site map is underneath but this is quite a lot of white space and so if you were viewing the Waitrose site on a netbook or small laptop (such as a Macbook like I was using) then you can easily miss the fact that the sitemap is actually present just underneath where you are viewing.

Waitrose Deliver – www.waitrosedeliver.com

Waitrose Deliver delivers hard to read 404 page

The Waitrose Deliver 404 page is not great, you get a hard to read error message in light grey text on a white background, an error message with a link to the home page plus a customer services telephone number and an email address. No navigation, no link to a sitemap, no search and no footer links plus did I mention the text was a bit difficult to read?

Waitrose Wine Direct – www.waitrosewine.com

Waitrose Wine Direct offers mature and full bodied 404 page

Waitrose’s Wine web site’s 404 page requests that you try using their search in the top right hand corner of the screen, provides links to the home page and to the customer services section and asks if you type in the URL correctly. The main navigation and footer navigation is all present too, which means that the user should have a fair chance of finding what they were looking for. A good 404 page.

Waitrose Jobs – www.waitrosejobs.com

Waitrose Jobs has a non-existent 404 page

This John Lewis branded web site does not have a custom 404 page and as such delivers a nasty server default page if a user gets lost, which is of no benefit at all.

Morrisons

Just the single web site for Morrison’s so let’s hope they make a good stab of it.

Morrisons – www.morrisons.co.uk

Morrisons 404 page - what 404 page?

Oh dear, no custom 404 page on their web site, if a user stumbles across a page not found error they will hit a default server 404 page. In my view that is offering a terrible user experience.

Somerfield

Not one of the major supermarket retailers and recently merged with the Co-Operative, Somerfield have just the one web site.

Somerfield – www.somerfield.co.uk

Somerfield's 404 page is a pretty good one

A straightforward 404 page set within the site so the main header including site navigation is shown across the top and the footer is displayed underneath. The 404 message itself is fairly simplistic with links to the home page and, for some reason, to the Somerfield Corporate site, which now redirects you to the Co-Operative’s Group portion of their site, as the 2 companies have merged.

The Co-Operative

Similar to Somerfield, perhaps that is why they have just merged, the Co-Operative has a single web site.

The Co-Operative – www.co-operative.coop

The Co-Op dishes out a decent 404 page

A fairly good 404 page, tells you why you have been presented with the page and what you can do about it including links to the site map, a link to the home page and also telling you to try the search on the home page. If that wasn’t enough there is a telephone number and a customer services email address.

There aren’t many 404 pages that we’ve seen that are better than this in my view.

Conclusions

A mixed bag of supermarkets gives us changeable results in the area of 404 pages. Sainsbury’s had 2 good 404 pages (on the Corporate site and Bank web sites) but the rest were pretty poor including their main web site. Waitrose wasn’t that much better with their Wine site being the highlight whilst Morrison’s was a complete non event. Somerfield’s 404 page was straightforward and the Co-Op’s was pretty good.

Overall, a sense of disappointment (again) with the major supermarkets and how lacking their sites are when it comes to the best practice use of something as simple and basic as a 404 page. Full marks go to the Co-Operative.

This is the second in my series of looking at supermarkets and how they use 404 pages to direct users if they get lost and so improve the overall user experience.

The supermarket that started this series off was Asda and the poor or nonexistent 404 pages that I found across a lot [...]]]>

Tesco.com web site

This is the second in my series of looking at supermarkets and how they use 404 pages to direct users if they get lost and so improve the overall user experience.

The supermarket that started this series off was Asda and the poor or nonexistent 404 pages that I found across a lot of their sites. You can read that post here and if you do it may help you to understand the context of this one.

If you are not sure what 404 pages are and why web sites should have them (especially major UK retailers) then you can read my introduction to 404 pages here.

So last time it was Asda and this time around we are reviewing Tesco’s web sites, the biggest UK supermarket chain.

Tesco’s Web Sites

Tesco doesn’t have as many individual web sites as Asda with around 14 in total, as opposed to Asda’s collection of 25 web sites. I have included each of Tesco’s 14 web sites in the analysis below.

No 404 Page

As with 9 of Asda’s web sites, there are a good proportion of the Tesco sites that do not have a custom 404 page. This means that when a user hits the 404 page they will just see a very unfriendly and unbranded server default 404 page, which is of practically no use to a lost user at all.

The server default 404 page was true of the following Tesco web sites:

1. Tesco Direct

Tesco Direct 404 page

Tesco Direct is a large online catalogue web site with a huge range of products. It is likely that an incorrect link or mistyped URL could result in the user hitting a 404 page, which is an unsightly Internet Information Server (IIS) server default page. Consider the Asda Direct web site from my previous analysis, which shows just how a nice custom 404 page could work for an online catalogue web site with main sections broken down into categories, almost like a site map. This gives users the ability to head to the next most relevant destination after they hit a 404 page rather than just an unfriendly default page.

2. Tesco Get Involved

Tesco Get Involved 404 page

This is Tesco’s community web site, demonstrating what Tesco is involved in, their charity work and urging us to volunteer. The web site uses the standard IIS 404 page but if you mistyped the URL so perhaps tried http://www.tescogetinvolved.com/Homes.aspx instead of http://www.tescogetinvolved.com/Home.aspx you get an ever more horrible ‘Server error in ‘/’ Application’. Sounds horrendous doesn’t it?

3. The Nutri Centre (Tesco)

Nutri Centre 404 page

The Nutri Centre have several stores within Tesco Extra’s up and down the country. Their web site, however, does not have a custom 404 page and is another odd result if you mistype a URL. For instance, if I mistype the About Us page URL and enter http://www.nutricentre.com/t-abouts.aspx instead of the correct URL of http://www.nutricentre.com/t-about.aspx, I get a page with the content area blank telling me, ‘you are here’ without any message to say what has gone wrong.

In my experience users do try to input URLs directly and can often mistype them. If a page URL is entered that is not recognised then a nice 404 page should appear and not a result that is obviously not intended. If you mistype a URL whilst on a product page then you do get a message telling you that the product could not be found.

The one I find the most surprising from the above list is Tesco Direct, which is a huge web site and could really make use of a decent 404 page to direct users to all the different sections that make up the Tesco Direct site or to complete a search if they cannot find a particular product that they are looking for.

Mixed Results

These next 2 Tesco web site offer some mixed results in that there is a custom 404 page in there somewhere but it is mixed up with server default 404 pages and some other weirdness depending on what you stumble across.

1. Tesco Entertainment

Tesco Entertainment 404 page

The Tesco Entertainment web site has recently been relaunch and sports a nice new design. Previously, the web site just had a server default 404 error page and so would have appeared in the first section of this post. However, the new web site has a custom 404 page, wahey!

Hold on, don’t get too excited as there is a bit of weirdness ahead. There is a custom 404 page in the Home section that tells me that something is wrong if I try to go to a URL such as http://www.tescoentertainment.com/store/browse/home/testing/. It is not a great example but we’ll take it at this stage.

If you go back to the root of the site and try http://www.tescoentertainment.com/testing/, you get a horrible server default page again. Why doesn’t the correct 404 page display here?

2. Tesco Photo

Tesco Photo 404 page

Tesco Photo is Tesco’s photo processing web site and looks like the older style Tesco web site. There are certainly a couple of odd things that I would hope they could eradicate if Tesco pushed a new photo site live.

The start page is http://www.tescophoto.com/wpp/tesco/welcome.jsp so if we amend that to http://www.tescophoto.com/wpp/tesco/testing we get a nasty server default 404 page. Going to the root (http://www.tescophoto.com/testing) as we did with Tesco Entertainment also produces the same server default 404 page, good consistency so far but bad 404 page.

Let’s just start this next bit by stating that all URLs as far as I could tell by a quick browse ended with an extension of .jsp. If I pretend to mistype a URL such as http://www.tescophoto.com/wpp/tesco/view_tour2.jsp instead of http://www.tescophoto.com/wpp/tesco/view_tour.jsp I do get a custom 404 page (again, why can’t this 404 page display for the entire site?) with a message that tells me to make sure that the URL has the correct extension and goes on to say that all URLs have the extension .html and not .htm.

Not so I’m afraid, as the site uses .jsp and not .html or .htm. Even if I amend my original URL to http://www.tescophoto.com/wpp/tesco/view_tour.html or http://www.tescophoto.com/wpp/tesco/view_tour2.html I am returned to the server default 404 page and lose the slightly nicer custom 404 page.

I know that this is just a case of the Tesco Photo site updating its error messages but I think it does highlight the importance of what you are telling lost users to do. In this case, a user would get so confused by this message and I don’t think it would make any sense to them at all.

So that is 5 web sites out of a total of 14 that either do not have a custom 404 page at all or have mixed results. This equates to 36% of Tesco’’s web sites, which is exactly the same as Asda’s 36% of web sites not having a custom 404 page.

Custom 404 Pages

Right, we are now onto the the next 9 of Tesco’s web sites, which all have custom 404 pages in place. Let’s see how good they are.

1. Tesco

Tesco 404 page

The 404 page on the main Tesco web site is a very basic 404 page but it does say sorry, explains what may have happened and gives you a link to the home page plus a customer service number and another link to an online enquiry form.

The enquiry form link is a bit odd because it takes you to a page with lots of phone numbers and postal addresses but no online enquiry form that I could fine. At the least the home page link does take you to the Tesco home page.

I think that Tesco are missing an opportunity to better direct their site visitors that may be getting lost and perhaps a link to their sitemap page here http://www.tesco.com/shopping/ would be more useful.

2. Tesco Finance

Tesco Finance 404 page

Offers an abrupt, ‘The page you are trying to reach does not exist.’ message and a link back to the Tesco Finance home page. Not the most polite of 404 pages but the main navigation is present, footer links plus a ‘Got a question’ link that opens an FAQ in a new window. Overall it is better than a lot of Tesco’s web sites but could be nicer to people who are a bit lost.

3. Tesco Compare

Tesco Compare 404 page

A reasonable stab at a 404 page with a short message and a link to the site map. Links to the main sections of the web site are presented below the message plus the main navigation and footer is retained so a lost user should be able to find roughly where they need to go.

4. Tesco Insulation

Tesco Insulation 404 page

Tesco also sell insulation for the home and the site does have a custom 404 page but in all honesty it is not much better than a server default page. The page is not branded as Tesco but rather as a company called Creo, which I believe must be their hosting provider. The 404 page itself is also not especially helpful in that it has no links back to the home page, or to a sitemap, or to complete a search, and so offers no assistance to help the user in where they might want to go next.

5. Tesco plc

Tesco plc 404 page

The Tesco plc web site is Tesco’s corporate home, covering their corporate responsibility, investor information and media details. The 404 page for such an important web site is straightforward and contains the details that you would expect including a standard error message with a link back to the home page. The site displays this set within the site navigation so that a user can still navigation to any section, to the sitemap or complete a search.

Overall, this 404 page is absolutely fine.

6. Tesco Diets

Tesco Diets 404 page

The Tesco Diets web site is concerned with online dieting, weight loss and healthy eating. It has a 404 page but this is branded as eDiets and not Tesco Diets. The text on this 404 page is about eDiets including links to the eDiets home page, eDiets news page and an eDiets email address. Clicking on the link to the eDiets home page actually takes the user back to the Tesco Diets home page. Is that confusing for a user do you think? I think so too.

7. Tesco Opticians

Tesco Opticians 404 page

The Tesco Opticians web site uses the same layout for their 404 page as the main Tesco web site. So there is some consistency at least between the two web sites, which is a good thing. Other than that the comments are the same in that the 404 page is very basic, it does say sorry, explains what may have happened and gives you a link to the Tesco Opticians home page plus a customer service number. In this case there is no link to an enquiry form.

8. Tesco Energy

Tesco Energy 404 page

Tesco Energy explains iteself as being an energy price comparison and carbon offsetting service. The 404 page displays a logo for Xelector (who much provide the price comparison engine that this web site is based on) and so is not Tesco branded. Furthermore, the 404 page is very basic, it consists of a short message and an email address to contact if you think you have stumbled across an error. However, the email address isn’t Tesco’s either, it is webmaster@xelector.com.

As a user of the Tesco Energy site, I would expect that if I get lost and find myself on the 404 page then I would see a Tesco Energy 404 page with Tesco Energy contact details and/or links back to the Tesco Energy web site. I don’t expect to be presented with whoever Xelector are and what they have to do with Tesco Energy.

In my view, this 404 page does not really help the user at all so there is little point in having it.

9. Tesco Travel Store

Tesco Travel Store 404 page

The Tesco Travel Store presents another 404 page with no Tesco branding and not much help. The error message tells you to click back or close down your browser (and when I open it again will I go back to the Tesco Travel Store? Not likely) or ‘click here to go back’. The click here link does not work.

I notice that the 404 page has a Lastminute.com favicon and this is consistent with the Tesco Travel Store being powered by Lastminute.com. I would expect a much better 404 page to be honest, as the Lastminute.com web site has quite a good 404 page.

Conclusions

So that concludes my roundup of Tesco’s 14 web sites and their use (or lack of use) of custom 404 pages. Overall, I am really disappointed that such a major UK retailer seems to be struggling with basic good practice in the proper use of 404 pages and the benefits that they offer to users of their web sites.

Considering the 14 web sites, I found 3 sites to have no 404 page, 2 to have a kind of strange halfway house approach to 404 pages and 9 to offer a semblance of a 404 page but Tesco Insulation, Tesco Diets, Tesco Energy and Tesco Travel Store were all awful examples.

From the remaining 5 web sites Tesco.com and Tesco Opticians had OK 404 pages and Tesco Finance, Tesco Compare and Tesco plc were all quite good. So from the original 14 web sites, only 3 were found to have quite good 404 pages but even those could be better in my view.

Next up we’ll have a look at Sainsbury’s, Waitrose and Morrison’s.