According to statistics from W3Schools.com, usage of Google Chrome has now risen to 16.7%.

That figure is in many ways not surprising because Google Chrome is an excellent web browser, is fast and does not crash as frequently as other browsers. What is surprising is that, whilst Chrome [...]]]>

Are you testing your websites with Google Chrome?

According to statistics from W3Schools.com, usage of Google Chrome has now risen to 16.7%.

That figure is in many ways not surprising because Google Chrome is an excellent web browser, is fast and does not crash as frequently as other browsers. What is surprising is that, whilst Chrome is still a long way behind Firefox at 46.4%, it is now ahead of IE8 (15.6%) and a long way ahead of IE7 (7.6%) and thankfully IE6 (7.2%), which is finally fading away.

Google Chrome’s share of the market continues to increase whilst all the major web browsers, except Opera, which picked up from 2.1% to 2.3% over the last month, are on a steady decline, including Firefox and Safari.

Now I’m not suggesting that this trend will continue, browser usage does fluctuate as new versions come out and entirely new browsers are launched.

What I am getting at in terms of Google Chrome’s greater market share, up from 10.8.% in January to 16.7% in July, is to make sure you are consistently testing your sites in Chrome.

Chrome Ahead of IE8

As I pointed out in a recent Tweet (please follow TestingWebSites by the way), the number of people using Google Chrome is now above IE8 and so you should be regularly testing in Chrome to pick up on any layout inconsistencies.

And there are quite a few that I have spotted when testing newly launched websites recently.

My belief is that the surge in Chrome usage over the last few months had caught a few web designers and developers out who traditionally test in IE6, IE7, IE8 and Firefox. I know from recent work with several digital agencies plus graphic design agencies and freelancers that there are still many who do not always test in Google Chrome.

Test With Google Chrome

If you don’t have Google Chrome installed already then you can get it from Google.com/chrome, available for PC, Mac or Linux. You can install it alongside Firefox and IE8 on a PC so can have all 3 accessible at once.

Google Chrome uses the WebKit rendering engine, which is also used by Safari and a number of lesser used browsers. It is possible that by testing on Google Chrome that you may find issues also associated with the Safari browser.

If for some reason you don’t want to install Google Chrome then using a browser checking tool such as Adobe Browserlab or CrossBrowserTesting.com will help you pick up on any issues.

I recently covered 3 browser checking tools or you can view all 9 browser testing tools in the Testing Tools Directory, around half of which support Google Chrome although none yet support Chrome on a Mac as far as I can tell.

Do you currently include Google Chrome in your browser testing?

Browser checking tools are important and can be a life saver, as they allow you to test or check your website in a variety of different web browsers.

It is not possible for many web designers and developers to [...]]]>

Browser checking tools allow you to test your website in a variety of different web browsers easily

Browser checking tools are important and can be a life saver, as they allow you to test or check your website in a variety of different web browsers.

It is not possible for many web designers and developers to have physical installations of each browser type and version on each operating system out there. I do know of digital agencies that have a number of testing PCs dotted around the development studio for quick testing of work or perhaps a testing server with 8 or 10 different browser versions and operating systems installed.

But for many of us, especially freelancers, having this number of browsers and operating systems at our finger tips is not possible unless we fork out money for the hardware ourselves and install the required browsers.

For instance, I have 3 computers, running Windows 7, Windows Vista and Mac OSX 10.5. On those computers I can test using the following browsers – IE7, IE8, Chrome, Firefox 3.6 (PC), Opera 10, Safari 3 (Mac) and Firefox 3 (Mac).

So that is a decent number of browsers, but there are times when I need to test on IE6 or a different version of Firefox, Safari or Chrome that I don’t have running on any of my machines. Plus, when new browsers are released, I have to decide whether I can accommodate it within my existing setup or whether I need to do things differently.

Thankfully, the last couple of years have seen a number of good browser checking software solutions launched, which allow me to check the website over in pretty much whichever browser I want installed on any of the main operating systems.

Here are 3 of the most popular browser checking tools taken from the relevant category of our Testing Tools Directory, which contains 9 browser testing tools in total.

CrossBrowserTesting.com

CrossBrowserTesting.com allows live testing and automated screenshots

Allows live testing on an OS and Browser of your choice plus the ability to take screenshots across a range of browsers.

Features:

• Offers live testing – for in depth testing, using the browser you need remotely
• Automated screenshots – to identify display issues
• From 150 minutes per month
• 1 week free trial
• Pricing starts at $20 per month

When you sign up you get access to the live testing and automated screenshot features, whilst all packages come with a 1 week free trial. Packages are based on a usage allowance with $20 per month getting you 150 minutes.

Browsrcamp

Browsrcamp focuses on Mac browsers

Focusing on web browsers running on a Mac operating system, such as Safari, Firefox and Flock, which allows those who don’t use a Mac to test the compatibility of their websites on Mac browsers.

Features:

• Gain access to 12 browsers on a remote machine including Safari, Firefox, Opera and Internet Explorer plus other lesser known browsers.

With prices starting at $3 for a 2 day subscription, Browsrcamp represents good value for money for website testing on a wide range of Mac browsers.

Adobe Browserlab

Adobe Browserlab is free for a year if you sign up to CS Live before the end of April 2011

Now a part of Adobe’s CS Live, Browserlab integrates with CS5 allowing you to access it from Dreamweaver and starting testing straightaway.

Features:

• Zoom – allows you to zoom in to see more detail and spot bugs more easily
• Rulers and Guides – display rulers along the x and y axes and display a guideline at the point of the screenshot
• Screenshot Delay – you can take a screenshot several seconds after the page loads allowing time for flash animations to start to play and show content
• Customisable settings – configure up to 10 browsers sets for different projects
• Multiple viewing modes – compare 2 browsers side by side or overlay one over the top of another with the Onion Skin view
• Covers all leading browsers

If you sign up by end of April 2011 you get Adobe Browserlab free for a year as part of CS Live, which also includes CS Review, Acrobat.com and a couple of other online services.

Conclusion

I think each of the 3 browser testing services has its merits. Browserlab from Adobe has some good features, getting free use for a year is also a big bonus and, even if you don’t use their Creative Suite then Browserlab is still a good standalone testing suite.

CrossBrowserTesting.com was pretty much the first service to move beyond just taking screenshots and allow you to access the browser itself to conduct real testing. They have the widest range of operating system and browser combinations and so are worth a look from that point of view.

I would use Browsrcamp for any specific Mac browser that you can’t get on either Browserlab or CrossBrowserTesting.com or if you just need access for a short period of time, as $3 is peanuts for 2 days access.

I thought I would share this small piece of advice. It might be obvious to a lot of people but is not always apparent or sometimes we forget or even run out of time to pay enough attention to it.

The advice is simply this:

When you report a bug or issue and the developer tells you [...]]]>

I thought I would share this small piece of advice. It might be obvious to a lot of people but is not always apparent or sometimes we forget or even run out of time to pay enough attention to it.

The advice is simply this:

When you report a bug or issue and the developer tells you it’s fixed, always check that it is actually fixed.

Sounds obvious doesn’t it?

Think back to times when you have taken the word of the developer that a particular bug has been fixed and decided not to test it or verify that it had actually been completed. Ok, it sounds a bit strong that you knowingly decided not to test the bug fix, you could have been swayed by a couple of factors. Perhaps that developer has always fixed bugs satisfactorily in the past or maybe you didn’t have the time to check the bug had been fixed properly and so gambled that it had been fully dealt with.

Whatever the reasons the result is that the bug fix did not get checked out and verified. For many issues, the fix put in place would simply have worked and that would be the end of the matter. That particular bug would never be heard of again, the gamble paid off.

But what are the chances that the fix did not work, or created a new previously undetected bug or that the developer misinterpreted the bug report and his or her fix is not what you expected. In my experience, the chances of things like this happening are pretty good (website testers are such a pessimistic bunch).

On occasion, bug ‘fixes’ like this actually have a bigger impact on the website than the original bug. They can make the situation worse rather than better. A minor bug that is improperly fixed can have a much bigger effect and without testing and verifying it properly you don’t know if that has happened this time or not.

Fixing A Bug And Making It Worse

To illustrate how it is easy to fix a bug and make it worse, consider the following.

Some work I have been managing recently involved using a fairly simple script. The way the script works is that you send a series of locations to it, in the form of a CSV file, and the script fetches data about each of those locations, which is then uploaded to an email marketing system.

The data that is returned by the script is dropped into personalised emails, which are regularly sent to thousands or even tens of thousands of people at a time.

The script is not the greatest of solutions but it is a fairly quick and straightforward piece of programming, which generally works well.

I reported a bug that for one particular location, no data was being returned by the script. We had to manually add the data for that one location, which was frustrating and time consuming but not the end of the world. A developer investigated the bug and soon after pronounces the issue fixed. The information is relayed to me that the bug is fixed. Everyone rejoices (not really, everyone just gets on with the next job).

When I check the script over, send some location data to it and review the results I find that data is indeed now being returned for the offending location. However, it is the wrong data. There are 2 locations in that particular city and the script is returning data for the other location.

Just to recap, the original bug was that no data was returned by the script for that single location. An inconvenience in that the missing data had to be manually added. The bug ‘fix’ has actually introduced a 2nd bug, which is that the incorrect data is now being returned for that single location, a more significant bug than the previous one if it went unnoticed, as thousands of individuals would have had incorrect personalisation details incorporated into emails they received from the organisation creating a great deal of confusion.

It just goes to show that when someone tells you a bug has been fixed, please always check. Something may indeed have been fixed, just not always what you anticipated. On occasion, the situation may have actually been made worse and not better.

I am sure that there are lots of similar stories to this one where a bug fix did not fix the bug but perhaps made it worse. Feel free to share any in the comments.

Image credit BenSutherland

This ad hoc approach to testing means that it is possible to miss [...]]]>

As a website is being built, the step of writing or putting together a test plan is often skipped entirely and testing just ‘begins’. If a test plan is done then it is often completed immediately prior to testing and completed too quickly.

This ad hoc approach to testing means that it is possible to miss large sections of functionality and not be able to fully consider all permutations or variations of events and actions that need to be tested.

For most websites, there is a fairly long list of items that need to be tested as part of any new website development and in order to know what to test, when you tested it and whether those tests passed or failed, it is best to make a proper plan.

We are going to show you how to write a simple and straightforward test plan, which will provide a more systematic method for testing your website but the process outlined below is still essentially a manual one. We are not yet moving into using software tools to help identify and run tests, nor are we covering automated testing here, although I hope to write a post on these aspects in due course.

Writing A Test Plan

A typical test plan should identify all the areas of the website that you need to test before it launches, which you then follow through to conduct your testing, making notes of when you completed the test and adding any bug or issue reports to your bug tracking software along the way.

Start off with getting all the main sections down in a spreadsheet, divided between front end and admin or CMS if the website has one.

Then, start to break each main section down into the constituent parts that you need to test, concentrate on the important parts of the website first such as the home page, which is likely to be one of the most visited pages and the main areas of functionality, which are most likely to be the most used.

For instance, in the case of an ecommerce site, this would include individual tests for items such as adding products to the shopping basket, removing products from the shopping basket, updating the shopping basket quantities, being able to enter the checkout process and completing the checkout process (depending on your website there may be several steps to test within the checkout process).

You should be asking yourself questions for when you start testing the checkout process from the point of view of the user such as:

• Can I complete my order?
• Can I go back and amend my order?
• Do I understand where I am in the checkout process?
• Can I clearly see what I am ordering and how much it will cost, including delivery and taxes?
• Do I get a receipt or email confirmation?

As the owner of the website there will be different questions such as:

• Does the payment processing work correctly?
• What happens if the incorrect card details are entered?
• Does the order confirmation process work correctly?

And so on, when putting a testing plan together you are testing the website from the point of view of the user and also from the point of view of the owner or operator of the website. You may also have specific usability testing lined up but the primary reason for the test plan as far as this blog post is concerned is to test the functional aspects of the website to ensure they work correctly. Does the website do what it says it does? For instance, if the checkout process displays the order details including taxes and delivery then does it show the correct details for both tax and delivery?

Another important area to test is the site search or product search, as this is likely to be used a lot on the website.

Make sure that you test all the forms within the website, such as enquiry forms and newsletter signup forms. Test that the form can only be submitted after the mandatory fields have been completed and that the form is emailed to the correct email address or saved to the database correctly.

Once you go through and think about each section of the website along the lines of the above then you should have compiled a decent list in your spreadsheet of areas and items to test. Add columns for each browser that you need to test each item on, which will be the web browsers that you have agreed to support as part of the project. This would normally cover the major web browsers such as IE6, IE7, IE8, Firefox 3, Chrome and Safari but may include others too depending on the usage of your target market (or existing audience if you have those statistics). There are a range of browser checking tools that can help you with this aspect.

For the admin system or CMS that powers the website, add a row to your test plan for each function. Going back to our ecommerce example, this means that there would be tests for add new product, edit existing product, remove product, view order, edit tax rate, etc. depending on what the system was capable of.

You could break these tests down further to test individual elements of adding a new product. For instance, is there validation present to check that mandatory fields are filled in when adding a product? Does the product image have to be a certain size or dimensions? Does the product title have to be shorter than a certain number of characters? All these items could occupy separate rows in your test plan.

Of course, if you were to break your test plan down to this extent for a large website or CMS then the result will be a long spreadsheet of items to test, which will take a great deal of time to test effectively. You will need to decide whether you have the time available to test in this level of detail or if some areas can be tested more quickly.

A few other items that you may wish to make sure you have as part of every test plan are the following, in no particular order:

• Ensure your HTML and CSS code validates to W3C guidelines
• Test accessibility of the website
• Test the SEO items that you have planned for to make sure they are in place
• Test against the specification for the project that was signed off to ensure that everything has been completed that was agreed
• Test against designs and/or wireframes or prototype html mockups that were completed to make sure the finished website matches the signed off designs or wireframes
• Take into account any changes requested (should be in the form of change control documents)
• Complete any security tests that are required

You may wish to have a look at our list of 10 different forms of website testing to make sure you have covered everything. The list can go on indefinitely, there is always something that can be tested a bit more, but at some point it has to be halted at a relevant point otherwise the testing will never get done.

List or Flow Chart?

Some aspects of the website lend themselves to having a flow chart in place that describes all the permutations or possibilities that need to be tested. This is especially helpful if one action then presents another set of possibilities. When testing without a flow chart it can become extremely difficult to remember which permutations you have tested and which you have not.

Having a flow chart or mindmap worked out that shows you all the possibilities allows you to think more clearly about what you are testing, what you expect the outcome to be and understand what you have tested and what is remaining to test.

Conclusion

If there is too much testing to do for one person then divide it up amongst other people with instructions on what you want testing and what to do with bugs when they are found. Make sure they follow their bit of the test plan and feed back on what they find, handing bugs to you or inputting them into your bug tracking software themselves.

As you test then fill in the test plan to show what you tested and when. If you keep the document client friendly then it can be shown as proof of what testing was completed prior to launch.

Testing is a vital part of any website development and having a decent plan before you start testing saves a great deal of time and effort whilst making sure that all the required testing gets done.

I go on to suggest some solutions but what I really [...]]]>

Over on my main company website and blog, www.webdepend.co.uk, I’ve put up a post aimed at graphic design agencies but may relate to many types of agencies or organisations that produce websites and experience bottlenecks when it comes to fully testing and launching those sites.

I go on to suggest some solutions but what I really wanted to put across to graphic design agencies are what I see as the benefits of outsourcing their website testing to an external company.

The benefits put forward in that post may be arguments that you agree or disagree with. Either way, I would be interested to gain your feedback – would you consider or do you outsource any testing to another party or do you intend on making sure that all testing remains in-house? What are the pros and cons of either approach in your view?

You can read the full post here:

http://www.webdepend.co.uk/2010/05/08/website-testing-problems-for-graphic-design-agencies/

One of the first posts I wrote, which described a typical Web Testing process, mentioned that a bug [...]]]>

An area that has not been looked at in much detail yet on this blog is what software or tools you can use to record and track bugs, issues or defects that you find when testing a website.

One of the first posts I wrote, which described a typical Web Testing process, mentioned that a bug tracking system ‘can be hugely beneficial, as you can set priorities for each bug and then understand how many bugs you have at each priority level. It gives you a much better understanding of how many bugs you have to fix before you can launch and calculate how long you expect that process to take.’

There are a good range of bug tracking applications out there, a lot of them web-based, which are vital when tracking bugs and associated information about each issue. Here we take a look at 4 of the main players in bug tracking software, which are also available in our Testing Tools Directory.

Why Use Bug Tracking Software?

Once you start testing a website you can come up with all sorts of bugs and issues that need addressing. I’m talking a bit more than the odd typo or broken image. Perhaps you find problems with the site search returning strange results or the checkout process not working correctly or there are specific issues in different browsers that work fine in other browsers. There are several different reasons to use an application that tracks bugs and feature requests.

Complicated Bugs

As bugs start to get a little more complex it is important to hand the developer all of the required details about each bug so that he or she can fix it in a reasonable amount of time, without chasing around trying to reproduce it or get a screenshot or URL showing where that bug was found. Bug tracking software gives you a structured format to report bugs so that you include the URL where you found the bug, provide a full description and a screenshot and what browser or operating system you were using. These details will shorten the amount of time that a developer needs in order to squash the bug and improve their productivity. Get more bugs squashed in less time.

Lots of Bugs

Alongside the complicated bugs will still be the relatively quick and simple items to fix such as the odd layout problem, typo, broken image or link, etc. These bugs, whilst they do need to be fixed, are sometimes not as important as other bugs and, when presented in a list or on a spreadsheet, can often be completed ahead of more important items because the developer naturally follows the list through from start to finish. Bug tracking software allows you to prioritise bugs so that developers work on the most critical issues first and leave the less important bugs until later. Rearranging priorities in a list or on a spreadsheet gets too cumbersome when dealing with a large number of bugs but issue tracking software handles this aspect with ease.

Improved Workflow

With bug tracking software you can assign bugs to different developers so everyone knows what they are working on. Once those bugs are completed then the developer assigns them back to you for checking or verifying. If you are happy you can close the bug, if not then reopen it and give it back to the developer. Do this for each bug and you can understand exactly where each bug is in the process, how many are assigned, how many are to be verified, etc. It allows you to have a much better handle on exactly where your web project is at instead of continually poring over your list or spreadsheet and trying to make an educated guess.

This improved workflow eases pressure on you and your developers and improves productivity because developers can just get on with fixing bugs or working on feature requests. It also means that you are not constantly updating lists and spreadsheets with the latest status of all the bugs reported.

Bug Archives

Once bugs are fixed, tested and closed they can be referred to at any time so that you can see when the bug was reported, when it was fixed, who fixed it and what the developer did to fix it (assuming they fill their comments in correctly). This provides an extremely useful archive if you ever need to refer to a particular bug again instead of trying to go back through emails or out of date spreadsheets to find details on that bug or feature request.

Some bug tracking systems allow time tracking so you can see how much time was spent on fixing a bug. Using bug tracking software ensures that developers become more accountable because their comments and times are recorded and can easily be referred to.

Bug Tracking Software

Now that we’ve established what bug tracking software offers and why web project managers and testers should incorporate it as part of most web projects (definitely on all large web projects and probably on most small to medium projects too) let’s have a look at some of the main bug tracking applications currently being used.

My specific experience is with using Bugzilla so let’s start with that one.

Bugzilla

Bugzilla - classic open source bug tracking application

Bugzilla was originally released by Netscape in 1998 and so has been around for a long while now. This open source software is one of the first bug tracking applications that spring to mind and, although it takes some setting up and the interface is a bit tricky to use, it has everything you need to record and track bugs effectively.

It is possible to sidestep the web interface by using a desktop client that synchronizes with the central bugzilla software. My specific experience with bugzilla was using the Deskzilla client on a Mac, which worked well.

As Bugzilla is open source then it is free to use although it needs to be installed and set up on a server in order to be ready to use.

Unfuddle

Unfuddle - web-based bug tracking software

Unfuddle is a hosted bug tracking application that also includes Git hosting and Subversion hosting allowing you to create your own secure code repositories. There are also some project management tools included such as tracking your project against specific milestones and time tracking is available in the more expensive corporate and enterprise packages.

Bugs, issues and feature requests are raised as tickets in the Unfuddle system and each ticket includes workflow management and an audit trail so you can keep track of who made changes to a ticket and when these changes were made.

If you need the code repositories and project tracking functionality along with a straightforward bug tracking system then Unfuddle could be the answer. There is a free package available that includes 1 project for 2 users. After that prices start at $9 per month.

Trac

Trac - open source issue tracking software with project management tools and a wiki

Trac is also open source software that combines an issue tracking system with a wiki and project management features such as milestones and being able to document the roadmap for a web project or software development. Trac can also serve as a web interface for version control systems giving you a usable front-end for Subversion or Git.

Tracking progress of individual bugs, issues or feature requests is straightforward with each one having its own numbered ticket with the ability to filter tickets by severity, project component, version or owner. Keeping on top of progress is also easy with the ability to set milestones and being able to see how far you are from reaching each milestone.

Trac has a good set of features and is a lightweight and flexible system that should be very useful for most web development projects. It runs on most operating systems but does require some setting up and this is perhaps its biggest downside that I can see if you want to start tracking bugs straightaway.

FogBugz

FogBugz - web-based or installed bug tracking software with large feature set

FogBugz is available as a hosted application, called FogBugz on demand, or as installed software on either Windows, Mac or Unix operating systems.

There are several parts to FogBugz, the issue tracking section makes it very easy to report bugs and you can even create bugs via email or create bugs quickly without having to fill in everything so you can get the bug into the system. Whilst I think this is a good feature you do need to make sure you go back and complete all the details before you assign the bug to a developer to work on. Everything is saved in a complete case history for future reference.

But issue tracking is not all that Fogbugz does, there is a project planning section where you can outline tasks, break larger tasks into subtasks, enter estimates and set milestones. Fogbugz also has something called Evidence-Based Scheduling, which looks at what tasks are being estimated and how long they actually took to complete and then plot a probability curve to work out whether a milestone can be hit or not. The FogBugz Wiki allows you to keep all project documentation in one place and there are also a whole load of plugins available to extend the functionality plus FogBugz can integrate with another piece of software from the same company (Fog Creek Software) called Kiln, which is a version control system.

FogBugz is the most expensive of the bug tracking software applications featured here with prices starting at $25 per user per month for FogBugz on demand or $999 for 5 users for the installed version of the software.

Conclusion

Whichever of the above applications you use, as long as you use one of them and get your workflow organised in terms of how you report, record and assign bugs then you will find that it becomes a breeze to keep track of exactly where you are at in your website testing. You won’t miss any issues (as long as they are reported and entered into the bug tracker), you can prioritise showstoppers or critical bugs easily to fix those first and archive completed bugs for future reference.

If I had to pick one of the applications above then I would go for either Unfuddle or FogBugz (on demand) because I like web-based applications as there is nothing really to set up or install. Both of these have a trial although both cost money whereas Bugzilla and Trac are free but require time installing and setting them up on a server.

FogBugz has the better overall feature set and I am intrigued as to how the Evidence-Based Scheduling works, as I have not come across that before. But it terms of a cheap, easy to use, bug tracking system that is ready to go and will do the job, I would pick Unfuddle with the caveat of going for FogBugz if I had a bigger team (and didn’t mind paying more money).

At the moment that testing subject of choice [...]]]>

When I’m reading blog posts, news articles or Twitter discussions concerning the testing of websites, the article or discussion in question actually relate to a single aspect of testing and gives the impression that there is only one form of website testing that we should be concerned with.

At the moment that testing subject of choice is known as Conversion Rate Testing, or sometimes called CRO (Conversion Rate Optimisation). This would normally mean that the tests being carried out are Split Tests or Multivariate Tests, using software such as Google Weboptimizer.

Don’t get me wrong, conversion rate testing is really important and can greatly increase the number of conversions on a particular page, which can raise sales, enquiries, profits or whatever aspect of conversion you are trying to improve.

But there are other forms of testing websites, quite a few in fact, and some that are quite basic that get overlooked. I wanted to give a brief description for each type of testing to help you understand all the areas that can (and should) be tested when launching a new website, rolling out a set of updates or at least checking on a regular basis.

Different Forms of Testing

Functional testing

This form of testing aims to ensure that the website works correctly or as it was intended to. You click on a link, does the website display the correct page? You complete a search, does the website return the correct search results? You add a product to the basket, does the website add the correct product to the basket? Functional testing is going through the entire website, including content management system or admin area, to make sure that each function within the website is performing as it should be.

Browser compatibility testing

Different web browsers display web pages differently from each other. This is one of the least helpful aspects of the Internet and browser compatibility testing or browser checking involves testing the website on each of the major browsers (or each browser that you have decided to support), which generally includes IE6, IE7, IE8, Firefox 3, Chrome, Safari 3 and Safari 4 on both PCs and Macs. You could possibly include Opera 9 and Opera 10 in that list in addition to testing your website on Linux as well as Windows and OSX. You could also check your analytics to see which browsers are most commonly visiting your site. There are several decent browser checking tools listed in our testing tools directory.

Usability testing

Usability testing assumes that your website works correctly and displays in each major browser correctly. However, a website that functions correctly is not necessarily a usable website for your target audience. Usability testing enables you to show your website to a group of users, ask them to perform certain actions and see how they accomplish those tasks. There are many reasonably priced usability testing tools on the market that have made this type of testing much more straightforward, many of which are listed in our testing tools directory. It is possible to gain great insight into how people actually use your website and a lot of the experts agree that you can pick up most usability issues when testing with only 5 users.

Accessibility testing

Accessibility testing is extremely important to make sure that your website complies with best practice and accessibility guidelines plus legislation surrounding accessible websites, which is in force in both the US and UK. In the US there is Section 508, which is part of the Rehabilitation Act and in the UK we have the Disability Discrimination Act or DDA. A fully accessible website not only benefits those users that would otherwise have difficulty using your website but search engines will also be able to navigate and understand your content more easily, thereby helping your search engine rankings. To help you test your website’s accessibility there are some accessibility testing tools available in our directory although manual testing is still recommended.

Performance testing

Performance testing is an area that has always been important but possibly even more so now that Google is starting to incorporate the performance of a website into its ranking algorithms. There used to be an 8 second rule, which believed that a user will wait no longer than 8 seconds for a page to download before they click back and try another website. That rule is a long way out of date, it is now no more than 1 or 2 seconds. Performance testing can show you when your website is slowing down, which pages are taking time to load and what element of the page is causing the problem. Performance testing tools are covered in our testing tools directory.

Stress or load testing

There is a difference between stress and load testing in that stress testing concerns itself with trying to break the system you are testing whilst load testing aims to find out how much volume you can put through the system. Both of these forms of testing are generally for larger websites and web applications. A range of stress and load testing tools can be found in our testing tools directory.

Site monitoring

A really simple test that you can put in place is to automatically monitor your website to check for downtime or outages. Nobody likes a website to be down including users and search engines. Regular downtime will lower confidence in your website and they could be affecting your traffic levels and therefore sales or enquiries without you knowing. A selection of site monitoring tools can be found in our testing tools directory.

Conversion rate testing

The current favourite form of testing, and an important one too. Conversion rate testing or optimisation can greatly improve the amount of visitors your website converts into enquiries and/or sales. If your SEO brings in 100 visitors a day and your site currently converts 5 of those into becoming customers then, with successful conversion rate testing, you could convert more visitors into customers and make more sales. Conversion rate testing normally takes the form of running split or multivariate tests on your website, many of which are listed in our testing tools directory.

Security testing

Websites can have security holes that could be extremely damaging for your business if they were to be found and exploited. Security testing aims to find where those security holes exist so that you can do something about them.

Spelling and Grammar checking

It is always useful to fully read through all text on the website to ensure there are no typos or grammar mistakes. You will also spot how you can improve the readability of text and other areas that perhaps need some attention. Also make sure you read through all messages that appear when signing up for a newsletter, adding products to a basket, or registering including any error messages. Simple changes to the wording of an error message into a nicely worded error message can make a big difference.

So there you have it, website testing can take on many forms and all of them should be completed to a greater or lesser degree. At least by completing each type of testing you can be aware of any issues or areas for improvement so then properly plan out what action you are going to take and when. Good luck with whatever testing you complete.

I’ve taken some time to update, reorganise and expand the testing tools page so that it become more of a directory rather than just a long list. This makes it easier for me to maintain and add to plus easier (hopefully) for you [...]]]>

Now contains a total of 96 website testing tools spanning 14 different categories.

I’ve taken some time to update, reorganise and expand the testing tools page so that it become more of a directory rather than just a long list. This makes it easier for me to maintain and add to plus easier (hopefully) for you to find the testing tools that you are looking for.

The Testing Tools page now serves as the main entrance into the directory with each category having its own page. Each category will continue to have more software and tools added over time plus there will be additional category or two coming along shortly.

Because I’ve been doing a bit of research into Usability Tools, this category is the main beneficiary of the reorganisation and now contains 29 usability testing applications. This is mostly due to the great blog post at Useful Usability, which also gives you pros, cons and pricing information for the 24 usability tools they feature.

Another category that has grown significantly is Split Or Multivariate Testing Tools, which now shows 16 applications. Some of this increase is down to a great comparison resource over at Which Multivariate? This newly released site has reviews and comparison charts of all the major split and multivariate testing software tools out there.

The full list of categories contained in the updated testing tools directory are:

1. Validation and Accessibility Testing Tools
2. Browser Checking Tools
3. Screen Resolutions Tools
4. Link Checking Tools
5. Site Performance Testing Tools
6. Stress or Load Testing Tools
7. Usability Testing Tools
8. Split or Multivariate Testing Tools
9. SEO Tools
10. Web Analytics
11. Online Surveys
12. Site Monitoring Tools
13. Software Testing Tools
14. Web Development Tools

I will continue to pick up on the huge number of testing tools being released and add to each category as time allows. If you spot a testing tool, which you feel is worth me adding to the directory, then please do contact me. At some point I will get around to adding a dedicated submission form if there are enough people willing to add testing applications.

Feel free to retweet, digg or add to delicious as you like or let me know any feedback (good or bad) in the comments.

I have come to the end of my blog series on how the major UK supermarkets use custom 404 pages to improve the user experience.

The supermarket that started this off was Asda, when I stumbled across the fact that the main Asda web site did not have a custom 404 [...]]]>

Example of www.asda.co.uk's 404 page

I have come to the end of my blog series on how the major UK supermarkets use custom 404 pages to improve the user experience.

The supermarket that started this off was Asda, when I stumbled across the fact that the main Asda web site did not have a custom 404 page and took me to the horrible server default 404 page. At the time I couldn’t believe it, I thought all major retailers would have had this basic principle of web development covered and be doing a good job of 404 pages.

Then I realised that Asda had quite a few other web sites so I started investigating those, found more web sites with no custom 404 pages and the list grew and grew until I had 25 Asda web sites in total, 9 of which do not have a custom 404 page at all.

If Asda are that bad, how do the other supermarkets compare?

The obvious next choice was Tesco. I started investigating their web sites with some trepidation – if Asda have 25 web sites then how many are Tesco going to have? Thankfully, the total is ‘only’ 14, 3 of which have no custom 404 page, including Tesco Direct, which is a huge online catalogue web site that would really benefit from a decent 404 page, and 6 have poor 404 pages, including Tesco Insulation, Tesco Diets, Tesco Energy and Tesco Travel Store, which were all terrible.

To wrap up the series I travelled around the web sites for Sainsbury’s, Waitrose, Morrison’s, Somerfield and Co-Operative and so covered all the major UK supermarkets.

The table below shows each supermarket, how many web sites they have in total and then the state of their 404 pages. I have included a percentage figure at the foot of the table, showing how many web sites of that supermarket’s total have good 404 pages, as a percentage.

Supermarkets 404 page use

You will see that Asda and Tesco are pretty close, with about a fifth of their web sites having what I would term as good 404 pages. Sainsbury’s are worse with only 11% (1 out of 9) whilst Waitrose has a quarter of its sites in the ‘Good’ category (1 out of 4).

There is also a figure for the percentage of web sites from that supermarket’s total without custom 404 pages altogether. Asda is pretty poor with 36%, which is 9 of its 25 web sites and Sainsbury’s is also lacking in this area with 44% of its sites without a custom 404 page, or 4 out of 9. Tesco comes next with 3 of its 14 sites without a custom 404 page, or 21%. Finally, Morrison’s single web site does not have a custom 404 page so they score 100% in this aspect.

Conclusion

The main thing to draw from all this is that the overall quality when it comes to the use of 404 pages in UK supermarkets is very poor. I had expected a great deal more from these major companies that obviously spend a huge amount of time, effort and expense on their multitude of web sites.

Whilst some of the supermarket sites do genuinely have good 404 pages, a lot are terrible or simply non-existent. The subject of 404 pages is possibly on the dull side for many (the novelty is wearing thin for me too after this study) but my belief is that they really do add to the user experience and a bad or non-existent 404 page really can have a negative effect on this experience.

Users can hit a 404 error for many reasons and I have discussed previously why a custom 404 page is a good idea. When a user does come across a 404, the important bit is what that user does next. What the owner of the web site should want them to do is to stay on their site, continue browsing until they reach the final goal for that web site – usually ordering something or making an enquiry. How does the user do that if all the 404 page shows is the server default version or does not help them in any way?

It is like reaching a crossroads with no signpost or map or hitting a dead end and having to go back the way you came, neither of which are good experiences for the user.

Supermarkets, and their in-house teams or external digital agencies, really need to look at the quality of the 404 pages that they are producing, often as part of a large scale redesign, and determine whether the result is good for the user or not. In the majority of cases at present, it is not.