Testing your Web site

14 February 2008

Rationale for user testing

User testing is the best way to find out if your Web site works.  It will help you to understand if your headings work, if your links make sense and if your structure is OK.  Best of all, it will allow you to look at your site with fresh eyes.  It will surprise you.

For usability testing, I generally contact about six people and ask them to volunteer about an hour of their time.  In return, I offer them a $20 voucher to a good bookstore.  It doesn't recompense them for their time, but it does show that I appreciate their help.

I will go to their house or work (their preference) and ask them to use the Global Collaborative to answer about five questions.  I will tell them that it doesn't matter whether they get the answers right or not.  We are testing the Web site, not them, so we are interested in how they use the Web site to look for answers.  I find that it is useful to talk about "taking the Web site for a test drive".

Then I watch what they do.  I watch as closely as I can, and I take notes.  I keep quiet and respond as neutrally as possible to any questions.  I only intervene if the test goes off the rails badly (if they go to some other site, for example).

In the past, I have recorded their actions and the issues revealed on a notepad.  I will do that again.  Other people video the screen as well.  This is particularly good, as you can show other people the problems later. If you don't have access to a digital video camera and a tripod, then you can often substitute screen shots to demonstrate issues.

It doesn't take many testers to spot the main issues.  This is not a process that will reveal every single flaw.  But it will reveal the main issues.  We are not trying to find every single issue - just knocking of the biggest ones will give us a big boost.  It will also help us to understand how new designs might be better.  As such, we should also make the results available to the designers. 

You can get further information about this sort of testing process in Guerrilla HCI (Human-Computer Interaction) by Jakob Nielsen, 1994.

User testing process

Here are five example questions that I have used for usability testing.  NB: The format is prettier than this when I give it to the testers.  In the example below, I have used the Global Collaborative as the example site.  You could do this just as easily with your part of the Global Collaborative.

1. How many organizations are there on the Global Collaborative?  Write down the number.
2. Find something that you are personally interested in on the Global Collaborative.  Write down the address.
3. What organization published this information?  Write down the organization name.
4. Contact that organization and tell them what you think.
5. Using this user name and password, please sign on to the Global Collaborative.
   User: test2test
   password: bigger32fork
   

It is important to keep your questions simple.  You are not trying to trick the participants.  You just want something that will (a) get them moving around the site and (b) everyone involved can agree are representative of the sorts of thing that people might do.

I try to mix closed questions ("How many organizations...") with open questions ("Find something interesting...").  However, sometimes questions that are too open don't work very well because people flounder.  A better example might be:

• Think of a topic that you think will be represented on the Global Collaborative.  Using the Global Collaborative, find something that you didn't know about that topic.

One important point is to  try to avoid words that will lead the testers too much.  For example, if all your contact details are labeled 'contact', try not to use that term in a question.  Instead, ask people to 'Send a message', rather than 'contact them'.  This is particularly true if you are asking people to search for things.

I often use my girl-friend to check that my questions are sensible, and that they will work as well as I hope.

I start the testers from different parts of the site:

• The front page.
• Google or Yahoo! (their choice) - in which case, the first question becomes "Find the Global Collaborative".
• The 404 page.
• A page deep in the site (as if they had followed a link or a search result).
• The "I'm sorry, but you need to be signed in to see that content" page.

Finding users

Testers should represent your target audiences. For my purposes, testers should be people who would normally be interested in the Global Collaborative. It would be great to draw upon:

• Academics working in the relevant social sciences.
• Students working in the relevant social sciences.
• People working for NGOs.
• Politicians and their advisers.
• Journalists.
• Members of the general public who are interested in these sorts of issues.

I try to include someone who is elderly and someone who has English as their second language, since these people bring fresh insights into the mix. It is also very important to include people who are using adaptive technologies, rather than the standard set-up of screen, keyboard and mouse.  This helps to understand how people with disabilities will use the site.  It also means that the designers understand that accessibility is important, and gives them the freedom to think beyond their usual conventions.

I try to avoid people who build Web sites as a profession and particularly people who do user testing as part of their work. Mostly this is based on recommendations from professional usability people, rather than any bad experiences on my own part.

However, time and social networks will limit who you can reach. Generally, I look for potential testers amongst my friends and acquaintances.  I send out e-mail invitations and advertise on my blog.

Reporting

Even if you are the only person working on your Web site, you should make an effort to write up your results.  It will help to clarify your thoughts.  It will also help you in the future, when you are trying to understand how your site has grown.  And it will absolutely help the person who takes care of the site after you.

If you are working with others, then you will need to report your results so that they can understand what needs to be changed.  However, they probably won't read your whole report.  Mostly, they will read the summary and look at any images that you have of particular problems that they don't understand.  That's OK - they are busy people. The main purposes of writing the report are to:

• Clarify your thoughts.
• Present a clear indication of what issues have been discovered.
• Provide evidence to convince people who don't understand any particular issue.
• Provide documentation for the future.