And now, the end is near…welcome to the final installment of my quickie-guide to running software usability testing sessions. If you’ve read the last three articles then at this point you have the software you want to test, some non-coworker users set up to test it, and a room to do the testing in.
Today we cover how to run the actual session. Recall the goal of your usability testing session: to observe typical users working with your software in a typical fashion, so you can note which parts of the software they find problematic. Now that you have completed all the preparation, you can begin.
Avoid giving the users a short tutorial, or any sort of unnecessary background information. Remember, they’re supposed to be average, novice users of this software, so any sort of demo would be an unfair advantage and consequently affect your observations. Just give them the tasks and let them go.
When users get frustrated or stuck (and they will) it’s tricky to discern the actual cause of the problem, and more importantly, why user are having this problem. It would help if you could read the user’s mind, but unless you’re a Betazoid, mind-reading is probably not your strong suit (and if it is, why the heck are you wasting your time developing software?).
The next best thing is convincing your users to speak their minds to your waiting ears.
So while users are doing their test tasks, keep reminding them to ‘narrate’ their work. For example: “Since I want to change the font type, I’m looking through the toolbar. But I can’t find the right button, so now I’m going to try the hover-help…but none of them say anything like ‘change font’ so I’m scanning the menus…but there’s still nothing useful, so I’m either going to have to look through the help docs or punch the monitor.”
In this piece of narration, if nothing else, you learn what this user’s typical search pattern is for completing simple tasks.
However, unlike my brother-in-law, most people aren’t used to constantly narrating what they’re doing as they do it, so you will have to regularly remind them. Ironically, the more trouble they’re having with the task the more thinking they will probably have to do, which means they will likely drop the less important chore of narrating. (Something about poor process monitoring in humans, I guess.)
An additional means of encouraging users to talk out loud is to employ the co-discovery method. This involves working with two users at a time so that as a team they can discuss how to complete the task while you just listen in. You’ll probably still have to remind them to talk out loud, but hopefully it won’t be as often.
There will also be times when you will have to gently probe the user for deeper information about the problem they’re having.
If they say, “I’m stuck,” this is a good place to ask why. In addition, do not neglect the users’ body language — they may hesitate before clicking on something. Why? Were they concerned they would make an unrecoverable change, or something else? Sometimes users will make a face (like they smell three-day old fish) so watch the expressions, and when they seem confused or unhappy ask them to explain.
Remember also that the language of your questions will shape their answers. You should avoid questions like, “This stinks, doesn’t it?” because you’re effectively asking, “Only an idiot wouldn’t agree that this stinks, and you’re not that sort of idiot, are you?”. And even if the user answers “No, this doesn’t stink,” they still haven’t told you it’s any good. So avoid yes/no questions. Even though you sound like a therapist, often the best question is, “How do you feel about this?” You haven’t biased users with your opinion, rather you’ve encouraged them into being open and honest.
Be sure to ask why users feel the way they do. Imagine they say, “I don’t like this pink icon.” If you don’t ask why, the reason might be “I prefer icons in earth tones,” rather than, “Yellow suggests a warning to me more so than pink.”
And thus endeth the lesson. I wish I had more space, but there are entire books dedicated to what I’ve tried to capture in four short articles. I think I’ve given you the basics though — enough to remove the fear and get you started.
Eventually, you will start to customize the process to your own skills and resources. And you’ll only get better with practice.
Now go forth, and create more usable software.
Blau is a software designer working in the User-Centered Design area at the IBM Toronto Lab. He can be reached at ablau@ca.ibm.com.