Recently I was talking to someone who is relatively new to the field of usability and user experience. He has developed a web application and wanted some ideas for getting feedback from users. He commented that he was planning on sending out a survey to users to see what they think about the web application. That was his plan for user testing. I’m so entrenched in the concept of usability and user testing that I have to stop sometimes and remember that not everyone else is.
“Well, you do have other choices besides doing a survey, you know”, I said.
“Oh, really?” he asked, “like what?”.
“I’ll send you some ideas,” I replied, and then I thought, “That would make a good blog post”, and, here we are.
1. “Traditional” moderated usability test – Let’s start with the most well-known and most used method of getting feedback from users. In a moderated usability test the user sits down in front of the software, web site web application, or other product that you are testing and uses the product, site or item to get one or more tasks done. The tester designs the test with real-life scenarios and asks the user to use the product or tool or site to go through and actually do the scenarios. The user is asked to talk out loud while they are completing the scenarios, so that the tester can understand what they are thinking and experiencing as they complete the activities they have been asked to do. It’s called moderated because there is a facilitator to moderate the testing.
It’s important in a moderated usability test that:
- Users must be representative of the actual user. It doesn’t work to use you or friend in the next cubicle, or your sister. The idea is to have a representative user try to use the site or product to get real tasks done.
- Although you may be collecting other data, such as time to complete the task or number or types of errors made, the main data comes from the comments users make while they are working (called the “think aloud” technique).
- Tests are done one-on-one. This isn’t a focus group.
- Some facilitators “probe” with questions during the test, but this is tricky to do. You don’t want your questions to influence the user. Some facilitators wait until the tasks are completed before asking questions (called “de-briefing”).
Pros: Gives you lots of great data on what the usability issues are
Cons: Fairly expensive to conduct. You do these one at a time, so if you are testing 10 users that’s a lot of your time to be at the sessions, plan them, analyze and report on data, etc. You may also need to pay for recruiting users and you need to give them an “incentive” (pay them in some way with cash, gift certificates etc). Continue Reading →