Unmoderated Usability Testing is becoming a very popular tool in the User Experience profession. It allows for design teams to gather feedback from a large pool of people early and often, especially during times where the design team simply doesn’t have direct access to their user base. Yes, there are downsides to doing Unmoderated Testing when considering it as a replacement to Moderated Testing, but I don’t see Unmoderated Testing as a replacement. I think there is room for both techniques if you use them together, and you use the results from each to inform the other. Unmoderated Testing tools provide several unique advantages that either difficult to collect using traditional methods, or would require so much time that a projects timeline get too large to stomach.
Complementing Traditional Testing
Here at Perficient, performing traditional usability testing methods is part of our design process. We ensure that the designs we create on our projects are exposed to their target audience before ever putting the “Done” stamp on them. Lately, we’ve been pairing up our traditional usability testing methods with an extra round of unmoderated usability testing. We do this to ensure that the final design changes we make really are the most appropriate, but it also allows us validate the design with a larger audience. The benefits of doing this are clear when we get challenged with “But you only tested with 10 users? How do you know that change will work once we go live?” Doing this extra round of testing also allows us to ask different questions and collect a different kind of data.
Combining the Qualitative and the Quantitative
Traditional Usability Testing is all about the qualitative data. Sure we collect some quantitative during a moderated usability test, but nothing on the scale that unmoderated offers. After going through a round of moderated testing, you begin to want to ask different questions of the participants. Or explore different areas of the design that you didn’t get to during the usability tests. Throwing in the extra round of unmoderated testing gives you that chances to focus on s other non-mission critical features of a design, and provides you hard numbers on user path analysis and success/failure/abandonment rates. (And more depending on the tool.) Collecting this type of information from 50-100 additional participants would take weeks using traditional methods, using an unmoderated tool you can do it over a weekend. (Assuming you’ve got a large pool of potential users to recruit from.)
Unmoderated Tools Will Never Replace In-Person Testing
If you’ve even sat in the same room with a “user” and watched them use a product or service, you know that experience will never be replaced by an unmoderated online tool or online web conferencing service. The amount of information you can collect simply by watching a person’s body language can provide the design team with tons of insight and inspiration to improve upon the design that is being tested. We know, and understand, the value of this kind of usability testing. With the introduction to these new unmoderated tools, we are able to build upon the techniques we already know and use to learn more about our “users” and validate our ideas more often. The more we are able to do this, the better the experience will be for users once that design gets released out into the wild.
Resources For Unmoderated Research & Testing
UX Zeitgeist Remote Research Topic – http://rosenfeldmedia.com/uxzeitgeist/topics/remote-research
Remote Research – http://remoteresear.ch/
Pros and Cons of Remote Usability Testing – http://johnnyholland.org/2010/06/02/pros-and-cons-of-remote-usability-testing/
Nate Bolts IxD 10 Remote Research Presentation – http://vimeo.com/9841981
IAS 09 Recording Portable Research: Observing Users on the Go – http://www.boxesandarrows.com/view/ia-summit-09-day-1
