Recruiting real end users can be costly and time consuming for usability testing. For anyone but a user experience (UX) purist, the temptation to use employees for usability testing can be quite a temptation. Employees are accessible and already paid for, so why not use them? After all, they are users outside of work. So, is there really a problem?
If you are careful in your recruiting efforts, you can get valid data
I worked at a software firm several years ago that had 3,500 employees on-site, and yes, we did use employees for some of our website testing. It can be an acceptable solution if what you want to find out is more general user behavior like:
- Can they find a phone number?
- Is the mega menu easy to use?
- Do they think there is content below the fold?
- Do they like hero image slideshows?
However…
That “if” is a big one, so let me explain how we handled it before you report me to the UX police. We always stayed away from usability testing with:
- Developers
- Marketing professionals
- Management
- Corporate training professionals
- Human Resources
Why? People in those positions can tend to know too much about website development or might be prone to overthink their reactions during testing because of their role in the company. We found usability testing with employees from the following categories will give us face-value reactions:
- Operations
- Finance
- Sales
Remember, we had a campus of 3,500 employees, so filtering our employees to find UX testing participants still gave us enough participants to have some confidence in the data we collected.
Would it have been better to not test with employees?
Sure! In an ideal world where enough time and money was always available for usability testing, yes. However, there are times where practical and ideal are at odds with each other. The worst thing you can do is not do any testing. All testing has some risks, such as insufficient sample size or the participants acting differently because they know they are being tested. But even with these risks, taking the “risk concern” to the extreme and not performing any testing ensures that you don’t get any insight. Remember to keep things in perspective. Testing website usability is very different from testing the failure rate of a heart valve.
As a researcher, I always paid attention to any behavior or comments from the employees that seemed like insider knowledge. When that happened, we were careful to call it out to stakeholders as behavior that might be suspect. As always, we were hypervigilant in keeping the identities of our testing participants anonymous. These are some of the judgment calls that you make as a researcher when it comes to qualitative data. They are part of the process of usability testing.
If you have filled out screeners in the past, before being part of a focus group or some other type of testing, what you do for a living, your age, education, or employment status can play into whether you’re accepted into the research. If you have a large enough pool of possible participants from your employees, or the role they perform is far enough removed, then you might be able to take advantage of employee convenience.
Sometimes it’s possible to run with scissors, as long as you run slowly and let everyone around you know you’ve got scissors. And yes, it’s usually more ideal to have amazing research budgets and not use employees.
Okay, now you can call the UX police on me.
