Have you ever put your jacket on before your shirt? How did that go? The sequence of your routine tasks can make a big difference. Have you ever thought the answer to a question was obvious because of the questions that came before it? People are conscious of the sequence of how things happen. Hearing a siren causes you to look around. Not many people look around before they here the siren.
On a good day, we pay attention. We look for clues about what we should do next. How we answer one question can influence how we might answer the next. It’s human nature to want others to think well of us. No one wants to be accused of not paying attention. Our hope is that if we pay attention, we will be more likely to succeed.
In website usability testing, the sequence of tasks makes a big difference.
Imagine our usability testing process requires participants to complete eight website tasks. With each completed task, participants build their body of knowledge of the website to help them with the next task. So, if tasks are presented in the same sequence, earlier tasks could likely test less favorably than the later tasks. This is because research participants have probably seen or learned things in the earlier tasks that they can use to improve how they perform later tasks. Failing to control for this places your data in jeopardy. You can walk away from the usability testing process thinking that participants were a lot more successful with the later tasks than the earlier tasks. This might have far more to do with the sequence of the tasks than just the tasks themselves.
The extent to which our user experience testing reflects the real world is the extent to which it has validity. It’s more important to learn how our pages work with the first-time visitor than for the repeat visitor. We may never get that first-time user back and like they say, “you never get a second chance to make a good first impression.” When designing the task sequence for user experience testing, make sure to the best of your ability that every participant gets the tasks in a different sequence. This way you have the best chance to observe first-time user experiences for your tasks, and you are less vulnerable to sequence bias.
You can build a matrix to help control for sequence bias. Depending on the number of tasks and participants in your testing, it might be impossible not to duplicate. That’s okay, but try to keep duplication to a minimum. Consider using the matrix below as a starting point for fine-tuning the sequence of your tasks.
All users – whether they are aware or not – are influenced by patterns or conventions, whether it is how headlines are styled or where secondary navigation is placed. By the time a testing participant has completed their third task, they are going to become familiar with how you do things. They might have seen something that will give them an advantage for a future task in a previous task. Or, if they become frustrated during the testing, participants could perform worse as time goes on. Both of these scenarios can call the validity of your data into question because, at a certain point, you are analyzing the effects of the UX testing on the participant and no longer the individual tasks.
When a user is on your website, you have no control over when they visit any particular page. Some might have come from the home page, others from almost any other page on your site, and still others might have come directly from a Google search results page. Because of this, it is imperative that you structure your UX testing to let you observe participants completing tasks in every order imaginable. You want to observe two things: a first-time website user and a returning website user. You want your testing to cover all your important scenarios.
Remember, the degree to which your website usability testing reflects real-world user scenarios is the degree to which your data can be relied upon. The sequence tasks are performed in testing makes a difference in your data, and your data makes a difference in the conclusions you draw.
Think back to the last time you were driving and a car next to yours was moving into your lane before using the turn indicator. Sequence does make a difference. Work with your Perficient Digital UX researcher to structure your testing, and ensure your website has the best chance to create a good first impression.
