- Try Asking Less, But More Often and Earlier. If you want to learn about users’ engagement with your product over a longer-term experience, might you try asking your participants less, but more often?
- Sometimes It’s Better Not To Ask. Do you need to ask about everything, right now? Could you, maybe, postpone a difficult topic for another time, or even exclude it altogether?
- Think About Sharing Your Results. How about publishing more UX survey results? Perhaps not the whole data set, perhaps not even the report. But what about giving participants a headline summary of the key findings from the research, if they want it? This could be the way to help keep our users interested and involved in our ongoing projects.
The theme Jarrett noted in the sessions she attended was how much of the research was over significant time, from years to lifetimes. Approaches used by researchers in these long-term studies can inform UX research, which is most often short in duration and limited in scope. Jarrett derived these tips to improve our ability to conduct longer-term studies, the kind needed to evaluate an experience that takes place over time.
As I considered how I might apply these tips, the first brought to mind a study I did for a client recently to explore how their audience used online professional resources in the course of their work. Because the client was specifically interested in patterns of use, I proposed a diary study. The client liked the regularity of feedback that approach offers, but was rightfully concerned that the audience, highly specialized and very busy medical professionals, would be hesitant to add another task to their intense work days. We decided instead to schedule a series of 15-minute “diary calls” with participants that asked them to reflect on their use of online resources to inform their work in the 24 hours prior to our conversation.
In planning the study, I was a bit concerned about memory limitations and priming affects. I also wondered whether we could really cover enough ground in 15 minutes. As the study progressed, I was relieved to find these concerns relatively minor.
While participants may not have recalled all possible uses, in those they did share, participants told detailed stories of how they used specific information to support their work. They also tended to report consistent levels of detail and usage patterns across the calls. So the usage patterns were familiar and highly memorable.
The calls generally were at least a week apart. Participants did not adjust their behavior based on prior conversations. In fact, as the call prompted them to reflect on their activity, some expressed surprise that they did not think to do anything different. This likely relates to how critical finding the right information from trusted online sources quickly is to their task success. They were not going to jump to new sources or take time to develop new strategies within a week.
As for short duration, because the study was a series of similar questions asked over time, short, focused calls were plenty of time to ask the most meaningful questions, touching on the second insight from Jarrett. For an activity that added just one more responsibility to their days, participants seemed to actually enjoy the opportunity to reflect, and my client gained useful data that would have been difficult to gather using a more typical method.
The lessons that Jarrett shared made me reconsider the diary call study and will help me improve longer-term research efforts. I’d love to hear what long-term research techniques have worked well for you. Do you have an experience that illustrates these tips?
