At UPA 2011, Michael Rawlins, Lori Hawkins, and Jeff Sauro presented about Keystroke Level Modeling (KLM), a tool for estimating the actual movements and the time to perform each step that a particular UI design requires for users to complete a given task. KLM offers a way to analyze the time on task required by a design rather than attempting to empirically test time on task, which is difficult to do meaningfully with tight project time frames, limited numbers of participants, and the constrained interactions of many usability test situations. Furthermore, evaluating time on task during a typical usability test (without extra considerations) can usually only return meaningful data when the goals of the product focus on the first experience a user has, such as with a website or kiosks user interfaces. Time on task data from usability testing becomes less meaningful when evaluating applications where use over time must be considered, even when efficiency is a key usability goal. Since KLM can analyze the optimal possible time on task rather than the initial experience, it can provide a more accurate measure the success of a design where efficiency is key to success in a tool that will be used repeatedly over time.
The presenters also noted that, because it offers early indicators of the potential success of a design, KLM is useful for evaluating design direction with stakeholders. The consistent measurements provide reliable metrics to help stakeholders evaluate a design against established business and usability goals. While the core calculations are straightforward enough to do on the “back of the napkin”, setting up spreadsheets to calculate full task paths can take time and can seem intimidating. The presenters shared CogTool, a site where you can model the interactions and calculate the time completing tasks will take. This site allows you to focus on modeling the tasks, not the setting up spreadsheets.
One notable limitation is that new interaction models such as gestural interfaces like those used in mobile platforms – interactions that are very different from the keyboard and mouse inputs that were measured in 1983 – require research to develop accurate metrics. It also has some limitations with combined interactions types. The presenters shared the Composite KLM tool that expands the initial model to address these limitations.
I am excited by what this technique can add to my professional toolkit. I’m evaluating using this technique on my current project redesigning a web application that is used for timed customer support tasks. Being able to evaluate and demonstrate how the new designs reduce time on task compared to the existing UI will ensure that the design direction supports the business goals of the client. That’s a pretty powerful story to share. I hope you find the technique as helpful in your own work.
Keystroke Level Modeling: Another Usability Insight from UPA 2011
by Karen Bachmann on July 6th, 2011 | ~ minute read
