Part 2 in this series talked about defining the purpose for resolving UX debt. Why do your designs matter to you and your business?
This post will suggest ideas for the next step: assessment. What needs to be done to improve your designs? Nothing fancy. Just a map of where you are and where you want to go.
Assessments are categorized below as:
- Your UX team
- Qualitative research
- Quantitative research
- Competitive analysis
Assessment #1. Your UX team
Before assessing your designs, check the UX culture of your organization. Strategist Leah Buley has done wonderful work on this. Watch her “Modern UX Organization” presentation from the 2015 IXDA conference. She’s found that good UX teams have:
- Executive UX leadership. UX owns a seat at the C-level. It’s not a stepchild of marketing or engineering.
- Quality people and processes. This includes UX specialists with deep expertise in activities like ethnography and interaction design. User research and testing are musts.
- Customer Experience. The UX team thinks beyond digital channels. They influence all touchpoints in a customer journey, including non-digital (physical) experiences.
- Strategic planning. The UX team participates in business strategy and project planning. They are proactive, not reactive, with requirements.
Before assessing UX debt, it’s worth asking what your organization is missing from this list. To go one step further, what dysfunctions might exist instead? These may be:
- A leader espousing minimal or even bad UX processes
- A culture that prioritizes deadlines, cost savings, or other projects over UX
- Poor designers or other project members already in place
For example, I once worked at a company where new products were not just invented but also managed by the R&D department. Whenever the usability team ran product tests, their recommendations were politely overridden.
This is the sort of dysfunction that requires attention before any design analysis. It’s futile analyzing UX design if nobody takes heed.
Assessment #2. Qualitative research
User research helps define where your UX currently stands and what pain points may need fixing. To paraphrase Jared Spool of User Interface Engineering (UIE), your proximity to users directly affects the quality of your work. Qualitative research includes:
- Behavioral assessments that directly study your users. This can occur in a controlled environment like a usability lab. It can also be conducted in a “natural” environment using contextual inquiry (CI), whereby researchers act as apprentices watching users doing their normal tasks. Or it might involve “gonzo” research in which you experience a design for yourself. In an IDEO study of hospital rooms, ethnographers laid in gurneys and discovered the tedium of dispiriting hospital ceilings.
- Attitudinal assessments ask users’ opinions about your designs. Voice of the Customer research may include surveys, interviews and website (or app) feedback. Even social media, product reviews and forums can be culled for user sentiment.
Both approaches matter, but the one I feel is best for redressing UX debt is behavioral. Specifically, it’s necessary for stakeholders to see users struggle.
This is because many project stakeholders assume that you’re merely presenting your opinions, not the users’. It took me many years of unsuccessfully pitching recommendations to realize this.
So obligate your stakeholders to watch users fight a design. This has a higher visceral effect than any persona or PowerPoint. Most importantly it spotlights the user vs. the system, not your stakeholders vs. you.
You have probably heard stakeholders say, “We don’t have the time or budget for user research. (And besides, we already know what users want.)” This is like saying a map is too expensive when you’re planning a trip. It brings to my mind a recent baseball book “More Than a Season” by Royals’ executive Dayton Moore. Upon joining the Royals in 2006, Dayton planned to reach the World Series in eight years. Eight years. In the technology world of two-week sprints, daily standups and Slack chat, who plans that far?
Sometimes Agile methods devolve. Scrums stop measuring daily work against the Big Picture. Instead, they’re used as forums to brainstorm the Big Picture itself on a daily basis.
Whether you’re improving a UX design or a baseball team, you have to plan beyond the scrums or sprints. As business professor Michael Porter says, a successful strategy takes years to implement.
Assessment #3. Quantitative research
The dirty secret to UX design is that nobody can grade your work if you don’t measure the results. Quantitative research corroborates qualitative research and vice versa.
I’m sure you know that tools like Google Analytics and Adobe SiteCatalyst measure Key Performance Indicators (KPIs). If your UX design has effective KPIs defined already, then great. These should be measured to benchmark your current designs.
In my experience, though, organizations with UX debt tend to install these tools and use them wrongly. This may include:
- Improper implementations that report faulty data
- KPIs that lack actionable recommendations
- A hundred incomprehensible KPIs (paralysis by analysis)
It’s vital that analytics are set up correctly, and that everyone understands them. I once supported a website whose Google Analytics showed traffic spiking by hundreds of thousands of visits. There are a lot of smart web analysts certified in GA, and I am not one of them. Finally my manager discovered that one of our developers was pinging that domain with a test web service, because, well, why not? GA dutifully logged all those pings as visits. Three months of analytics, wrecked.
Invest in a professional analyst or in certification training. Your designs will thank you.
Assessment #4. Competitive Analysis
After you’ve graded your designs, the final step is context. How do your designs stack up against the competition? Competitive analysis comes last because you want to know what to compare, namely 1) your users’ needs (qualitative research) and 2) your KPIs (quantitative research).
Activities may include:
- Benchmark usability testing
- Benchmark user satisfaction scores (like Net Promoter Score)
- Competitive feature inventories
- Domain and page authority comparisons
- Backlink and Google PageRank comparisons
- Services like builtwith.com to see technologies in competing websites
These activities are much more useful than online tools that guesstimate website traffic. Such services are often wrong by exponential levels.
The most common pitfall I’ve seen is stakeholders pointing to another website or app and saying “We want that.” It’s superficial to stop there. Again, copying another UX means that the best you can do is match it; you’ll never beat it. What’s worse, you may incur new UX debt by shoehorning your requirements into a layout that doesn’t make sense.
Form should follow function, not the other way around. Competitive analysis has to go beyond finding the slickest carousel, Parallax scroll, or hamburger menu to copy.
Next Steps
The last part of this blog series will address How to connect the Start and End points in this diagram. What does it take in terms of teamwork and attitude? Hope this helps. Happy designing.
