Persuading Clients That the Need for User Research and Usability Testing Is Real

Getting an Unbiased View from Users

“The benefit of asking users—who are outside the project context—to try to complete tasks using a design solution is that users approach tasks naturally, without any particular bias,” reply Dan and Jo. “They do not have any political leanings or in-depth project knowledge that would influence the results. We gain access to users’ natural language, their life context, other bits of knowledge that they may use within the context of a task and a domain, and the personal and digital ecosystems that are at play. We may also discover a usability or design principle that addresses a user need.

“As experts who a client has brought onto a project, we may complement research data by identifying areas that we think a design needs to address. But our reports may focus too much on areas that are not important to users or, ultimately, the business. Also, by confirming what has concerned the business for some time, the voice of the user—or voice of the customer—often holds more weight with a business. So we should both do expert reviews and get user feedback. Ideally, we should blend our expert knowledge and user feedback together so we can tell a stronger story overall.”

Making the Case for Research Because We’re Not the User

“Because I’m not the user!” exclaims Carol. “Seeing is believing, and no one can show your client what works and what doesn’t, what pleases and what puzzles, what engages or bores the user better than the user. The beauty of the convenience and cost effectiveness of small usability studies is that, in a day or less, your client can learn from their users and apply that learning directly to product improvement.

“If your client doesn’t believe that you need to do research, ask them to watch just two usability sessions. During the first session, your client might have doubts about the quality of the participant. But after two sessions, in which they see the same issues cropping up, they may put their doubts aside. If you can get your client to watch three sessions, they’ll be hooked.

“If you can’t get your client to watch live usability sessions, do the next best thing and show them video highlights. A good 10-minute highlights tape can go a long way toward bringing the users’ experience to your client.

“Whenever I get asked to do an expert review, or heuristic evaluation, I always ask why the client wants an expert review rather than usability testing. If the answer is to clean up the product after the review, then test it, I know that they understand the strength of this two-pronged approach to building usability into products. But if the answer is that they don’t have the time or budget for testing with real users, push back and ask how much time and budget they have. Then show them how they can do testing within both their timeframe and their budget. If they can do only one research activity, make your best case that it should be testing with real users.”

“Since the bulk of my work is research and evaluation,” answers Cory, “I actually hear this question or a derivation of it fairly regularly. Stakeholders wonder why they shouldn’t just hire me to do a heuristic or expert review. My answer to them is that, while I can review a user interface with respect to general best practices, in most cases, I’m not the typical user. There are times when—from a best practices perspective—some element of a user interface may not appear to make sense. However, in doing a usability study with the target audience, I may find that these apparent issues don’t actually hinder users’ ability to complete key tasks. Conversely, even though a design may meet best practices, sometimes the typical users’ background, knowledge, or experience may prevent their fully understanding how to interact with the user interface as designed.”

Tailoring Your Answer to the Situation

“I’ve heard this comment many times in my interactions with project leadership,” says Leo. “I’ve given them several different responses, depending on how much data actually exists. Here are some example of responses that I’ve used:

• questioning whether there’s a data gap—‘Thank you for your vote of confidence! Yes, I’ve seen a lot of similar situations in the past. For example, if I understand your situation correctly, it seems like Project X, which I worked on, was very similar. Sadly, not all situations map one to one, so I won’t know how much of Project X could actually apply to your project unless we get some baseline information. Do you have much user research or data that could jump start that baseline effort?’
• using deflection—‘If I understand your situation correctly, it seems like Project X, which I worked on, was very similar.’ Assuming that they claim to have much of the user research completed: ‘Sadly, because that work was proprietary, we can’t just reuse it verbatim. We’ll want to do some light-weight calibration and validation of your situation to see what would work for you. Once we figure that out, I can reuse the appropriate design principles and patterns from prior projects as they apply.’
• blending new data with known data—‘Thank you. I have worked on similar projects in the past. But like business strategy, marketing, and other efforts whose goal is to expand your business’s reach, we’ll need to do some general work to get the lay of the land for a new UX engagement. Just as you wouldn’t embark on a new business venture without some research to guide you, we wouldn’t want to start investing in design and development without some guidance. I’m not talking about a massive research effort, depending on what your team already knows….’
• denying the honor—Thank you. While I have worked on similar projects in the past, the right approach for your project depends on what you expect to get done. I totally agree that, if this were limited to a best-practices consultation, I could offer you my expert opinion based on common design practices in this space. But, based on what I’ve heard you say, I think your project has more to it than that. Otherwise, I’d be happy to recommend some great resources for your team to use and save you a lot of money.”

Focusing on the Return on Investment for Research

“It’s important to note that user research is not always necessary,” replies Jordan. “Sometimes organizations already have research, data, or insights that could provide answers to the questions you’d intended to answer by conducting user research. I always recommend usability testing, but an alpha or beta launch could meet the same need. Neither of these approaches is required on every project. On the other hand, if a project does legitimately need user research or usability testing, you should focus on communicating the return on investment (ROI) of conducting research. Usually when organizations say something like ‘You’re the expert, we trust you,’ it means they hadn’t planned to conduct research or testing and are afraid of the cost or schedule implications. They’d rather not do anything that could put their timeline or budget in jeopardy.

“To avoid this situation, plan for research and testing from the beginning. Even if you’re not sure what form it will take, include it in your plan and earmark budget and time to do whatever you think may be necessary. If your clients are actually concerned about ROI, they’ll raise the issue during the planning phase of the project. Their argument might be something like: ‘The cost and time that conducting research and testing would require outweigh any value we’d get from it because we’ve already hired an expert who should know all of this.’

“As UX professionals, our expertise lies in information architecture, interaction design, usability testing, and user research—among other areas. We’re not capable of reading the minds of a particular organization’s customer base. But we are capable of conducting a study that we’ve designed to learn actual users opinions about things. Then we can use what we learn from our research to inform our decisions regarding information architecture and interaction design. Our motivation for doing user research and usability testing is to produce theories, then test them on real users. Each theory that you test should add to an organization’s understanding of their users. The value of gaining this understanding makes conducting user research worthwhile. If user research would deliver no value, you shouldn’t conduct it.

“To look at this another way, the intent of any product-design, Web site–design, or software-design project is either to optimize something or launch something,” continues Jordan. “If the intent is to optimize a user experience, we need to understand what users currently like and dislike about it. If the intent is to launch something new, we need to understand the most important elements of positive and negative user experiences with which the user has interacted in the past. So, regardless of the situation, user research and usability testing are often necessary.

“In the end, if a client is adamantly opposed to conducting user research, it may be an indication of organizational short-sightedness. Some organizations want user experiences that are good enough for the user; but that cater to the agenda of the organization itself. They won’t tell you this outright, but they’ll have an aversion to talking or listening to users or customers.

“Clients who have drunk their organization’s Kool-Aid often feel that their customers should be thankful to their organization—rather than their organization’s being thankful to their customers. These are date-and-dump clients who make good friends for a short time, on a specific project, but who aren’t visionaries. They’re clients who you’ll avoid once you’ve finished a project. They’re clients who drain you and make you feel horrible at the end of the day—because in the end, UX professionals want to create user-centric designs. Creating user-centric designs is hard when you’re working for an organization that puts its own wants and needs ahead of the users or customers.”

Communicating Why You Should Do User Research and Usability Testing

Tobias tells us, “There are five reasons why we need to involve users:

1. Without users, you cannot—by definition—do user-centered design. Without users, a good UX designer will create a design that follows best practices for screen design—patterns, font sizes, color contrast, layout and alignment rules, and other such things. That alone is of value, but it’s far from being enough.
2. Without learning about users’ needs, what requirements and needs are you designing against? Did you come up with them? In all cases where you’re not part of the target audience, that’s a bad idea. Unless you’re a total hero in fund management, oil exploration, rocket science, electron microscopy, or whatever kind of product domain you’re working on, you’re not representative of the users. Should you get user requirements and user needs from the stakeholders? How much do they actually know about the real context of use? How often have you heard a project sponsor say, ‘I used to do that job,’ but it was 20 years ago and his experience bears no resemblance whatsoever to the reality of today?
3. Without seeing users at work, you cannot fully understand the context of use. Even if you assume that the stakeholders have correctly acquired, analyzed, and documented valid, testable, and unambiguous user requirements, as someone on the receiving end of all this information, you may not have any experience with the product’s context of use. You’ll need to learn a lot of details and nuances about that context, but learning them from stakeholders would mean you’d get only a filtered version of reality. They might not be able to answer questions like: ‘How far away from the screen does the user stand when doing XYZ?’ or ‘Do the existing users really follow the protocol that the handbook dictates?’ Often, users don’t use a product in the way that stakeholders envisioned and planned—and the stakeholders may be completely unaware of this. I remember a project on which, during field observations, we invalidated a couple of key assumptions that the product owners had been 100% sure about.
4. Without understanding users’ needs, you cannot justify your design decisions. How could you possibly survive a design-review meeting with stakeholders if they have all the ammunition—the context knowledge—and you have only an empty gun? If they point to your design and ask, ‘Why is this UI element there? It’s not listed in the requirements!’ perhaps you’d say that you feel it would be useful. That’s not a very strong argument. But if you’ve been doing user research, you would be able to say: ‘This feature allows users to do XYZ, which we identified as a critical need when we were visiting users last month.’ If you’ve done usability testing, you could say, ‘During our first usability study, 90% of test participants did not understand how to do XYZ. This feature helps them do XYZ. We verified this during our second usability study; none of the participants had any issues with XZY.’ The best thing you can have as a UX designer is empirical evidence, because it trumps opinions and beliefs.
5. User research is fun, educational, and inspiring! For many, this is the best part of being a UX professional. It’s where the music plays that pulls our heartstrings.”