What do your product stakeholders really know about user research? In organizations in which User Experience is more mature or even completely institutionalized, there is a good chance that product stakeholders—for example, your product owners, product managers, business analysts, and developers—know at least a little something about user research and user-centered design practices.
But in organizations in which User Experience is in its infancy or corporate budgets underfund User Experience, you might hear someone on your product team say, “That’s the stuff you do with the one-way mirrors, right?” The majority of product teams are likely neither practicing nor evangelizing UX research. There might be a UX unicorn or two—a UX professional who does a little bit of everything, including design, strategy and research. In companies that have not fully embraced User Experience, product stakeholders do not have a firm grounding in UX research and do not truly understand what is necessary to capture the user’s story.
Champion Advertisement
Continue Reading…
According to Eric Schaffer, author of Institutionalization of UX: A Step-by-Step Guide to a User Experience Practice and founder of the UX education and consulting firm Human Factors International (HFI), executives typically take the wrong approach to building up a UX capability within their organizations. They usually hire 2% of the UX staff they really need, then when designs have not improved, let them go. They don’t know what to do with UX professionals because of their lack of experience working with them. Because of product stakeholders’ ignorance of users’ true needs in their industry domain and a lack of UX research, they form distorted views of users and their tasks, unwittingly injecting their own biases and assumptions into decision making during the product design process.
So what are some of the myths that product stakeholders perpetuate about UX research? In this article, which is Part 1 in a series of two articles, I’ll discuss two common myths that product stakeholders believe about user research, as well as the truths behind those myths:
Myth #1: It starts with the method, then everything else follows.
Myth #2: “We don’t need real users! Let’s just ask our coworkers to participate instead!”
Myth #1: It starts with the method, then everything else follows.
I simply don’t have enough fingers to count the number of times product people have told me how to do UX research—even before discussing what problems they are trying to solve or research questions they are trying to answer. Recently, a product owner asked me, “Can you send out a survey to ask our users about our product?”
Starting by choosing a research method even before you understand what questions your research needs to answer is crazy! But who can really blame the product folks? They are solution focused and their work is time constrained. So suggesting a method to follow is their way of getting one step closer to product launch, and they think it should be alright.
As UX researchers, we need to reverse engineer their prescribed method into research goals and hypotheses. This mental exercise lets you walk the suggested method backward to product stakeholders’ initial hypotheses and assumptions about users. UX researchers must transform stakeholder assumptions into clear, focused research goals and objectives. Figure 1 is a visual representation of this process and shows the relationship between stakeholder’s assumptions and UX research methods.
Figure 1—Deriving research methods from stakeholder assumptions
Recently, a key stakeholder sent me a list of research approaches he thought would help answer some of the key questions he had about his product. One of these approaches was: “A/B test messages for tone and length.” Deciphering why he wanted to do an A/B test would have been a waste of time. Don’t assume for a moment that a product owner truly knows why to choose an A/B test over any other method! The chosen method of research should be solely the decision of the UX researcher. Product teams need to trust UX researchers to make these critical decisions.
Understanding why he wanted to test messages for tone and length is a more relevant question and gets closer to the heart of his hypotheses and assumptions, as well as what he does not know and wants to learn more about. This product owner was actually interested in understanding what makes an effective message. Once I understood the real need for research, I concluded that face-to-face interviews would be the most effective way of accomplishing the research objectives and answering those burning research questions.
Myth #1 Debunked: It starts with understanding the hypotheses and assumptions of key stakeholders, then everything else—including the right research methods—follows.
Myth #2: We don’t need real users! Let’s just ask our coworkers to participate instead!
The worst thing to do would be to ask your coworkers to participate in UX research, then use their feedback as a part of the final research results. This is especially true of your fellow UX professionals! Coworkers are very likely to tell the moderator exactly what he or she wants to hear.
In contrast, when real users provide feedback on design ideas or interact with early prototypes during usability studies, they are significantly less biased and more likely to give unfiltered feedback. Getting unfiltered feedback from real or prospective users during UX research sessions is crucial. Rarely will a real user tell UX researchers everything they want to hear, not have any issues understanding things during a research session, or anticipate all of the researchers’ questions. (Although it does rarely happen that it seems practically as if a user has read the interview guide before starting their session.) The likely result of testing with real users is having a mix of participants who liked some things, but not others.
A recent study I conducted related to a relatively novel use of old barcoding technology. I found that user feedback was fairly mixed. Depending on a couple of key factors that we discovered during the sessions, users would be either more or less likely to see value in and adopt the technology in their daily work. Without talking to real users, we would not have realized these key factors that then informed the product’s UX design. Now that we’re aware of these factors, the product, design, and content teams can control them, targeting the right subset of users and using these factors as the basis for future enhancements. This knowledge will also help the product team know what they are up against when and if the product goes to market.
Myth #2 Debunked: Conduct UX research with real users. Testing with individuals who are not your real users leads to shaky, questionable conclusions and gives product stakeholders a false sense of confidence when making product decisions.
Conclusion
Product stakeholders are not entirely at fault for perpetuating myths such as these—especially if they’ve had negative experiences with UX research. They may not want to think about what goes into getting user insights, so end up making unfounded assumptions about UX research. They may view UX research as an obstacle rather than a partnership—just another bottleneck in getting their product to market. They may view UX research as a long, tedious process. Put yourself in their shoes. When there is a tight timeline for delivering new product features, if you believed UX research would take longer than would be possible in the timeframe, wouldn’t you be looking for ways to short-circuit the process? Rather than engaging in the UX research process, some product stakeholders would rather just check the box that says it’s done and move onto something else over which they have more control.
A important note to UX researchers—Don’t allow UX research to become a bottleneck that slows down the development process. As UX researchers, not only is it your job to discover the hypotheses and assumptions in stakeholders’ minds, but also to evangelize the value of UX research and partner with your stakeholders. Always engage stakeholders in the research process, and find ways of accomplishing your research goals in an efficient, focused manner, ensuring that you meet your stakeholders’ deadlines.
Stay tuned for Part 2 of this series, in which I’ll debunk more of the myths that product stakeholders believe about user research:
Myth #3: We don’t have the budget to do participant recruiting. It’s not worth the expense.
Myth #4: Apple never does user research, so why should we?
Myth #5: Forget about defining the problem! Let’s skip right to the solution!
Michael has worked in the field of IT (Information Technology) for more than 20 years—as an engineer, business analyst, and, for the last ten years, as a UX researcher. He has written on UX topics such as research methodology, UX strategy, and innovation for industry publications that include UXmatters, UX Mastery, Boxes and Arrows, UX Planet, and UX Collective. In Discovery, his quarterly column on UXmatters, Michael writes about the insights that derive from formative UX-research studies. He has a B.A. in Creative Writing from Binghamton University, an M.B.A. in Finance and Strategy from NYU Stern, and an M.S. in Human-Computer Interaction from Iowa State University. Read More
