In Part 1 of this two-part series, I debunked the following myths that some product stakeholders believe about user research and described the proper approach to follow:
Myth #1: It starts with the method, then everything else follows.
Myth #2: We don’t need real users! Let’s just ask our coworkers to participate instead!
When product stakeholders first engage user researchers on their projects, they may propose research methods without first considering their burning questions. Often, product stakeholders try to persuade researchers to take shortcuts by doing hallway testing with fellow employees. Choosing research methods before asking those burning questions or testing prototypes with your colleagues are recipes for UX disaster. To do proper user research, you need to talk to real users. Product stakeholders who give little thought or attention to the users of their product perpetuate the myth that it is okay for user researchers to take such shortcuts. They view these approaches to research as a way of simply checking the box, saying it’s done, while disregarding the value of any actionable learnings they might have gleaned from legitimate research.
Champion Advertisement
Continue Reading…
Now, in Part 2, I’ll debunk the following myths:
Myth #3: We don’t have the budget to do participant recruiting. It’s not worth the expense.
Myth #4: Apple never does user research, so why should we?
Myth #5: Forget about defining the problem! Let’s skip right to the solution!
Myth #3: We don’t have the budget to do participant recruiting. It’s not worth the expense.
Participant recruiting can be very expensive, especially if you use a third-party recruiting agency. On top of the cost of recruiting participants, they add extra fees such as processing incentives. Don’t feel that you need to use third-party recruiters! You don’t. Recruiting agencies don't really know who your users are. They merely take orders from the researcher or client who requests study participants. Your organization know its users better than anyone outside, so start handling participant recruitment from within your organization.
Recruiting from Within Your Organization
You can develop a participant-recruiting capability internally, within your own organization. Talk to your sales and service departments. Build relationships with key people who frequently talk and work directly with customers, clients, or users. Enabling them to do participant recruiting for you will save you time and money. Tell recruiters what you need, in terms of participant criteria. Indicate that you’ll give participants a small incentive for their time at the end of each research session. But, believe it or not, if your users are truly engaged in your company’s products, they’ll be very excited to speak with you, regardless of the incentive. This is especially true if their organization has made significant investments in your product.
At my company, we have a program called Come See for Yourself, which affords users the opportunity to engage with people from Research, Development, and Product by letting them watch them at work. By letting us come into their organization, users become part of the product-development process. This is a low-cost way of recruiting participants for exploratory research studies. The only recruiting costs are the incentives for users’ time and the time it takes to schedule the sessions. Through this program, users gain a greater appreciation for the development process and feel they are part of something larger. These users are often great participants for future studies and, ultimately, reduce your organization’s dependency on using third-party recruiting agencies. As long as you are mindful of not reaching out to them too often, they’ll be happy to offer you their insights—and their behaviors will inspire product teams to build better products.
Controlling Costs by Recruiting Prospective Users
Don’t just stop at recruiting users. Think about recruiting prospective users. Again, in-house resources can help you find these people. Who else in the general population would be likely to consider your product for their organization? Their needs may not be drastically different from your current users’ needs. In my organization, ADP, not only do we recruit users, we also recruit people who use competing products or other alternatives. Getting their input provides a more balanced, representative view of the user experience. Users who have been using your product for a long time may have a more myopic view of the user experience because they have been entrenched in it for so long. They may even be part of a captive audience because of the high cost of switching to another vendor. Or they may be jaded and have adapted to your system’s design and terminology, so may exhibit a narrower range of user expectations.
By managing your recruiting in house rather than using a third-party recruiter, you can gain better control over your recruiting costs. You’ll keep information about the costs of any incentives and intelligence about screening criteria solely in your recruiting database. This can be as simple as a spreadsheet or as complex as a third-party recruiting and research-panel solution. While many of these large-scale solutions can cost tens of thousands of dollars, you can amortize their cost across all of your company’s research resources. This can be especially beneficial to companies that have large research teams spread across the organization. One useful resource my organization currently uses for surveys and storing recruiting data is Qualtrics. Many other solutions exist. The best one for you really depends on what your recruiting needs are.
Avoiding Over-reliance on Third-Party Recruiters
Third-party recruiters save you time because you don’t have to call potential participants to recruit them and schedule their sessions on your own. But be careful: using them can become quite addictive, given the savings in time and effort. However, over-reliance on third-party recruiters can be a dangerous dependency, especially when budget purse strings tighten.
Using recruiters lets researchers focus on the things that matter more—for example, refining the interview guide, working with the design team to ready the prototype for testing, and engaging with stakeholders to ensure you’re meeting the goals of the study. If all that is worth giving up recruiting tasks and you have an unlimited budget, go with a third-party recruiter. But most organizations don’t have unlimited budgets for research. Plus, when budget strings really tighten and something big has to go, there is a good chance it might be user research. Unless you’ve been frugal about things like recruiting expenses and tools, researchers might have to find another way to get to talk to users.
Don’t become addicted to third-party recruiters! Make sure to hedge against the potential risk of losing your recruiting budget by thinking creatively about how internal resources such as your sales and service colleagues can help you with the heavy lifting.
Myth #3 Debunked: Participant recruiting is worth the expense. It does not have to cost as much as you think. Be creative and resourceful. Don’t rely on just one source for recruiting participants. Use multiple sources. Recruit internally by relying on your existing user-contact channels. Recruit externally by finding prospective users of your product or people who are using competing products. Work within your company’s budget constraints. To prepare for times when money gets tight, ensure your backup strategy for recruiting from within your organization is in good working order.
Myth #4: Apple never does user research, so why should we?
This myth could not be further from the truth. In a Fast Co.Design article from 2014, former Apple Senior Designer Mark Kawano says, “Everybody there is thinking about UX and design, not just the designers.” From the engineers to the product people to the senior leaders, thinking about design is in their DNA. Companies similar to Apple have been doing user research for years. Microsoft is a great example of another company whose culture has UX design and research built into its organization. They have a dedicated Web site just for recruiting potential research participants. When an organization’s daily decisions consider UX implications, user experience comes from the bottom up, not as an afterthought. The outcome: automatic, reflexive decisions based on what users really want and need, not a short-sighted patchwork of workarounds. If an organization doesn’t listen to its users, senior leaders will once their product fails to catch on because the team made decisions built on faulty assumptions about users.
Companies that say they do UX research and have all the fancy equipment deserve a deeper look. For example, the rise of innovation labs offers only visual proof that companies might be doing user research. Research firm CB Insights has published 65 examples of companies, both new and well established, that have opened innovation labs. Some of these are companies you would expect to see on a list like this: Google, Xerox, Oracle, and IBM. But there are also surprises such as Home Depot and the supermarket giant Tesco. My organization has opened innovation labs across the USA to jump-start efforts to rethink product experiences.
Just because all these companies are laying down hard cash on building labs, that doesn’t necessarily mean they’re doing actual user research. Their labs may have one-way mirrors and high-tech recording equipment, but some companies may have ulterior motives for opening these tech centers. For example, they may use them to upsell their existing customers or show them what might be coming in a future release. Only if you have an insider’s knowledge about these companies could you truly know whether they are listening to their customers through disciplined, well-orchestrated research programs. What’s really important is whether the practice of user research has been acculturated throughout the organization—and, if it has, what kind of impact the research is having on the company’s sales and strategy.
Myth #4 Debunked: Apple does UX research. It’s a part of their cultural blueprint—and it should be part of your organization’s culture, too. If you want to know for sure whether an organization’s culture embraces UX research, you will need to dig deeper—beyond seeing their expensive, depreciable lab investments. You will need to ask the people who work there.
Myth #5: Forget about defining the problem! Let’s skip right to the solution!
Where I work, there is a sticker inside the door handle that says: “Fall in love with the problem. Not the solution!”
Product stakeholders tend to fall in love with their solutions. When they are planning features to build, they can easily get stuck in the mire of user stories, happy-path scenarios, and edge cases, forgetting all about the problem they’re solving. Then, when the time comes for them to listen to all the rich, detailed insights about their product from exploratory research, they may remain fixated on understanding whether their solution is the right one. They can become victims of what psychologists call confirmation bias—grasping at any evidence from research that upholds their existing beliefs as definitive proof of their solution’s success and adopting it as a means of justifying their product decisions going forward. Their bias may cause them to ignore everything else or dismiss it as unimportant. This narrow view can eliminate a wealth of potential ideas for solving customers’ problems. Unfortunately, as long as people remain fixated on their preconceptions about the solution, they’ll never know whether alternative solutions are viable or perhaps even more desirable for their customers.
You cannot necessarily blame product stakeholders for being solution focused. After all, it’s a part of their job description. But, to counter their solution-centric view of the world, user researchers should be problem centric and remain laser focused on understanding the problem space. Is there a way to somehow align the interests of solution-obsessed product stakeholders and problem-loving researchers, so product stakeholders can identify solutions that solve real user problems?
Using Methods That Resonate Better with Product Stakeholders
Some research methods may resonate better with your product stakeholders than others. For example, one research method that helps product stakeholders leverage their solution-centric mindset during research is the jobs-to-be-done, or JTBD, method, pioneered by Anthony Ulwick. The underlying principle behind JTBD is that customers hire products to get jobs done for them. According to innovation guru, Anthony Ulwick, author of What Customers Want: Using Outcome-Driven Innovation to Create Breakthrough Products and Services:
“A company must first be able to figure out just what outcomes customers want the product to satisfy. In other words, companies need to figure out what jobs customers want to get done and how they measure success in getting a job done before they can determine what solutions customers want.”
If we think about products as people we hire, this implies that the customer has the choice to use your solution or someone else’s. JTBD lets us understand which solutions are better by measuring how well each solution solves a real problem—or, in the language of JTBD, a job. It ties the solution, or outcome, directly to the problem, allowing product stakeholders to better appreciate the problem. It helps them to understand the criteria that customers use to value a product and broadens their thinking to more creative solutions. Solutions, in the form of customer inputs, become “concise, actionable, and measurable.”
How does JTBD help product stakeholders overcome the pain of UX research? In the JTBD framework, product stakeholders can use statements of customers’ desired outcomes to help them generate a multitude of solutions. In a past research study with human-resources professionals who are responsible for getting new hires into their system, participants told me that one of their key user needs is getting the new-hire data into the system as quickly and accurately as possible.
A product stakeholder not using JTBD might interpret that as a need to provide a single solution that addresses both of these issues: Automate the entry of new-hire information into the HR system. Though that statement might be too vague to really know for sure. Framing the problem as a set of JTBD desired outcomes makes the statements clear and concise: Minimize the number of errors when entering new-hire data into the system.Increase the speed at which new-hire data gets into the system. Product people can immediately discern what outcomes HR professionals value and provide solutions that focus on those outcomes. While automating the flow of information into the HR system is important, stakeholders need to consider accuracy as well. Framing the solution as a JTBD desired outcome helps product stakeholders broaden their thinking to a range of possible solutions by pulling them out of their narrow mindset.
Myth #5 Debunked: Start by understanding the problem space before jumping to solutions. Learning about the users’ painpoints and what is not working for customers lets you discover their unmet needs. The real, rich stories you hear during research sessions can reveal a wealth of information about the problems your customers face when using your product. Considering what jobs users need to accomplish will ultimately broaden your thinking and help you fine-tune your solution. Frame problems in the language of product stakeholders by tying them to desired outcomes that are measurable, actionable, and concise.
Note to product stakeholders: Let solutions grow out of user needs. If you have a solution in mind, be aware of your biases and set them aside. The problems you discover through research can engender fresh solutions or refinements to existing solutions.
Conclusion
The five myths I’ve discussed in this two-part series are not easy for organizations to overcome. Teams must overcome many barriers to even begin debunking the myths that prevail in their organization. Financial barriers may prevent companies from investing in participant recruiting. Cultural barriers may reflect systemic issues that companies can solve only through major organizational overhauls. For example, corporate egomaniacs and their idolaters may believe they are the reincarnation of Steve Jobs.
If your organization suffers from any of the painful outcomes these myths perpetuate, when attempting to counter them, it is best to start small. Work within the bounds of what you can control and influence by building a tribe from the ground up. People at higher levels of your organization who witness small changes in mindset and see slow, gradual changes over time may be more accepting of them than fast, abrupt changes. Change won’t happen overnight, but it can happen over months and years. Change can start with you: the UX professional who debunks harmful myths that give User Experience a bad name.
References
Ulwick, Anthony. What Customers Want: Using Outcome-Driven Innovation to Create Breakthrough Products and Services. New York: McGraw-Hill, 2005.
Michael has worked in the field of IT (Information Technology) for more than 20 years—as an engineer, business analyst, and, for the last ten years, as a UX researcher. He has written on UX topics such as research methodology, UX strategy, and innovation for industry publications that include UXmatters, UX Mastery, Boxes and Arrows, UX Planet, and UX Collective. In Discovery, his quarterly column on UXmatters, Michael writes about the insights that derive from formative UX-research studies. He has a B.A. in Creative Writing from Binghamton University, an M.B.A. in Finance and Strategy from NYU Stern, and an M.S. in Human-Computer Interaction from Iowa State University. Read More
