In many respects, conducting UX research internationally is much the same as conducting studies in your own country. You can apply the same basic principles of human-computer interaction and UX research techniques in studying people’s behavior in any country. However, there are logistical challenges with international UX research and important differences that you need to consider. In this two-part series, I’ll provide advice about conducting international UX research studies.
Determining Who Should Moderate Your Sessions
Most UX researchers prefer to moderate their own sessions because it gives them maximal control over their research. However, for international studies, you may need to hire local UX researchers to moderate the sessions within each country in which you’re conducting research, while you observe and take notes. To determine whether you should moderate the sessions yourself or hire local moderators, ask yourself the following questions.
Champion Advertisement
Continue Reading…
Do You Speak the Participants’ Language Fluently?
To enable participants to freely express their thoughts and feelings, you should conduct the sessions in the language in which they’re most fluent and comfortable speaking. Even if participants speak a second language, don’t force them to speak that language if they would struggle to come up with the right words to express their thoughts. Therefore, if the participants speak your language fluently or you speak their language fluently, you can conduct the sessions yourself. If not, you would need to hire a local moderator who speaks the participants’ language fluently.
Do the Participants Speak Different Languages?
In some countries, different parts of the population speak various languages. So you might encounter some participants who speak one language and others who speak another language. Unless you speak both languages fluently, you should hire a local moderator to conduct the sessions in whatever languages you don’t speak. If that moderator speaks both languages, you should ideally have that person conduct all of the sessions for consistency.
For example, a couple years ago, I conducted UX research in India with a local moderator who spoke with the participants in English. Even though they all spoke English fluently, they sometimes slipped into speaking Hindi when they had difficulty thinking of how to express a particular thought in English. The moderator conversed with them in Hindi for a few moments, then summarized in English what they had said in Hindi—both as a way of confirming their meaning and to translate for us. For example, the moderator said, “So what you’re saying is: you don’t like how the….”
How Different Is the Participants’ Culture from Yours?
Even if you can speak the participants’ language fluently, is their culture different enough from your own that a local moderator would be better able to interact with and understand the participants? Would the participants feel more comfortable with and have a better rapport with a moderator from their own culture? Might the local moderator pick up on subtle cultural issues and cues that you would not notice? In such situations, a local moderator could redirect the flow of a session accordingly, ask appropriate follow-up questions, and have a better understanding of the findings.
What Is the Focus of Your Research?
In determining how important it is for you to understand the local culture, consider your research goals and methods. If you’re conducting usability testing on a Web site, application, or mobile app, your research goals would usually focus narrowly on evaluating the user experience of the user interface. Of course, there might be cultural issues that would impact the user experience, but the basic principles that determine how people interact with and understand technology are similar across countries—as long as they have similar levels of experience with technology. For example, a UX researcher from the United States, who speaks French fluently, could successfully moderate usability-testing sessions for a French Web site with French participants—without there being a large cultural gap between them that would affect the results.
However, if you’re conducting ethnographic research, for which your research goal is to understand the participants’ lives, work, culture, and use of a particular technology, it is much more important that the moderator already has some understanding of their local culture. For example, a UX researcher from the United States might be able to conduct ethnographic research in countries such as Canada or the UK. Even though these countries have cultural differences from the United States, their societies aren’t so different that the researcher couldn’t understand and overcome such issues. However, most UX researchers from the United States would have a more difficult time conducting ethnographic research in a very different culture such as Indonesia, unless they first studied the local culture extensively. In this case, it might be better to partner with a local moderator, who could help you understand the people and their culture.
What Is Your Research Budget?
Finally, if you’re not going to conduct the research sessions yourself, you must consider the costs of hiring a local moderator. If the moderator would conduct the sessions in a language that you and the other observers don’t speak, you would also need to hire a translator. Nevertheless, if you hire a local moderator, you might be able to observe the sessions remotely through online-meeting software, which would save you and the other observers the cost of traveling to the sessions.
Deciding Whether to Conduct Sessions in Person or Remotely
Whether you moderate the research sessions yourself or hire a local moderator, you can travel to conduct the sessions in person or conduct them remotely. You have the following options:
Everyone is remote. The moderator, participants, and observers conduct the sessions remotely, using online-meeting software. If you hire a local moderator, you and the observers can monitor the sessions through the online meeting.
The moderator and participants are in person together. Everyone else—including you if you hired a local moderator—would observe the sessions remotely through online-meeting software.
You must travel to in-person sessions. You might either moderate the sessions yourself or observe them as a local moderator conducts the sessions.
In deciding whether to travel to conduct or observe the sessions in person or to conduct or observe them remotely, consider the following questions.
What Are Your Research Objectives?
Are your research objectives narrowly focused on evaluating the user experience of a product or user interface, or is your goal to understand the participants’ context—including their behavior, work, personal lives, and culture? Remote UX research is very effective for evaluating the user experience of a product or Web site and interviewing participants to get their opinions about design concepts. In such cases, your objectives focus narrowly on the design and the user experience. However, conducting in-person sessions is much more appropriate if your objective is to understand your participants’ current context and behaviors.
Do You Have the Budget and Time for International Travel?
International travel is obviously much more expensive and time consuming than conducting research remotely. It’s also physically taxing to travel long distances and adjust to large time-zone differences. In addition to the travel time, consider the time and effort that is necessary to book travel, get a passport and visa, receive the necessary immunizations, and handle the legal issues of conducting business in another country.
At What Times Should You Conduct the Sessions?
Of course, it can be difficult to adjust to jet lag. But, when you travel to participants’ location to conduct UX research, your sessions should take place at reasonable times during their day or evening. With the help of caffeine, you can stay awake and be present for the sessions.
Remote research sessions might need to occur at inconvenient times for your time zone—such as the middle of the night. Therefore, it might not be possible for you to moderate those sessions effectively. If you hire a local moderator, you might be able to observe some late night or early morning sessions. Then the next day, you can watch the recordings of any sessions that occurred during the middle of your night.
In How Many Locations Do Your Participants Reside?
It’s much easier to travel to one country, then travel to multiple locations within that country, than it is to travel to multiple countries. The cost, time, and difficulty of traveling to multiple countries could make doing remote research much more practical. For example, I recently conducted remote research in seven different countries, with local moderators in the non-English speaking countries. It would have been far too expensive and time consuming to travel to all of those countries.
Hiring a Local Moderator
If you need to hire a local moderator, it’s crucial that you hire someone who is very good at moderating UX research. It can be difficult to give up the control of moderating your own sessions, so you would need to hire someone you can trust to do it well. There’s nothing more frustrating than to observe helplessly as a poor moderator ruins your careful research planning. Yes, you’ll be able to communicate with the moderator during sessions. Plus, you should correct any issues with the moderator between sessions, but you’ll have only so much control at that point.
Getting References
Ideally, hire someone who you’ve used before and found to be a great moderator. If you don’t have someone you can call on, there’s nothing more reassuring than getting referrals to a moderator from fellow UX researchers that you trust. It’s a good idea for companies to maintain lists and reviews of international moderators that you should use and which you should avoid.
Evaluating Possible Moderators
If you can’t get a referral from a colleague, you should interview and evaluate all potential moderators. Often, recruiting companies or research facilities recommend using their own moderators. However, it’s still important to evaluate these moderators to see whether they would be the right fit for your project.
First, review each moderator’s resume and background. Is their background in UX or marketing research? Note the research methods they’ve mentioned in their resume. Do these researchers use the terms user experience, UX, user research, or usability testing in their resumes? Or do their resumes mention only focus groups and interviews, which might indicate a marketing-research background? Look at the types of projects they’ve worked on to see whether they’re similar to your project.
Then, interview each of these moderators to learn about their experience. You might want to make this seem more like a casual conversation between colleagues rather than a formal job interview, but you would need to learn similar things as during an interview. Learn about their background, whether their focus has been mainly on UX or marketing research, which UX research methods they’ve used, what types of projects they’ve worked on, their level of experience with the subject matter of your research, how they deal with difficult participants, how they would prefer you to communicate with them during sessions, and how they would handle requests for changes to the research between sessions. In addition to assessing their experience and research skills, you should try to get a sense of what these moderators would be like to work with. Would they be pleasant and professional or difficult to work with?
Hiring a Translator
If you hire a local moderator to conduct your research sessions in a language in which you and your observers are not fluent, you would also need to hire a simultaneous translator. This person would listen to the sessions and translate what the moderator and participants say into your language, nearly in real time.
Simultaneous translation can be useful for either in-person or remote sessions. One method of conducting remote sessions is to run two online meetings simultaneously. In one meeting, the moderator and a participant conduct a session in their own language. You and your observers join this meeting only to observe, without connecting to the meeting’s audio. In the other online meeting, the translator speaks in your and the observers’ language. You and the observers connect to this meeting’s audio and listen to the translation. Thus, you and the observers can watch the session between the moderator and participant and listen to the session with the translated audio. This approach might sound a bit complicated, but it works very well.
It is important that you hire a good translator. In addition to being fluent in both languages, the translator must be able to listen to what the moderator and participant say, then quickly and accurately translate it to your language. The best translators use different voices or types of emphasis to indicate whether they are speaking for the moderator or the participant. Otherwise, it can sometimes be difficult to determine who is saying what.
Get recommendations and read reviews for all potential translators. Hopefully, your company maintains such information. Recruiting companies and research facilities usually have staff translators that they recommend. If these are reputable companies, you can trust that they would provide a competent translator. However, it’s still a good idea to review these translators’ resumes and interview them. You might want to give them a test to see how well they translate or watch a video of them translating research sessions in the languages your sessions require.
What Are Your Next Steps?
Once you’ve determined who should moderate your international UX research sessions and whether you should travel to the research sessions or conduct them remotely, your next steps are to recruit participants, prepare your local moderator for the sessions, set up the necessary technology, and observe the sessions. I’ll provide advice on these tasks and more in Part 2 of this series.
This couldn’t have come at a better time. I’m preparing to conduct focus groups for people around the world—grouped by region. We don’t have a budget for a translator or local moderator, so I’ll be moderating the sessions. The participants will be non-native English speakers with varying degrees of English proficiency. I’ve been considering whether to allow both typed and vocal responses or just typed—similar to a Webinar format. Also, what form of payment and currency is appropriate?
Excellent article. Thanks for sharing your knowledge. Your experiences match mine very well.
To summarize: When you do International UX Research:
Use a moderator who speaks the native language of the test participants fluently.
Use a moderator who understands the local culture.
If you use a local moderator, do a pilot-test session where you observe them.
Ask the local moderator to comment on your usability-test tasks; this works both ways: you may be able to improve the tasks and adapt them to the local culture, and the comments will give you an impression of the qualifications of the moderator.
Use simultaneous translation; get the best translator that money can buy.
But always keep in mind: The perfect is the enemy of the good, so nonoptimal UX testing is better than no UX testing.
Jim has spent most of the 21st Century researching and designing intuitive and satisfying user experiences. As a UX consultant, he has worked on Web sites, mobile apps, intranets, Web applications, software, and business applications for financial, pharmaceutical, medical, entertainment, retail, technology, and government clients. He has a Masters of Science degree in Human-Computer Interaction from DePaul University. Read More
