Avoiding User Research No-Shows

Using Reminders to Prevent No-Shows

Jon: It’s a safe bet that a single communication with participants isn’t enough in our modern culture of distraction. What does a good reminder communication look like for participants who have signed up? How can you ensure that participants actually see and read it?

Katie: We don’t rely just on using online communication. We also use voice communication to connect with research participants. Establishing a rapport with participants helps them feel more engaged and better understand the importance of their feedback. However, with fewer people answering phone calls from numbers they don’t know, it could be hard to connect in this way. A simple and effective way of ensuring you know whether participants have read your message and intend to show up for their session is to ask them to reply to a confirmation question.

Patricia: I agree with Katie’s thoughts on how to reach participants. Depending on a study’s timeframe, how far in advance and how often you should send reminders can vary. There’s a delicate balance between wanting to stay in contact with participants, but ensuring you don’t overwhelm them. For participants I’ve never interacted with before, I’ve found that they generally appreciate receiving a reminder the week before a study, reiterating the study details and what you expect of them and providing additional instructions.

These reminders are usually email messages, but I close them by asking participants to let me know that they’ve received and understood the message and providing options for responding by email, text, or phone, so they feel in control of how we’re communicating. Then, the day before a session, I use whatever mode of communication a participant used in responding to my previous email message to request another brief, follow-up confirmation. I close that message by reminding the participant that I know life happens, so if any conflict arises with our meeting time, to please let me know. If I’m meeting with a familiar participant, I might decrease the number of reminders, but I still confirm a few days beforehand.

Jonathan: Of course, there are email communication tools—often for marketing—that track email opens and link opens. While sending reminders is generally good, there is a delicate balance between reminding participants and annoying them. In case people have completely forgotten about their session, it’s good practice to remind them about what they’ve signed up for, what you’ll expect of them, and when they should show up. Of course, the subject line of such an email message is quite important. Make sure it doesn’t sound spammy and that the email message sounds credible. Participants should get a sense of how the session would benefit them—whether it’s physical swag or getting to help improve a product they know and love. As both Katie and Patricia have mentioned, requiring that participants reconfirm that they’re still planning to attend can reduce heartache later.

Preventing No-Shows When Conducting Research at Events

Jon: At Rockwell, many of our best research opportunities come at training events and trade shows, when participants are likely only about a ten-minute walk from the study location. But no-shows still happen. Why do you think this is?

Katie: It could be that participants do not necessarily understand the importance of the sessions we set up. Some people might not realize we’re depending on them for these one-on-one sessions and think that others would join us if they’re not able to participate. Therefore, it is very important to emphasize that we’re setting up these sessions to accommodate their schedule, and we’re relying on them to show up to provide their feedback.

Jonathan: The cause of no-shows could be as simple as people forgetting or their getting distracted while on the show floor, or they might have agreed to participate before looking through the program schedule. They might encounter an exciting new product, technology, or presentation along the way. Being on a show floor could also give the erroneous impression that research events—similar to other events during the show—involve larger groups of people, so they might believe their presence wouldn’t be missed.

Patricia: A plethora of reasons might result in a no-show in such a situation, but I don’t think this is intentional. To these participants’ credit, if they’ve never participated in a research session before, they don’t know all the behind-the-scenes effort it takes to set one up. At large trade shows and training events, there is so much hustle and bustle and so many different events occupy people’s time, but there are no clocks! The events keep people’s attention on the main floor, so unfortunately, despite our best efforts, participants who have confirmed their participation sometimes do not show for their session, even if it’s only a short distance from the floor.

Handling Participants’ Being Late for Sessions

Jon: Let’s pretend that ten minutes have elapsed since the agreed-upon starting time for a study, but the participant still has not shown up. What’s your next move?

Patricia: If you haven’t already done so, take a deep breath and remember that any number of things could have happened. Then, look back over your correspondence with the participant to see whether there was possibly a miscommunication. Attempt to reach out to the participant, by any means available, to find out whether you provided incorrect information. But don’t take a panicked tone! Never blame the participant. Always take the fall as the researcher or coordinator. Hopefully, you’ll be able to get hold of the participant, who is on the way. Regardless, your next step is to review your test plan, decide how to adjust it to accommodate a shorter timeframe, and keep a friendly face on in case the participant arrives at any point during the designated session time.

Katie: Along with what Patricia says, if you aren’t able to get hold of participants, simply send them another email message, letting them know you’re still waiting for them and will be there until a specific time. If possible, you can also give them the option to reschedule—perhaps for a more convenient time online.

Jonathan: As both Katie and Patricia have mentioned, a participant could simply be delayed for any number of reasons or just be a bit lost. Ideally, you should send a quick reminder to the participant—whether by email, phone, or a text message—to verify their status and go over the time and place of their session. If you’re expecting the participant to meet you in person, you can also scope out the area to see whether there is anyone looking lost nearby, who may be looking for you.

Communicating with Participants Who Are Late

Jon: What is the ideal communication to a participant who has yet to show up—and may not show up at all?

Katie: Once you realize participants are not going to show up at all, send them an email message, asking whether they would like to reschedule. That way, if they were going to join extremely late or just forgot, they’ll have a way to follow up with you, and you won’t have to wait around for the entire session time. Say something like: “Hello, I’m sorry we missed each other today. We’re still very interested in your feedback. If you have [30 minutes] to talk with us in the next [week], we would be happy to schedule a time. Please let us know what times would work best for you!”

Patricia: To reiterate my sentiments regarding the last question, as the researcher, it’s my responsibility to let any blame fall on me. With that in mind, my ideal communication might look something like this: “Hello again, [participant’s name]. This is Patricia with [company name]. I’m [calling / texting / emailing] you regarding our research session today because I want to make sure that there was no mix-up regarding the information I sent you. We had scheduled our meeting for [time], at [location]. Since it’s past our starting time, I want check in and make sure you’re okay and see whether I can provide any further instructions on how to get to the [location]. Thank you!”

Jonathan: As Patricia mentioned, it’s important to frame any communication with a participant who is potentially a no-show as an oversight or error on your part. Encourage the participant to join the scheduled session and provide any information the participant would need to successfully join the session, whether physical directions or a dial-in number for a remote study.

Avoiding No-Shows When Conducting Remote Studies

Jon: Are there any tips you can share for avoiding no-shows for remote studies? Do your tactics change in any way in comparison to in-person studies?

Katie: For remote studies, I still use the same tactics that I described earlier. The biggest thing is making sure, whenever possible, that the participant is the one choosing the time. That way, you know the timing should work well, and you won’t schedule against things that might be hard for the participant to miss. Another thing I like to do is to send the invitation to just me and the participant, so that person knows I am relying on him or her to participate. Then, I send a separate invitation to stakeholders who are planning to listen in, providing the Zoom link. So it is clear to the participant that this is a one-on-one session, and I’m depending on that person to show.

Jonathan: As Katie was saying, remote studies can largely follow the same techniques we’ve already discussed for in-person research sessions. One nice thing about remote sessions is that you can be more flexible on timing on your end because you won’t have to travel to a particular location or worry about your participant not being able to find your building. Plus, doing a study remotely potentially makes it easier to use backups or floaters because your pool of participants is not restricted to those who can show up in person. Whether you’re doing a session remotely or in person, it really helps to send reminders and receive a confirmation from your participant before the session. If someone has not worked with you before, be prepared for technology issues relating to remote screen sharing or video conferencing. Keep some troubleshooting steps in mind in case your participants have trouble.

Patricia: I’ve actually found that I get fewer no-shows for remote studies—probably because participants don’t typically have to go far out of their way to attend a session. Nevertheless, my communications remain fairly consistent whether I’m recruiting for in-person or remote research. If anything, because remote studies place a burden on participants to have a handle on technological tools—such as how to open the Webinar link, check their settings for Web cams or other video equipment, and do screen sharing—I try to provide additional information on those tools in my communications, so they feel familiar and comfortable.

Handling No-Shows

Jon: Are there any additional tips or suggestions you can offer—not only for avoiding no-shows but also for navigating those situations in an agile way when they do occur?

Patricia: Be sure to plan for no-shows! This means planning for them both mentally and physically. As a user-research facilitator, it should always in the back of your mind that—no matter how well you’ve prepared or even over-prepared for your research sessions—there can always be no-shows, so take them in stride. When they do happen, see whether you can follow up with the participant to reschedule for a later time or seek out a different participant to fill the void. You should also plan for participants who might arrive late for sessions by preparing an abbreviated script or activity to use if you don’t have time to complete the full script. A great resource that I constantly return to is The Moderator’s Survival Guide: Handling Common, Tricky, and Sticky Situations in User Research, by Donna Tedesco and Fiona Tranquada!

Jonathan: No-shows do happen from time to time, regardless of how much you’ve prepared, and can be disappointing. As Patricia mentioned, if participants are delayed, being prepared pays off. In advance, assess what research activities you should prioritize for an abbreviated session. If participants fail to show up altogether, consider giving them a second chance by reaching out to see whether they want to reschedule. But if participants consistently fail to show up, consider placing them on a no-show list, then check against it when you’re building your next recruitment list. If your studies consistently have many no-shows, you might need to go back to the drawing board and determine how to change or modify your recruitment process to alleviate this issue.

Katie: The best tip I’ve received is to be sure to make a personal connection with your participants. Whenever you can, give participants a quick call to introduce yourself and talk about the research you’re doing. This conversation might also lead you to other participants. If you explain to the participants you’ve already selected what types of users you’re looking for, they might have contacts who would be interested in working with you as well. Ensuring that you’re respectful of people’s time and participation helps ensure that people would want to participate—perhaps across multiple studies!

Conclusion

As Patricia has pointed out, life happens. You cannot control every factor that could impact participants’ ability to show up for your user-research activities. However, you can strive to mitigate recurrences of no-shows through communication that is tactful and timely, but avoids overwhelming spam-wary participants, as Jonathan has suggested.

By establishing a personal connection with your participants, you make it more difficult for them to blow off your research activities. As Katie has recommended, communications that reinforce participants’ importance to the success of a research activity goes a long way toward making participants feel both valued and accountable. As UX professionals, our putting users front and center is critical to the success of our products and services. Structured user-research activities are often our best opportunities for gaining valuable face time with users—as long as they show up!