The European Survey Research Association Conference: When Long-Term Means Years
The European Survey Research Association had their conference in Lausanne, Switzerland, from July 18–22, 2011. I wanted to attend the conference as part of my research for my forthcoming book on surveys, because it presented an opportunity to learn about the current concerns of the survey methodologists.
I was also delighted that a current client joined me in a presentation of one of our projects. You can take a look at our presentation, “Usability Testing of Market Research Surveys,”
on SlideShare.
The conference included more than 150 sessions, each with three or four papers, in up to 12 parallel tracks, so it was impossible to attend more than a small selection of them. So what to focus on? As I looked back over my notes, I spotted a theme: many of these presentations were about research that happens over years, decades, or lifetimes. To give just two examples from this conference:
- Todd Hughes from the US Census Bureau talked about the usability tests they conducted in 2010—not to inform the 2010 Census, but looking ahead to 2020. These things take a long time to design.
- Stephan Huber, a professor at the Institut für Bildungsmanagement und Bildungsökonomie in Switzerland, talked about the latest developments in a study of Swiss teenagers that started in 1854.
In this review of the conference, I’m going to discuss what I learned from three other presentations that gave me ideas for my own longer-term studies.
UX Tip #1: Try Asking Less, But More Often and Earlier
Jennifer S. Barber of the Population Studies Center at the University of Michigan talked about a study of 18- and 19-year-old women and what experiences might lead to pregnancy—especially unintended pregnancy—that is, unwanted or unexpectedly earlier than planned. Unintended pregnancy can be a health risk for mother and baby. The challenge is that, once a woman is pregnant, it’s too late to ask her about the finer details of her life in the months before the pregnancy.
So they set up a panel to track around 1000 women over five years. But what factors to look at? Ideally, the researchers wanted to know pretty much everything about the women’s lives, and one solution might have been to do periodic interviews that covered, well, just about everything.
They started with an hour-long interview with each woman at the start of the project. (I just realized that’s at least 25 person-weeks of interviewing.) The temptation might have been to continue with similar hour-long interviews at rather long intervals throughout the study. Instead, they opted for weekly 5-minute interviews—with a mixture of standard and variable questions each week—and gave participants the option of answering questions on the Web or by phone. This has worked very well. Five minutes isn’t too onerous for anyone; the variety of questions helps to keep the interview interesting; panelists like the regular incentive, even though it’s small—just $1 per interview, plus a small bonus every 5 weeks.
Tip for UX studies—If you want to learn about users’ engagement with your product over a longer-term experience, might you try asking your participants less, but more often?
