The World Wide Web offers a virtual space in which immigrants can construct a hybrid culture. For example, immigrants may express their identity through a virtual diaspora community. Such a virtual community may offer discussion boards, social networking, links, and other content, bringing together the immigrant’s original culture with that of the host society in near synchronous interaction. The sites and types of interactions in which participants choose to engage depend on a complex set of factors that have both cultural and functional import.
Recent research in usability attests to a growing recognition of the importance of understanding how migration affects and is affected by networked technology. HCI (Human-Computer Interaction) researchers Irina Shklovskii, Janet Vertesi, and Silvia Lindtner argue that cross-cultural interconnectedness across the globe necessitates a new approach to usability, which they call transnational HCI. Transnational HCI is the application of knowledge, practices, and methods from the field of Human-Computer Interaction to the analysis of immigrant users or users who routinely interact electronically across cultures that are separated by distance. At the heart of transnational HCI is a user base that has a flexible, multi-faceted sense of cultural identity. Transnational HCI focuses less on anthropological, rhetorical, historical, and sociological factors—the why of cultural interaction—and more on how users interact with technology and each other to accomplish specific tasks.
In this article, I’ll demonstrate how transnational HCI can guide usability for immigrants through data that I gathered during a study of Russian-American users on the World Wide Web. According to Ameredia, the Russian-American population represents the second largest ethnic market in the United States—behind Mexicans—comprising 10.3% of the total foreign-born population.
As people with footholds in two or more cultures and high degrees of cultural awareness, Russian-Americans regularly use a variety of Web resources having different cultural identities to accomplish specific tasks. According to ISO standards and UX methods, to what extent and in what ways do multiple cultural identities affect user interactions among Russian-Americans? In what ways could the type of study that I conducted inform a transnational HCI approach? And finally, what do the results of my study suggest UX professionals could do to better meet the needs of immigrant users?
Measures of Usability
In my study, I applied ISO-based, usability-testing principles to Web sites for the Russian-American immigrant user group. According to the ISO 9241-11 definition of usability, testing measures three things:
- effectiveness—the accuracy and completeness with which users achieve specific tasks
- efficiency—the number of resources, including time, that users expend in completing their goals
- satisfaction—the degree to which users feel an application, service, or site meets their hopes and expectations
In my study, I compared measures of effectiveness, efficiency, and satisfaction across the cultural domains of the Web sites.
Effectiveness
I measured effectiveness in terms of how often users correctly completed the tasks. Using this measure, user performance was best on the Russian sites, with 93% of participants successfully completing tasks. In second place, the US sites had a completion rate of 87%, while the completion rate for the Russian-American sites was 80%.
Efficiency
Often, by setting task goals prior to usability testing, it’s possible to establish optimal completion times for the tasks. For example, a development team may decide that it wants users to be able to complete tasks within a certain timeframe or may ascertain an optimal completion time from previous studies. Because of the scarcity of prior research among this immigrant group and the fact that the Web sites were not developed specifically for this study, it was not possible to establish optimal completion times for tasks.
As an alternative, I used the lostness metric to determine task efficiency. As Tom Tullis and Bill Albert discussed in their book Measuring the User Experience, lostness measures the optimal number of page visits that are necessary to complete a task against the total number of pages a user has visited. Task efficiency depends on the extent to which a user finds an optimal path to information. The lower the lostness score, the less lost the user felt on a Web site. The lostness data I’ve provided here includes only successfully completed tasks.
Users were most efficient with the Russian-American sites, which had a lostness score of 0.14. The Russian sites were next with a score of 0.19; followed by the US sites with a score of 0.21. Participants were more efficient using the Russian-American Web sites than the US sites.
User effectiveness correlated with efficiency somewhat among the Russian-American and Russian sites, indicating that the more users successfully completed tasks, the more efficient they were. For US sites, this correlation was weaker, though still positive, which may indicate that these sites were designed in such a way that inefficient users were still able to complete the tasks, but that this is not true of the Russian and Russian-American sites.
Satisfaction
The study rated user satisfaction in two ways. First, users comparatively rated their satisfaction with a site against the other sites, on a scale from 1, most satisfactory, to 3, least satisfactory. Users rated the Russian sites the best, with a score of 1.7, followed by the US sites, at 1.9, and the Russian-American sites, at 2.1. Users’ satisfaction rankings for Web sites strongly correlated with their effectiveness measures, indicating that the more successful users were in completing tasks, the more likely they were to rank a site highly. No significant correlation existed between user ranking and efficiency.
Second, I rated user satisfaction according to the frequency and content of open-ended user remarks. I subdivided comments into positive and negative categories. Users contributed 305 comments about the Web sites. I totaled the positive and negative comments, then calculated a percentage grade against the total number of comments for each cultural category. The US Web sites had the highest proportion of positive comments, at 75%; followed by the Russian sites, at 50%; and lastly, the Russian-American sites, at 38%.
| Russian-American | Russian | American | |
|---|---|---|---|
Effectiveness |
3 |
1 |
2 |
Efficiency |
1 |
2 |
3 |
Satisfaction |
|||
Rank |
3 |
1 |
2 |
Number of Positive Comments |
3 |
2 |
1 |
Note—1 indicates the best in a category; 3, the worst.
I further analyzed the comments, determining that they fell into three primary categories;
- organization—including navigation, findability, and technological quality
- content—including the aesthetics, quality, and integrity of the content, the prevalence of advertising, and its relevance to the purported topic
- cultural sensitivity—including cultural markers such as artwork and the provision of Russian or English language
The US Web sites received the highest proportion of positive comments, with 81% for organization and 69% for content; followed by the Russian Web sites, with 47% for organization and 42% for content; and last, the Russian-American sites, with 19% for organization and 39% for content. The Russian-American and Russian Web sites had the highest number of positive comments for cultural sensitivity, with 79% of comments being positive.
