With the exotic, mixed realities that futurists and science-fiction writers have envisioned seemingly just around the corner, it is time to move beyond questions of technical feasibility to consider the value and impact of turning the realities of everyday social settings and experiences inside out. As with all new technologies as they move from the stage of technical probe to social probe, this AR transformation will happen case by case and context by context, involving many factors beyond the direct reach of UX design. However, as a result of the inherently social nature of augmented reality, we can be sure the value and impact of many augmented experiences depends in large part on how effectively they integrate the social dimensions of real-world settings, in real time.
Psychologists describe people’s ability to engage in and adapt to social situations and interactions well as social maturity or social intelligence,
distinguishing social maturity from other aspects of maturity such as age, intelligence, and emotional acuity. I believe we can use the idea of social maturity to help frame and design the coming wave of social augmented experiences. In an earlier column on UXmatters, “Inside Out: Interaction Design for Augmented Reality,” I said that the majority of then-current AR experiences relied on a small set of interaction patterns, limiting their possibilities. I also suggested that missing patterns or antipatterns—such as Loner, Tunnel Vision, and Secondhand Smoke—lower the potential value and relevance of AR experiences. In combination, these AR interaction patterns and antipatterns can sometimes result in antisocial experiences and behavior within social settings. Or, using our borrowed metric, AR experiences often show fairly low levels of social maturity or intelligence.
Since broad-scale augmented reality is still in its pioneer stage, I think it’s more constructive to say that the social maturity of current augmented experiences is similar to that of a young child who is learning the complex rules and norms that determine socially acceptable behavior. With unevenly developed abilities and understanding, fitting into social situations is very difficult. Augmented reality is experiencing similar challenges. This is no slight. The social aspects of many more mature media are still neither fully evolved nor understood. However, it does mean that there is much ground to cover for augmented reality to reach social maturity. Looking ahead, there are two possibilities for the future of augmented reality:
- Augmented reality could mature socially and develop a proper understanding of social dynamics, enabling the creation of AR experiences that can successfully integrate with the social sphere—thus, providing truly social augmented experiences—and accelerating the growth and relevance of augmented reality.
- Without substantial social integration, augmented reality might remain restricted to a class of specialized utilities that are better suited for focused, asocial or semisocial activities like technical reference—one of the primary applications of augmented reality from the beginning.
Once we can look back on this period in the evolution of augmented reality from the future, in retrospect, perhaps we’ll perceive the question of social maturity as an inflection point in the overall development of AR experiences.
One example of a newly feasible AR capability for mobile devices that has substantial social dimensions is facial recognition—using computer vision to identify the people in a camera’s field of view, then presenting information about them. Recently, Google launched Google Goggles—which uses their very large product, image, location, and price databases to identify items like books and CDs—with its latent facial-recognition capabilities deliberately disabled. At the moment, Google believes it is exhibiting greater social maturity by publicly acknowledging the newly possible interactions, but not supporting them.
On the other hand, Comverse’s Face.com and TAT’s Recognizr application, shown in Figure 3, are wholly based on facial recognition and allow the quick retrieval of linked information such as social-network profiles, status updates, and shared content. While both Face.com and Recognizr are opt-in services and offer the ability to manage both your participation and your image’s associated profile information, their value premise is dependent on the very same ability Google has chosen to disable.
We can understand the business reasons for both decisions, but simply turning a capability on or off fails to address the contextual layers and nuances of social and emotional settings. Nor does it present the opportunity to learn from and refine the interactions augmented reality allows us to design. What, though, does socially mature mean for facial recognition?
Let’s consider some of the AR facial-recognition experiences coming to market, calling attention to the interactions, relationships, and contextual models we must consider when designing a new technology we’re about to introduce into complex, living-and-breathing social settings.
The core usage scenario for current AR facial-recognition product offerings goes something like this: A user, holding a smartphone or other device camera, focuses on a person’s face, in close view, long enough to register and recognize the person’s facial image, then waits while the service interrogates that person’s digital identity profile—which he or she has possibly defined, but may be ad-hoc and aggregated. The user then lowers the phone and visually scans the profile or other information the facial-recognition application has retrieved, perhaps choosing to focus on a specific pool of Facebook or another service’s shared-lifestream content, perusing it to get a sense of a person’s status, interests, and usual activities. Finally, the user may decide to engage in conversation with that person—or may perhaps choose not to do so.
The contextual model for such an interaction assumes that interrupted or delayed conversations are socially acceptable—though asynchrony is one of the most frustrating aspects of poorly mediated interactions. (‘Can you hear me now…?’) This scenario also depends on the directed, gaze-based, device-powered scanning of faces at close, personal distances. For strangers who have no affiliation, I’d wager this experience is too similar to having a policeman stop you and ask you for your identification for most people to consider it an acceptable social interaction. Even among people with existing, but weaker ties like the indirect relationships that loosely link colleagues in a large organization, this feels like a depersonalizing and inherently suspicious way of greeting someone. Using a device in this way also creates a physical barrier between people and shifts the focus of one’s attention away from the person to the device and the information it presents. This hardly feels like an improvement over the everyday act of introducing yourself or joining a conversation group at a party.
Contrast this AR experience with the interaction models and user experience of the facial-recognition capabilities of desktop photo-management tools like iPhoto or Web-based services like Picasa. By default, these applications are much further removed from live social contexts, relying on asynchronous augmentation that neither disrupts the flow of social interactions nor invokes the same level of social meaning. They offer similar capabilities, but their social impact is much lower.
To be fair, it’s true that AR facial recognition and profile data retrieval could feel appropriate or even enhance the value of a complete experience in many social settings—for example, at a mixer event or when playing a pervasive game, attending or making a public presentation, or entering a shared workspace as a new member for the first time. But how can we know when and how to augment social experiences?
