XR Design: Imagination, Immersion, and Presence

Levels of Immersion

Typically, the realms of AR, MR, and VR are progressively more immersive. In AR, we are first and foremost experiencing physical reality, but with the added twist of imaginative reality being overlaid on top of actual reality. AR is the least immersive XR experience. In MR, we have some ability to augment our reality by adding imaginative functionality that lets us visually manipulate objects—for example, creating a focus effect on gaze and a blur effect for our peripheral vision. MR is more immersive than AR, but less immersive than VR. Finally, VR lets us enter whole new worlds. In the realm of VR, we do much more than simply add imagination to our base reality; we can create an entirely new base reality in any way we could possibly desire. The only limit is our imagination.

Presence and Immersion

Every XR experience should start by defining presence, which is achieved when the user truly feels that she is somewhere in particular. This is a desirable effect within VR. To create a proper VR experience, you must first establish the setting the user is inhabiting. If you do not firmly establish where the user is, she might not feel that she is anywhere in particular, causing an eerie sense of disorientation and presenting a huge barrier to the rest of the planned experience.

Presence demonstrates where the user is at any given moment. The more clearly you can establish presence, the more immersive an experience will be for the user. When the user confidently knows where she is, you allow her to feel that she is actually there. Self-doubt in the context of physical orientation breaks our sense of immersion. Immersion comprises many factors, but is highly dependent on presence.

Designing for Presence in Virtual Realities

Achieving presence within VR is one of the most fundamental challenges that VR designers encounter—albeit a worthy challenge. With presence, the user no longer has the perspective of the third eye, but that of the first-person actor, who is fully immersed in the virtual environment.

So how do we achieve presence? There are two primary factors to consider when designing for presence in VR: place illusion and plausibility illusion.

Place illusion convinces the user that she is actually in a particular environment, while plausibility illusion persuades the user that the experience she is having in that place is believable without a doubt. There are quite a few factors to keep in mind when designing for place and plausibility. Let’s go over a few.

The first factor that I’d like to talk about is sensory modality. Within the interactions you design, how many of the user’s sensory capabilities are you engaging? A good mix of visual, auditory, and haptic stimulation can make the user feel less removed from the real world, but not if the use of multiple senses feels forced. Designers must creatively plan multimodal interactions so they feel necessary and enjoyable.

A common mistake of early VR experiences has been to be overly concerned with 360-degree space and rely too heavily on perfecting just the visuals. Be sure to consider how auditory and haptic modalities can enhance the overall experience by complementing and expanding upon its visual component. The goal should be to design a fluent, cohesive, multimodal experience that appropriately engages several senses in giving and receiving information within an XR experience. This is particularly important in achieving place illusion. When users engage their senses in your designed experience just as they would in real life, they begin to reach out and touch things as if they were really there, without hesitation and, most importantly, without questioning the mechanics necessary to pursue their desired actions.

In addition to creating multimodal XR interactions, it is important to ensure that responses to the user’s actions are as expected. For example, if the user throws a rock at the bad guy in a VR action game, his reaction should be anger and perhaps to threaten the user. This would be the expected reaction. However, if the bad guy were to instead pick up the rock, as well as a few other rocks, and start juggling, the user would begin to doubt the veracity of the experience, breaking plausibility illusion. Carefully considering each and every possible user action and designing a believable system reaction is key in achieving plausibility illusion and strengthening presence within an immersive experience.

Designing for Presence in Augmented Realities

Designing for presence in AR is a bit different from designing for presence in VR. In AR, the real world is the canvas. So, rather than convincing the user that she is actually in a world that is entirely separate from the one she knows to be real, AR relies on maintaining a connection to the real world. Instead of escaping the real world, AR uses it as a canvas. So we must define presence in a way that maintains some connection to the real world by demonstrating an awareness of certain attributes that we know to be true.

This doesn’t mean that, as an AR designer, you should decrease the amount of magic you apply when augmenting, enhancing, or somehow modifying the user’s perception of the real world. However, it does mean accommodating the real-world objects, physical laws, and other constraints that surround the augmented experience you’re designing. The two should feel meshed. At every opportunity, AR designers should look for ways in which users can interact with augmented elements in ways that acknowledge the surrounding, real-world elements.

Acknowledgment of the real world requires thinking about the real-world space in which the user resides, the context of use it implies, the emotions it may evoke, and the actions or behavioral patterns that may be common to this space.

For example, if you are augmenting an office space, you might consider some of the behaviors and emotions that are typical within this context. In the office, users might commonly check their email and look at their calendar. Providing augmented calendar and email-notification user interfaces that the user could swipe into focus to quickly glance at updates might be perfect for this context of use. However, it might not be as useful or desirable within the context of a tennis match on the weekend—a very different context in which users are deliberately trying to escape the emotions that are characteristic of their behaviors within an office context.

Things That Break Presence

Whenever a user must stop to think about the mechanics of how to carry out a desired action, presence begins to deteriorate. These mechanics include sifting through a menu to find a particular item or action and wondering which gesture or control to use to initiate the desired action.

In addition to creating less than transparent interaction mechanics, XR designers also run the risk of reducing presence when the following discontinuities occur:

• Virtual spaces fail to suggest where the user can physically go.
• Nonplayable characters (NPCs) lack a conversational tone or cannot provide the user with adequate responses to inquiries that are likely predictable based on location and story context—assuming a story-type experience.
• Real-life interactions—such as opening and closing doors or picking something up and throwing it—demand unusual user inputs or behave in a way that is vastly different from the real world.

There are quite a few more presence-breaking pitfalls that I could discuss, but the ones I’ve noted here are some key things to watch out for.

Wrapping Up

XR is an incredibly powerful new tool for bridging the gap between imagination and reality. It achieves this most effectively when full immersion occurs, a solid sense of presence exists, and a multimodal experience looks, feels, and sounds believable. These are foundational concepts to keep top of mind when you’re moving from designing experiences for traditional mediums to designing the magical world of XR.

In the next part of this series, I’ll cover some additional concepts to keep in mind, as well as some of the more practical considerations regarding the early stages of XR design.