In my previous column on extended reality (XR), I discussed some of the bigger-picture themes that have led to the creation of virtual reality (VR), augmented reality (AR), and mixed reality (MR), as well as some of the fundamental building blocks that determine the effectiveness of an XR experience. In this installment of my XR mini-series, I’ll continue by discussing some core considerations to keep in mind when designing an XR experience—from how to approach designing the first minute of an experience to experience-wide design decisions, including user agency, interaction capabilities, and virtual personas.
The First 60 Seconds
When designing for augmented or virtual reality, particularly the latter, you must introduce the user to an experience that may range from slightly different—for example, fairly passive AR—to wildly different—very imaginative VR. In doing so, you’ll likely include some sort of narrative and a set of interactions that lets the user pursue that narrative.
Champion Advertisement
Continue Reading…
The narrative could be a story that is far removed from the life we know, as in the case of a video game or interactive media that takes place in another world or a different version of our own world. However, it could simply be a new way of looking at the reality we already know— for example, an informative AR video that visitors can view or play when near a statue in a park.
Regardless of the user’s level of immersion, if XR designers are to provide a successful experience, they have a fairly small window of time in which to engage two things: curiosity and affordances.
In the first 60 seconds, set the stage for what is to come throughout the rest of the experience. Within the first minute or so, the experience that users encounter should enable them to develop a mental model of the entire experience.
This doesn’t necessarily mean users should expect to know certain specifics about the rest of the experience—such as story details—but they should firmly understand where they are, why they are there, and how they can interact with the experience to move themselves forward.
That final point is perhaps most critical: you must clearly afford methods of interaction to the user in the first minute. For example, will your XR app experience frequently make use of multimodal feedback or multimodal methods of user input? If so, design your experience to demonstrate these interactions in the first minute.
Above and beyond setting expectations for methods of input and output, firmly indicate and demonstrate what goals you hope the user will be able to achieve with your XR app and design an early-stage experience to build curiosity around those goals. Give the user the knowledge and the desire to move forward.
Demonstrating Affordances and Building Curiosity
To ensure the user progresses through an XR experience, you must design for
goal orientation—What is the user trying to accomplish? What does the user need to know and become familiar with to engage in interactions or narratives that advance the experience from beginning to end?
procedural orientation—How do we, as XR designers, make sure the experience unfolds cohesively? What design elements should we include to advance the experience—with or without user interaction?
feedback—What sort of diegetic feedback can we provide to users as they take steps toward the interactions or change agents advance the experience? It is important to provide natural, but immediate feedback to let users know whether they are currently heading down the right path or should rethink their actions.
In a practical sense, regardless of what type of XR app you are creating, the experience should start off by introducing goals—for example, save the princess, lose weight, create a piece of art, or socialize with distant friends—then, provide a narrative that motivates users to achieve those goals. The XR designer should create an experience that encourages emotional curiosity and quickly demonstrates affordances, giving the user a mental model of what agency they have within the experience. This sets the stage early for what sorts of things the user can do later on.
A great way to teach the user how to interact with the experience is by introducing a dilemma or challenge of sorts. This motivates the user to solve the problem creatively, using their newfound abilities within the particular experience. Challenges are an engaging way of demonstrating interaction methods and building curiosity all at once.
Considerations for Experience-wide Interactions
Once you’ve presented the key interaction affordances and helped users to develop a clear mental model of the experience within the first 60 seconds, that mental model of how to interact with the system and what to expect helps them to succeed throughout the rest of the experience.
But what shapes the rest of the experience? How do you decide what its many interactions will be? Next, let’s talk a bit about user agency, interaction capabilities, and virtual personas.
User Agency
Agency is the empowerment of users to impact their experience—users’ ability to influence the world around them. It can be either global or local.
With global agency, you are giving users the power to create impact that affects the entire experience, regardless of a particular scene. Local agency empowers users to take action in a way that impacts a particular scene or area they currently inhabit within your XR world and is characterized by a more limited sense of power.
In planning your XR experience, it is important to determine how much agency you would like to afford users during each phase of the experience. But remember, agency is power. It has a great influence on the dynamic between users and the extended-reality world they inhabit. So, the XR designer should carefully distribute that power in a way that helps the intended experience unfold.
More power isn’t always better. The right amount of power depends on the type of relationship you want to create between the user and the XR environment. While empowerment is a wonderful capability that highlights the creative wonders that often characterize XR experiences, sometimes restriction is the better way to create the user-to-environment dynamic that best supports the experience you are designing.
Levels of User Agency
One option is to remove user agency altogether. This lets you take users on a directed path, controlling their view, and not allowing any interactive capabilities that could alter the scene. If the goal of your experience would benefit from being directive rather than empowering, this level of user agency could be a viable option. Generally, however, this is not the type of exciting, immersive experience that makes XR enjoyable. Nevertheless, it has its place, depending on the XR designer’s intentions for the experience.
Moving beyond a directed experience, you can opt to give users varying amounts of local agency, empowering them to affect the scene they are in. The level of local agency can vary from the ability to interact with the scene in minor ways such as moving around, to being able to drastically alter the environment around them by picking things up and moving them. However, local agency, in contrast to global agency, is always constrained by scene-level user power.
Beyond local agency, you have the option of giving users global agency. This may entail users’ being able to affect not only individual scenes that they currently inhabit but also parts of the experience that they’ll experience later. This can lead to users’ being able to create their own unique journey for the experience.
The level of agency you provide the user depends on the experience you are trying to achieve—in terms of its being directive versus enabling freedom—and how empowerment plays a role in the design of your local and global experiential intentions.
Decreasing agency sets the stage for more directive or suggestive experiences, while increasing agency also increases user freedom and choice. Of course, the latter implies many potential paths within the experience, so carefully consider the design and development effort that this increased depth of experience would require.
Interaction Capabilities and Virtual Personas
Once you’ve decided the appropriate level of agency, the question then becomes how to go about giving users the ability to take agency. Be creative, but intentional in using gaze, movement, sound, motion, and controller input as tools the user can employ to take agency.
But remember, above all else, to be sure to design affordances that inform users of the amount of agency available to them, while also teaching them the mechanics of engaging in the available types of agency—especially in the first 60 seconds of the XR experience. Nothing sours an experience like getting stuck on a challenge an hour into the experience because you require users to employ a method of interaction they didn’t even know existed. Firmly establishing the laws of the new world you’re creating is critical to user happiness.
One way of helping users understand their interaction capabilities is by showing them who they are. What form does your user take in your experience? Will the user be human, with familiar capabilities and limitations? A bird that can fly? A wizard with spells at her disposal?
Creating an understanding of who the user is and what role that user is taking on within a given XR experience is one of the easiest ways to afford what users might be able to do. This also kickstarts users’ progress in learning interaction mechanics when they are pursuing the mini-challenges you’ve created at the beginning of the experience for onboarding the user.
While virtual personas help in forming a mental model of users’ interaction capabilities, they are also a great way to set the mood and style of the experience. If the personas are human, what is their sense of fashion? How might this play into the mental model you’re trying to instill in the experience? How do the physical motions your virtual characters take to represent users’ actions fit with the mood you are developing in the scene?
These are just a few considerations. There are many more. More broadly, carefully consider how you’ll virtually represent the user—and whether the virtual representations are in accord with the intended experience, predicted user desires, and the level of agency you intend to offer.
In Summary
It is important not to dive into scene-level specifics right away. Instead, consider the experience as a whole—based on the overarching experiential intentions of your design. This includes considering how much user agency you want to afford to users—both locally, within particular scenes, as well as globally. Also, consider the types of interactions you want to provide to users, the persona that encapsulates them, and how these influence user mood and the general feel of the experience.
Of course, once you’ve established these experiential building blocks, you must remember to design the first 60 seconds of the initial scene in a way that generates a mental model of the experience to come. By demonstrating affordances and building curiosity, you’ll prevent users from leaving your experience and keep them engaged and moving forward.
These are some of the most fundamental design considerations for creating a new XR experience. In my next column, I’ll cover additional design considerations at a more granular level—such as locomotion and disorientation, spatial-audio grounding, and foveated rendering.
Dash designs usable, enjoyable digital experiences that are driven by research and guided by the needs and desires of internal and external stakeholders. In his work, he draws upon his past experience in startups, UX consulting, internships, and freelancing, as well as the wealth of UX knowledge he gained through his journey to earn an MS in HCI. From concept to launch, Dash incorporates Lean and full-cycle UX tools and methods. He is always excited by future opportunities to play his part in delivering innovative digital solutions. Read More
