Agile Development Is No Excuse for Shoddy UX Research

The Heart of the Matter: The Backlog

The fundamental problem is that some think UX research has no rightful place in a sprint backlog. Agile/Scrum teams often view UX research as extraneous to development work. They consider development their key undertaking and first priority. UX research is perhaps a distant third priority, after design. The biggest obstacle to UX research routinely becoming part of the backlog is the unfortunate perception that it’s a bottleneck that impedes the progress of design and development. Team members ask, “Wouldn’t designers and, consequently, developers end up twiddling their thumbs, waiting for time-consuming research to take place?”

Well, yes, they might if your conception of agile is actually just a series of mini-waterfalls. If your sprints consist of linear hand-offs from research to design, then design to development, research would cause a holdup. But if your Scrum team works with a true agile mindset, it won’t.

Here’s an example: Let’s say an agile team is in the early stages of designing a jobs Web site. UX research is under way, but nowhere near complete. The personas are still sketchy, and user requirements are not fully fleshed out. All the team currently has to go on are assumptions about who the users are, what their needs are, and how to best serve those needs, for example:

“Our Web site caters to university and college students who are looking for part-time jobs to fund their education. The assumption is that they are interested in short-term work, with low barriers to entry. The competitive Web sites are too generic, don’t cater to this specific job market, and fail to help users automatically match their profiles with potential job opportunities.”

What should the designers and developers do while waiting for the results of UX research? Instead of just waiting and complaining about UX research holding them back, the designers could begin interacting with researchers at the very outset of a project. At the very least, they could gain insights into the team’s basic assumptions and hypotheses. Or better still, the designers and other team members could participate in some of the research to learn about users firsthand. Designers could simultaneously work on some preliminary sketches, basing them on whatever they’ve learned from the research so far—no matter how rudimentary. If the developers got those sketches right away, they could start working on their software architecture and code, implementing those sketches—even though they’re likely to change later on. Remember, agile welcomes change. If everyone is working in parallel, no one is just waiting.

Of course, working in this way requires a high degree of open communication and collaboration on the team—another basic tenet of agile!

What if UX research were later to reveal that the primary user requirement is not merely finding jobs, but finding jobs within walking or biking distance of their home because very few members of the target user group own cars? The team could quickly add new user stories to the backlog to cover additional features relating to job search by distance and location.

There’s really no need to do big, up-front UX research; no need to wait for the research results; and no need to take shortcuts in doing research either. But you must consistently add UX research and any actionable items resulting from it to the backlog, prioritize the research, and undertake research work like any other work that occurs during sprints.

When planning a project, you must treat all work without distinction—whether research, design, or development. Encapsulate research in backlog items—epics, user stories, or tasks—and ensure that the team completes the highest-priority work, irrespective of whether it relates to research, design, or development.

Agile teams need to become truly interdisciplinary, not just in their composition, but also in their disposition.

Planning Your Research

Like design, UX research should ideally be iterative. While it may make sense for certain types of research such as evaluative research to iterate over shorter time spans—as Krug suggests, test early and test often—other types of research such as inquiries into sustained user behavior or changing trends might span relatively longer periods.

Before making UX research part of a team’s agile cadence, it is helpful to categorize your research, then see how you can best time-box it to fit into the backlog. To that end, we suggest organizing UX research into three tracks, as shown in Figure 1:

1. Strategic track
2. Tactical track
3. Validation track

Strategic Track

The strategic track comprises any generative research that can help you shape the product vision, strategic goals, and product roadmap and identify the right target users. Research on this track is usually ongoing and exploratory. Typically, you’ll begin this kind of research early, prior to design and development, then continue your research over relatively extended, iterative cycles, spanning months.

Tactical Track

The tactical track comprises short-term, exploratory research that helps you define specific product features and desired experience outcomes. It helps product owners prioritize the work in the product backlog. You can do this research iteratively, within the timespan of a sprint, which is generally just a few weeks. This research helps you determine what features to include, modify, or sunset in any given release.

Validation Track

As the name of this track suggests, validation entails evaluative research and testing, which proves or disproves assumptions, validates design decisions, determines any usability and accessibility issues, and measures user satisfaction and emotional response. Ideally, you should do research belonging to this track during every sprint cycle—which are usually two to three weeks in duration. This research could include usability testing, cognitive walkthroughs, or heuristic evaluations.

Of course, the names of the tracks we’ve suggested are generic. You could label or categorize them differently to suit the dynamics of your project. The purpose of these tracks is to help you better estimate and organize your research effort.

Adding UX Research to the Backlog

We’ve seen as many variations of a product backlog as the number of agile teams we’ve worked with. Each team uses slightly different labels and hierarchies for backlog items, as well as ways of structuring and organizing them.

Most commonly though, a Scrum product backlog comprises work items in the form of epics, user stories, and tasks. User stories—or simply, stories—are user requirements that a team can realize within the span of a sprint. Epics are stories that are too large in scope or complexity to complete them within one sprint. Tasks, the smallest work unit in the backlog, are activities team members need to accomplish to complete user stories.

There are some formal approaches that can be helpful in incorporating UX-research and design work into product backlogs—for example, Jon Innes’s UXI matrix or Jeff Patton’s user story–map backlog. These are definitely worth looking at. But, if you’d like to keep things simple and avoid adding too much overhead, you can just organize your research into the buckets, or tracks, we suggested earlier: Strategic, Tactical, and Validation.

For backlog items that are likely to extend over a longer period of time, follow these steps when adding them to the backlog:

1. Add a UX Research epic to the backlog.
2. Later, during backlog-grooming sessions, gradually start breaking the epic down into UX-research stories, which are essentially parts of the larger research effort that you can accomplish within a single sprint.

Adding Strategic-Track Backlog Items

Backlog items that fall under the strategic track are likely to extend over a longer period of time and, thus, necessitate your first writing a UX Research epic that you’ll later break into user stories. Be mindful that you should be able to time-box one or more of these stories into a single sprint.

UX-research stories in the strategic track could be about formulating research questions, formulating a hypothesis, creating a screening questionnaire, recruiting participants, conducting a first round of interviews, or compiling and analyzing data, for example. You get the idea.

A UX-research epic or story does not differ greatly from other epics or stories, except that you should write them from the perspective of the consumer of that UX research—for example, the design team, product strategist, or business leadership. In other words, they must highlight need, intention, and outcome.

Here’s a template you can use when writing a UX-research epic or story:

In the context of <a scenario/situation>, the <consumer of the research> wants to observe/understand <a phenomenon> or get an answer to <a research question>, so that <the desired outcome>.

An example of a UX Research epic in this format might be:

In the context of a lower-than-anticipated conversion rate of 10%, the product strategist wants to understand why users are failing to complete registration and get through the profile-setup steps, so she can develop some strategies to encourage users to complete the registration and profile-setup process.

Alternatively, you can use the following example template for each of the UX-research stories within an epic:

In reference to <research epic #>, we want to <subset/step of the research>, so that <the desired outcome>.

Here’s an example of a UX-research story that’s part of our example epic:

In reference to research epic #123, we want to conduct contextual interviews with six participants so we can discover any issues with the registration and profile-setup process.

Adding Tactical-Track Backlog Items

For backlog items you’ve identified and placed under the tactical track, if you expect the research to extend beyond the length of a sprint, you’ll again need to create a UX Research epic, then break the epic down into research stories. But, if you can time-box the research into only one sprint, just create a single UX-research story, using the template for UX Research epics.

Here’s an example of a standalone UX-research story:

In the context of tree-based location selection on the Job Search page, the product owner and design team need to know whether it is overly complex and would result in user confusion and a suboptimal user experience, so, if necessary, the design team can explore ways to improve or replace that feature.

Adding Validation-Track Backlog Items

For backlog items under the validation track, if you’ll need to validate more than one user story—for example, conduct usability testing relating to the design and functionality of multiple features that you’ve specified in various user stories—you may want to add a single UX-research story to the backlog, using this template:

When the <user type> views or interacts with the <page|section|component> to <task>, we want to <observe/understand/record something> with respect to <criteria >, so <the desired outcome>. Refer to: <Story # or Stories #, #, #>

Here’s an example validation-track story:

When a registered job seeker views or interacts with the Job Search page to search by location, we want to observe whether the user can interpret the search results and understand which jobs are within walking or biking distance, so the UX and visual designers can determine whether the format and visual design of the search results are optimal. Refer to: Story #123, Story #456.

Alternatively, if you need to validate only one user story, we’d suggest that you instead add it as an acceptance criterion. Acceptance criteria are statements that accompany a user story and must be true to consider the story complete.

For example, consider this user story:

As a user who has signed in, I would like to have a way to easily change distance units from miles to kilometers, and vice versa, so when I search for jobs, I can view the distance to potential job locations in the units I prefer.

One acceptance criterion for this story could be a validation criterion, for example:

A test user, when asked to change distance units, can promptly locate a link to User Preferences and successfully change the units from Miles to Kilometers, or vice versa.

Of course, these are just suggested epic and story templates. You can tweak them for specific research items, according to your needs.

Prioritizing and Planning UX Research

Once you have incorporated UX research into your product backlog, prioritize the research in the same way you would any other stories. Backlog grooming should include time for going through the UX-research stories with all relevant stakeholders and fleshing out any additional details, stakeholder expectations, acceptance criteria, or dependencies. The product owner should determine the return on investment (ROI) of the UX-research items and prioritize them in the backlog accordingly.

During each sprint-planning meeting, discuss and estimate the UX-research stories with the entire team. Initially, it may be a challenge for team members who aren’t UX researchers to vote on story sizing for research stories. But you should encourage them to make a guesstimate and discuss their vote with you, so you can help them to develop a better understanding of UX-research activities over time and to become more engaged with UX research.

Once team members have assigned the UX-research stories to themselves, encourage them to add tasks—definite steps they’ll take toward completing the stories. They should also report their progress on these tasks during standups.

A Few Agile Research Tips

Make your research findings easily consumable. In the spirit of agile, don’t rely on heavy documentation in reporting your research findings. Research results that get trapped in 100-page PDFs or on team wikis will likely never see the light of day, so won’t have the desired impact. It is important that you deliver your research findings directly to the product team and stakeholders, during work sessions that accommodate discussions and dialogue and result in concrete actions and logical next steps. A good forum for this discussion might be a sprint review or demo session at which all team members and stakeholders are present.

Conclusion

UX design does not happen in a vacuum, but stems from context, which comprises specific business needs, user aspirations, and technical constraints. UX research helps designers gain an understanding of that context—of what they need to design, for whom, and why. Research can also help validate a design and prove or disprove a team’s assumptions.

Some key outcomes of good UX research include, but are not limited to, gaining a competitive edge; improving customer acquisition, user engagement, and retention; getting a better return on investment; and reducing time and financial investments. In other words, UX research delivers all the same benefits as agile. Thus, UX research and agile are different means to the same end. So this arranged marriage could actually work!