Five Best Practices for Becoming a Data-Driven Design Organization, Part 1

Best Practice #2: Moderated Testing of Early Designs

Our user research has enabled us to improve our information architecture and interaction models—early in the software-development process before committing development resources. The results of this research have been better outcomes for both customers and the business. Early, iterative testing of UX designs provides insights from a customer-experience perspective, helping the business to understand whether and how a design solution meets the business goals and requirements.

The Challenge or Opportunity

At an overarching level, our goal was to build a data-driven culture that makes customer centric–design decisions. So I identified and developed CXD strategy methods that would provide the information we needed to drive decisions. Exercising our influence to help the organization become more customer centric required our continual commitment, patience, and persistence. By leveraging windows of opportunity for research, I began building the organization’s appetite for gathering early learnings to help drive decisions—using associated metrics when available—and support business goals.

The Learnings

Once a CXD team has created designs based on customer-journey profiles, it is extremely important to test those designs with people who match the targeted profiles. Many still view design as subjective. (While those of us with training in the field of CXD understand the science behind design, educating others is still necessary.)

The best way to validate designs or design elements—for both your CXD team and your stakeholders—is to be as rigorous as possible in matching test participants to approved customer profiles. In this way, you can validate design decisions and changes. Each research activity builds on previous insights, giving you a database of archived information to mine. It also establishes a history of design decisions that your team has made based on research.

Using an iterative design process, your UX team can gather early feedback on and determine the usability of specific design options, choose the best design options for targeted user profiles, and apply your research findings to improve designs. Moderated, in-house usability testing let us obtain early feedback on design options for new ecommerce flows, from participants who fit the customer profile we had agreed on. The results of our research provided clear design direction and supported our final, data-driven design decisions, which were based on participant feedback.

The Research Process

Since we had no user researchers on our staff, I hired an outside agency to conduct the usability testing. Best practice indicates that designers should not test their own designs—they’re too close to the project. I arranged to use online conferencing to conduct research with each participant or group of participants. This was a first for my organization, even though it is an Web conferencing company. This approach delivered numerous advantages during testing:

• We reduced the complexity of scheduling logistics.
• We were able to involve more participants because geography was not an issue.
• The test costs were almost inconsequential.
• We could use document sharing and video conferencing.
• It enabled us to include observers from anywhere in the organization.

To guide the research, I instructed the CXD team to leverage the issues we had uncovered during the customer-journey project and use them to inform design decisions. Based on our previous customer-journey research, we were able to create clickable, walk-through prototypes of several design options.

We invited twelve participants who matched the target profile to be part of the research. Each of the research sessions lasted no more than two hours and comprised scenarios and tasks that were typical of what a customer would do in a given context—for example, signing up for the service or finding information. We followed a standard think-aloud protocol, in which participants share what they are thinking while using a design solution. The moderator walked each participant through the prototypes. Stakeholders—in particular, Product Managers and Program Managers—dialed in and listened, but could not  interact with the participants. There was great value in their seeing what potential customers said about their experience.

The Results and Business impact

Using another project-appropriate, customer centric–research method provided value in three ways:

• By revealing customer thinking, it built credibility in the organization.
• The CXD team and the broader organization learned another approach for customer-centric research.
• It provided data on which to base our design decisions.

Increasing the number of design decisions that were based on customer research contributed to our building a customer-centric organizational culture.

By letting us hear first-hand feedback from potential and current customers, our research gave us a new touchpoint that added to our organization’s understanding and enabled us to create more robust personas. Through this process, we built greater organizational confidence as we leveraged our research results in making final design decisions. This made the work of building consensus among stakeholders a lot easier. Our discussions were data driven. The research did the convincing. This approach results in quicker, more directionally accurate design decisions and fewer design iterations, which translates to faster time to market.

Our designers learned to gather early feedback from research, with participants matching our targeted user profiles, and to apply the learnings when iterating on their designs. The research process also demonstrated to stakeholders and management the value of gathering early feedback from customers. Our research had a positive impact on the business and enabled us to accurately execute on business goals and reduce the cost of design cycles. Plus, we increased the quality of the customer experiences—as our Web metrics demonstrated. From one research activity to the next, we gained a series of useful insights.

The Canary in the Coal Mine

Honestly, writing about these five best practices has helped me to reflect on how far we’ve come as an organization in building our customer-centric focus. Both the metrics that we track and the way we leverage them in discovering insights demonstrate our organization’s growth. As does our organization’s support for customer-research efforts of various kinds. Cross-functional teams now devote more time to reviewing the research data and leverage the data when brainstorming roadmaps and design directions.

However, I would wager that most of our CXD professionals have often felt like the proverbial canary in the coal mine that chirps to alert miners to problems. Ours can be a lonely voice because our professional colleagues are not always trained to hear and attend to our advocacy for the customer experience. Thus, our journey has been a bumpy road, along which I’ve learned how to speak to and influence naysayers. Since I was originally hired to bring CXD expertise to the organization, I felt that I was responsible for finding ways to help my colleagues tune into what our CXD team brought to the discussions. I knew they weren’t tone deaf!

In the second and final part of this series of articles, I’ll share how using some additional customer-research methods and implementing right-sized processes has helped to tune the ear of the organization over time. Plus, I’ll describe how building a CXD team that is both synergistic and diverse is critical to developing a holistic customer-experience design strategy.