Design
We conduct usability testing to implement data-driven experience decisions.

Usability testing delivers the cold-hard truth of a product or site's effectiveness.

User testing is a broad term encompassing dozens of different applications dependent upon the defined needs. In order to understand the right test for the right product or site, it's important to consider timelines, budget, and objectives. With a tight timeline, limited budget, but ambitious objectives, usability testing is often the best method to employ.

At the highest level, these are the three main steps of usability testing:

01

Set the Plan

Set out test scripts from the start based on the decisions that need to be made later.

02

Do the Work

Ensure all participants are recruited and ready, with an executional team trained for the task at hand.

03

Understand the Results

Gather the usability test data in a meaningful way that enables data-driven experience decisions.

User Research Matrix Natural Use User Research Matrix Out of Context User Research Matrix Scripted Use User Research Matrix Hybrid
Hover over key →
Tap key →
Key →
Natural Use (In Context)
Out-of-context Use
Scripted Use
Hybrid
Landscape of user research methods adapted from Christian Rohrer

Usability is about people and how they understand and use things, not about technology.

Steve Krug

Set the Plan

Step 01

User journeys and job stories are the fundamentals of content strategy that enable usability testing to be conducted. They are the compass with which to measure user success, and each task within the usability testing plan should map against a desired business or user outcome.

When writing the usability test scripts rooted in the key task journeys, there are a set of standard rules to follow, including:

✅ Do
  • Ask open-ended questions for qualitative insights
  • Encourage the user to talk through their thoughts, actions, and opinions
  • Let the user take the path that they have chosen
  • Remove bias from your statements
❌ Don’t
  • Employ leading questions
  • Reveal information the user needs to discover on their own to complete another task
  • Ask what the user would prefer to see / what they want
  • Ask vague questions that won’t provide value to the results of the test

In addition to writing the tasks, it’s important to recruit participants who are representative of your site users. This may mean multiple user groups depending on the product.

When planning a usability test, recruit participants and write test tasks that map back to user and business goals.

Do the Work

Step 02

In order to verify that the variables in the test are limited, it's critical to ensure the team proctoring the exam is small and purposeful. Usually, this team will consist of two roles: a test facilitator and a notetaker.

These tests can be run in many ways both virtually and in-person, however the crucial piece is to reduce the influence of any confounding factors.

During a usability test, the facilitator asks questions and gives tasks for the participant to perform, while the notetaker records verbal and behavioral feedback.

Understand the Results

Step 03

The outcomes of the test include recordings, notes documents, and quick stoplight pass/fail documentation. These three deliverables build the foundation for understanding user trends and critical data points that will define the product roadmap moving forward.

With this high-level understanding, we then have a basis with which to ask deeper, more specific questions about usability. This is accomplished by aggregating the stoplight metrics and mapping them against the qualitative data that was received during the tests, including documentation of unoptimized paths and spontaneous verbal feedback on process flow and visual representation.

Assess the data with quantitative and qualitative measures to identify common issues and their severity, and find opportunities to fix them.

Building a Roadmap

Once the data has been thoroughly analyzed, it's then clear to experience designers what the roadmap will look like. There are generally three big buckets of tasks that are born here:

  1. Quick Wins
  2. Feature Additions
  3. Strategic Reorganization

From these three groups, we're able to create a task list and resource strategy based on the upcoming available timeline and budget for product improvements.

Build a product roadmap based on the outcomes from the usability test, existing analytics, and internal resourcing.