​Project Profile

Strategic user research frameworks:

Speed, breadth and depth

Background: Detail telephoto in Pompidou Museum of Modern Art, Paris. © Dan Abelow.

Project Profile

Strategic user research frameworks:

Three benefits are speed, breadth and depth

Industry-leading companies must continuously solve new UX (User Experience) problems. To scale this process, a user research framework is an effective solution. This solves many UX issues at once, and raises UX performance across an entire company's operations.

As this UX expert was engaged to solve many problems across an industry-leading company's website, a User Research Framework was developed. This was so effective the company applied it to over 90 UX projects that delivered rapid and consistent improvements:

  • Performance grew to processing 90% of $21B orders and providing 80% of support.
  • Improved user success and customer satisfaction in all areas, including products, support, content, media, navigation, search, training and jobs.
  • Half the projects were collaborations with individual businesses within the company, providing broad and consistent improvements across many online businesses.

Speed: Online testing frameworks:

The old adage of "Faster, Cheaper, Better" is evolving swiftly. Today's biggest driver is "Faster," which comes from agile, scrum, lean and similar processes. UX's responsibility remains "Better," but speed and scale are now crucial.

As this company's numerous web teams advanced all areas of the company's online businesses, services and online tools, UX was responsible for making both specific and broad improvements rapidly.

This "need for speed" led to an online testing framework. Once designed and pilot tested, user tests could be set up, run and reported rapidly. Managers and developers quickly learned what worked, what didn't and the top priorities for improvements.

In addition, this often multiplied the number ​of users tested. Where 5 or 10 users were tested before, 50 or 150 real users might be included in an online framework test.

This online testing framework shrank weeks of UX work to days, and sometimes even shrank days to hours. 

Breadth: Consistent UX metrics quickly show status, issues and needed improvements

A custom UX metrics framework was developed to add "guidance at a glance" to UX research, analysis and reports. 

This measurement framework was based on a custom set of UX KPI's. These drove a consistent UX strategy and management across multiple kinds of user research and deliverables.

About half of the 90 UX studies were collaborations with individual businesses throughout the company. This produced a consistent and familiar UX for customers across all areas of the company's online businesses, products, support, purchasing and services. 

From each study, that business's managers, designers and developers immediately saw what works, the problems, and their priority UX improvements needed. 

Each study delivered clear steps to "move the meter." That made UX a normal part of development, driving a steady stream of UX improvements across the company.

Highlights from a UX Framework Test: What do managers and developers learn?

Right away, readers see that 137 users did 7 core tasks. KPI's include  5 performance metrics and 6 problem metrics.

The demographic profile shows that these 137 real users are representative across geographies, experience levels, and frequency of use.

While the overall user success was 83%, the 3 tasks with biggest user problems had user success rates of 64%, 71% and 76%.

Numerous tables make it easy to see where to focus. Red metrics show areas that need attention. White is on target. Green metrics show areas raised above the current UX target.

The top 3 problem tasks are reported separately. In each task, red rows show the specific steps where users had problems. These steps are ranked by "Priority to Fix." (The issue to fix at each step are on other slides). 

Depth: Advisory Board of major customers

This UX Framework included a Customer Advisory Board. These customer input events were two-day in-person meetings that included key representatives from a company's major customers and business partners.

These Customer Advisory Board Meetings were located in each key market worldwide. We planned, facilitated and reported each meeting as a focused, high-priority event.

Each meeting provided in-person opportunities for major customers and partners to express their ideas, thoughts, concerns and needs on the company's new plans for its online product information, services and online businesses.

Numerous small-group breakouts connected relevant company managers with their primary customers. These provided deep feedback on the specific visions, designs, features and walkthroughs of each area's new plans to expand the company's online capabilities.

This hands-on customer input, along with with UX designs, tests and other research, validated and improved many types of future plans before and during development.

Markets where we have run in-person events with major customers to obtain input on new plans before developing them.

As the results from a first UX study grew to over 90 UX projects, we built and ran a strategic user research framework. This new competitive capability grew the online businesses with the speed, breadth and depth they needed.