Usability testing shouldn’t be something that is done only when you have a generous budget. Whenever you undertake a website redesign, or just make significant changes to an existing site, you should test those changes with users — ideally, with actual customers or representatives of your website’s user base.

This article is designed for organisations that don’t have an in-house UX team or a partner UX agency, that would like to increase their knowledge on how to add value through usability testing.

In this article:

  1. Why test with users?
  2. What usability testing is, and what it isn’t
  3. Selecting appropriate test participants
  4. How many participants?
  5. What are we going to test against?
  6. What you’ll need to run your budget usability test
  7. The value of expert usability testing

Why test with users?

It can be easy to imagine that you can put yourself into your users’ shoes and think just as they do. But the reality is, when you’re a stakeholder, you develop all kinds of blind spots to things that can trip up your visitors. Knowing your website so well, you breeze past aspects that aren’t as clear as they could be, that might cause a new visitor to stumble. When users stumble too often, unless yours happens to be the only service of its kind, they’ll quickly leave and go hunting for something that works better.

What usability testing is, and what it isn’t

It’s important to highlight the difference between targeted usability testing and addressing the overall experience of a product. The latter is a much broader and deeper consideration, relating to many contextual factors, such as how the user comes to interact with your product, the expectations they arrive with, what they’re doing at the same time as using it, their life situation, and so on.

What we’re discussing in this article is the effort to identify things that could cause undue thinking or frustration, or allow mistakes to be made by people using the website.

A few examples include:

  • Unclear navigation, that makes it difficult for people to find what they’re looking for, or that leads people to places they didn’t expect;
  • Insufficient user controls, for example, inability for users to exit a process that they entered into by mistake, or to undo errors;
  • Tasks take too long to complete
  • Issues with consistency, i.e. the presence of controls that appear differently from page to page, or controls that look the same or similar to others that work differently.
Usability testing

Usability testing can be conducted in person or remotely.

Selecting appropriate test participants

Ideally, you’ll use participants who represent people who currently use, or could end up using your website. Some services help you recruit participants based on the criteria you provide (for a fee), such as People for Research, or Liveminds.

If you have the means to contact and arrange tests with your customer base directly, all the better.

Is time with/direct access to relevant participants a roadblock to testing/ You could conduct usability testing with almost anyone: coworkers, family, or friends, and you’ll likely identify the most critical usability concerns (provided your tasks are well-written – more on this later), although you might not get the same quality of insights.

How many participants?

Research suggests that testing with more than 5 users in each round of testing is a waste of resources. The reason put simply is that after the first 5 users, the number of repeated insights increases dramatically. It’s worth noting, however, that this ‘rule of 5’ applies specifically to qualitative usability testing, That’s where you’re focusing on user’s thoughts, perceptions, and expectations whilst using a tool, and not on user research or testing as a whole.

It’s best to limit the number of participants, keeping your test lean and affordable, to free up your remaining budget/time to conduct further tests, on improved versions of your design.

What are we going to test against?

Before you’re ready to start testing, you need to define a specific set of tasks that you will ask each participant. This will give you a clear benchmark to compare across all users.

Examples of tasks you might set participants:

  • Find a blog post about how to bake a cake
  • Send a message to the website owner
  • Locate the cheapest blue jumper in stock

Writing good quality usability testing tasks is not easy. If you can only involve UX experts in one part of the process, we recommend you make it this one. If you must do it all yourself, you should be aware of some mistakes to avoid.

What you’ll need to run your budget usability test

  • To run a remote test, you can use Zoom, or another videoconferencing tool, set to record (don’t forget to ask permission from your participants to record them). To run an in-person test, you just need a quiet room
  • Ideally, a partner who can observe silently and take notes
  • 5 participants — for qualitative usability testing
  • Notepad & pen
  • List of tasks
  • A timer, to record how long each participant takes to complete each task
  • Ideally, one person to facilitate, and another to observe and take notes
  • Incentives (depending on whom you select, you should offer incentives to recognise people’s investment of time in helping you. This can be anything from cash to gift vouchers)

The value of expert usability testing

We hope this has provided a simple introduction to usability testing, and demonstrated that there’s plenty that you can do without calling on UX experts. However, to get the best results, and to reduce the risk of drawing the wrong conclusions from your research, you should seek support from professionals who know how to:

  • Ask the right questions, in a way that avoids leading participants
  • Interpret what the user does and what they say — which can often be at odds with one another
  • Report on the outcomes of the tests and make design recommendations
  • Execute on the recommendations — i.e. having learned about what seems to be wrong, what do we do about it? That’s where design comes in.

Have you experimented with usability testing in your organisation? What successes or challenges have you encountered? Let us know on Twitter.