All you need to know about usability testing

What is Usability Testing?

Before we dive deeper into the discussion, let’s first define what usability testing is. It’s an examination of how users interact with a product to identify its strengths, weaknesses, and issues which need fixing urgently. Usability testing revolves around quality improvement which is based on five components:

• Learnability – how easy or difficult it is for users to complete a task the first time they use your solution
• Efficiency – how efficient is it to perform a task?
• Memorability – how simple is it to return to using your product after not using it for weeks or months?
• Errors or bugs – is error recovery quick and easy?
• Satisfaction – is your design enjoyable to use?

You might be wondering – is it worth conducting a usability test? We’ll get to it right now.

The Main Benefits of Usability Testing

Conducting usability testing regularly will bring you significant benefits, such as:

• Enhanced user experience – when it comes to product selection, customers have plenty of options. Good CX is absolutely necessary to prevent app switching
• Improved conversion – shortening the path to conversion will have a positive impact on boosting your conversion rate
• Revenue increase – usability issues can result in revenue losses
• Showing commitment to great UX – working towards providing a seamless UX will show your customers that you care about them

Different Types of Usability Testing

There isn’t just one usability test, there are various types. Let’s take a look at the main ones to help you choose the most suitable option for your business.

1. In-person usability testing

   This occurs when you’re physically in the same room as your participants. There are different in-person usability testing methods which you use, including:

   • Lab testing: introduces a controlled environment where testers can be evaluated based on the target market characteristics. You can conduct it in-house or in a 3rd party facility.
   • Guerilla testing: is about volume and involves testing with as many random people as possible.
   • Observational usability testing: the participants are testing your product in a quiet location while you act as an observer but without being physically present. You may use screen recording or video recording to observe users’ behavior.
   • Eye-tracking: see what parts of the interface get the most focus and which parts are overlooked.

2. Remote usability testing

   As the name indicates, it takes place when you and your participants are in two separate locations. Here are the most common forms of remote usability testing:

   • Phone interview: one-on-one sessions where researchers interview participants through the phone. Both parties can ask questions.
   • Digital card sorting: users receive digital cards and are asked to sort them in a way that will make the most sense to them. This helps with designing the right information architecture.
   • Session recordings: using screen recording tools like LiveSession enables you to remotely test ideas and observe how users interact with your interface.
     
     
     

3. Exploratory testing

   Users are given the opportunity to freely explore the product – you can check how they interact with the interface. Testing happens naturally and the evaluator has no influence over the participants.

4. Comparative testing

   It’s used to compare your product against two or more solutions. It can help you generate new ideas, features, and functionalities.

5. Assessment testing

   It lets you conduct a general usability test – it usually comes after exploratory testing, and helps you assess how your design impacts usability.

Website Usability Testing - How to Do it Right

Now that you know what usability testing is, and what methods you can use to conduct it effectively it’s time to talk about website and mobile usability testing.

Website usability testing is about checking how visitors interact with your website. It’s done to improve the retention rate, reduce development and support costs, assess what information is missing from your website, and to boost sales.

How to Conduct a Website Usability Test

There are a number of steps that you should take to ensure the high effectiveness of your website usability test.

Step 1: Plan for your test

Decide on the scope – figure out what parts or aspects of your website you would like to evaluate. Is it the copy, navigation, or maybe certain apps?

What’s the purpose? – before you begin your test, you must decide on its goal. Is it to verify whether no information is missing from your website or to check if it’s easy to move from one page to another?

Schedule and location – are you planning to conduct your test remotely or in-person?

Participants – who will you invite to the tests, and how many participants will you need? Try to match your participants with your target audience.

Scenarios – there are two things you must prepare: scenarios and questions which you should hand out to participants.

Source: Unsplash

Step 2: Run the test

You can select from two methods:

Unmoderated remote testing – great if you have a limited budget as no physical equipment is required to conduct the test, you don’t have to limit yourself in terms of participants’ location, and there are no timezone constraints.

Moderated or over-the-shoulder testing – you can closely observe the participants and notice their expressions, users can ask you questions if tasks are unclear to them, you get feedback instantly and can even ask participants to perform extra tasks if necessary.

Step 3: Analyze the findings

In order to put data into good use, you need to analyze your results. For this purpose, you should:

• Gather all your data and write down the main facts
• Segment issues that were identified into common categories like login issues
• Assign similar issues to the appropriate category
• List all issues and prioritize them in line with their business or UX importance

A Few Tips to Website Usability Testing

How many participants? – according to Jakob Nielsen, the precursor of website usability, it’s best to use no more than 5 participants.

Don’t judge too quickly – don’t implement any changes unless you notice a clear pattern of similar issues

Record your testing session – recording your sessions will help you keep track of the whole process. For this purpose, it’s worth using a session recording tool like LiveSession.

Test early and frequently – the earlier you test the smaller your development costs will be and errors will be easier to fix.

Run a website usability survey – including a survey on your website can help you boost customer satisfaction. Not only can you ask your visitors whether they found what they were looking for but you can also ask them what should be improved.

Now, let’s take a look at how usability testing is performed for mobile devices.

What Is a Mobile Usability Test and Why Do You Need It

Understanding how to properly conduct usability testing for mobile is as important as it is for desktop. In fact, it’s predicted that by 2025 nearly 75% of the world will primarily use their smartphones to connect to the Internet.

On the other hand, the competition for mobile users is already fierce, as there are jointly over 5 million apps in the Google Play and iOS App Store. Given these numbers, your mobile user experience should be a top priority!

The Advantages of Mobile Usability Tests

Thanks to mobile usability tests, among others, you will:

• Find areas that require improvement (for example, bugs or errors). In order to do this you can, for instance, use a session recording tool like LiveSession to observe how your users interact with your website on their mobile device.
• Optimize mobile development costs. If you understand which features your users utilize the most or which areas require critical improvements, you’ll be able to spend your development budget wisely.
• Improve your retention rate. By understanding your users’ behavior and goals, you’ll be able to prevent them from ceasing from using your app.

Source: Unsplash

Now, here are the steps you should take if you’d like to conduct usability tests for mobile devices:

Mobile Usability Testing Process

Here are the five steps you’ll need to take to ace your usability tests:

Step 1: Define your objectives

As in the case of the above-mentioned website usability test, you’ll need to define your goals. We advise you to schedule a team meeting to discuss what you already know vs what you want to find out about your users or app. The result should be a list of areas you’ll want to prioritize in the test.

Step 2: Make a list of tasks

Now that you know which topics you want to collect insights on, build out your questions around them. Organize tasks around various scenarios to test the most important features or use cases, in and out.

Step 3: Create your documentation

Before you proceed, write down all of the goals and tasks established in Step 1 and 2.

Step 4: Select the right participants

Don’t rush through this step! Make sure to select the right participants who are representative of your users. Only then will the mobile usability test provide relevant results.

Step 5: Choose the methodology and run the test

Decide whether you’d rather conduct a remote or in-house test and whether they should be moderated or not (we’ve mentioned these types earlier). As users complete the tasks, write down any observations. Also, collect insights from them after they’ve completed the test. This data will be the basis for your usability testing report, which we discuss later in this guide.

You can read about the entire mobile usability testing process in a dedicated section.

Top Usability Testing Questions You Need To Ask Your Audience

Now that we’ve reviewed what website usability is and what it looks like for websites and mobile products, let’s proceed to one of the most important elements of usability testing – asking the right questions. In fact, collecting qualitative and quantitative feedback from your users is as important as observing how they interact with your app or website.

That being said, it’s time to have a look at the types and examples of ready-to-use usability testing questions.

4 types of usability testing questions

User testing questions can be divided into four groups – screening, pre-test, in-test, and post-test.

Screening usability questions

Screening questions will help you pre-qualify your users for your UX testing. When you work on your usability testing, you should write down all the criteria your ideal participant should meet.

Among others, you should ask about:

• Demographic data, such as age, gender, occupation, and level of education, or firmographic data, such as company size, industry, etc. This information will help you, for instance, determine if you’re targeting the right occupation from the right company size.
• Tech-literacy, i.e., how easy it is for your users to use certain devices. Asking such a question will help you, for instance, choose only those people who have high tech skills.

Pre-test usability questions

Pre-test questions will help you further narrow down or group your users. These include:

• Behavioral questions, for example: “how often do you engage with this (or similar) products?”
• What do you use the website or app for? – to understand separate use cases
• Which features do you use most? – to identify users, who are already familiar with your product vs those who are new to it
• What problems does this product help you solve? – to understand and group user goals

Next, you should proceed to…

In-test usability questions

These questions should be inquired as users engage with your product or website. Among others, we recommend that you ask:

• When you log on, what’s the first thing you do? – to understand user motivation
• Do you like the interface? Is it easy to use? – to collect general impressions and nail down any UX/UI flaws
• Why did you decide to leave this page? – to understand exit intent

Last, but not least, you should also question your users after they’d completed the usability test.

Post-test questions

Ask your testers about their impressions. Particularly, focus on the following:

• Given your experience, how frequently would you use this product/feature? – to find out how valuable the product is for users
• What were your feelings about the instructions? – to optimize user testing sessions in the future
• How likely are you to recommend this website or app to others (on a scale from 0-10)? – the so-called Net Promoter Score question, which will help you evaluate customer loyalty levels and understand key issues
• Do you know anyone that would like this product? – to find out how you can better your ideal customer profile

You can find a full list of recommended usability testing questions in a dedicated chapter of this guide.

How to Become Great at Usability Reporting

With all the work you’ve put into organizing your usability test, you’ll now also need to ensure that your findings are put to use by relevant teams. There’s no better way of doing this than by creating a thorough usability testing report! Below, we review what usability reporting is, what it should include, and why it’s indispensable in your user testing.

What a Usability Testing Report Is

In the most simple terms, a usability test report is a document you prepare after evaluating all the feedback and insights from sessions with users. These include, among others, your own notes and observations, any evidence data (for instance, session recordings), as well as quantitative and qualitative feedback from participants.

Don’t worry if this sounds like a complicated process – there are tried-and-tested usability report templates. You can follow the structure below:

Executive summary

The goal of the executive summary is to quickly onboard anyone who picks up your report to understand what it covers and what the key findings were. A paragraph will suffice – you’ll go into detail in the next sections.

Background summary

This section should be written in a way that allows anyone who hasn’t been involved in the usability test to quickly understand why you’ve decided to run the test. Among others:

• What the test objectives were
• What products/features were tested (for example, website or app)
• Who the testers were

Methodology

Next, explain what methodology and tools you applied. Did you use user session recordings? Or maybe you performed lab tests? This is the section where you should also specify all the questions you asked your participants.

Audience

Describe your participants as thoroughly as possible. Make sure to include their demographic/firmographic data and why you’ve decided to recruit them in the first place.

Test results

Create a detailed description of the results. To improve readability, we highly recommend that you include a table that clearly specifies all key data – task completion rates, the overall number of participants, types of assignments they were assigned, etc.

Findings and recommendations

Make a bulleted list of findings and a separate list for recommendations. The results should be tangible – i.e. it’s no place for assumptions, only theses that have been verified. You can also create a list that segments your findings via user scenarios.

In this guide, we’ve dedicated an entire section to usability testing reports, so make sure to take a look!

Usability Testing Tools to Help you Optimize Your UX

While usability testing might seem daunting at first, there are a bunch of usability testing tools that will make the entire process a lot easier. Here are the ones that we recommend using:

1. Mixpanel

   It’s your go-to solution if you want to gain a deep understanding of how users interact with your product, how and when they convert, and what changes you must incorporate to retain your users.

   

Source: Angel.co

2. Google Analytics

   A tool known by every marketer – it lets you check your conversion rates, find your best-performing landing pages as well as those which need improving, discover traffic sources, and gain insights about your visitors. You can create detailed reports which you can easily share with your team.

   

   Source: Neil Patel

3. LiveSession

   LiveSession is a session replay software that lets you check how visitors engage with your website and spot any pressing issues which prevent them from converting. As a result, you can eliminate any bugs and roadblocks which improve the overall UX. LiveSession offers a free trial, so be sure to check it out.

   Create a seamless user experience and improve your conversion rate. Try LiveSession today.

   Test all LiveSession features for 14 days, no credit card required.

   Sign up free
   

   Source: LiveSession

4. Unbounce

   It offers a simple to use, drag-and-drop landing page and pop-up builder – you don’t need any coding skills to use it. It’s great for A/B testing, however, what makes it really unique is its AI-powered Smart Traffic and Unbounce Conversion Intelligence features. They give you access to insights necessary to understand how users engage with your product.

   

   Source: ThemeForest

5. Usability Hub

   Usability Hub is a user testing and usability research hub which lets you get feedback from real people and ultimately improve the UX of your app or website. It includes features like first-click tests, design surveys, preference tests, and many others.

   

   Source: UXpro

If you’re hungry for more information about usability testing tools then check out this article, where we discuss more solutions along with their benefits.

Summary

As you’ve just seen, usability testing is essential if you want to outpace your competitors and consistently provide your users with great UX. By incorporating it as a continuous process, you’ll be able to improve your customer satisfaction levels, adjust your product to the ever-changing market, and optimize product development costs, among other benefits.

Now, if you truly want to master usability testing, we highly recommend that you read the remaining chapters of this guide. This section was just introductory reading, and there’s still much more to uncover about effective website and mobile usability testing, creating useful, comprehensive usability reports, drafting the right questions for testers, and more!

Click here to proceed to Chapter 1: What is Usability Testing? Definition and a Review of the Best Methods.

Chapter 1

What is usability testing?