How to Become Great at Usability Reporting

What Is a Usability Testing Report and What Should It Include

Let’s assume you have just conducted a usability test on your product and are now wondering – how should I analyze the results? The answer lies in creating a report which is the by-product of your usability testing. Usability testing and reporting go hand in hand – you cannot conduct great usability testing without efficient reporting of your findings. While usability testing is conducted to discover valuable insights directly from the end-users, a usability report is a summary of the findings.

It’s a document prepared after evaluating the qualitative and quantitative data gathered during a usability test, which offers findings and recommendations that help in improving your product’s UX.

Now that we know the definition of a usability report, let’s talk about what every report should include.

What to Include in Your Report

A usability testing report covers several variables. Here are some factors to include in your report:

Background summary

The background summary should explain the main reasons for conducting the test and define which products were tested (websites, mobile apps, etc.). It should also offer information on the test participants, testing location, and timestamps of when the study was conducted. A brief outline of all the steps taken to perform the test, the type of testing material used, and any observed setbacks should all be noted in the background summary.

Methodology

Whether you employ session recordings or offer head-to-head testing, it is always worth including how the usability test was conducted. Also, mention the types of questions and whether all four – screening, pre-test, in-test, and post-test questions – were handed out to the participants.

Audience

Describe your participants, their demographics, and why they were recruited in the first place. Providing a summary of their demographics e.g. age, profession, sites visited, and internet usage is fine, but you should refrain from using full names due to privacy concerns.

Test results

Test results are the key ingredients of a usability report. In your report, you should create a table, which features a summary of the task completion rates, number of participants, types of assignments, and other data.

Findings and recommendations

Make a list of all your findings and recommendations. These should be real findings that you observed, and not merely assumed. Also, you can either make one overall list of your findings or have them segmented by scenarios.

Executive summary of the usability testing

Placing an executive summary at the top of your report will help instantly onboard anyone who comes across your report in the future.

With all of the above in mind, let’s now take a look at the key metrics your report should include.

What Usability Metrics Should You Focus On

Usability metrics are the standards of measurements and come in handy when you are trying to gauge the efficacy of a product. Without them, it would be virtually impossible to create a testing report. Here are some of the most important metrics to include in your reports:

Success rate

Per the Nielsen Norman Group, the success rate is the simplest usability metric. It is usually defined by the percentage of tasks that users completed successfully. User success gives insight into how well your product resonates with the end-users, and is the main reason why a usability test is conducted in the first place. The higher the percentage of completed tasks by the users, the higher the chances of your product’s popularity.

The number of errors

Listing the average number of times an error occurred when a user was performing a given task is a good metric to focus on. Errors can be accidental where the user makes a typo, picks the wrong reservation dates for a flight, clicks on the wrong image, etc. Albeit the fact that there cannot be a completely perfect outcome with an interactive test, listing the average number of errors does point you in the right direction for diagnostic purposes. Scoring each error with a severity level and short description is also a good idea in understanding the functionality of your tested product.

Time-based efficiency

This is a key metric to list in your report as it gauges the amount of time it takes for a user to complete a certain task or use a certain feature. This, in turn, measures the efficiency of your product. The less time a user spends performing a task, the higher the effectiveness of your solution.

User’s subjective satisfaction

This includes the subjective thoughts of your participants regarding their level of comfort, behavior, and acceptability of product usage. This metric can be measured on either a quantitative or a qualitative scale to extract insights for your report.

For usability testing to be successful, your findings must be clear and up to the point. Let us now look at a few tips to make sure your reports follow this mantra.

6 Tips to Usability Reporting

The following tips will be useful for everyone on your team in identifying the key problems with your product and providing resolutions:

1. Include visuals

   We recommend that you include visuals in the reports. Most people have a short attention span and keeping them engaged with long text descriptions can be really hard. Instead, providing annotated screenshots or, better yet, video clips from your session recordings, will deliver a greater impact.

2. Keep it short

   No one likes to read pages and pages of formal documents. Keeping your reports concise, to the point, and avoiding technicalities is the key to getting the desired outcome.

3. Prioritize usability issues

   Avoid discussing every single issue that was encountered during the test. It is also a good idea to assign priorities to your issues when compiling them in the report. This way the higher priority issues are looked at first to ensure a reduction in production backlog.

4. Use what you learned

   It is important to take away constructive insights from your report and put them to good use. Implementing the findings into positive solutions will result in a much better UX.

5. Take advantage of quotes

   To make your reports more credible, try to incorporate user quotes. Adding direct quotes from the participants clarifies the user viewpoint and makes your reports powerful. They also add character to your report and keep its readers engaged.

6. Remember about the next steps

   Always summarize your report with the proposed next steps. Having a solid plan of action to improve your product will enhance your report. Propose discussions and presentations that you should organize in the future based on the report findings. This will not only promote teamwork but will also pave the way for product optimization.

Once you are armed with the above-mentioned great tips, try investing in a tool that can compile and organize your data in the best way possible.

Three Tools That Help with Usability Reporting

Do you want your readers to understand the findings in your report easily? How about keeping them engaged from start to finish? In order to do so, we recommend that you invest in a professional reporting tool. Here are the top three tools you should consider for your usability reporting:

Google Data Studio

Google Data Studio is Google’s proprietary built-in visualization tool, which is free to use. It displays your quantitative findings by importing a .xls or a .csv file from your Google Sheets. The best part is that it is completely synchronized with your files. Every time your source files are changed, all data on Google Studio gets updated as well.

Source: Google Data Studio

Xtensio

This tool allows a collaborative approach. You can edit and work on your data live along with your team members. It also offers a library of templates, that let you design an instant and visually-appealing usability report.

Source: Xtensio

Chartblocks

Chartblocks comes in handy when you are dealing with time constraints. It also makes it easy to merge and import data from multiple sources and create usability charts.

Source: Chartblocks

Summary

Let’s peek into the key findings that were discussed in this article:

• Usability testing and usability reporting go hand in hand – you cannot conduct great usability testing without providing efficient reporting.
• A usability test report is a document that is prepared after evaluating the qualitative and quantitative data gathered during a usability test. It offers findings and recommendations that eventually help in improving your product’s UX.
• A usability report should include metrics like background summary, methodology, audience, test results, findings and recommendations, and an executive summary of the underlying usability test.
• Main metrics to focus on when creating a usability report: success rates, number of errors, time-based efficiency, and user’s subjective satisfaction.
• Various tips for creating a usability report: including visuals, keeping the report concise, prioritizing usability issues, using what you learned, taking advantage of direct user quotes, and crafting your next steps.
• The three top tools that greatly help with usability reporting include Google Data Studio, Xtensio, and Chartblocks.

To sum up, your product is part of your brand identity and should not be compromised. Conducting usability testing and writing great usability reports is a sure-shot way of improving your product, tweaking the weak spots, and reaching your target audience with a stronger, better-crafted offer.

Chapter 4

9 Usability Testing Tools You Need To Optimize Your UX