.svg)
Usability Testing for QA Teams: A Session Script, Findings Template, and Priority Matrix You Can Copy
Learn with AI
At some point a product manager or a designer is going to turn to the most user-focused person on the team and say: can you run a usability test on this? That person is usually a manual tester, because manual testers already spend their days thinking about how real people use software.
If that is you, this guide is the practical version of what to do: how to find participants, how to run the session without leading them, how to write up findings that developers actually act on, and how to decide which UX problems matter enough to fix now. There is a session script template you can copy near the end.
THE SHORT ANSWER
How to run a usability test that actually gets acted on:
• Recruit 5 to 8 users who match your real user profile, not colleagues who already know the product.
• Run a think-aloud session: give one task at a time, then stay quiet and let people struggle.
• Write findings as what you observed, not what you concluded ("3 of 5 users could not find X," not "the UI is confusing").
• Prioritize with a severity-and-frequency matrix, not gut feeling.
• Log critical findings as defects in the same tracker as functional bugs so they actually get fixed.
There's a full session script you can copy near the end of this guide.
What Usability Testing Actually Is
Usability testing is watching real people try to complete real tasks with your product, and noting where they struggle. That is the whole thing. You are not asking whether the feature works to spec. Functional testing, one of the more established types of QA testing, already answered that. You are asking whether a person who has never seen this screen can actually use it.
This is the most human of all testing types. Automation cannot do it. AI cannot do it. A script cannot tell you that a user paused for four seconds, frowned, and then clicked the wrong button with visible frustration. Someone has to watch, and someone has to understand what they saw. That is the part that makes it valuable and the part that makes it yours.
Functional testing asks: does the button work? Usability testing asks: can the user find the button, understand what it does, and trust what happens when they click it? Both matter. They are not the same test.
The Process at a Glance
Five stages, start to finish. The rest of this guide walks through each one.
| The Usability Testing Process Five stages, start to finish |
|
|---|---|
| 1. RECRUIT | 5 to 8 users matching your real user profile. |
| 2. PREP | Tasks, script, and recording setup. |
| 3. FACILITATE | Think-aloud, observe, and stay quiet. |
| 4. DOCUMENT | Findings tied to tasks, not opinions. |
| 5. PRIORITIZE | Severity and frequency, then hand off to developers. |
How to Recruit Participants
You need fewer people than you think. Five to eight participants will surface the large majority of usability problems, per Nielsen Norman Group's research. After that, you start seeing the same issues repeat and the value of each extra session drops fast. Do not wait to line up twenty people. Five is enough to start.
What matters far more than the number is who they are. Recruit people who match your actual users. Testing a tool for accountants on a room full of developers will give you confident, wrong answers. If you cannot reach real users, get as close as you can and be honest in your write-up about who you tested with.
-
Existing users: ask customer success or support for a few willing customers. The closest thing to the real thing.
-
Internal proxies: colleagues from outside the product team who have not seen the feature. Not ideal, but workable for early rounds.
-
Recruiting tools: services exist to source matched participants for a fee, useful when internal options run out.
One rule: never use someone who helped build the thing. They know where everything is. They cannot get lost, which means they cannot tell you where real users will.
How to Run a Think-Aloud Session
The think-aloud method is the backbone of usability testing. You give the participant a task and ask them to narrate their thoughts out loud as they work through it: what they are looking for, what they expect to happen, what is confusing them. Their running commentary is the data.
Your job as facilitator is mostly to stay quiet. This is harder than it sounds. When you watch someone struggle with something obvious, every instinct says to help. Do not. The moment you guide them, you have lost the finding. The struggle is the point.
What to do
- Give one task at a time, framed as a goal, not as instructions. Say "buy a blue shirt in your size," not "click the catalog, then filter by color."
- Stay silent while they work. Count to ten in your head before you say anything.
- When they go quiet, prompt only with neutral phrases: "What are you thinking right now?" or "What did you expect to happen there?"
- Note the moments of hesitation, the wrong turns, and the workarounds. Those are your findings.
- Record the session if the participant consents. You will miss things live, and a recording lets you show developers the actual moment of friction.
What not to do
- Do not answer their questions during the task. Turn it back: "What would you do if I were not here?"
- Do not explain how the feature is supposed to work. If it needs explaining, that is the finding.
- Do not ask leading questions like "wasn't that easy?" Ask "how did that feel?" and let them answer.
A Session Script You Can Copy
Run every session the same way and your findings become comparable across participants. Here is a script template that has worked across many teams. Adapt the tasks to your product and keep the structure.
Usability Session Script: [Feature Name], [Date]
Intro (2 min)
"Thanks for helping. We are testing the product, not you. There are no wrong answers."
"Please think out loud as you go. Tell me what you are looking at and what you expect to happen."
"I cannot help you during the tasks, because I need to see what you would do on your own."
Warm-up (2 min)
"Before we start, tell me a bit about how you currently handle [the problem the product solves]."
Tasks (20 to 30 min), one at a time, framed as goals
Task 1: [Goal stated as an outcome, e.g. "Create an account and log in."]
Task 2: [Goal, e.g. "Find and apply a discount code at checkout."]
Task 3: [Goal, e.g. "Update your notification settings."]
(Observe. Stay quiet. Note hesitations, wrong turns, workarounds.)
Wrap-up (5 min)
"What was the most frustrating part?"
"If you could change one thing, what would it be?"
"Anything you expected to find that was not there?"
Close
"Thank you. This is genuinely useful."
How to Document Findings Developers Act On
This is where most usability tests fail. The session goes well, the tester writes "the checkout is confusing" in a doc, and nothing changes. A developer cannot act on "confusing." They can act on a specific observation tied to a specific moment.
Write each finding as what you observed, not what you concluded. "Three of five users could not find the promo code field and gave up looking" is actionable. "The checkout has poor UX" is not. The first tells a developer exactly what to fix. The second starts an argument.
A findings table keeps it concrete and sortable:
| Where | What was observed | Impact | Priority |
|---|---|---|---|
| Task 2: Checkout | 3 of 5 users could not find the promo code field | Blocks the task, many users | Critical |
| Task 1: Sign up | 2 users hesitated at password rules, eventually succeeded | Slows, some users | Medium |
| Task 3: Profile | 1 user disliked the avatar color | Cosmetic preference | Backlog |
Tie every finding to the task where it happened and the number of users who hit it. Frequency is evidence. One person disliking a color is a preference. Three people failing the same task is a pattern, and patterns get fixed.
How to Prioritize UX Bugs
Not every usability finding deserves a fix, and not every fix deserves to happen this sprint. The trap is treating all UX feedback as equally urgent, which floods the backlog and gets the whole report ignored. Prioritize on two axes: how badly it hurts the user, and how many users it hits.
| Affects many users | Affects few users | |
|---|---|---|
| Blocks the task | 🔴 Fix now Critical UX bug |
🟡 Fix soon Schedule it |
| Slows / annoys | 🟡 Fix soon Schedule it |
🟢 Backlog Note and revisit |
A problem that blocks the task for many users is a critical UX bug and it should be treated like any other critical defect. A cosmetic preference that bothered one person goes in the backlog and probably stays there. Most findings land somewhere in between, and the matrix gives you a defensible way to sort them that holds up when a developer or a PM pushes back.
Severity is not the same as how strongly the user complained. A quiet user who silently failed the task is a more important finding than a loud user who got mildly annoyed and then succeeded. Watch what they do, not just what they say.
Logging Usability Findings Alongside Functional Bugs
Usability findings tend to live in a separate document from your functional bugs, which is exactly why they get forgotten. They sit in a slide deck nobody reopens while the functional bugs flow through the team's tracker and get fixed.
The fix is to put them in the same place. In Katalon True Platform, you can log a usability finding as a defect right alongside functional bugs, with the same severity field, the same priority, and the same path into Jira or Azure DevOps. A critical usability bug that blocks checkout sits in the same queue as a critical functional bug, because to the user, both stop them from buying the shirt.
This is the practical reason to treat usability testing as part of your quality process rather than a separate research activity. When the finding is a tracked defect with a severity and an owner, it gets fixed. When it is a line in a report, it gets read once and lost.
If you are also being asked to manually test AI features, the same detective-work mindset applies to a very different problem. Katalon's guide to testing AI applications manually is worth reading next.
|
FAQs
How many participants do you need for a usability test?
Five to eight participants, matched to your real user profile, will surface the large majority of usability problems. Beyond that number, sessions start repeating the same issues and each additional participant adds less value. Who you recruit matters more than how many: a small group of real users beats a large group of people who already know the product.
What is the think-aloud method?
Think-aloud is the core technique in usability testing. The participant narrates their thoughts as they attempt a task: what they expect, what confuses them, what they're looking for. The facilitator stays mostly silent and only prompts with neutral questions like "what are you thinking right now?" The participant's running commentary is the actual data.
How is usability testing different from functional testing?
Functional testing checks whether a feature works to spec: does the button do what it's supposed to do? Usability testing checks whether a real person can find that button, understand what it does, and trust what happens when they click it. Both are necessary and neither substitutes for the other.
How do you prioritize usability findings?
Score each finding on two axes: how badly it hurts the user (blocks the task versus slows or annoys) and how many users it affects (many versus few). A finding that blocks the task for many users is a critical UX bug and should be fixed immediately. A cosmetic issue that bothered one person can sit in the backlog.
Should usability findings be tracked in the same place as functional bugs?
Yes. Usability findings that live in a separate slide deck or report tend to get read once and forgotten. Logging them as defects, with the same severity and priority fields as functional bugs, in a tool like Katalon True Platform, keeps them in the same queue developers already work from, so they actually get fixed.
