Using NN/g research to achieve ~98% usability issue detection through synthetic testers

How many synthetic testers do I need?

At Uxia, we redefine usability testing by using synthetic testers - AI-powered testers capable of evaluating your product designs in minutes.

A frequently asked question is:

“How many synthetic testers do I need?”

The well-known Nielsen Norman Group article “Why You Only Need to Test with 5 Users” explains that in human usability tests, each additional user brings fewer new insights.

But with synthetic testers, the constraints of human testing disappear.
Based on the same NN/g mathematical model, 10 synthetic testers allow Uxia to uncover ~97-98% of usability issues, offering better coverage without increasing cost or time.

This article explains why.

What the Nielsen Norman Group Research Says

In Jakob Nielsen’s foundational article, the core idea is that usability tests follow a predictable curve:

Each new user uncovers fewer new usability issues than the previous one.

NN/g formalises this using Nielsen & Landauer’s model:

Problems found = N × (1 − (1 − L)ⁿ)
where:

• N = total number of usability issues

• L = probability one user finds a given issue

• n = number of users

In typical studies, L ≈ 31%, meaning:

• 5 users uncover ~85% of issues

• More users = diminishing returns

NN/g then recommends five users per iterative round, not because five is magic but because human testing becomes expensive after that point.

Why Uxia Uses 10 Synthetic Testers, Not 5

Synthetic testers flip the economics of research completely:

• No recruitment

• No incentive costs

• Instant scalability

• Repeatable, consistent evaluation

• Ability to run multiple variations automatically

Because the constraints of human testing disappear, we’re no longer limited to “five” by cost but we are still guided by the same underlying mathematics of problem discovery.

Here’s why 10 synthetic testers is optimal for Uxia:

1.Ten Testers Surface Up to ~98% of Issues

With synthetic testers, the L value is significantly more stable (more consistent detection of problems across runs than humans).

Plugging n = 10 into the Nielsen/Landauer curve yields:

~98% of issues uncovered

This dramatically increases confidence while keeping the test lean and fast.

Where human testing hits diminishing returns, synthetic testers benefit from volume because adding testers costs virtually nothing, all Pro Plans include up to 10 testers per test. If you want even more insights and accuracy, Custom Plans allow as many testers as needed.

2.Synthetic testers allow diversity of perspectives

One limitation of NN/g’s “5 users” rule is that it assumes a single, homogeneous user group.

At Uxia, we take the user description you provide - such as demographics, social factors, behaviours, etc - and enrich it with publicly available online data to build a detailed, realistic persona profile.

From there:

• Uxia generates multiple high-fidelity synthetic users

• Each tester reflects a different interpretation of your target audience

• You can choose up to 10 of those testers

• You select the exact mix of user profiles you want to evaluate your product

This means the testers you run are tailored representations of your real audience, not abstract AI users.

Instead of fixed personas like “novice” or “expert,” you define the user reality - and Uxia brings it to life through synthetic users built to match your market.

With 10 testers, we can guarantee multiple user styles are represented - something you cannot reliably achieve with 5 human testers unless you segment your sample.

3.They Are Consistent and Repeatable

Humans vary by:

• experience

• mood

• fatigue

• interpretation

Synthetic testers don’t.

This allows us to scale from 5 to 10 without introducing human-variability bias.

4.Synthetic Testing Is Extremely Fast

A 10-person human usability study takes from hours to days to be completed depending on your target audience.

A 10-synthetic-tester study in Uxia takes ~10 minutes.

With speed no longer a bottleneck, 10 testers is the new optimal baseline.

5.Testing with 10 Users Comes at No Extra Cost

With Uxia, increasing the number of testers doesn’t increase the price. All Pro plans include testing with up to 10 synthetic users, allowing you to achieve higher coverage without any additional fees.
If you need to run tests with more than 10 users, you can upgrade to a Custom plan that scales to your needs.

Conclusion

Nielsen Norman Group’s research teaches us that you don’t need huge sample sizes to uncover most usability issues.

Uxia elevates this idea using synthetic testers - and makes it possible to increase effectiveness by testing with 10 synthetic users, delivering:

• Fast insights

• Scalable evaluation

• Affordable research

• High consistency

• Behaviour-diverse testing

• ~98% issue detection with 10 testers

If you want to understand how synthetic testers compare to human testers - and see a real example - check here.