Qualitative vs Quantitative CRO Research: How to Find What to Test

Most guides that rank for conversion research methods stop at a tidy definition: quantitative data is numbers, qualitative data is opinions, use both. Useful for a pub quiz, useless on a Monday morning when someone asks which page you should test first. They also skip the two things that actually decide whether your research is any good in Britain: how the consent banner mangles your data, and how few sessions you genuinely need to watch before a pattern is obvious.

This is the discovery piece. It sits before any test gets built. The job here is narrow and worth doing well: find the leaks, understand why they happen, and turn what you learn into a ranked list of things to try. Quantitative research tells you where the problem is. Qualitative research tells you why it is happening. You find what to test by reading them together, then prioritising.

The split that actually matters

Forget “numbers versus opinions”. The honest division is by the question each method answers.

Used alone, each one misleads you. Analytics tells you the mobile checkout converts at half the desktop rate, then goes quiet. Watch eight recordings of mobile abandoners and you see the postcode lookup fails on Safari, or the discount field demands a code nobody has. The number found the wound; the recording told you what cut it.

Start quantitative: find where the money leaks out

Begin with analytics because it is cheap, fast, and stops you researching pages that do not matter. The goal of this pass is a shortlist, not an explanation.

In GA4, build a funnel exploration across your primary conversion path and segment it by device and by traffic source. Note that GA4 renamed “conversions” to key events in 2024, so set those up first. You are hunting for three things: the step with the steepest drop, a segment that performs far worse than the rest (mobile is the usual culprit), and pages with high traffic and weak conversion, which are your best targets because a small percentage gain moves real volume. Our companion guide on conversion funnel optimisation walks through reading those drop-offs step by step.

One British caveat the US guides ignore entirely: your analytics is only as complete as your consent banner lets it be. Under PECR and UK GDPR, visitors who decline analytics cookies are not in your GA4 numbers at all, so a “20 percent drop” might be 20 percent of a consented subset that skews towards more engaged users. The 2025 Data (Use and Access) Act relaxes the position for genuinely low-risk analytics, but the safe operating assumption today is still that some traffic is missing from every chart. Treat quantitative findings as directional, not gospel, and let qualitative work confirm them.

Then qualitative: find out why

Once you have a shortlist of leaky pages, switch to understanding them. Three methods, in rough order of effort.

Session recordings and heatmaps. Watch real sessions of people who abandoned the leaky step. You are not auditing the whole site; you are watching abandoners on one page. Click and scroll heatmaps show where attention goes and where it dies; rage clicks and dead clicks flag broken or misleading elements. Microsoft Clarity does all of this and is genuinely free with no traffic caps or paid tiers, which is why it is the default starting point for low-budget teams. Hotjar is now part of Contentsquare, and Google Optimize is gone, so the free tooling landscape in 2026 is simpler than older articles suggest.

On-site surveys. A one-question poll on the leaky page, or a post-conversion survey, gets you reasons in the visitor’s own words. Keep it to one to three questions and time it contextually: exit-intent on a checkout, or immediately after purchase. The single biggest mistake is asking hypotheticals (“what features would you like?”), which produce wish lists nobody acts on. Ask about specific past behaviour instead. Copy-paste starting set:

• “What stopped you from completing your purchase today?”
• “What almost stopped you from buying?”
• “What were you hoping to find that you didn’t?”
• “Did you have any concerns about pricing, trust or payment security?”

User testing. Give five people a task (“find a kettle under thirty pounds and add it to the basket”) and watch where they stall. This is where small samples earn their keep.

How many recordings, surveys and testers? (real numbers)

The ranking pages say “qualitative samples are small” and leave it there. Here are figures you can plan against.

Jakob Nielsen’s well-known model, with co-author Tom Landauer, found that testing with just five users surfaces about 85 percent of an interface’s usability problems. The maths is problems found = N(1 − (1 − L)^n), where a single user typically catches around 31 percent of issues. The practical lesson, from Nielsen Norman Group, is to run several small tests rather than one large one.

For interviews and open-ended research, academics talk about saturation, the point where new participants stop telling you anything new. Guest, Bunce and Johnson’s well-cited 2006 study found basic themes commonly land by around 12 interviews, with more heterogeneous studies needing more. For CRO you rarely need the high end.

Translated into a session-recording rule of thumb: watch recordings of abandoners on one leaky page until you have seen the same friction three times, then stop. In practice that is usually 8 to 15 sessions, not the hundreds people imagine. If you are still seeing brand-new problems after 20, your page has bigger issues than any single test will fix.

When you cannot A/B test, qualitative IS your programme

Most research guides quietly assume you have the traffic to run experiments. Plenty of UK sites do not. If you convert fewer than roughly 500 times a month, a classic A/B test will take months to reach significance, if it ever does. You can check whether your traffic can carry a test with our sample size calculator and the minimum detectable effect calculator before committing.

When the maths says no, do not give up on optimisation; change the method. Lean almost entirely on qualitative research and obvious-fix changes: ship the things recordings and surveys prove are broken, and measure the before-and-after on the metric directly rather than splitting traffic. You lose statistical certainty, but a checkout that errors on Safari does not need a 95 percent confidence interval to justify fixing.

The bridge: turning findings into ranked tests

This is where competing articles wave their hands and say “form a hypothesis”. The chain is more explicit than that, and it runs in one direction:

1. Finding. An observation backed by evidence. “Eight of twelve mobile abandoners on the delivery step tried to tap a field that is not actually editable.”
2. Prioritise. You will have more findings than capacity. Score them. ICE (Impact, Confidence, Ease) or PIE (Potential, Importance, Ease) work for quick triage; CXL’s PXL framework replaces gut-feel scoring with binary questions (“is the change above the fold?”, “is it backed by user data?”) to cut subjectivity. Our CRO programme guide has worked PIE and ICE tables with real numbers.
3. Hypothesis. Phrase the top finding as a falsifiable statement: “Because mobile users mistake the static field for an input, making it a real editable field will reduce delivery-step abandonment.”
4. Test. Now, and only now, you build the experiment.

Peep Laja’s widely quoted line is that CRO is “80 percent research, 20 percent experimentation”. The point of the research half is precisely this master list of evidenced, prioritised findings. Skip it and you are A/B testing button colours because someone in the meeting had a hunch.

A two-week research sprint you can copy

For a single leaky page or flow:

• Days 1 to 2: GA4 funnel and segment analysis. Confirm where the drop is and which segment is worst.
• Days 3 to 9: Set up Clarity, let recordings accumulate, add a one-question exit survey to the leaky page. Run five user-testing sessions if you can.
• Days 10 to 12: Watch abandoner recordings until friction repeats, read survey responses, write up findings with evidence attached.
• Days 13 to 14: Score findings, write hypotheses for the top two or three, hand them to whoever builds the test.

The deliverable is a written list of problems, each tied to its evidence and a hypothesis. That document is the whole reason to do research, and it is what the pages currently ranking for this term never actually show you how to produce. For the wider picture of how this feeds a repeating optimisation loop, see the conversion rate optimisation guide.

Frequently asked questions

Which should I do first, qualitative or quantitative? Quantitative first, then qualitative. Analytics gives you a shortlist of pages and steps worth investigating, which stops you spending hours watching recordings of pages that convert fine. Once you know where the leak is, qualitative research tells you why.

How many session recordings should I watch? Watch recordings of abandoners on one page until you have seen the same friction repeat about three times, usually 8 to 15 sessions. If new problems keep appearing after 20, the page needs broader work than a single test.

Is five users really enough for user testing? For finding usability problems on one flow, yes. Nielsen and Landauer’s model puts five users at roughly 85 percent of an interface’s issues, and several small tests beat one large one. Five is a starting point, not a hard ceiling.

Do I need paid tools for conversion research? No. GA4 covers the quantitative side and Microsoft Clarity covers heatmaps and session recordings with no traffic caps or paid tiers. You can run a complete research sprint without spending anything.

How does UK privacy law affect my research data? Visitors who decline analytics cookies under PECR and UK GDPR are absent from your GA4 figures, so your numbers describe a consented subset, not everyone. The 2025 Data (Use and Access) Act eases this for low-risk analytics, but treat quantitative findings as directional and confirm them qualitatively.

What survey question should I ask on a page where people drop off? Ask about specific past behaviour, not hypotheticals. “What stopped you from completing your purchase today?” beats “what features would you like?”. Keep it to one to three questions and trigger it on exit-intent or just after conversion.