What CRO is and what it is usually sold as instead
Three things are routinely marketed as conversion rate optimisation without being it. Knowing the difference protects your budget.
| Frequently sold as CRO | Why it is not |
|---|---|
| Applying best practices | A best practice is the average result of other people's tests on other people's audiences. It is a reasonable source of hypotheses and it is not evidence about your page. |
| Installing a heatmap tool | Heatmaps describe behaviour. They do not explain it and they cannot prove a change caused an improvement. They are an input to CRO, not CRO. |
| Redesigning the page | A redesign changes forty variables at once. If it wins you cannot say why and cannot repeat it. If it loses you cannot say which part was wrong. That is a rebuild, not an experiment. |
Real CRO produces two things a redesign cannot: a measured effect size and a piece of durable knowledge about your buyers that outlives the specific page.
The honest traffic conversation, before anything else
A/B testing needs volume. This is arithmetic rather than opinion and it is the question most agencies avoid because the honest answer disqualifies a share of their prospects. Here is what it actually takes to detect a change with 95% confidence and 80% power.
| Current conversion rate | Lift you want to detect | Visitors needed per variant | Realistic verdict |
|---|---|---|---|
| 2% | 20% relative | Around 19,000 | Needs roughly 38,000 visitors per test. Most B2B landing pages cannot supply this. |
| 5% | 20% relative | Around 7,600 | Workable at around 15,000 visitors per month per test. |
| 5% | 10% relative | Around 30,000 | Small lifts are expensive to prove. Do not chase them at low volume. |
| 10% | 20% relative | Around 3,600 | Comfortable. Roughly 7,200 visitors per test. |
| 5% | 50% relative | Around 1,200 | Big swings are cheap to detect. Test bold changes when volume is limited. |
The practical rule: if a page produces fewer than about 200 conversions per month, classical A/B testing will waste your money and produce results you should not believe. That does not mean you have no options. It does mean the honest options are different.
| If your volume is | What we recommend instead |
|---|---|
| Under 200 conversions a month | Qualitative research, session replay review, user testing and bold redesign of the whole page based on findings. Measure before and after as a directional signal rather than a proof. Test big changes only. |
| 200 to 1,000 conversions a month | A/B testing on high impact elements only: offer, hero message, form structure and pricing presentation. One test at a time. Expect a test to run three to six weeks. |
| Over 1,000 conversions a month | A continuous programme. Multiple concurrent tests on separate pages, a maintained backlog and a shipping cadence of two to four tests a month. |
How we run a programme
| Stage | Question it answers | What we do |
|---|---|---|
| 1. Measure | Where exactly is the loss? | Funnel analysis in GA4, segmented by device, traffic source and returning versus new. Most pages have one segment dragging the average down and it is usually mobile. |
| 2. Research | Why are they leaving? | Session replay, scroll and click analysis, form field analytics, exit intent surveys and five to eight moderated user sessions on the live page. |
| 3. Hypothesise | What might fix it? | Every finding becomes a written hypothesis in a standard format, scored and ranked into a backlog you can see. |
| 4. Test | Did it work? | One variable per experiment where volume allows. Sample size fixed in advance. No stopping early because the numbers look good on day three. |
| 5. Ship and learn | What do we now know? | Winners deployed. Losers documented. Every result written up with the effect size, the confidence level and what it teaches about your buyers. |
How we write a hypothesis
A hypothesis that cannot be wrong is not a hypothesis. Every test in your backlog is written in the same structure so that the result is interpretable regardless of which way it goes.
Because session replay shows 61% of mobile visitors abandon at the third form field, which asks for company size, we believe that moving company size to a post submission qualification step will increase mobile form completion by at least 15% measured by completed submissions divided by form starts, on mobile only, and we will be wrong if completion rises but sales report a drop in lead quality.
That last line matters more than the rest. A test that lifts conversions and lowers lead quality is a loss dressed as a win and it is the most common way CRO programmes damage a business while reporting success.
Prioritising the backlog
You will always have more ideas than traffic to test them. We score each hypothesis on three dimensions and test in rank order rather than in order of who suggested it.
| Factor | What we score |
|---|---|
| Potential | How much room the page has to improve at this point in the funnel, based on where the measured loss actually is |
| Importance | How much traffic passes through this element and how qualified that traffic is. A change above the fold on your highest spend campaign outranks a footer tweak permanently |
| Ease | Build effort, technical risk and whether it needs design, copy or developer time. Cheap tests that ship this week beat expensive tests that ship next quarter |
What we test, in the order that usually pays
- The offer itself. What you ask for and what you give in return produces larger swings than any element on the page. Moving from "book a demo" to "see a two minute product tour" changes commitment level, not just wording.
- The hero message. The first fifteen words decide whether the rest gets read. This is also where a different message can be tested cheaply against a control.
- Form structure. Field count, field order, multistep versus single step and whether the qualifying question sits before or after the submission.
- Proof placement. Not whether you have testimonials but whether the right proof sits directly after the claim it supports.
- Pricing presentation. Anchoring, plan order, annual versus monthly default and whether pricing appears at all before a conversation.
- Friction removal. Navigation on a paid landing page, competing calls to action and anything that offers an exit before the argument is finished.
Where results come from and where they do not
Two honest expectations. First, most tests do not win. In a well run programme roughly one test in five produces a clear winner and that is normal rather than a failure. The value of the other four is knowing which assumptions are false before you build a strategy on them. Second, button colour tests do not move businesses. If an agency's case studies are all micro element wins, they are testing where it is safe rather than where it matters.
When testing reveals that the page's underlying argument is wrong rather than its execution, the correct move is not another element test. It is to rebuild the page from scratch and treat the new version as a fresh control.
One variable is worth checking before you test anything else: page speed is a conversion variable and a slow page loses visitors before any of your copy has a chance to be judged.
Tools we work in
We work in your existing stack where one exists rather than forcing a migration. Testing platforms we use regularly include VWO, Optimizely and Convert. Analytics and behavioural research typically run through GA4, Microsoft Clarity, Hotjar or PostHog. Where server side testing is required for performance reasons, we implement at that layer instead of running a client side script that slows the page it is testing.
Reporting
- Every test written up with sample size, effect size, confidence level and segment breakdown
- Losing and inconclusive tests reported in the same detail as winners, because a suppressed loss is how a programme starts lying to itself
- A running knowledge document of what has been established about your buyers, which is the asset that survives after the individual pages change
- Monthly review call covering results, backlog reprioritisation and what the last month changed about the plan
