What is A/B testing, and does my small business need it?
The short answer
Almost certainly not yet. A/B testing shows two versions of a page to different visitors and measures which performs better — but it needs enough traffic to tell a real difference from luck, and most small business sites do not have it. Below a few thousand visitors a month you will be reading noise. Fix the obvious things instead; they are bigger wins and they are free.
On this page
What it actually is
You have two versions of a page — say one with the price shown and one without. Visitors are split randomly between them, and you measure which produces more enquiries.
That randomisation is the entire point. Without it you are comparing this month against last month, and any difference could be the season, a holiday, a competitor's campaign or the weather. Splitting the same traffic at the same moment removes all of that.
Done properly it is the only way to know a change helped rather than to believe it did.
Why most small businesses should not bother yet
Here is the arithmetic nobody puts in the sales pitch.
To detect a meaningful difference between two versions you need enough conversions in each group that the gap could not plausibly be chance. As a rough guide, that means hundreds of conversions per version, which for a site converting at a few per cent means thousands of visitors per version, over a period short enough that nothing else changed.
A local business getting six hundred visitors a month and twelve enquiries would need years to run one clean test. And a test run for years is not a test, because the business, the market and the traffic have all moved.
The failure mode is worse than wasted effort. You run a test for three weeks, version B has nine enquiries to version A's six, and you conclude B is fifty per cent better. It is not. That gap is well inside what random chance produces at those numbers, and you have just made a permanent decision on a coin flip — while feeling more confident than before you started, which is the expensive part.
What to do instead
At small-business traffic levels, the wins are not marginal — they are structural, and you do not need a test to know they help.
Test the contact form works. Not an A/B test, an actual test. A broken form is a conversion rate of zero — see my contact form stopped working.
Put a price on the site. The single biggest change most small business sites can make, and you do not need statistical significance to know that a visitor who cannot tell whether they can afford you will leave.
Make it load in under three seconds on a phone on mobile data.
Ask for one thing per page, worded as what the visitor gets.
Answer the question people actually arrived with, which usually means a page per service rather than one combined page.
Each of those is a known improvement rather than a hypothesis. Do all five before you consider testing anything, because a test can only find small differences and you still have large ones available.
The version of testing that does work at small scale
Not A/B testing — sequential comparison with an honest caveat.
Change one thing. Note the date. Compare the next three months against the previous three, same source of traffic, and be explicit with yourself that seasonality is a confounder. It is a weaker method and it does not prove anything. But it is directionally useful, costs nothing, and it is what a small business can actually run.
The discipline that makes it work is changing one thing at a time and writing down what you changed and when. Most businesses cannot say what they altered six months ago, which makes any comparison meaningless regardless of method.
When it genuinely becomes worth it
- You have a few thousand relevant visitors a month, consistently
- The page you want to test is on the path to an outcome you can count
- You have already done the structural fixes above
- A small percentage difference is worth real money to you
Until then, the honest answer is that you would be buying a tool that produces numbers your traffic cannot support — which is worse than no numbers, because confident wrong answers get acted on.
What this looks like in Helm
The prerequisite for any version of this — proper testing or the sequential comparison above — is being able to change a page and see what happened.
Changing it is a field and a Go Live: the price, the button wording, the headline, the page text. Which matters because the method depends on iteration, and iteration dies if each change is a support ticket.
Seeing what happened is in the same dashboard: entry pages, sources, form submissions, and date ranges with a comparison overlay so this quarter against last quarter is two clicks rather than a spreadsheet.
One honest note: Helm does not do split testing. There is no traffic-splitting engine, and for the sites it serves that is the right call — the tool would produce numbers the traffic cannot support. If you grow to the point where genuine A/B testing pays, you will want a dedicated tool for it, and we would tell you so.
The short version
A/B testing is a real technique that needs real traffic. Below a few thousand visitors a month you will be measuring luck and acting on it confidently, which is worse than not measuring. Do the five structural fixes first — they are bigger, free, and certain. Then change one thing at a time, write down when, and compare quarter against quarter until you are big enough for the real method.