How to Run an A/B Test: Proper Experiment Design for Ads and Websites

An A/B test shows two versions to the same audience at the same time and measures which works better. It looks simple, but poor design leads to wrong decisions.

Proper design

• One variable: headline or image, not both.
• A clear hypothesis: “A benefit-first headline raises clicks.”
• Enough sample: at least 100 conversions per version, or traffic sized with a calculator.
• Duration: at least one full week to capture day-of-week effects.
• No early stopping: early differences are usually noise.

What to test

Start with high-impact elements: the offer, headline, hero image, CTA text, form length. Small variables such as colour and button radius come last.

Record

Document every test with hypothesis, result and lesson; the test library is the team’s institutional memory.