The Disciplines That Matter
- Test one thing at a time. Change a single variable, such as imagery, copy or incentive, so you can attribute any change to it. Don’t change the reward and the imagery in the same test.
- Prioritise by impact, ease and volume. Favour changes that are likely to move results, quick to make, and where you have enough traffic to reach a conclusion.
- Always be testing. Treat it as continuous optimisation: when one test concludes, start the next.
- Give both variants equal exposure. Promote referral across post-purchase, homepage and email so the test gets enough traffic.
Common Pitfalls
- Testing too many things at once.
- Stopping a test before it concludes.
- Acting on incomplete results, or cherry-picking the data.
Which Metric to Use
Decide the winner on Conversion rate for higher-volume audiences (it tracks purchasing directly) or Share rate for lower-volume audiences (it measures share behaviour and concludes sooner). See how A/B tests work.Proven Test Ideas
Real results from live referral programmes, and good starting points for your own tests:
Results vary by brand and audience, so treat these as hypotheses to test rather than guarantees.
See Also
Set up an A/B test
Set up an A/B test.
Manage an A/B test
Manage an A/B test.