Start an A/B test against a live version, or split a draft section into two variants, then configure how the test runs and concludes.
There are two ways to start an A/B test, depending on whether you are testing against something that is already live. Either way, you configure the test in a short wizard on the section you are testing. On a draft with no test yet, the insights panel also nudges you with a Run an A/B test recommendation. For the concepts behind these, see how A/B tests work.
Test Against a Live Version (Control Versus Variant)
Use this when you have a live version and want to test a change against it. The live setup becomes the control.
1
Start the Test from the Live Version
On the live version, click Set up A/B test (it also appears as a recommendation in the insights panel). Mention Me creates a new draft version based on the running version.
Starting an A/B test from a live version.
2
Pick the Baseline and Name the Version
If the live version already has more than one variant, choose the baseline variant to test against. Enter a Version name (internal), then click Create test. Mention Me creates a new draft version based on the live version, with your chosen baseline as the control.
3
Set Up the Test on a Section
On the section you want to test (for example Rewards or Sharing), click its + A/B test button. The baseline stays as the control (Variant A); edit Variant B to create the change you want to test.
Split a Section on a Draft (Variant Versus Variant)
Use this when you are building something new and want to compare two fresh ideas, with no live baseline.
1
Split the Section
On a draft version, click the + A/B test button on the section you want to test (for example Rewards or Sharing). The section splits into Variant A and Variant B.
2
Edit Each Variant
Edit both variants so they differ. Neither is a control here; the two are compared as equals.
You can A/B test six sections of a version, each via its + A/B test button: Branding, Images, Rewards, Sharing, Referrer journey and Referee journey.
Setting up a test on a section opens a wizard with three steps.
1
Overview
A summary of what will happen: your current setting becomes the control (or Variant A), you edit the other variant, and traffic is split 50/50. Click Next.
The A/B test setup wizard, Overview step.
2
How It Runs
Choose the test metric that decides the winner, Conversion rate or Share rate (the recommended option is marked). Then set how fast the test should run:
For a test against a live baseline, pick a test speed: Normal (highest confidence), Fast, or Fastest. Each shows its confidence target and estimated duration.
For a variant-versus-variant test with no baseline, tell Mention Me your weekly order volume (Less than 200 orders a week or More than 200 orders a week). This sets the confidence target, and the test runs for a fixed 8 weeks.
Choosing the test metric and speed.
3
When It Concludes
Decide what happens for each outcome:
If a winner is found. The winning variant is promoted automatically.
If a better variant is found (but no winner). Choose Allocate to the best performer (recommended), keep the control, or pick a specific variant.
If there’s no meaningful difference. Choose to keep the control or allocate to the variant.
Next, make your variants meaningfully different and refine the test before launch: edit Variant B, use View differences, and adjust or remove the test from the section’s manage menu. See manage an A/B test.
