A/B testing troubleshooting

Setup Issues

Issue	What to do
”The A/B test variants are identical. Differentiate them to run a meaningful A/B test.”	Edit Variant B so it differs from the control. See manage an A/B test.
”This A/B test is missing required settings. Set the test metric and what happens on each outcome.”	Reopen the wizard via Adjust test settings and complete the metric and outcome steps.
The promotion option is missing	A time-limited promotion cannot run an A/B test. Use a standard version.

I Can’t Edit the Test

You cannot edit a version while its A/B test is live, so set the metric, speed and outcomes before you launch. The control variant is always view-only; edit the other variant.

The Test Is Taking a Long Time

A test runs to a maximum of 8 weeks (occasionally up to 12 where a return or cooling-off window applies). Most winners are found within about 4 weeks.
For low-volume audiences, choose Share rate as the test metric, or pick a faster speed (which uses a lower confidence target). With very low volumes, it is often better to follow referral best practices than to split limited traffic between variants.

There’s No Clear Winner

If the test reaches its maximum duration without a clear winner, Mention Me applies the outcome you chose in the When it concludes step (for example keep the control, or allocate to the best performer). You can review the results and switch traffic to the other variant once.

Still Stuck?

Contact support.

Getting Started

Customers

Campaigns

Reporting

Data & Uploads

A/B testing troubleshooting

Setup Issues

I Can’t Edit the Test

The Test Is Taking a Long Time

There’s No Clear Winner

Still Stuck?

​Setup Issues

​I Can’t Edit the Test

​The Test Is Taking a Long Time

​There’s No Clear Winner

​Still Stuck?

Setup Issues

I Can’t Edit the Test

The Test Is Taking a Long Time

There’s No Clear Winner

Still Stuck?