Skip to main content
As part of the improvements to A/B testing introduced in summer 2025, Mention Me updated the metrics used to evaluate test performance. This article explains the changes, why they were introduced, and how they differ from existing metrics.

​
What Changed?

Mention Me introduced three new metrics for tests:
  • Variation share rate
  • Variation purchase rate
  • Variation conversion rate
These are used specifically within A/B tests or single variant testing. The existing share, purchase, and conversion rates remain unchanged at the campaign, market, and programme levels. The original metrics are calculated from a broader user base - those enrolled in any offer - regardless of which variation they saw. The new metrics only consider customers who were shown a specific test variant, offering more targeted and reliable insights during experiment evaluation.

​
Metric Comparison

​
How Does This Help?

The new metrics more accurately measure the impact of a specific offer by exclusively tracking users who:
  • Were shown that exact variant
  • Decided to refer (share)
  • Earned purchases from their shared offers
This filtering prevents unrelated impressions, enrolments, and conversions from skewing test results. In A/B testing, this reduces noise and allows faster, clearer conclusions. While the new metrics are used within experiments only, the original metrics still serve as a complete picture of campaign success and continue to be used in programme-level reporting. You may notice that unique impressions can exceed total enrolments. Along with the metric changes, this might result in numerical differences compared to what you’re used to - but is expected and more accurate for experimentation.

​
Why Would a Customer See One Offer and Be Enrolled in Another?

Referral links are valid for a set period (typically 3-6 months). If a referee visits after this time, they’re shown the most current offer - so they may convert via a different offer than the referrer originally promoted. In A/B testing, this leads to an attribution ambiguity: should the original offer be credited, or the new one shown? For experiment integrity, Mention Me’s updated metrics attribute only to the experience actually displayed to the customer. This makes A/B test evaluation more precise. These scenarios are still valuable and are fully included in overall campaign metrics outside of test performance reports.
Rolled-forward example journey showing offer attribution across time

Rolled-forward example: how offer attribution works across time

Last modified on July 13, 2026