Sample ratio mismatch (SRM)

A sample ratio mismatch occurs when the number of users actually observed in each arm of an experiment differs from the intended allocation, for example a planned 50/50 split arriving as 52/48 by more than chance would explain. It is checked with a chi-square test comparing observed counts to expected counts under the intended ratio.

SRM is a diagnostic signal, not a result: its presence means something in the experiment's plumbing is broken, biased randomization, a bot filter treating one arm differently, a redirect or page-load failure that drops users from one variant more than the other, or logging that undercounts one group, and it invalidates the experiment's other metrics until fixed, because whatever caused the imbalance likely also biased who ended up in each group.

Practitioners treat SRM checks as a mandatory gate before reading any other result: a significant mismatch, commonly flagged well below a p-value of 0.001 since even small true biases are worth catching, should stop analysis and trigger an investigation into instrumentation, redirects, or exclusion logic, rather than being explained away as noise. Ignoring an SRM and interpreting the headline metric anyway is one of the most common causes of a false experiment conclusion.