# Margin of Error

**Category:** metrics  
**Short Description:** Maximum expected difference between sample estimate and true value. A measure of the precision of a sample estimate.  
**Last Updated:** 2026-07-07T00:00:00Z

## Definition

Margin of error represents the maximum expected difference between a sample-based estimate and the true population value, given a specific confidence level. In advertising, it helps quantify the reliability of metrics and determines required sample sizes for meaningful testing.

## Formula

**Formula:** `Margin of Error = Critical Value × Standard Error`

Half the width of the confidence interval — how far your sample estimate could sit from the true value at a given confidence level; it shrinks as the sample grows.

## Calculation

**Formula:** `Critical Value × Standard Error`

**Explanation:** For advertising metrics, the critical value depends on desired confidence level (typically 1.96 for 95%), and standard error accounts for sample size and variance. Larger sample sizes generally lead to narrower, more reliable intervals. The result is in the same units as the metric being estimated (a CTR margin of error is in percentage points).

### Components

- **Critical Value**: The value based on the desired confidence level
- **Standard Error**: The measure of sample estimate precision

## Industry Benchmarks

| Segment | Typical Range | Median | Notes |
| --- | --- | --- | --- |
| CTR test at 95% confidence, ±0.5 pp target | n ≈ 15,000 – 25,000 / variant | ~19,000 | Assumes 2% baseline CTR; lower baselines need more traffic. |
| Conversion-rate test, 3% baseline, ±0.3 pp | n ≈ 10,000 – 12,000 / variant | ~11,000 | Tighter MOE targets explode sample requirements — plan MDE before the test. |
| Stakeholder reporting (directional) | ±5% – ±10% relative on ROAS | ±7% | Acceptable for forecasting; too wide to pick creative winners. |

**Sources:** Evan Miller A/B sample-size calculator, Standard binomial MOE tables, Marketing analytics practice

## Examples

- A 3% conversion rate measured on 10,000 visitors: standard error = √(0.03 × 0.97 / 10,000) ≈ 0.17%, so margin of error at 95% confidence = 1.96 × 0.17% ≈ ±0.33 pp — the true rate plausibly sits between 2.67% and 3.33%
- CTR of 2.5% with ±0.3% margin of error at 95% confidence
- ROAS projection accuracy within ±15% for budget forecasting

## How AdSights Helps

**Tracking Margin of Error:** Margin of error is the difference between 'Variant B looks better' and 'Variant B is measurably better.' AdSights shows per-variant sample progress against the MOE you need for your minimum detectable effect — so teams don't call tests at 500 conversions when the plan required 5,000. When a creative test is underpowered, AdSights surfaces it before budget is reallocated to a noisy leader.

## FAQs

### How do you calculate the margin of error for a conversion rate?

For a rate p measured on n observations, MOE = z × √(p(1 − p) / n), where z is 1.96 for 95% confidence. Worked example: a 3% conversion rate on 10,000 visitors gives a standard error of √(0.03 × 0.97 / 10,000) ≈ 0.17%, so the margin of error is 1.96 × 0.17% ≈ ±0.33 percentage points — the true rate plausibly sits anywhere from 2.67% to 3.33%. Any variant difference smaller than that spread is inside the noise.

### What margin of error is acceptable for ad tests?

Smaller than the effect you're deciding on — that's the only rule that matters. If two variants differ by 0.5 pp and each estimate carries a ±0.4 pp margin of error, the comparison is effectively a coin flip regardless of who leads. For creative decisions, size the test so the MOE is at most half the minimum lift you'd act on; for directional stakeholder reporting, ±5–10% relative is usually tolerable.

### How does sample size affect the margin of error?

Margin of error shrinks with the square root of sample size, so precision gets expensive fast: halving the MOE takes four times the data. A 3% conversion rate carries roughly ±0.33 pp at 10,000 visitors, ±0.17 pp at 40,000, and ±0.08 pp at 160,000. This square-root law is why tight creative tests need surprisingly large budgets, and why chasing very small lifts on low-traffic campaigns rarely pays for the measurement.

## Related Terms

### Similar Terms

- **[Confidence Interval](/resources/glossary/metrics/confidence-interval)**: Margin of error defines the width of confidence intervals

### Component Terms

- **[Statistical Significance](/resources/glossary/metrics/statistical-significance)**: Helps determine when differences exceed margin of error
- **[Sample Size](/resources/glossary/metrics/sample-size)**: Larger samples reduce margin of error for more precise estimates
- **[Population Mean](/resources/glossary/metrics/population-mean)**: Margin of error bounds how far a sample estimate can plausibly sit from the population mean

## Featured in topic hubs

- [Experimentation & Statistics](/resources/topics/experimentation)