AdOpt experiment dashboard

Two sources, each used for what it is good at. Elasticsearch carries the arm assignment, so it supplies every ratio and uplift. MySQL carries the billed ledger, so it supplies every absolute dollar figure.

Arms

• Baseline = control — roughly 5% of requests.
• Optimized = adopt_thompson_balanced — roughly 95%.
• Grouping uses model_group, the randomised assignment, not treatment.

Why per 1M requests, not per impression

AdOpt changes how many impressions you win, so impressions are an outcome of the test, not a fixed denominator. Requests are the randomisation unit, so every rate is divided by SUM(sample_weight) across both log streams (no_bid weight 100, has_bid weight 10). Measured per impression, a winning treatment can look like a loss.

Which denominator a breakdown uses

Fields beginning win_ — endpoint, preset, AdOpt flags — only exist on documents where a bid won. So the request count inside one of those buckets is really a win count, and a win is an AdOpt outcome. Dividing by it would silently turn every rate into a per-impression figure and flip the sign of the uplift. Those breakdowns are therefore normalised against the arm's total client requests, the same denominator the headline figures use.

Request-level fields — country, SSP, device, publisher — appear on every document, no_bid included, so their own bucket count is a genuine request count and is used directly.

Money in Elasticsearch

Gated on is_impression = 1. Revenue is SUM(cost_w) and cost is SUM(rev_w) — the two materialised fields are stored swapped relative to their names. Prices are CPM, so both are divided by 1000.

Booked dollars

From <client>.stats: revenue = dsp_spend, cost = ssp_spend. This is the whole book, both arms together, and it is the number to quote externally. Elasticsearch margins usually read a little under booked margin, because cost there is our bid while we clear at the second price.

Extra dollars created

The treatment share of booked money, minus what that same traffic would have earned at the baseline rate: booked$ × treatment_share × u / (1 + u), where u is the per-request uplift for that metric and treatment_share comes from the arms' revenue split in Elasticsearch.

Rows marked "allocated"

The ledger only breaks out SSP, SSP company and endpoint. For any other breakdown — country, device, publisher — booked dollars are apportioned by that slice's share of Elasticsearch revenue. Same for every filtered view. Rows sourced straight from the ledger are marked direct.

The uncertainty band

The ± figure is a Poisson proxy on impression counts. It reflects sample size only, not price dispersion, so read it as a noise floor rather than a true confidence interval. Baseline is only ~5% of traffic, so thin slices move around a lot; rows under 500 baseline impressions are greyed out.

Coverage

Elasticsearch keeps about 7 days, so uplifts are unavailable before that. MySQL goes back much further, so booked dollars still render for older ranges. All times are UTC.