Metric definitions

This page states how Apex turns raw events into the numbers on the results page, the results API, and apex experiments results. Use it when you reconcile raw events with reported numbers.

Results window

• The window runs in UTC. It starts at the later of the first launch and the last results reset. It ends when the experiment ended, or now, rounded down to the minute.
• Changing the allocation, weights, or variations starts a new assignment epoch for visitor bucketing, but it does not move the results window. To count only data after such a change, reset the results.
• Custom date ranges on the results page use the shop's reporting time zone.
• If the experiment uses a weekday filter, only visitors whose first exposure fell on a selected weekday are counted.
• Results only include the experiment's environment, which is production by default.

Visitor

A visitor is a distinct visitor ID with at least one exposure event for that variation inside the window. Exposure events are experiment views and rendered personalizations. Being assigned without seeing the experiment, or firing a goal event, does not make someone a visitor.

• A visitor exposed to two variations counts in both.
• Bot traffic is always excluded.
• QA-session traffic is excluded unless you explicitly include it.
• Placeholder visitor IDs that cannot identify a real person are excluded from visitor counts.
• For purchase goals, an order that Apex matched to a variation on the server without a tracked exposure adds its visitor to that variation's population.

Conversions

• Count once: each exposed visitor counts at most once per variation in the window.
• Count every: every qualifying event counts, so conversions can exceed visitors. Purchase and revenue goals always count distinct buyers. A value goal follows its own setting: set to count every, it sums the value of every qualifying event.
• Only goal events at or after the visitor's first exposure to that variation in the window count. Earlier conversions are dropped, and the results page shows a warning when this is material.
• Duplicate events with the same event ID count once.

Resetting results starts a new window and re-arms count-once goals. A reset "from now" also starts a new assignment epoch: returning visitors are re-bucketed with the same deterministic hash, so they keep their variation unless the allocation changed. A reset to a past date keeps the current epoch.

Purchases

A purchase conversion is a distinct buyer, not an order. An order counts for an experiment when all of these hold:

1. The order belongs to the variation recorded on the order's first event. It does not depend on which variation the visitor saw last. One order can count in several experiments.
2. The order was first seen inside the window.
3. The same visitor was exposed to that variation in the window before the order, or the visitor has no exposure to that variation at all and the order was matched on the server.

There is no attribution expiry: any order from the first exposure to the end of the window counts. Orders placed before exposure, and orders from visitors exposed only before the window started, are not counted.

Orders are identified by their order key, falling back to the event ID. Repeat events for the same order within 60 seconds of its first sighting are merged. Zero-value orders are included.

Line-item-filtered goals count only orders from exposed visitors whose captured products match the filter. Orders without product data are excluded unless the goal is set to include them.

Revenue

• Revenue is converted into the requested currency at the daily rate for the order date.
• Orders without a currency are assumed to be in the shop currency, unless that setting is turned off.
• Orders that cannot be converted still count as purchases but are left out of money totals.
• The revenue read is withheld only when no attributed order can be valued.
• Revenue statistics cap single orders at the 99th percentile so one outlier cannot decide a test. Reported totals are not capped.

Conversion rate

Conversion rate is conversions divided by the variation's population. For purchase goals the population includes the server-matched order visitors described above. For count-every goals the rate is events per visitor and can exceed 100%.

In the API, arms[].primaryRate and primaryConversions always use unique converters, even when the primary goal counts every event. The goal's counting field states which basis the statistics use.

Quality checks

Sample-ratio mismatch (SRM). When enabled, Apex tests whether the observed split between variations differs implausibly from the configured allocation. A mismatch blocks winner and loser verdicts and auto-stop. Confidence values stay visible for diagnosis.

Population inconsistency. When a variation has more unique converters on the primary goal than visitors in its population, the numbers cannot be valid. The API then removes primary conversions, rates, lift, confidence, the time series, segments, the funnel, and revenue inference. The results page hides the headline rates, lift, confidence, and primary series, and marks the primary goal row with a warning.

See experiment results and metrics and goals.