Rheo documentation
Core concepts

Experiment

Channel-scoped A/B test. Flow, banner, or code content kinds serve published experiences or host parameters.

Purpose

An experiment splits traffic on a single channel across two or more variants. Its content kind is chosen when the experiment is created and does not change. A flow experiment serves a published flow on each variant. A banner experiment serves a published banner on each variant (holdout control may resolve with control: true and render nothing). A code experiment serves a variant key and a parameter object. Rheo assigns each end user to one arm and keeps that assignment stable across sessions.

Each experiment has a primary metric (flow completion, paywall purchase, or a scoped custom SDK event). Rates, lift, significance, and the winning-variant call all use that metric. See Experiments.

Experiments require Grow+ and count toward your workspace limit on concurrent experiments (draft, running, and pending-decision slots combined).

Status lifecycle

StatusMeaning
DraftConfiguring variants, weights, and primary metric; not serving traffic
RunningReady to serve. The channel assigns users only after you attach this experiment
Pending decisionEnd time reached; traffic continues with frozen weights until you promote a winner or extend
StoppedExperiment ended. If the channel was serving it, the channel is pinned to a variant or cleared. If the channel was serving something else, that assignment stays.

Serving an experiment is a channel assignment. Starting an experiment does not attach the channel. Assign the running experiment on that channel when it should serve traffic.

Experiment detail page showing variant arms, traffic weights, status badge, and completion stats comparison

Statistical design

While configuring a draft, you can set baseline rate, minimum detectable effect (MDE), significance level, and statistical power so the sample size planner estimates how many people you need per arm. After the run, the dashboard compares arms to the control variant on the primary metric, with confidence intervals, lift, Bonferroni-adjusted significance, and diagnostics for uneven traffic split.

Operator workflow

  1. Create an experiment on a channel. Choose a content kind, then add variants, weights, and a success metric. The first variant is the control until you pick another.
  2. For a flow experiment with a custom event that needs a layer or step, pin variants, then pick the target on the draft Primary metric card. A code experiment uses one success event name instead.
  3. Start the run. Assign the running experiment on the channel when it should serve traffic.
  4. Monitor stats on the experiment detail page. Flow experiments also appear in per-flow analytics.
  5. At the end, pin a variant or clear the channel if it is serving this experiment, or extend the run.