The life of an experiment

From draft to conclusion: scheduling, pausing, ending with a winner, rolling the winner out, and undoing all of it.

An experiment moves through explicit states: draft → scheduled → running → paused → concluded → archived. Every transition is an action someone takes, recorded in the experiment's timeline with who and when — there is no state a test can drift into.

Draft

The wizard only ever produces a draft. Nothing is live, nothing is measured, and everything is editable — including the things that lock later, like template suffixes, split URL destinations, the split mode, and the primary metric. A Next steps checklist on the experiment page lists exactly what still has to be true before this experiment can start; publishing with open blockers is refused with the list, not with a generic error.

Before starting, the QA & Preview tab gives you a preview per variant — with a QR code for checking on a phone — on the real storefront, visible only in your own tab. For template and split URL tests the preview puts your own session in that variant: no assignment, nothing stored, nothing counted. It also runs pre-flight checks on the experiment itself, so a broken selector or a missing destination is named before any visitor could meet it.

Scheduled and running

Start now, or set a start date, an end date, or both; a scheduled experiment goes live on its own at the start time and stops measuring at the end time. Either date can be cleared while it still matters.

While running, each part of the experiment has its own edit — identity, traffic split, schedule, where it runs, metrics — and every edit shows a Review changes summary before saving, stating its consequence: whether it reaches the storefront on the next publish, or moves the attribution parameter of every variant. What would corrupt the read is locked, and the interface says so in place rather than letting you find out later.

Pause stops assigning new visitors without concluding anything — visitors see the original page, nothing measured is lost; Resume continues. Both are one click on the experiment page.

Ending

Ending an experiment happens in two explicit decisions. First, name the outcome: pick the winning variant, or end without one. Then choose what happens to the storefront:

End test: the experiment stops, the storefront returns to the original experience, and the report is preserved as the record.
Roll out & validate: the winning variant is served to everyone, and Proa keeps watching it for a validation period to confirm the rolled-out winner performs as the experiment said it would. If it seems to underperform, the experiment page raises an alert instead of staying quiet. The rollout is reversible at any point — Revert rollout returns the store to the pre-experiment state, even after the experiment is concluded.
Make permanent: the winner stops being an experiment variant and becomes a regular customization of the site, and, when the winner has its own content, that content is downloadable so it can also live in your theme.

Keeping these distinct is the point: stopping a test, shipping a winner, and adopting a winner for good are three different decisions, and the interface never merges them behind one button.

Auto-apply

Optionally, per experiment, Proa can close the experiment on its own: you set a minimum number of days (1 to 30), and only a decisively significant winner on the primary metric — with enough visitors and orders behind it — is ever applied. A merely-leading variant is not. When it triggers, the winner is rolled out with the same validation watch as a manual rollout, and you get an email naming what was applied and why. It is opt-in per experiment, never a default.

Running it again

Duplicate creates a new draft with the same setup, so an experiment you have already run becomes the starting point for the next question — nothing from the original's data carries over, and the copy warns you if running both would make them collide.

The escape hatches

Everything above assumes an orderly life. When something is wrong now: pausing stops new assignments in one click, rollback returns the whole site to a previous published version, and the kill switch stops everything — each reaching visitors in under a minute. Experiments are only cheap to run because undoing one is cheaper still.