Experiment Methodology
Choose how Ours Privacy reads your experiment results and when to wait before making a decision.
Ours Privacy helps you compare versions of a page or experience. For most experiments, you can leave the default setting selected and focus on the question you care about: is the new version helping people complete the goal?
Choose How to Read Results
Bayesian
Bayesian is the default. It updates as visitors arrive and is the best fit for most landing-page, CTA, and content experiments.
Frequentist
Frequentist is for teams that want to decide the audience size before launch, then wait until that plan is complete.
Sequential
Sequential is for conversion experiments that need to be checked while they are running.
Pick the method while the experiment is still a draft. After you start it, the choice is locked so that the result stays comparable over time. If you need a different plan, clone the experiment and start a new one.
When a Conversion Takes Time
By default, a visitor appears in Bayesian results as soon as they see a variant. That is useful for goals that happen right away, such as a button click or form submit.
If people often convert several days later—such as booking an appointment after comparing options—turn on Wait 7 days for outcomes before starting the experiment. Ours Privacy then gives every visitor seven days to complete the goal before including that visitor in the decision-making result.
While those seven-day windows are still open, Results shows Finalizing outcomes. This is expected: recent visitors are visible in the delivery counts, but Ours Privacy is still giving them time to convert. The setting is locked when the experiment starts.
What the Counts Mean
- Visitors are the people who saw a version of your experiment.
- Impressions are the number of times a version was shown. One person can create more than one impression.
- Conversions are people who completed the goal you chose after seeing a version.
This is why visitor and impression counts can be different. When you use a seven-day outcome window, Results also shows how many visitors are ready to compare out of everyone who has seen the experiment. Ours Privacy uses people—not repeat page views—to compare variants.
What You’ll See in Results
Results makes it clear whether there is enough information to make a decision:
- Collecting data means the experiment needs more visitors.
- Finalizing outcomes means visitors have arrived, but their chosen conversion window has not finished.
- Ready means Ours Privacy can evaluate the experiment using the plan you chose.
- No conclusion means there is not yet a clear enough result to call a winner.
Options for Analytics Teams
If your team has a reporting standard or an analyst helping with experiment design, it can choose a more specific method:
- Frequentist lets you set a visitor goal before launch. It supports conversion and value goals, and an optional CUPED setting that can make a test more efficient by using earlier visitor activity.
- Sequential is for conversion goals only. It is designed for teams that need to check the result during the experiment.
These options are deliberate choices, not automatic upgrades. Ours Privacy never changes your method after the experiment starts.
Related Reading
How is this guide?