Open Experiment Standard
Section 7 · OES Spec

Results

Machine-readable, granular, and per-metric — not just an overall summary.

Results are the raw record of what was measured: exposures, sample sizes, per-metric comparisons with intervals and effect sizes. Each metric result links back to the metric definition above so a consumer can interpret it without ambiguity.

Fields

FieldTypePurpose
results.sampleSizesobjectUnits exposed or analyzed by variant.
results.exposuresobjectExposure counts by variant.
results.metricResultsobject[]Array of metric-level results.
results.segmentResultsobject[]Optional segment-level results.
results.timeSeriesResultsobject[]Optional daily or hourly cumulative results.
results.variantComparisonsobject[]Treatment-vs-control comparisons.

Notes

  • Each `metricResults` entry MUST reference a `metricId` defined in the `metrics` array.
  • Frequentist results SHOULD include either a confidence interval or a standard error.
  • Bayesian results SHOULD include either a credible interval or a probability of improvement.

Example

{
  "metricId": "activation_rate",
  "role": "primary",
  "comparison": {
    "baselineVariantId": "control",
    "variantId": "new_onboarding"
  },
  "baselineValue": 0.421,
  "variantValue": 0.447,
  "absoluteDifference": 0.026,
  "relativeDifference": 0.0618,
  "confidenceInterval": {
    "level": 0.95,
    "lower": 0.008,
    "upper": 0.044
  },
  "pValue": 0.004,
  "resultStatus": "positive",
  "decisionImpact": "supports_ship"
}