Section 10 · OES Spec

Quality checks

Trust checks as a first-class part of the standard.

Quality checks tell a future reader whether the result should be trusted at all. Each check is its own object — typed, severity-graded, and carrying enough observed/expected data that a reader can re-derive the verdict by hand.

Fields

Field	Type	Purpose
qualityChecks[].checkTyperequired	string	e.g. sample_ratio_mismatch, exposure_health, invariant_metric.
qualityChecks[].status	enum	pass, warn, fail, not_run.
qualityChecks[].severity	enum	low, medium, high, critical.
qualityChecks[].observed	object	Observed values.
qualityChecks[].expected	object	Expected values.
qualityChecks[].pValue	number (0–1)	Where applicable.
qualityChecks[].message	string	Plain-English explanation.

Notes

Common check types: sample_ratio_mismatch, exposure_health, assignment_health, event_volume, invariant_metric, runtime_errors, bot_filtering, carryover_risk, novelty_risk, interaction_risk, data_freshness.

Example

{
  "checkType": "sample_ratio_mismatch",
  "status": "pass",
  "severity": "high",
  "observed": {
    "control": 50123,
    "treatment": 49987
  },
  "expected": {
    "control": 0.5,
    "treatment": 0.5
  },
  "pValue": 0.61,
  "message": "No evidence of sample ratio mismatch."
}