One seeded, ground-truth retrieval simulation, run in the Python source-of-truth and the Go port. v2 adds calibrated per-passage relevance in the reward, grounded abstention, and a load-aware sizer that prunes the expensive stage. The two languages agree on every effect — the win is a property of the design, not one simulation.
| Python — source of truth | Go — port | |||||
|---|---|---|---|---|---|---|
| Metric | v1 | v2 | Δ | v1 | v2 | Δ |
| Learned-policy precision Share of served passages that are truly relevant, once the policy converges. v2's reward finally sees calibrated relevance instead of a coverage-biased judge. | 67.6% | 82.2% | ▲ +14.6 | 84.6% | 95.9% | ▲ +11.3 |
| Abstention recall Truly-unanswerable queries that v2 declines to answer. v1 has no abstention at all — it always answers. | 0.0% | 100% | ▲ +100 | 0.0% | 100% | ▲ +100 |
| False-abstain rate Answerable queries v2 wrongly declined — the cost of abstention. It stays at zero: the floor never fires on a good query. | 0.0% | 0.0% | — none | 0.0% | 0.0% | — none |
| Expensive-stage depth Passages sent to the costly rerank / synthesis stage from a deep k=8 arm. The survival-product sizer prunes the low-relevance tail before it is paid for. | 8.00 | 3.00 | ▼ −62% | 8.00 | 3.00 | ▼ −63% |
| Precision after the sizer Precision of what survives the gate. The passages the sizer removed were the wrong ones, so precision rises to 100%. | 37.5% | 100% | ▲ +62.5 | 37.5% | 100% | ▲ +62.5 |
Honest by construction The v2 additions are opt-in, so v2-with-flags-off is byte-for-byte v1 — the A/B is a flag toggle in one process, not two codebases.
40-seed average A single run is dominated by bandit exploration luck, so precision is averaged over 40 seeds; every number here is deterministic and reproducible.
β = 0.9 headline Precision is shown at the calibration-trust ceiling (shipped default 0.5); the lift grows monotonically with how much the reward trusts calibrated relevance.
Two runtimes, one methodology Absolute precision differs because each runtime uses its own strategy set — but both show a double-digit lift, full abstention, and a ~62% depth cut.
// ground-truth simulation · 600-query stream · precision measured on answerable, answered queries only · reproduce: github.com/redevops-io/context-runtime