Detection becomes repair: the curve is fitted on one forecaster, frozen, and named

The last release detected the violation and then served the certified state
anyway. A validator that changes nothing is decoration, so `servable:false` is
now load-bearing: an artifact that fails its policy returns
ARTIFACT_POLICY_BLOCKED with no number, and every probability-derived claim goes
with it. The gate sits inside the resolution, not beside the flag that turns the
shadow on, so no environment variable can reach past it — a test asserts
`resolve` never reads process.env at all. Shadow and live consume the SAME
decision, differing only in which promotion stage they demand.

Era mismatch still resolves to VERSION_MISMATCH rather than the new state. "This
artifact belongs to a different forecaster" is more precise than "policy
blocked", and the existing state already says it exactly.

THE REPAIR. `currentEraSource` filters on model_version in the QUERY, taking the
era from config/modelVersion so the query, the artifact and the validator all
read one identity. Measured on the actual fitted set, not a second count:
6,069 current-era rows, 0 wrong-era.

The procedure was then certified on current-era rows ONLY — four walk-forward
folds, training strictly before each evaluation block, 0 future rows in train on
every fold. All four improve; pooled n=3,108 gives Brier 0.24701 -> 0.24323,
delta -0.00378, CI [-0.00619,-0.00147] excluding zero; ECE falls in every fold.
Mapping spread inside support is 0.001-0.018. The prior mixed-era certification
did not substitute for this.

Policy B selected. A (era-filtered 65/35) and B (all current-era) are
statistically indistinguishable, A-B = +0.0001 CI [-0.00029,+0.00048], but B has
the better ECE (0.0064 vs 0.0109) and the holdout existed to certify the
PROCEDURE — it is not permanently withheld from the artifact that ships.
withheld_from_fit is 0.

FROZEN. `mlb-hits-isotonic@2026-09-03`: 6,069 rows, training_cutoff 2026-09-01
(distinct from fit_as_of 2026-09-03 — the newest observation admitted is not the
eligibility bound), 12 knots, source_digest 25919c16…, knot_digest 5ae940ea…,
served_curve_digest c24a9dc5…, 8 curve steps, 924 bytes, committed as JSON.

The runtime no longer fits. It loads. A test greps the service for fitIsotonic,
fromLedger and loadRows and requires all three absent, because the old behaviour
meant a user's number could move with no version, no review and no rollback, and
a past Read could not be reconstructed because its curve no longer existed.
New settled outcomes are forward evidence now; they cannot touch this curve.

Independent reconstruction from the declared training contract alone — fresh
read, fresh digest, fresh fit — reproduces every digest and the curve byte for
byte. Calling the builder twice would only have proven the builder deterministic.

Promotion is a frozen source constant. A snapshot cannot promote, a settlement
cannot promote, a successful fit cannot promote, and dropping a file into the
artifacts directory promotes nothing. Stage is APPROVED_FOR_SHADOW; live is
explicitly false.

Two coverage holes found by their own teeth. The promotion guard could be
deleted with every test still green, because the promoted file naturally agrees
with itself — extracted as `acceptFile` and tested on the case `load()` cannot
reach. And `validate(null)` returned no `servable` field at all, which is falsy
at a call site and so would have read as correct while asserting nothing.

Shadow OFF. Live OFF. CALIBRATION_DEPLOYED []. No frontend change.
Suite 404/404, 5,634 passed, 4 skipped. Teeth 26/26 + 10/10 + 23/23.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CQJeAG8vcDoL5zkiaJyVb8
This commit is contained in:
Kev
2026-09-03 01:06:18 -04:00
parent 5cad851922
commit be8e16aca9
16 changed files with 1299 additions and 242 deletions
+11 -13
View File
@@ -71,21 +71,19 @@ logicTooth(1, 'runtime SHA verification waits for a snapshot despite working run
// ── 2 — the estimator must carry a reconstructable identity ──────────────
injectionTooth(2, 'runtime estimator lacks a reconstructable artifact identity',
'src/services/model/probabilityContractService.js',
` knot_count: Array.isArray(fitted.map) ? fitted.map.length : null,
knot_digest: digest(fitted.map),`,
` knot_count: null,
knot_digest: 'static-placeholder',`,
'tests/unit/probabilityContract.test.js');
'src/services/model/fitPolicy.js',
` if (!artifact.knot_digest || !artifact.served_curve_digest) violations.push(VIOLATION.NO_ARTIFACT_IDENTITY);`,
` if (false) violations.push(VIOLATION.NO_ARTIFACT_IDENTITY);`,
'tests/unit/artifactGovernance.test.js');
// ── 3 — the artifact must be tied to the certified model era ─────────────
injectionTooth(3, 'runtime artifact differs from the certified artifact',
'src/services/model/probabilityContractService.js',
` model_version: contract.model_version,
fit_as_of: fitted.cutoff || null,`,
` model_version: 'engine1@some-other-era',
fit_as_of: fitted.cutoff || null,`,
'tests/unit/probabilityContractShadow.test.js');
'src/services/model/artifactRegistry.js',
` estimator_type: 'isotonic',
stage: promoted.stage,`,
` estimator_type: 'low_param',
stage: promoted.stage,`,
'tests/unit/artifactGovernance.test.js');
// ── 4 — point-in-time evidence ───────────────────────────────────────────
injectionTooth(4, 'dynamic refit uses future settlement evidence for an earlier Read',
@@ -127,7 +125,7 @@ logicTooth(25, 'shadow flag parses loosely or defaults ON', () => {
const strict = pcSrc.includes("String(raw || '') === '1'")
&& snap.includes("probabilityContract').shadowState().shadow === 'ON'");
const scoped = snap.includes("&& sp === 'mlb'");
const statScoped = snap.includes("{ sport: 'mlb', stat: 'hits' }");
const statScoped = snap.includes("sport: 'mlb', stat: 'hits'");
return { caught: strict && scoped && statScoped,
detail: `strict '1' compare=${strict}; sport-scoped=${scoped}; stat-scoped=${statScoped}` };
});