A digest names the curve; it does not vouch for the procedure that makes the next one
Artifact identity was the previous tranche's answer. It is not certification. `knot_digest` says WHICH mapping ran and nothing about whether tomorrow's refit deserves the same trust — that one would get its own digest and be equally "identified" while fitted on anything at all. So the object being certified is named: VYNDR certifies a PROCEDURE, not a frozen curve. A frozen curve goes stale against a live model and has to be replaced by hand on no schedule; "fit past, apply forward" is procedural by construction. `fitPolicy.POLICY_V1` declares it — data selection, horizon, algorithm and version, minimum rows, model-version restriction, sport, stat, and a support contract that a refit may NOT widen. Each artifact still carries its own digest. `fitPolicy.validate` is the Step-22 gate: an artifact does not become servable because the algorithm ran. It refuses a widened support, a wrong era, a wrong estimator, a thin fit, a missing identity or a missing training cutoff, and `servable` is false whenever any violation stands, with no override argument. The statistical bars stay where they already live in calibrationRegistry — this is not a second governance system. MEASURED, AND THE REASON LIVE SERVING STAYS BLOCKED: production does not match the declaration. `loadSettledRows` applies no model_version filter, so at fit_as_of 2026-09-02 the fit drew 6,084 rows from 9,361 settled — all 3,292 from the superseded engine1@2026-07-20 plus 2,792 current-era. 54.1% of the served map is fitted on a forecaster it was never certified for, while the artifact declares the current era. That is a provenance contradiction, not a performance claim: era-filtered scores 0.24065 against pooled 0.24068 on 1,120 out-of-sample rows and both intervals span zero. It is blocked because nothing prevents the next era change from repeating it, and because the freshness lag grows. `era_restricted` is answered STRUCTURALLY, not by an extra read — the query is in this service and applies no filter, so the artifact records ERA_NOT_RESTRICTED rather than claiming a restriction that did not hold. An unverified restriction is recorded as a violation, because "we did not check" is exactly the state production is in. The violation does NOT distort the shadow. Flipping every row to UNCERTIFIED would make the shadow measure the violation instead of the contract, so the policy state rides beside the resolution and a test asserts the shadow still reads CERTIFIED_CALIBRATED at 0.65 and UNCERTIFIED at 0.91. Nothing serves. CALIBRATION_DEPLOYED still []. Shadow still OFF (the production variable remains absent — the probe reads configuration_source "default"). Suite 402/402, 5,607 passed, 4 skipped. Teeth 10/10 + 23/23. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CQJeAG8vcDoL5zkiaJyVb8
This commit is contained in:
@@ -72,15 +72,19 @@ logicTooth(1, 'runtime SHA verification waits for a snapshot despite working run
|
||||
// ── 2 — the estimator must carry a reconstructable identity ──────────────
|
||||
injectionTooth(2, 'runtime estimator lacks a reconstructable artifact identity',
|
||||
'src/services/model/probabilityContractService.js',
|
||||
` knot_digest: digest(fitted.map),`,
|
||||
` knot_digest: 'static-placeholder',`,
|
||||
` knot_count: Array.isArray(fitted.map) ? fitted.map.length : null,
|
||||
knot_digest: digest(fitted.map),`,
|
||||
` knot_count: null,
|
||||
knot_digest: 'static-placeholder',`,
|
||||
'tests/unit/probabilityContract.test.js');
|
||||
|
||||
// ── 3 — the artifact must be tied to the certified model era ─────────────
|
||||
injectionTooth(3, 'runtime artifact differs from the certified artifact',
|
||||
'src/services/model/probabilityContractService.js',
|
||||
` model_version: contract.model_version,`,
|
||||
` model_version: 'engine1@some-other-era',`,
|
||||
` model_version: contract.model_version,
|
||||
fit_as_of: fitted.cutoff || null,`,
|
||||
` model_version: 'engine1@some-other-era',
|
||||
fit_as_of: fitted.cutoff || null,`,
|
||||
'tests/unit/probabilityContractShadow.test.js');
|
||||
|
||||
// ── 4 — point-in-time evidence ───────────────────────────────────────────
|
||||
@@ -128,6 +132,30 @@ logicTooth(25, 'shadow flag parses loosely or defaults ON', () => {
|
||||
detail: `strict '1' compare=${strict}; sport-scoped=${scoped}; stat-scoped=${statScoped}` };
|
||||
});
|
||||
|
||||
// ── 9 — the fit policy must not drift silently ───────────────────────────
|
||||
injectionTooth(9, 'current fit policy silently changed before measurement',
|
||||
'src/services/model/fitPolicy.js',
|
||||
` model_version: 'engine1@2026-08-07-fullwindow',
|
||||
data_selection: Object.freeze({`,
|
||||
` model_version: 'engine1@2026-07-20',
|
||||
data_selection: Object.freeze({`,
|
||||
'tests/unit/fitPolicy.test.js');
|
||||
|
||||
// ── 10 — a refit may not become servable just because it ran ─────────────
|
||||
injectionTooth(10, 'future refit becomes servable without a validity gate',
|
||||
'src/services/model/fitPolicy.js',
|
||||
` /** A policy-invalid artifact is NEVER servable. There is no override. */
|
||||
servable: violations.length === 0,`,
|
||||
` servable: true,`,
|
||||
'tests/unit/fitPolicy.test.js');
|
||||
|
||||
// ── 10b — support may not be widened by a refit ──────────────────────────
|
||||
injectionTooth(26, 'a refit widens the region it is trusted in',
|
||||
'src/services/model/fitPolicy.js',
|
||||
` violations.push(VIOLATION.SUPPORT_WIDENED);`,
|
||||
` { /* widening allowed */ }`,
|
||||
'tests/unit/fitPolicy.test.js');
|
||||
|
||||
const unreachable = [
|
||||
{ id: 17, name: 'UI displays raw exact confidence for UNCERTIFIED', why: 'no live-serving UI path exists; nothing consumes served_probability yet' },
|
||||
{ id: 18, name: 'live EV/Kelly/VALUE consumers bypass served_probability', why: 'live consumers are unchanged by design in this tranche — tooth 16 asserts none exist' },
|
||||
|
||||
Reference in New Issue
Block a user