Detection becomes repair: the curve is fitted on one forecaster, frozen, and named
The last release detected the violation and then served the certified state anyway. A validator that changes nothing is decoration, so `servable:false` is now load-bearing: an artifact that fails its policy returns ARTIFACT_POLICY_BLOCKED with no number, and every probability-derived claim goes with it. The gate sits inside the resolution, not beside the flag that turns the shadow on, so no environment variable can reach past it — a test asserts `resolve` never reads process.env at all. Shadow and live consume the SAME decision, differing only in which promotion stage they demand. Era mismatch still resolves to VERSION_MISMATCH rather than the new state. "This artifact belongs to a different forecaster" is more precise than "policy blocked", and the existing state already says it exactly. THE REPAIR. `currentEraSource` filters on model_version in the QUERY, taking the era from config/modelVersion so the query, the artifact and the validator all read one identity. Measured on the actual fitted set, not a second count: 6,069 current-era rows, 0 wrong-era. The procedure was then certified on current-era rows ONLY — four walk-forward folds, training strictly before each evaluation block, 0 future rows in train on every fold. All four improve; pooled n=3,108 gives Brier 0.24701 -> 0.24323, delta -0.00378, CI [-0.00619,-0.00147] excluding zero; ECE falls in every fold. Mapping spread inside support is 0.001-0.018. The prior mixed-era certification did not substitute for this. Policy B selected. A (era-filtered 65/35) and B (all current-era) are statistically indistinguishable, A-B = +0.0001 CI [-0.00029,+0.00048], but B has the better ECE (0.0064 vs 0.0109) and the holdout existed to certify the PROCEDURE — it is not permanently withheld from the artifact that ships. withheld_from_fit is 0. FROZEN. `mlb-hits-isotonic@2026-09-03`: 6,069 rows, training_cutoff 2026-09-01 (distinct from fit_as_of 2026-09-03 — the newest observation admitted is not the eligibility bound), 12 knots, source_digest 25919c16…, knot_digest 5ae940ea…, served_curve_digest c24a9dc5…, 8 curve steps, 924 bytes, committed as JSON. The runtime no longer fits. It loads. A test greps the service for fitIsotonic, fromLedger and loadRows and requires all three absent, because the old behaviour meant a user's number could move with no version, no review and no rollback, and a past Read could not be reconstructed because its curve no longer existed. New settled outcomes are forward evidence now; they cannot touch this curve. Independent reconstruction from the declared training contract alone — fresh read, fresh digest, fresh fit — reproduces every digest and the curve byte for byte. Calling the builder twice would only have proven the builder deterministic. Promotion is a frozen source constant. A snapshot cannot promote, a settlement cannot promote, a successful fit cannot promote, and dropping a file into the artifacts directory promotes nothing. Stage is APPROVED_FOR_SHADOW; live is explicitly false. Two coverage holes found by their own teeth. The promotion guard could be deleted with every test still green, because the promoted file naturally agrees with itself — extracted as `acceptFile` and tested on the case `load()` cannot reach. And `validate(null)` returned no `servable` field at all, which is falsy at a call site and so would have read as correct while asserting nothing. Shadow OFF. Live OFF. CALIBRATION_DEPLOYED []. No frontend change. Suite 404/404, 5,634 passed, 4 skipped. Teeth 26/26 + 10/10 + 23/23. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CQJeAG8vcDoL5zkiaJyVb8
This commit is contained in:
@@ -0,0 +1,137 @@
|
||||
'use strict';
|
||||
|
||||
/**
|
||||
* artifactRegistry — WHICH FROZEN CURVE IS ACTIVE, AND WHO DECIDED.
|
||||
*
|
||||
* ── WHY THE RUNTIME NO LONGER FITS ───────────────────────────────────────
|
||||
* A certified refit PROCEDURE does not mean the active production curve should
|
||||
* mutate every snapshot. It did: `probabilityContractService` called the fitter
|
||||
* on every run, so the served mapping silently changed as outcomes settled —
|
||||
* unreconstructable after the fact, and capable of moving a user's number with
|
||||
* no version, no review and no rollback.
|
||||
*
|
||||
* The procedure is certified separately (walk-forward, current era only). What
|
||||
* ships is ONE frozen artifact, fitted once, committed as JSON, loaded here.
|
||||
*
|
||||
* ── WHY A COMMITTED FILE ─────────────────────────────────────────────────
|
||||
* The curve is ~900 bytes. Git already gives immutability, versioning, review
|
||||
* as a diff, and rollback — the same seam
|
||||
* `supabase/schema/model_snapshots.columns.json` uses. A mutable environment
|
||||
* variable could not hold the curve honestly, and a new service would be a
|
||||
* second governance stack for one small file.
|
||||
*
|
||||
* ── PROMOTION IS A HUMAN ACT ─────────────────────────────────────────────
|
||||
* `STAGE` below is a source constant. A snapshot cannot promote. A settlement
|
||||
* cannot promote. A successful fit cannot promote. Adding an artifact file does
|
||||
* nothing until this map names it.
|
||||
*/
|
||||
|
||||
const fs = require('fs');
|
||||
const path = require('path');
|
||||
const fitPolicy = require('./fitPolicy');
|
||||
|
||||
const DIR = path.join(__dirname, 'artifacts');
|
||||
|
||||
const STAGE = Object.freeze({
|
||||
NONE: 'NONE',
|
||||
APPROVED_FOR_SHADOW: 'APPROVED_FOR_SHADOW',
|
||||
APPROVED_FOR_LIVE: 'APPROVED_FOR_LIVE',
|
||||
});
|
||||
|
||||
/**
|
||||
* THE PROMOTION TABLE. Editing this is the promotion.
|
||||
*
|
||||
* mlb:hits is APPROVED_FOR_SHADOW — it may be evaluated and persisted as a
|
||||
* shadow candidate. It is deliberately NOT APPROVED_FOR_LIVE: no user-facing
|
||||
* number comes from it, and moving it there is a separate decision requiring
|
||||
* its own evidence.
|
||||
*/
|
||||
const PROMOTED = Object.freeze({
|
||||
'mlb:hits': Object.freeze({
|
||||
artifact_id: 'mlb-hits-isotonic@2026-09-03',
|
||||
stage: STAGE.APPROVED_FOR_SHADOW,
|
||||
}),
|
||||
});
|
||||
|
||||
const keyOf = (sport, stat) => `${String(sport || '').toLowerCase()}:${String(stat || '').toLowerCase()}`;
|
||||
const fileFor = (artifactId) => path.join(DIR, `${String(artifactId).replace(/[@:]/g, '_')}.json`);
|
||||
|
||||
/**
|
||||
* THE PROMOTION GUARD. A file is accepted only if it declares the id it was
|
||||
* promoted under.
|
||||
*
|
||||
* Exported because it is the one rule whose failure case cannot be reached
|
||||
* through `load()` on the happy path: the promoted file naturally agrees with
|
||||
* itself, so deleting this check changed nothing observable and a teeth
|
||||
* injection came back green. The rule it enforces is that dropping a file into
|
||||
* the directory, or editing its id, cannot promote anything.
|
||||
*/
|
||||
function acceptFile(raw, promoted) {
|
||||
if (!raw || !promoted) return false;
|
||||
return raw.artifact_id === promoted.artifact_id;
|
||||
}
|
||||
|
||||
const cache = new Map();
|
||||
|
||||
/**
|
||||
* Load the promoted artifact. Returns null when nothing is promoted, when the
|
||||
* file is missing, or when the artifact fails its own policy — never a partial
|
||||
* or a fallback, because a fallback here is a curve nobody certified.
|
||||
*/
|
||||
function load(sport, stat) {
|
||||
const key = keyOf(sport, stat);
|
||||
if (cache.has(key)) return cache.get(key);
|
||||
|
||||
const promoted = PROMOTED[key];
|
||||
if (!promoted) { cache.set(key, null); return null; }
|
||||
|
||||
let raw;
|
||||
try { raw = JSON.parse(fs.readFileSync(fileFor(promoted.artifact_id), 'utf8')); }
|
||||
catch { cache.set(key, null); return null; }
|
||||
|
||||
if (!acceptFile(raw, promoted)) { cache.set(key, null); return null; }
|
||||
|
||||
// RE-VALIDATED AT LOAD, not trusted from the file. The file records what the
|
||||
// builder concluded; this is the runtime asking the same question again, so a
|
||||
// hand-edited `servable: true` cannot smuggle an invalid artifact into use.
|
||||
const check = fitPolicy.validate(
|
||||
{ ...raw, estimator_type: 'isotonic' },
|
||||
{ era_counts: (raw.era_audit && raw.era_audit.era_counts) || null },
|
||||
);
|
||||
|
||||
const out = Object.freeze({
|
||||
...raw,
|
||||
estimator_type: 'isotonic',
|
||||
stage: promoted.stage,
|
||||
approved_for_shadow: promoted.stage === STAGE.APPROVED_FOR_SHADOW || promoted.stage === STAGE.APPROVED_FOR_LIVE,
|
||||
approved_for_live: promoted.stage === STAGE.APPROVED_FOR_LIVE,
|
||||
fit_policy_valid: check.valid,
|
||||
fit_policy_violations: Object.freeze(check.violations),
|
||||
servable: check.servable,
|
||||
});
|
||||
cache.set(key, out);
|
||||
return out;
|
||||
}
|
||||
|
||||
/**
|
||||
* Serve from the FROZEN CURVE, never from a refitted map.
|
||||
*
|
||||
* The curve is a step table at 0.001 granularity, which is exact for `p_win`
|
||||
* (quantised to 3dp at source). Outside the certified bands it returns null —
|
||||
* the curve does not extend past what was certified.
|
||||
*/
|
||||
function applyCurve(artifact, p) {
|
||||
if (!artifact || !Array.isArray(artifact.served_curve)) return null;
|
||||
const x = Number(p);
|
||||
if (!Number.isFinite(x)) return null;
|
||||
const inBand = (artifact.certified_bands || []).some(([lo, hi]) => x >= lo && x < hi);
|
||||
if (!inBand) return null;
|
||||
let v = null;
|
||||
for (const [from, val] of artifact.served_curve) { if (x >= from) v = val; else break; }
|
||||
return v;
|
||||
}
|
||||
|
||||
/** Test seam only — the promotion table itself is never writable at runtime. */
|
||||
function __resetCache() { cache.clear(); }
|
||||
|
||||
module.exports = { STAGE, PROMOTED, load, applyCurve, fileFor, acceptFile, __resetCache };
|
||||
@@ -0,0 +1,69 @@
|
||||
{
|
||||
"artifact_id": "mlb-hits-isotonic@2026-09-03",
|
||||
"procedure_version": "mlb-hits-isotonic-refit@v1",
|
||||
"sport": "mlb",
|
||||
"stat": "hits",
|
||||
"model_version": "engine1@2026-08-07-fullwindow",
|
||||
"fit_as_of": "2026-09-03",
|
||||
"training_cutoff": "2026-09-01",
|
||||
"fit_n": 6069,
|
||||
"withheld_from_fit": 0,
|
||||
"source_digest": "25919c160c59cc738b9b4cb9c6d62c48249d5daff5d2c37c7892cd47349a930a",
|
||||
"algorithm": "isotonic-pav",
|
||||
"algorithm_version": "calibration.fitIsotonic@2026-08",
|
||||
"knot_count": 12,
|
||||
"knot_digest": "5ae940ea163b7da2",
|
||||
"served_curve": [
|
||||
[
|
||||
0.5,
|
||||
0.518976
|
||||
],
|
||||
[
|
||||
0.541,
|
||||
0.550427
|
||||
],
|
||||
[
|
||||
0.562,
|
||||
0.570656
|
||||
],
|
||||
[
|
||||
0.613,
|
||||
0.574792
|
||||
],
|
||||
[
|
||||
0.648,
|
||||
0.581454
|
||||
],
|
||||
[
|
||||
0.67,
|
||||
0.607595
|
||||
],
|
||||
[
|
||||
0.684,
|
||||
0.625298
|
||||
],
|
||||
[
|
||||
0.798,
|
||||
0.63806
|
||||
]
|
||||
],
|
||||
"served_curve_digest": "c24a9dc5c2a96068",
|
||||
"certified_bands": [
|
||||
[
|
||||
0.5,
|
||||
0.8
|
||||
]
|
||||
],
|
||||
"era_audit": {
|
||||
"era_counts": {
|
||||
"engine1@2026-08-07-fullwindow": 6069
|
||||
},
|
||||
"current_era_rows": 6069,
|
||||
"wrong_era_rows": 0
|
||||
},
|
||||
"approved_for_shadow": true,
|
||||
"approved_for_live": false,
|
||||
"fit_policy_valid": true,
|
||||
"fit_policy_violations": [],
|
||||
"servable": true
|
||||
}
|
||||
@@ -0,0 +1,95 @@
|
||||
'use strict';
|
||||
|
||||
/**
|
||||
* currentEraSource — THE CANONICAL TRAINING SET FOR A CALIBRATION ARTIFACT.
|
||||
*
|
||||
* ── THE INVARIANT THIS EXISTS TO ENFORCE ─────────────────────────────────
|
||||
* An artifact declaring `model_version = X` must be fitted ONLY on observations
|
||||
* whose `model_version` is X. Formally:
|
||||
*
|
||||
* artifact.model_version = X => distinct(training_rows.model_version) = {X}
|
||||
*
|
||||
* The defect this replaces: `calibrationService.loadSettledRows` applies no
|
||||
* model filter, so at fit_as_of 2026-09-02 the chronological 65% drew all 3,292
|
||||
* rows from the superseded engine1@2026-07-20 plus 2,792 current-era rows —
|
||||
* 54.1% of a map that declared the current era. A calibrator corrects a specific
|
||||
* forecaster; fitted on a different one it is measuring something else.
|
||||
*
|
||||
* The legacy loader is deliberately untouched: other contracts still depend on
|
||||
* its historical behaviour, and changing it would silently move them.
|
||||
*
|
||||
* ── ONE MODEL IDENTITY ───────────────────────────────────────────────────
|
||||
* The era comes from `config/modelVersion` and is threaded through: the QUERY
|
||||
* filters on it, the ARTIFACT records it, and the VALIDATOR compares them. No
|
||||
* second hardcoded default — that is exactly how ledgerService and
|
||||
* retentionService drifted apart.
|
||||
*/
|
||||
|
||||
const crypto = require('crypto');
|
||||
const { paginate } = require('../../utils/safePaginate');
|
||||
|
||||
/** Fields that define a training observation. Anything else is database noise. */
|
||||
const CANONICAL_FIELDS = Object.freeze(['id', 'p_win', 'outcome', 'game_date', 'model_version']);
|
||||
|
||||
/**
|
||||
* Load the eligible current-era settled observations.
|
||||
*
|
||||
* `before` is STRICT — a Read may never train on its own outcome.
|
||||
* The quarantine exclusion mirrors the legacy path so the two remain comparable.
|
||||
*/
|
||||
async function loadRows(sb, { sport, stat, modelVersion, before } = {}) {
|
||||
if (!sb) throw new Error('currentEraSource.loadRows: no client');
|
||||
if (!sport || !stat || !modelVersion || !before) {
|
||||
throw new Error('currentEraSource.loadRows: sport, stat, modelVersion and before are all required');
|
||||
}
|
||||
const rows = await paginate(
|
||||
() => sb.from('ledger_entries')
|
||||
.select('id, p_win, outcome, game_date, quarantine_reason, model_version')
|
||||
.eq('sport', sport).is('user_id', null).eq('stat', stat)
|
||||
.eq('model_version', modelVersion) // THE RESTRICTION
|
||||
.in('outcome', ['hit', 'miss']).not('p_win', 'is', null)
|
||||
.lt('game_date', before), // strictly before
|
||||
{ key: 'id', pageSize: 1000, label: `currentEraSource(${sport}/${stat}/${modelVersion})` },
|
||||
);
|
||||
return rows
|
||||
.filter((r) => !(r.quarantine_reason || '').startsWith('nontakeable_book'))
|
||||
.map((r) => ({
|
||||
id: String(r.id),
|
||||
p: Number(r.p_win),
|
||||
won: r.outcome === 'hit' ? 1 : 0,
|
||||
date: String(r.game_date),
|
||||
model_version: r.model_version,
|
||||
}))
|
||||
.filter((r) => Number.isFinite(r.p));
|
||||
}
|
||||
|
||||
/**
|
||||
* Audit the era composition OF THE ACTUAL FITTED SET.
|
||||
*
|
||||
* Deliberately computed from the rows that were fitted, not from a separate
|
||||
* approximate count — a second query can agree with the wrong set.
|
||||
*/
|
||||
function eraAudit(rows, modelVersion) {
|
||||
const counts = {};
|
||||
for (const r of rows || []) counts[r.model_version || 'unknown'] = (counts[r.model_version || 'unknown'] || 0) + 1;
|
||||
const wrong = Object.entries(counts)
|
||||
.filter(([era, n]) => era !== modelVersion && Number(n) > 0)
|
||||
.reduce((s, [, n]) => s + Number(n), 0);
|
||||
return { era_counts: counts, current_era_rows: counts[modelVersion] || 0, wrong_era_rows: wrong };
|
||||
}
|
||||
|
||||
/**
|
||||
* DETERMINISTIC SOURCE DIGEST over the EXACT fitted observation set.
|
||||
*
|
||||
* Sorted by row id so database return order cannot change it, and built from
|
||||
* the canonical fields only — so it moves when a row is added, removed, or has
|
||||
* its outcome, probability or model identity changed, and not otherwise.
|
||||
*/
|
||||
function sourceDigest(rows) {
|
||||
const canon = (rows || [])
|
||||
.map((r) => [String(r.id), Number(r.p).toFixed(6), Number(r.won), String(r.date), String(r.model_version)])
|
||||
.sort((a, b) => (a[0] < b[0] ? -1 : a[0] > b[0] ? 1 : 0));
|
||||
return crypto.createHash('sha256').update(JSON.stringify(canon)).digest('hex');
|
||||
}
|
||||
|
||||
module.exports = { loadRows, eraAudit, sourceDigest, CANONICAL_FIELDS };
|
||||
@@ -47,6 +47,17 @@ const STATE = Object.freeze({
|
||||
CERTIFIED_EMPIRICAL_BAND: 'CERTIFIED_EMPIRICAL_BAND',
|
||||
UNCERTIFIED: 'UNCERTIFIED',
|
||||
UNSUPPORTED: 'UNSUPPORTED',
|
||||
/**
|
||||
* The contract is certified but the ARTIFACT that would answer failed its
|
||||
* governance policy, or is not promoted for this usage.
|
||||
*
|
||||
* A distinct state, deliberately. UNSUPPORTED means "we never certified this
|
||||
* sport/stat"; INVALID means "your input was malformed". Neither is true
|
||||
* here: the contract exists and the input is fine, and today's artifact is
|
||||
* the thing that is not usable. Folding it into either would destroy the
|
||||
* diagnosis at exactly the moment someone needs it.
|
||||
*/
|
||||
ARTIFACT_POLICY_BLOCKED: 'ARTIFACT_POLICY_BLOCKED',
|
||||
VERSION_MISMATCH: 'VERSION_MISMATCH',
|
||||
INVALID: 'INVALID',
|
||||
});
|
||||
@@ -156,6 +167,8 @@ function resolve(read = {}, deps = {}) {
|
||||
// names the exact mapping that ran. Absent when no artifact was supplied,
|
||||
// never invented.
|
||||
artifact: deps.artifact || null,
|
||||
artifact_id: (deps.artifact && deps.artifact.artifact_id) || null,
|
||||
procedure_version: (deps.artifact && deps.artifact.procedure_version) || null,
|
||||
reason: null,
|
||||
};
|
||||
|
||||
@@ -181,6 +194,28 @@ function resolve(read = {}, deps = {}) {
|
||||
if (raw === null || raw < 0 || raw > 1) {
|
||||
return { ...base, probability_state: STATE.INVALID, reason: 'raw probability absent or out of range' };
|
||||
}
|
||||
// ── GOVERNANCE IS LOAD-BEARING ─────────────────────────────────────────
|
||||
// `servable: false` has to MEAN something mechanically, or the validator is
|
||||
// decoration. An artifact that failed its policy may not produce a certified
|
||||
// state, and no environment variable can override this: the gate is here, in
|
||||
// the resolution, not beside the flag that turns the shadow on.
|
||||
//
|
||||
// ONE piece of logic for both usages (Step 4): future live serving consumes
|
||||
// the same decision, differing only in which promotion stage it demands.
|
||||
if (deps.artifact) {
|
||||
const a = deps.artifact;
|
||||
const usage = deps.usage === 'live' ? 'live' : 'shadow';
|
||||
const promoted = usage === 'live' ? a.approved_for_live === true : a.approved_for_shadow === true;
|
||||
if (a.servable !== true) {
|
||||
return { ...base, probability_state: STATE.ARTIFACT_POLICY_BLOCKED,
|
||||
reason: `estimator artifact failed its fit policy: ${(a.fit_policy_violations || []).join(', ') || 'unservable'}` };
|
||||
}
|
||||
if (!promoted) {
|
||||
return { ...base, probability_state: STATE.ARTIFACT_POLICY_BLOCKED,
|
||||
reason: `estimator artifact is not promoted for ${usage} use` };
|
||||
}
|
||||
}
|
||||
|
||||
if (!inCertifiedRawBand(contract.certified_bands, raw)) {
|
||||
// NO RAW FALLBACK. This is the whole point of the module.
|
||||
return { ...base, probability_state: STATE.UNCERTIFIED, reason: 'raw value lies outside certified estimator support' };
|
||||
|
||||
@@ -1,135 +1,50 @@
|
||||
'use strict';
|
||||
|
||||
/**
|
||||
* probabilityContractService — fit the certified estimator, point in time.
|
||||
* probabilityContractService — LOAD THE FROZEN ARTIFACT. DO NOT FIT.
|
||||
*
|
||||
* REUSES the existing fitter and its "fit on settled history strictly before
|
||||
* today" discipline. What it does NOT reuse is `calibrationService.calibrate()`,
|
||||
* whose gate is evaluated in CALIBRATED-OUTPUT space and whose else-branch
|
||||
* serves RAW. Both of those are the blocked contract; only the MAP is taken.
|
||||
* This used to call the fitter on every snapshot, so the active mapping changed
|
||||
* silently as outcomes settled: a user's number could move with no version, no
|
||||
* review and no rollback, and a past Read could not be reconstructed because the
|
||||
* curve that produced it no longer existed anywhere.
|
||||
*
|
||||
* SUPPORT COMES FROM THE CERTIFIED ARTIFACT, NOT FROM TONIGHT'S FIT. The bands
|
||||
* in `probabilityContract.MLB_HITS` were adjudicated on a three-way split and
|
||||
* are a fixed property of that adjudication. Letting a nightly refit widen its
|
||||
* own support is how an estimator certifies itself.
|
||||
* The refit PROCEDURE is certified separately and offline
|
||||
* (`scripts/certify-current-era-procedure.js`), and produces ONE committed
|
||||
* artifact (`scripts/build-current-era-artifact.js`). At runtime there is no
|
||||
* fitter, no ledger read, and nothing to drift: `artifactRegistry` loads the
|
||||
* promoted JSON and the served value comes from its frozen curve.
|
||||
*
|
||||
* A consequence worth stating: new settled outcomes are FORWARD EVALUATION
|
||||
* evidence. They may inform a future candidate. They cannot alter this curve.
|
||||
*/
|
||||
|
||||
const crypto = require('crypto');
|
||||
const cal = require('./calibration');
|
||||
const pc = require('./probabilityContract');
|
||||
const fitPolicy = require('./fitPolicy');
|
||||
|
||||
/** Short, stable content digest. Full sha256 truncated — collision risk here is
|
||||
* irrelevant and 16 hex chars keeps the per-row payload small. */
|
||||
const digest = (obj) => crypto.createHash('sha256')
|
||||
.update(JSON.stringify(obj)).digest('hex').slice(0, 16);
|
||||
const registry = require('./artifactRegistry');
|
||||
|
||||
/**
|
||||
* THE SERVED CURVE — the complete served function inside certified support.
|
||||
*
|
||||
* `p_win` is quantised to three decimals at the source
|
||||
* (`analyzeViaEngine1`: Math.round(pWin * 1000) / 1000), so a step table at
|
||||
* 0.001 granularity is not a sample of the mapping — it IS the mapping, for
|
||||
* every input that can actually occur. Six steps, ~200 bytes.
|
||||
*
|
||||
* Storing it on the row makes a Read reconstructable WITHOUT re-deriving the
|
||||
* training set. That matters because settled rows can be re-settled
|
||||
* (`re_settled_at`), so a later refit at the same cutoff is not guaranteed to
|
||||
* reproduce the same map — and a claim you can only verify when the inputs
|
||||
* happen not to have moved is not a reconstructable claim.
|
||||
* @returns {null|{contract, artifact, resolve}} null when nothing is promoted
|
||||
* for this sport/stat, or the promoted artifact fails to load. Null means
|
||||
* NOTHING is served — never a fallback curve, which would be a mapping nobody
|
||||
* certified.
|
||||
*/
|
||||
function servedCurve(map, bands) {
|
||||
const steps = [];
|
||||
let prev = null;
|
||||
for (const [lo, hi] of bands) {
|
||||
for (let x = lo; x < hi - 1e-9; x += 0.001) {
|
||||
const raw = Math.round(x * 1000) / 1000;
|
||||
const v = cal.applyIsotonic(map, raw);
|
||||
if (v === null) continue;
|
||||
const rounded = Math.round(v * 1e6) / 1e6;
|
||||
if (rounded !== prev) { steps.push([raw, rounded]); prev = rounded; }
|
||||
}
|
||||
prev = null; // bands are independent segments
|
||||
}
|
||||
return steps;
|
||||
}
|
||||
|
||||
/**
|
||||
* @returns {null|{estimate, fit_n, fitted_through, contract}} null when there
|
||||
* is not enough settled history — and null means NOTHING is served, never
|
||||
* "pass raw through".
|
||||
*/
|
||||
async function build(sb, { sport = 'mlb', stat = 'hits', before = null, ...opts } = {}) {
|
||||
async function build(_sb, { sport = 'mlb', stat = 'hits', usage = 'shadow' } = {}) {
|
||||
const contract = pc.contractFor(sport, stat);
|
||||
if (!contract || !sb) return null;
|
||||
if (!contract) return null;
|
||||
|
||||
const svc = opts.calibrationService || require('./calibrationService');
|
||||
const fitted = await svc.fromLedger(sb, { sport, stat, before, ...opts });
|
||||
if (!fitted || !fitted.map) return null;
|
||||
|
||||
// ── ARTIFACT IDENTITY ──────────────────────────────────────────────────
|
||||
// The runtime REFITS PER SNAPSHOT against `game_date < todayEt()`, so the
|
||||
// mapping changes as outcomes settle. That is point-in-time correct going
|
||||
// forward — a Read can only ever have seen settlements strictly before its
|
||||
// own day — but without an identity a served number could not be tied to the
|
||||
// function that produced it, and "isotonic" would be a label rather than a
|
||||
// claim. This is the identity.
|
||||
const curve = servedCurve(fitted.map, contract.certified_bands);
|
||||
// ── THE FIT-POLICY CHECK (Step 22) ─────────────────────────────────────
|
||||
// An artifact does not become servable because the algorithm ran. The era
|
||||
// composition is known STRUCTURALLY, not by an extra read: this service
|
||||
// applies no model_version filter today, so the restriction demonstrably did
|
||||
// not hold and the artifact records that rather than claiming it did.
|
||||
//
|
||||
// `era_restricted` is a fact about the QUERY, and the query is right here.
|
||||
const eraRestricted = false; // calibrationService.loadSettledRows applies no model filter
|
||||
const draft = {
|
||||
estimator_type: contract.estimator_type,
|
||||
model_version: contract.model_version,
|
||||
fit_n: fitted.fit_n ?? null,
|
||||
training_cutoff: fitted.fitted_through || null,
|
||||
knot_digest: digest(fitted.map),
|
||||
served_curve_digest: digest(curve),
|
||||
certified_bands: contract.certified_bands,
|
||||
};
|
||||
const policy = fitPolicy.validate(draft, eraRestricted
|
||||
? { era_counts: { [contract.model_version]: fitted.fit_n } } : {});
|
||||
|
||||
const artifact = Object.freeze({
|
||||
estimator_type: contract.estimator_type,
|
||||
estimator_version: contract.estimator_version,
|
||||
certification_version: contract.certification_version,
|
||||
model_version: contract.model_version,
|
||||
fit_as_of: fitted.cutoff || null, // the exact lt(game_date) bound
|
||||
training_cutoff: fitted.fitted_through || null, // last date INSIDE the fit
|
||||
fit_n: fitted.fit_n ?? null,
|
||||
knot_count: Array.isArray(fitted.map) ? fitted.map.length : null,
|
||||
knot_digest: digest(fitted.map),
|
||||
served_curve: curve, // complete over certified support
|
||||
served_curve_digest: digest(curve),
|
||||
// WHICH PROCEDURE, AND WHETHER IT HELD. Recorded on every artifact so a
|
||||
// later reader sees the policy state of the fit that produced the number,
|
||||
// not just the number's fingerprint.
|
||||
fit_policy_version: policy.policy_version,
|
||||
fit_policy_valid: policy.valid,
|
||||
fit_policy_violations: Object.freeze(policy.violations),
|
||||
// A policy-invalid artifact is never servable. Nothing serves today, so
|
||||
// this is a declaration; it becomes load-bearing the moment serving exists.
|
||||
servable: policy.servable,
|
||||
});
|
||||
const artifact = registry.load(sport, stat);
|
||||
if (!artifact) return null;
|
||||
|
||||
return {
|
||||
contract,
|
||||
artifact,
|
||||
fit_n: fitted.fit_n,
|
||||
fitted_through: fitted.fitted_through,
|
||||
cutoff: fitted.cutoff,
|
||||
/** The estimator, and only the estimator. No gate, no fallback. */
|
||||
estimate: (p) => cal.applyIsotonic(fitted.map, p),
|
||||
/** Resolve one grade through the full contract. */
|
||||
fit_n: artifact.fit_n,
|
||||
fitted_through: artifact.training_cutoff,
|
||||
cutoff: artifact.fit_as_of,
|
||||
/** The frozen curve. Two calls return the same value, forever. */
|
||||
estimate: (p) => registry.applyCurve(artifact, p),
|
||||
resolve(read) {
|
||||
return pc.resolve({ ...read, sport, stat },
|
||||
{ estimate: (p) => cal.applyIsotonic(fitted.map, p), artifact });
|
||||
{ estimate: (p) => registry.applyCurve(artifact, p), artifact, usage });
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
@@ -871,13 +871,26 @@ function mergeProbabilityContract(rows, contract) {
|
||||
certification_version: res.certification_version,
|
||||
model_version: res.model_version,
|
||||
reason: res.reason,
|
||||
// WHICH FITTED FUNCTION PRODUCED THIS. The estimator refits per
|
||||
// snapshot, so the certification version alone cannot identify the
|
||||
// mapping that ran. `served_curve` is the complete served function over
|
||||
// certified support at p_win's own 3dp granularity, so the row is
|
||||
// reconstructable without re-deriving a training set that may since
|
||||
// have been re-settled.
|
||||
artifact: res.artifact || null,
|
||||
// WHICH FROZEN ARTIFACT PRODUCED THIS — its IDENTITY, not its body.
|
||||
// The curve is committed in the repository and addressable by
|
||||
// `artifact_id`, so embedding it on every row would store the same ~900
|
||||
// bytes thousands of times per snapshot to say something the id already
|
||||
// says. `source_digest` pins the exact observation set it was fitted
|
||||
// on, so the row remains reconstructable.
|
||||
artifact: res.artifact ? {
|
||||
artifact_id: res.artifact.artifact_id,
|
||||
procedure_version: res.artifact.procedure_version,
|
||||
model_version: res.artifact.model_version,
|
||||
fit_as_of: res.artifact.fit_as_of,
|
||||
training_cutoff: res.artifact.training_cutoff,
|
||||
fit_n: res.artifact.fit_n,
|
||||
source_digest: res.artifact.source_digest,
|
||||
knot_digest: res.artifact.knot_digest,
|
||||
served_curve_digest: res.artifact.served_curve_digest,
|
||||
certified_bands: res.artifact.certified_bands,
|
||||
stage: res.artifact.stage,
|
||||
servable: res.artifact.servable,
|
||||
} : null,
|
||||
derived: derived ? {
|
||||
available: derived.available,
|
||||
ev_pct: derived.ev_pct,
|
||||
|
||||
@@ -860,12 +860,15 @@ async function runSnapshot(sport, opts = {}) {
|
||||
let probContract = null;
|
||||
if (require('./model/probabilityContract').shadowState().shadow === 'ON' && sp === 'mlb') {
|
||||
try {
|
||||
// No database client: the artifact is a committed file, not a fit. There
|
||||
// is nothing to read and nothing that can drift between snapshots.
|
||||
const pcs = deps.probabilityContractService || require('./model/probabilityContractService');
|
||||
const sbc = deps.supabase || require('../utils/supabase').getSupabaseServiceClient();
|
||||
probContract = sbc ? await pcs.build(sbc, { sport: 'mlb', stat: 'hits' }) : null;
|
||||
probContract = await pcs.build(null, { sport: 'mlb', stat: 'hits', usage: 'shadow' });
|
||||
console.log(probContract
|
||||
? `[probability-contract] shadow armed — isotonic, fit n=${probContract.fit_n} through ${probContract.fitted_through}, certified raw [0.50,0.80)`
|
||||
: '[probability-contract] shadow armed but NO estimator (thin history) — nothing would be served');
|
||||
? `[probability-contract] shadow armed — frozen artifact ${probContract.artifact.artifact_id}`
|
||||
+ ` (${probContract.artifact.stage}), fit n=${probContract.fit_n} through ${probContract.fitted_through},`
|
||||
+ ` knots ${probContract.artifact.knot_digest}, certified raw [0.50,0.80)`
|
||||
: '[probability-contract] shadow armed but NO PROMOTED ARTIFACT — nothing would be served');
|
||||
} catch (e) {
|
||||
probContract = null;
|
||||
console.warn('[probability-contract] shadow build failed (snapshot continues):', e.message);
|
||||
|
||||
Reference in New Issue
Block a user