The mapping that ran now has a name, and the row carries it
Runtime probes say the fleet is on a8de676, one process generation. But
"isotonic" was a label, not a claim: probabilityContractService refits per
snapshot against `game_date < todayEt()`, so the mapping changes as outcomes
settle, and nothing on a row could say WHICH mapping produced its number.
The artifact now has an identity:
estimator_type / estimator_version / certification_version / model_version
fit_as_of the exact lt(game_date) bound 2026-09-02
training_cutoff last date INSIDE the fit 2026-08-21
fit_n / knot_count 6,084 / 28
knot_digest d9d571d728ba76de
served_curve the COMPLETE served function over [0.50,0.80)
served_curve_digest
The served curve is not a sample. p_win is quantised to three decimals at the
source, so a step table at 0.001 granularity is the mapping itself for every
input that can occur — six steps, ~200 bytes. Storing it makes a Read
reconstructable WITHOUT re-deriving a training set that may since have been
re-settled, and a claim you can only verify when the inputs happen not to have
moved is not a reconstructable claim.
Proven, not asserted: the production construction path run twice gives an
identical digest, and an INDEPENDENT reconstruction — re-walk 9,361 settled
ledger rows at the declared bound, refit from scratch — reproduces
d9d571d728ba76de exactly, 28 knots for 28.
A teeth injection found a real defect behind a coverage hole. `resolve` checked
the CONTRACT's model era and never the ARTIFACT's, so a mapping fitted for a
different era could be recorded beside a served number with every test green.
Both the era and the estimator type are now checked, and a mismatch serves
nothing rather than serving quietly.
OBSERVED AND NOT CHANGED: calibrationService splits 65/35 to certify its own
bands, a step this contract does not consume because support comes from the
frozen artifact. So the served map is fitted through 2026-08-21 while 3,277
more recent settled rows sit unused, and that lag grows with history. Changing
it would change the fitted function, which this tranche froze.
Shadow still defaults OFF. CALIBRATION_DEPLOYED still []. served_probability is
referenced by nothing outside the contract layer — asserted by a tooth.
Suite 401/401, 5,593 passed, 4 skipped. Teeth 23/23 (prior) + 7/7 (new).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CQJeAG8vcDoL5zkiaJyVb8
This commit is contained in:
@@ -78,12 +78,30 @@ describe('the shadow reaches the row', () => {
|
||||
});
|
||||
|
||||
describe('the shadow changes NOTHING that is served', () => {
|
||||
const SERVED = ['p_win', 'confidence', 'grade', 'ev_pct', 'value', 'takeable', 'side', 'line'];
|
||||
it('leaves every served field byte-identical', () => {
|
||||
const before = [row(), row({ p_win: 0.91, grade: 'B+' }), row({ side: 'under', p_win: 0.55 })];
|
||||
it('leaves EVERY key byte-identical except probability_contract', () => {
|
||||
// Diffing a NAMED LIST of served fields only proves the fields someone
|
||||
// thought to list. Diff every key, so a field added later is covered by
|
||||
// this test on the day it appears.
|
||||
const before = [row(), row({ p_win: 0.91, grade: 'B+' }), row({ side: 'under', p_win: 0.55 }),
|
||||
row({ p_win: null, grade: null }), row({ model_version: 'engine1@2026-07-20' })];
|
||||
const snapshot = JSON.parse(JSON.stringify(before));
|
||||
const after = retention.mergeProbabilityContract(before, contract);
|
||||
after.forEach((r, i) => { for (const k of SERVED) expect(r[k]).toEqual(snapshot[i][k]); });
|
||||
after.forEach((r, i) => {
|
||||
const keys = new Set([...Object.keys(r), ...Object.keys(snapshot[i])]);
|
||||
keys.delete('probability_contract');
|
||||
expect(keys.size).toBeGreaterThan(8); // the diff must be non-vacuous
|
||||
for (const k of keys) expect(r[k]).toEqual(snapshot[i][k]);
|
||||
});
|
||||
});
|
||||
|
||||
it('adds exactly ONE key and no others', () => {
|
||||
const before = row();
|
||||
const after = retention.mergeProbabilityContract([before], contract)[0];
|
||||
const added = Object.keys(after).filter((k) => !(k in before));
|
||||
const removed = Object.keys(before).filter((k) => !(k in after));
|
||||
expect(added).toEqual([]); // declared null upfront
|
||||
expect(removed).toEqual([]);
|
||||
expect(after.probability_contract).not.toBeNull();
|
||||
});
|
||||
|
||||
it('does not mutate the input rows in place', () => {
|
||||
@@ -124,3 +142,44 @@ describe('the flag is OFF by default', () => {
|
||||
expect(src).toMatch(/CALIBRATION_DEPLOYED\s*=\s*Object\.freeze\(\[\s*\]\)/);
|
||||
});
|
||||
});
|
||||
|
||||
describe('the artifact must agree with the contract, not merely accompany it', () => {
|
||||
const cal = require('../../src/services/model/calibration');
|
||||
const svc = require('../../src/services/model/probabilityContractService');
|
||||
const map = cal.fitIsotonic(Array.from({ length: 900 }, (_, i) => {
|
||||
const p = Math.round((0.35 + (i % 60) / 100) * 1000) / 1000;
|
||||
return { p, won: ((i * 2654435761) % 1000) / 1000 < (0.5 + 0.45 * (p - 0.5)) ? 1 : 0, date: `d${i % 14}` };
|
||||
}), { minTotal: 200 });
|
||||
const build = () => svc.build({}, { calibrationService: { fromLedger: async () => ({
|
||||
map, fit_n: 900, fitted_through: '2026-08-21', cutoff: '2026-09-02' }) } });
|
||||
|
||||
it('the built artifact declares the CERTIFIED era and estimator', async () => {
|
||||
const b = await build();
|
||||
expect(b.artifact.model_version).toBe(pc.MLB_HITS.model_version);
|
||||
expect(b.artifact.estimator_type).toBe(pc.MLB_HITS.estimator_type);
|
||||
});
|
||||
|
||||
it('an artifact fitted for another era serves NOTHING, even on a matching read', () => {
|
||||
const r = pc.resolve({ sport: 'mlb', stat: 'hits', model_version: ERA, p_win: 0.65 },
|
||||
{ estimate: () => 0.6, artifact: { model_version: 'engine1@2026-07-20', estimator_type: 'isotonic' } });
|
||||
expect(r.probability_state).toBe(pc.STATE.VERSION_MISMATCH);
|
||||
expect(r.served_probability).toBeNull();
|
||||
});
|
||||
|
||||
it('an artifact of the wrong estimator type serves NOTHING', () => {
|
||||
const r = pc.resolve({ sport: 'mlb', stat: 'hits', model_version: ERA, p_win: 0.65 },
|
||||
{ estimate: () => 0.6, artifact: { model_version: ERA, estimator_type: 'low_param' } });
|
||||
expect(r.probability_state).toBe(pc.STATE.VERSION_MISMATCH);
|
||||
expect(r.served_probability).toBeNull();
|
||||
});
|
||||
|
||||
it('and the row records the mismatch rather than dropping it', () => {
|
||||
const bad = { fit_n: 1, fitted_through: 'x',
|
||||
resolve: (read) => pc.resolve({ ...read, sport: 'mlb', stat: 'hits' },
|
||||
{ estimate: () => 0.6, artifact: { model_version: 'engine1@2026-07-20', estimator_type: 'isotonic' } }) };
|
||||
const [r] = retention.mergeProbabilityContract([row()], bad);
|
||||
expect(r.probability_contract.probability_state).toBe(pc.STATE.VERSION_MISMATCH);
|
||||
expect(r.probability_contract.served_probability).toBeNull();
|
||||
expect(r.probability_contract.raw_model_probability).toBe(0.65);
|
||||
});
|
||||
});
|
||||
|
||||
Reference in New Issue
Block a user