The mapping that ran now has a name, and the row carries it
Runtime probes say the fleet is on a8de676, one process generation. But
"isotonic" was a label, not a claim: probabilityContractService refits per
snapshot against `game_date < todayEt()`, so the mapping changes as outcomes
settle, and nothing on a row could say WHICH mapping produced its number.
The artifact now has an identity:
estimator_type / estimator_version / certification_version / model_version
fit_as_of the exact lt(game_date) bound 2026-09-02
training_cutoff last date INSIDE the fit 2026-08-21
fit_n / knot_count 6,084 / 28
knot_digest d9d571d728ba76de
served_curve the COMPLETE served function over [0.50,0.80)
served_curve_digest
The served curve is not a sample. p_win is quantised to three decimals at the
source, so a step table at 0.001 granularity is the mapping itself for every
input that can occur — six steps, ~200 bytes. Storing it makes a Read
reconstructable WITHOUT re-deriving a training set that may since have been
re-settled, and a claim you can only verify when the inputs happen not to have
moved is not a reconstructable claim.
Proven, not asserted: the production construction path run twice gives an
identical digest, and an INDEPENDENT reconstruction — re-walk 9,361 settled
ledger rows at the declared bound, refit from scratch — reproduces
d9d571d728ba76de exactly, 28 knots for 28.
A teeth injection found a real defect behind a coverage hole. `resolve` checked
the CONTRACT's model era and never the ARTIFACT's, so a mapping fitted for a
different era could be recorded beside a served number with every test green.
Both the era and the estimator type are now checked, and a mismatch serves
nothing rather than serving quietly.
OBSERVED AND NOT CHANGED: calibrationService splits 65/35 to certify its own
bands, a step this contract does not consume because support comes from the
frozen artifact. So the served map is fitted through 2026-08-21 while 3,277
more recent settled rows sit unused, and that lag grows with history. Changing
it would change the fitted function, which this tranche froze.
Shadow still defaults OFF. CALIBRATION_DEPLOYED still []. served_probability is
referenced by nothing outside the contract layer — asserted by a tooth.
Suite 401/401, 5,593 passed, 4 skipped. Teeth 23/23 (prior) + 7/7 (new).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CQJeAG8vcDoL5zkiaJyVb8
This commit is contained in:
@@ -219,3 +219,111 @@ describe('probabilityContractService', () => {
|
||||
expect(built.resolve({ model_version: ERA, p_win: 0.95 }).probability_state).toBe(pc.STATE.UNCERTIFIED);
|
||||
});
|
||||
});
|
||||
|
||||
describe('artifact identity — the mapping that actually ran', () => {
|
||||
const cal = require('../../src/services/model/calibration');
|
||||
const mk = (seed) => cal.fitIsotonic(Array.from({ length: 900 }, (_, i) => {
|
||||
const p = Math.round((0.35 + (i % 60) / 100) * 1000) / 1000;
|
||||
return { p, won: ((i * seed) % 1000) / 1000 < (0.5 + 0.45 * (p - 0.5)) ? 1 : 0, date: `d${i % 14}` };
|
||||
}), { minTotal: 200 });
|
||||
const fitted = (map, over = {}) => ({ calibrationService: { fromLedger: async () => ({
|
||||
map, fit_n: 900, fitted_through: 'd13', cutoff: '2026-09-03', ...over }) } });
|
||||
|
||||
it('the same evidence reconstructs the same artifact, digest for digest', async () => {
|
||||
const map = mk(2654435761);
|
||||
const a = await svc.build({}, fitted(map));
|
||||
const b = await svc.build({}, fitted(map));
|
||||
expect(a.artifact.knot_digest).toBe(b.artifact.knot_digest);
|
||||
expect(a.artifact.served_curve_digest).toBe(b.artifact.served_curve_digest);
|
||||
expect(a.artifact.served_curve).toEqual(b.artifact.served_curve);
|
||||
});
|
||||
|
||||
it('DIFFERENT evidence produces a different identity — the digest is not decorative', async () => {
|
||||
const a = await svc.build({}, fitted(mk(2654435761)));
|
||||
const b = await svc.build({}, fitted(mk(40503)));
|
||||
expect(a.artifact.knot_digest).not.toBe(b.artifact.knot_digest);
|
||||
});
|
||||
|
||||
it('carries the point-in-time bound and the training cutoff, distinctly', async () => {
|
||||
const a = await svc.build({}, fitted(mk(2654435761)));
|
||||
expect(a.artifact.fit_as_of).toBe('2026-09-03'); // the lt(game_date) bound
|
||||
expect(a.artifact.training_cutoff).toBe('d13'); // last date inside the fit
|
||||
expect(a.artifact.fit_n).toBe(900);
|
||||
expect(a.artifact.knot_count).toBeGreaterThan(0);
|
||||
});
|
||||
|
||||
it('the served curve IS the served function over certified support, not a sample', async () => {
|
||||
const a = await svc.build({}, fitted(mk(2654435761)));
|
||||
const lookup = (raw) => {
|
||||
let v = null;
|
||||
for (const [from, val] of a.artifact.served_curve) if (raw >= from) v = val;
|
||||
return v;
|
||||
};
|
||||
for (let x = 0.50; x < 0.80 - 1e-9; x += 0.001) {
|
||||
const raw = Math.round(x * 1000) / 1000;
|
||||
const r = a.resolve({ model_version: ERA, p_win: raw });
|
||||
expect(r.served_probability).toBe(Math.round(lookup(raw) * 1000) / 1000);
|
||||
}
|
||||
});
|
||||
|
||||
it('the curve covers ONLY certified support — never the unsupported tail', async () => {
|
||||
const a = await svc.build({}, fitted(mk(2654435761)));
|
||||
for (const [from] of a.artifact.served_curve) {
|
||||
expect(from).toBeGreaterThanOrEqual(0.50);
|
||||
expect(from).toBeLessThan(0.80);
|
||||
}
|
||||
});
|
||||
|
||||
it('a resolution with no artifact records null rather than inventing one', () => {
|
||||
const r = pc.resolve(read(0.65), { estimate: iso });
|
||||
expect(r.artifact).toBeNull();
|
||||
expect(r.served_probability).not.toBeNull();
|
||||
});
|
||||
});
|
||||
|
||||
describe('point in time — a Read can only see settlements before its own day', () => {
|
||||
const calSvc = require('../../src/services/model/calibrationService');
|
||||
|
||||
/** Records the filters actually applied, so this tests behaviour not source. */
|
||||
function recordingClient(rows) {
|
||||
const applied = [];
|
||||
const q = {
|
||||
select: () => q, eq: (c, v) => { applied.push(['eq', c, v]); return q; },
|
||||
is: (c, v) => { applied.push(['is', c, v]); return q; },
|
||||
in: (c, v) => { applied.push(['in', c, v]); return q; },
|
||||
not: (c, o, v) => { applied.push(['not', c, o, v]); return q; },
|
||||
lt: (c, v) => { applied.push(['lt', c, v]); return q; },
|
||||
lte: (c, v) => { applied.push(['lte', c, v]); return q; },
|
||||
gte: (c, v) => { applied.push(['gte', c, v]); return q; },
|
||||
order: () => q, limit: () => q,
|
||||
range: async () => ({ data: rows, error: null, count: rows.length }),
|
||||
then: (res) => res({ data: rows, error: null }),
|
||||
};
|
||||
return { applied, client: { from: () => q } };
|
||||
}
|
||||
|
||||
it('bounds the training walk STRICTLY BEFORE the cutoff, never at or after it', async () => {
|
||||
const { applied, client } = recordingClient([]);
|
||||
await calSvc.loadSettledRows(client, { sport: 'mlb', stat: 'hits', before: '2026-09-02' });
|
||||
const bound = applied.filter((a) => a[1] === 'game_date');
|
||||
expect(bound.length).toBeGreaterThan(0);
|
||||
// strictly-less is the whole discipline: `lte` would admit same-day games
|
||||
expect(bound.some((a) => a[0] === 'lt' && a[2] === '2026-09-02')).toBe(true);
|
||||
expect(bound.some((a) => a[0] === 'lte')).toBe(false);
|
||||
expect(bound.some((a) => a[0] === 'gte')).toBe(false);
|
||||
});
|
||||
|
||||
it('the artifact records the bound it was fitted under, so a Read names its own evidence horizon', async () => {
|
||||
const cal = require('../../src/services/model/calibration');
|
||||
const map = cal.fitIsotonic(Array.from({ length: 900 }, (_, i) => {
|
||||
const p = Math.round((0.35 + (i % 60) / 100) * 1000) / 1000;
|
||||
return { p, won: ((i * 2654435761) % 1000) / 1000 < (0.5 + 0.45 * (p - 0.5)) ? 1 : 0, date: `d${i % 14}` };
|
||||
}), { minTotal: 200 });
|
||||
const built = await svc.build({}, { calibrationService: { fromLedger: async () => ({
|
||||
map, fit_n: 900, fitted_through: '2026-08-21', cutoff: '2026-09-02' }) } });
|
||||
expect(built.artifact.fit_as_of).toBe('2026-09-02');
|
||||
expect(built.artifact.training_cutoff).toBe('2026-08-21');
|
||||
// and the two are DIFFERENT questions — the bound, and the last date inside it
|
||||
expect(built.artifact.fit_as_of).not.toBe(built.artifact.training_cutoff);
|
||||
});
|
||||
});
|
||||
|
||||
@@ -78,12 +78,30 @@ describe('the shadow reaches the row', () => {
|
||||
});
|
||||
|
||||
describe('the shadow changes NOTHING that is served', () => {
|
||||
const SERVED = ['p_win', 'confidence', 'grade', 'ev_pct', 'value', 'takeable', 'side', 'line'];
|
||||
it('leaves every served field byte-identical', () => {
|
||||
const before = [row(), row({ p_win: 0.91, grade: 'B+' }), row({ side: 'under', p_win: 0.55 })];
|
||||
it('leaves EVERY key byte-identical except probability_contract', () => {
|
||||
// Diffing a NAMED LIST of served fields only proves the fields someone
|
||||
// thought to list. Diff every key, so a field added later is covered by
|
||||
// this test on the day it appears.
|
||||
const before = [row(), row({ p_win: 0.91, grade: 'B+' }), row({ side: 'under', p_win: 0.55 }),
|
||||
row({ p_win: null, grade: null }), row({ model_version: 'engine1@2026-07-20' })];
|
||||
const snapshot = JSON.parse(JSON.stringify(before));
|
||||
const after = retention.mergeProbabilityContract(before, contract);
|
||||
after.forEach((r, i) => { for (const k of SERVED) expect(r[k]).toEqual(snapshot[i][k]); });
|
||||
after.forEach((r, i) => {
|
||||
const keys = new Set([...Object.keys(r), ...Object.keys(snapshot[i])]);
|
||||
keys.delete('probability_contract');
|
||||
expect(keys.size).toBeGreaterThan(8); // the diff must be non-vacuous
|
||||
for (const k of keys) expect(r[k]).toEqual(snapshot[i][k]);
|
||||
});
|
||||
});
|
||||
|
||||
it('adds exactly ONE key and no others', () => {
|
||||
const before = row();
|
||||
const after = retention.mergeProbabilityContract([before], contract)[0];
|
||||
const added = Object.keys(after).filter((k) => !(k in before));
|
||||
const removed = Object.keys(before).filter((k) => !(k in after));
|
||||
expect(added).toEqual([]); // declared null upfront
|
||||
expect(removed).toEqual([]);
|
||||
expect(after.probability_contract).not.toBeNull();
|
||||
});
|
||||
|
||||
it('does not mutate the input rows in place', () => {
|
||||
@@ -124,3 +142,44 @@ describe('the flag is OFF by default', () => {
|
||||
expect(src).toMatch(/CALIBRATION_DEPLOYED\s*=\s*Object\.freeze\(\[\s*\]\)/);
|
||||
});
|
||||
});
|
||||
|
||||
describe('the artifact must agree with the contract, not merely accompany it', () => {
|
||||
const cal = require('../../src/services/model/calibration');
|
||||
const svc = require('../../src/services/model/probabilityContractService');
|
||||
const map = cal.fitIsotonic(Array.from({ length: 900 }, (_, i) => {
|
||||
const p = Math.round((0.35 + (i % 60) / 100) * 1000) / 1000;
|
||||
return { p, won: ((i * 2654435761) % 1000) / 1000 < (0.5 + 0.45 * (p - 0.5)) ? 1 : 0, date: `d${i % 14}` };
|
||||
}), { minTotal: 200 });
|
||||
const build = () => svc.build({}, { calibrationService: { fromLedger: async () => ({
|
||||
map, fit_n: 900, fitted_through: '2026-08-21', cutoff: '2026-09-02' }) } });
|
||||
|
||||
it('the built artifact declares the CERTIFIED era and estimator', async () => {
|
||||
const b = await build();
|
||||
expect(b.artifact.model_version).toBe(pc.MLB_HITS.model_version);
|
||||
expect(b.artifact.estimator_type).toBe(pc.MLB_HITS.estimator_type);
|
||||
});
|
||||
|
||||
it('an artifact fitted for another era serves NOTHING, even on a matching read', () => {
|
||||
const r = pc.resolve({ sport: 'mlb', stat: 'hits', model_version: ERA, p_win: 0.65 },
|
||||
{ estimate: () => 0.6, artifact: { model_version: 'engine1@2026-07-20', estimator_type: 'isotonic' } });
|
||||
expect(r.probability_state).toBe(pc.STATE.VERSION_MISMATCH);
|
||||
expect(r.served_probability).toBeNull();
|
||||
});
|
||||
|
||||
it('an artifact of the wrong estimator type serves NOTHING', () => {
|
||||
const r = pc.resolve({ sport: 'mlb', stat: 'hits', model_version: ERA, p_win: 0.65 },
|
||||
{ estimate: () => 0.6, artifact: { model_version: ERA, estimator_type: 'low_param' } });
|
||||
expect(r.probability_state).toBe(pc.STATE.VERSION_MISMATCH);
|
||||
expect(r.served_probability).toBeNull();
|
||||
});
|
||||
|
||||
it('and the row records the mismatch rather than dropping it', () => {
|
||||
const bad = { fit_n: 1, fitted_through: 'x',
|
||||
resolve: (read) => pc.resolve({ ...read, sport: 'mlb', stat: 'hits' },
|
||||
{ estimate: () => 0.6, artifact: { model_version: 'engine1@2026-07-20', estimator_type: 'isotonic' } }) };
|
||||
const [r] = retention.mergeProbabilityContract([row()], bad);
|
||||
expect(r.probability_contract.probability_state).toBe(pc.STATE.VERSION_MISMATCH);
|
||||
expect(r.probability_contract.served_probability).toBeNull();
|
||||
expect(r.probability_contract.raw_model_probability).toBe(0.65);
|
||||
});
|
||||
});
|
||||
|
||||
Reference in New Issue
Block a user