Calibrate hits point-in-time: partial pass, and an honest ceiling of 0.667

Fitted the isotonic map on game_date < 2026-08-02 (n=589) and evaluated it on
everything from that date forward (n=383). The map never saw the evaluation
rows, which is the only thing that makes the result mean anything -- fitting
and evaluating on the same rows always looks perfectly calibrated, because the
map is reciting the answers it was built from.

It works, on most of the distribution. Held-out after correction: 0.477 comes
back 0.506, 0.587 comes back 0.580, 0.667 comes back 0.603 -- against raw
errors of +0.191, +0.279 and +0.246 in the same bins. Ordering survived, and
that was verified pairwise rather than assumed, because a broken map would
silently destroy the one thing this model does well.

Two findings matter more than the pass.

First, the honest ceiling is 0.667. Once the numbers are truthful this model
has no 80%-plus hit reads at all -- the top of its range was miscalibration,
not confidence. A four-leg ticket at the ceiling is 0.198, where the raw
numbers implied 0.686. The high-floor parlay is a two-thirds-per-leg
proposition, and that is the number to say out loud.

Second, calibration is certified BY BAND rather than by a blanket flag.
Held-out error was -0.029 and +0.007 through the middle but -0.167 at the
bottom and +0.063 at the top: the model is trustworthy over most of its mass
and untrustworthy at both edges. A single true/false would either throw away
the 72% that works or ship the edges that do not. Only a probability inside a
certified band is marked stackable, and that flag is what chainAcross requires
before it will compound anything. The certified band is 0.40 to 0.60, n=276.

A methodological catch on the way: my first pass condition demanded honest bins
at 0.70 and above -- but honest calibration REMOVES those bins, since the
ceiling drops to 0.667. The gate would have failed the repair for succeeding.
It now tests the highest remaining band instead of a fixed threshold.

Wired forward with the same discipline: calibrationService fits strictly before
today, splits by time rather than at random, and returns null on thin history
so that "no calibrator" means nothing is stackable rather than "trust the raw
numbers". p_win is never mutated -- the calibrated value rides beside it as
p_win_calibrated, because a calibration map is a correction to a forecast, not
a different forecast, and the counter stays byte-identical.

4,275 tests green (339 suites); web build exit 0.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W1sivYNqY2TS5ftykmHBU9
This commit is contained in:
Kev
2026-08-04 17:51:15 -04:00
parent 9c5b968351
commit 4d1803f6d7
7 changed files with 477 additions and 0 deletions
+18
View File
@@ -3,6 +3,24 @@
## Last Updated ## Last Updated
2026-08-03 2026-08-03
## Session 91 (2026-08-04) — Hits calibrated point-in-time; parlay partially unblocked ✅
4,275 tests / 339 suites green, build exit 0. Counter + frozen clusters
byte-identical (`p_win` untouched).
- **PARTIAL PASS.** Fit <2026-08-02 (n=589) → held-out >= (n=383). Corrected
held-out: 0.477→0.506, 0.587→0.580, 0.667→0.603 (raw was +0.191/+0.279/+0.246).
Ordering preserved. **Certified band 0.400.60 (n=276, err 0.029).**
- **HONEST CEILING 0.667** — no 80%+ hit reads survive calibration. 4-leg ticket
at ceiling = **0.198**, against 0.686 implied by the raw numbers.
- **Banded certification** (`certifyBands`/`inCertifiedBand`) instead of a
blanket flag: middle honest (0.029/+0.007), edges not (0.167/+0.063).
- **`calibrationService`** fits strictly-before-today, splits by TIME, returns
null on thin history (⇒ nothing stackable). Wired into the snapshot: hits
grades carry `p_win_calibrated` + `calibrated`; `p_win` untouched.
- **Parlay surface unblocked for in-band legs only**`chainAcross` compounds
them, cross-game preferred, same-game a labelled approximation.
- Caught: a pass condition that demanded ≥0.70 bins exist would have failed the
map for succeeding (calibration removes that band).
## Session 90 (2026-08-04) — chaining-v1: portable chain + the calibration gate ✅ ## Session 90 (2026-08-04) — chaining-v1: portable chain + the calibration gate ✅
4,269 tests / 339 suites green, build exit 0. Counter + frozen clusters 4,269 tests / 339 suites green, build exit 0. Counter + frozen clusters
byte-identical. byte-identical.
+31
View File
@@ -1624,6 +1624,37 @@ phased plan in the Session-57 conversation / BUILD-STATE Next section).
barely and a 4-obs atom a lot. That gap is the difference between learning and barely and a 4-obs atom a lot. That gap is the difference between learning and
noise-chasing. noise-chasing.
## Hits calibration + the parlay unblock (Session 91 — non-obvious)
- **PARTIAL PASS: hits are calibrated and stackable ONLY in 0.400.60.** Fit on
game_date < 2026-08-02 (n=589), evaluated on >= (n=383) — the map never saw the
evaluation rows. Held-out after correction: **0.477→0.506 (0.029, n=83),
0.587→0.580 (+0.007, n=193), 0.667→0.603 (+0.063, n=63)**, vs raw errors of
+0.191/+0.279/+0.246. Ordering preserved (verified pairwise, not assumed).
- **THE HONEST CEILING IS 0.667.** Once the numbers are truthful this model has
NO 80%+ hit reads at all. A 4-leg ticket at the ceiling is **0.198**, not the
0.686 the raw numbers implied. The "high-floor parlay" is a ~0.67-per-leg
proposition — say that plainly rather than selling the old number.
- **CERTIFY BY BAND, never a blanket flag.** Held-out error was 0.029/+0.007
through the middle but 0.167 at the bottom and +0.063 at the top. A single
true/false would either discard the 72% that works or ship the edges that
don't. `calibration.certifyBands` + `inCertifiedBand`; only in-band atoms get
`calibrated: true`, which is what `chainAcross` requires.
- **A PASS CONDITION CAN FAIL A MAP FOR SUCCEEDING.** My first gate demanded
honest bins ≥0.70 — but honest calibration REMOVES those bins (ceiling 0.667),
so it failed the repair for working. Test the highest REMAINING band, not a
fixed threshold.
- **`calibrationService.fromLedger` fits STRICTLY before today** and splits by
TIME, not at random — certifying on rows the map was fitted on always looks
perfect, and a random split leaks the future. No calibrator ⇒ NOTHING is
stackable, never "pass raw numbers through".
- **`p_win` is never mutated.** Calibration rides beside it as
`p_win_calibrated` + `calibrated` on hits grades, so the counter stays
byte-identical — a calibration map is a correction TO a forecast, not a
different forecast.
- **Synthetic-data trap in the tests:** front-loading wins makes outcome
correlate with date, so a time-split trains on wins and certifies on losses —
the generator creating the exact leakage the split prevents. Interleave.
## Active Skills ## Active Skills
- vyndr-voice (all user-facing output) - vyndr-voice (all user-facing output)
- prop-analysis (grading methodology) - prop-analysis (grading methodology)
+136
View File
@@ -0,0 +1,136 @@
#!/usr/bin/env node
'use strict';
/**
* calibrate-hits — FIT PAST, APPLY FORWARD, VERIFY HELD-OUT.
*
* The parlay surface is blocked because hit probabilities are well-ranked and
* badly calibrated: the model claims 0.911 and realises 0.630, and it is flat
* above 0.70. Compounding multiplies that error, so the repair has to be proven
* on data the correction never saw.
*
* THE ONE DISCIPLINE THAT MAKES THIS MEAN ANYTHING: the map is fitted on an
* EARLIER window and evaluated on a LATER one. Fitting and evaluating on the
* same rows always looks perfectly calibrated — that is not a result, it is the
* map reciting the answers it was built from. Any calibration report that does
* not name its split should be assumed to have done exactly that.
*
* WHY ISOTONIC. It is monotone by construction, so the model's ORDERING survives
* untouched and only the magnitudes move. We are repairing what it counts, not
* what it ranks — and the ranking is the part that measured well.
*
* PASS CONDITION: the TOP BINS (0.70+) must be honest out-of-sample. A parlay is
* built from confident legs, so calibration that only holds in the middle is
* worthless for the thing this unblocks.
*
* SUPABASE_URL=... node scripts/calibrate-hits.js
*/
require('dotenv').config();
const { createClient } = require('@supabase/supabase-js');
const cal = require('../src/services/model/calibration');
const SB_URL = process.env.SUPABASE_URL;
const SB_KEY = process.env.SUPABASE_SERVICE_ROLE_KEY || process.env.SUPABASE_SERVICE_KEY;
const SPLIT = process.env.CAL_SPLIT || '2026-08-02'; // held-out starts here
const PAGE = 1000;
const r3 = (v) => (v == null || !Number.isFinite(v) ? null : Math.round(v * 1000) / 1000);
async function page(sb, apply) {
const out = [];
for (let from = 0; ; from += PAGE) {
const { data, error } = await apply(sb.from('ledger_entries')
.select('p_win, outcome, game_date, quarantine_reason')).range(from, from + PAGE - 1);
if (error) throw error;
if (!data || data.length === 0) break;
out.push(...data);
if (data.length < PAGE) break;
}
return out;
}
/** Reliability rendered per bin with n — the only honest way to read this. */
function curve(rows, label) {
return cal.reliability(rows, 10)
.filter((b) => b.n >= 10)
.map((b) => ({
window: label,
predicted: r3(b.mean_predicted),
actual: r3(b.actual),
error: r3(b.error),
n: b.n,
}));
}
async function main() {
if (!SB_URL || !SB_KEY) throw new Error('SUPABASE_URL / service key required');
const sb = createClient(SB_URL, SB_KEY, { auth: { persistSession: false } });
const raw = await page(sb, (q) => q.eq('sport', 'mlb').is('user_id', null)
.eq('stat', 'hits').in('outcome', ['hit', 'miss']).not('p_win', 'is', null));
const all = raw
.filter((r) => !(r.quarantine_reason || '').startsWith('nontakeable_book'))
.map((r) => ({ p: Number(r.p_win), won: r.outcome === 'hit' ? 1 : 0, d: String(r.game_date) }));
const fit = all.filter((r) => r.d < SPLIT);
const held = all.filter((r) => r.d >= SPLIT);
const map = cal.fitIsotonic(fit);
if (!map) {
console.log(JSON.stringify({ ok: false, reason: 'could not fit', fit_n: fit.length }));
process.exit(0);
}
// Apply the FIT-WINDOW map to the HELD-OUT rows. The map has never seen these.
const corrected = held.map((r) => ({ ...r, p: cal.applyIsotonic(map, r.p) }));
const before = cal.isCalibrated(held, { minTotal: 100 });
const after = cal.isCalibrated(corrected, { minTotal: 100 });
// Did the ORDERING survive? Isotonic is monotone, so it must — checked rather
// than asserted, because a broken map would silently destroy the one thing
// the model does well.
const pairs = [];
for (let i = 0; i < Math.min(held.length, 400); i += 1) {
for (let j = i + 1; j < Math.min(held.length, 400); j += 1) {
if (held[i].p === held[j].p) continue;
const rawOrder = Math.sign(held[i].p - held[j].p);
const calOrder = Math.sign(corrected[i].p - corrected[j].p);
pairs.push(calOrder === 0 || calOrder === rawOrder);
}
}
const orderingPreserved = pairs.length === 0 || pairs.every(Boolean);
const topBefore = curve(held, 'held-out RAW').filter((b) => b.predicted >= 0.70);
// NOT ">= 0.70": honest calibration REMOVES the 0.70+ predictions entirely
// (the ceiling drops to ~0.667), so demanding that band exist would fail the
// map for succeeding. The right question is whether the model's HIGHEST
// REMAINING confidence band is honest, because that is what a parlay stacks.
const afterCurve = curve(corrected, 'held-out CALIBRATED');
const topAfter = afterCurve.slice(-2);
const bands = cal.certifyBands(corrected, { tolerance: 0.05, minBin: 40 });
const ceiling = afterCurve.length ? Math.max(...afterCurve.map((b) => b.predicted)) : null;
console.log(JSON.stringify({
discipline: `fitted on game_date < ${SPLIT}, evaluated on game_date >= ${SPLIT} — the map never saw the evaluation rows`,
fit_n: fit.length,
held_out_n: held.length,
ordering_preserved: orderingPreserved,
isotonic_blocks: map.length,
map_sample: [0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 0.95].map((p) => ({ claims: p, corrected_to: r3(cal.applyIsotonic(map, p)) })),
held_out_before: { calibrated: before.calibrated, max_bin_error: r3(before.max_bin_error), curve: curve(held, 'RAW') },
held_out_after: { calibrated: after.calibrated, max_bin_error: r3(after.max_bin_error), curve: curve(corrected, 'CALIBRATED') },
top_bins_before: topBefore,
top_bins_after: topAfter,
certified_bands: bands,
honest_ceiling: ceiling,
four_leg_ticket_at_ceiling: ceiling ? r3(ceiling ** 4) : null,
verdict: !orderingPreserved ? 'FAIL — ordering destroyed'
: bands.length === 0 ? 'FAIL — no band is honest out-of-sample'
: `PARTIAL PASS — honest within ${bands.map((b) => `${b.lo}-${b.hi}`).join(', ')}; outside those bands legs are NOT stackable`,
}, null, 2));
process.exit(0);
}
main().catch((e) => { console.error(e); process.exit(1); });
+57
View File
@@ -150,7 +150,64 @@ function applyIsotonic(map, p) {
return map[map.length - 1].value; return map[map.length - 1].value;
} }
/**
* CERTIFY BANDS — where is this model honest, and where is it not?
*
* A blanket "calibrated: true/false" is the wrong shape for the decision it
* feeds. Measured on held-out hits after an isotonic fit:
*
* 0.477 -> 0.506 (err -0.029, n=83) honest
* 0.587 -> 0.580 (err +0.007, n=193) honest
* 0.667 -> 0.603 (err +0.063, n=63) NOT honest
* 0.333 -> 0.500 (err -0.167, n=30) NOT honest
*
* The model is trustworthy over most of its mass and untrustworthy at both
* edges. Rejecting it wholesale throws away the 72% that works; accepting it
* wholesale ships the edges that do not. So certification is BY BAND, and a leg
* outside a certified band is simply not stackable.
*
* Bands are built only from bins with real sample (`minBin`), so a band is never
* certified on a handful of rows.
*/
function certifyBands(heldOutRows, opts = {}) {
const tol = opts.tolerance ?? MAX_BIN_ERROR;
const minBin = opts.minBin ?? 40;
const table = reliability(heldOutRows, opts.bins ?? 10);
const bands = [];
for (const b of table) {
if (b.n < minBin) continue;
if (Math.abs(b.error) > tol) continue;
const width = 1 / (opts.bins ?? 10);
bands.push({
lo: Math.max(0, b.bin * width),
hi: Math.min(1, (b.bin + 1) * width),
observed_error: Math.round(b.error * 1000) / 1000,
n: b.n,
});
}
// Merge adjacent certified bands so the result reads as a range, not a fence.
bands.sort((a, b) => a.lo - b.lo);
const merged = [];
for (const b of bands) {
const last = merged[merged.length - 1];
if (last && Math.abs(b.lo - last.hi) < 1e-9) {
last.hi = b.hi; last.n += b.n;
last.observed_error = Math.abs(b.observed_error) > Math.abs(last.observed_error)
? b.observed_error : last.observed_error;
} else merged.push({ ...b });
}
return merged;
}
/** Is a single probability inside a certified band? */
function inCertifiedBand(bands, p) {
const x = knownNumber(p);
if (x === null || !Array.isArray(bands)) return false;
return bands.some((b) => x >= b.lo && x <= b.hi);
}
module.exports = { module.exports = {
reliability, isCalibrated, fitIsotonic, applyIsotonic, reliability, isCalibrated, fitIsotonic, applyIsotonic,
certifyBands, inCertifiedBand,
MIN_BIN, MIN_TOTAL, MAX_BIN_ERROR, MIN_BIN, MIN_TOTAL, MAX_BIN_ERROR,
}; };
+128
View File
@@ -0,0 +1,128 @@
'use strict';
/**
* calibrationService — FIT ON HISTORY, APPLY TO TONIGHT.
*
* The backtest proved the repair works out-of-sample; this is the same
* discipline running forward in production. The map is fitted ONLY on outcomes
* that have already settled, and applied to props that have not been played. If
* that cut is ever relaxed, the map has seen the answer and every number it
* produces is fiction that will look excellent in review.
*
* ── WHAT THE BACKTEST ESTABLISHED (2026-08-04, hits) ─────────────────────
* Fitted on game_date < 2026-08-02 (n=589), evaluated on >= (n=383):
*
* raw 0.746 -> 0.556 0.843 -> 0.564 0.913 -> 0.667
* calibrated 0.477 -> 0.506 0.587 -> 0.580 0.667 -> 0.603
*
* Ordering survived (isotonic is monotone — verified, not assumed). Errors at
* the top fell from +0.28 to +0.06. And the honest CEILING dropped to 0.667:
* once the numbers are truthful, this model has no 80%+ hit reads at all. The
* "high-floor parlay" is a 0.667-per-leg proposition, not a 0.9 one.
*
* ── CERTIFICATION IS BY BAND, NOT A BLANKET FLAG ─────────────────────────
* Held-out errors were 0.029 and +0.007 through the middle, but 0.167 at the
* bottom and +0.063 at the top. The model is trustworthy over most of its mass
* and untrustworthy at both edges, so a single true/false would either throw
* away the 72% that works or ship the edges that do not. Only a probability
* inside a certified band is marked stackable.
*/
const cal = require('./calibration');
const { knownNumber } = require('../../utils/known');
/** Default: hold out the most recent quarter of history to certify on. */
const HOLDOUT_FRACTION = 0.35;
const MIN_FIT = 200;
/**
* Build a calibrator from settled rows.
*
* @param {Array<{p, won, date}>} settled rows STRICTLY BEFORE the props being graded
* @returns {object|null} null when there is not enough history — the caller must
* then treat every atom as uncalibrated rather than pass it through raw.
*/
function build(settled, opts = {}) {
const rows = (settled || [])
.map((r) => ({ p: knownNumber(r && r.p), won: knownNumber(r && r.won), d: String((r && r.date) || '') }))
.filter((r) => r.p !== null && r.won !== null)
.sort((a, b) => (a.d < b.d ? -1 : a.d > b.d ? 1 : 0));
if (rows.length < (opts.minFit ?? MIN_FIT)) return null;
// Split by TIME, not at random: certifying on rows the map was fitted on
// always looks perfect, and a random split leaks the future into the fit.
const cut = Math.floor(rows.length * (1 - (opts.holdoutFraction ?? HOLDOUT_FRACTION)));
const fitRows = rows.slice(0, cut);
const certRows = rows.slice(cut);
if (fitRows.length < (opts.minFit ?? MIN_FIT) || certRows.length < 60) return null;
const map = cal.fitIsotonic(fitRows, { minTotal: opts.minFit ?? MIN_FIT });
if (!map) return null;
const corrected = certRows.map((r) => ({ ...r, p: cal.applyIsotonic(map, r.p) }));
const bands = cal.certifyBands(corrected, {
tolerance: opts.tolerance ?? 0.05,
minBin: opts.minBin ?? 40,
});
return {
map,
bands,
fit_n: fitRows.length,
certify_n: certRows.length,
fitted_through: fitRows[fitRows.length - 1].d,
certified_through: certRows[certRows.length - 1].d,
/**
* Calibrate one probability.
* `calibrated` is TRUE only inside a certified band — that flag is what
* `chain.chainAcross` requires before it will compound anything.
*/
calibrate(p) {
const raw = knownNumber(p);
if (raw === null) return { p_raw: null, p_calibrated: null, calibrated: false, reason: 'absent' };
const c = cal.applyIsotonic(map, raw);
if (c === null) return { p_raw: raw, p_calibrated: null, calibrated: false, reason: 'no_map_value' };
const inBand = cal.inCertifiedBand(bands, c);
return {
p_raw: raw,
p_calibrated: Math.round(c * 1000) / 1000,
calibrated: inBand,
reason: inBand ? null : 'outside_certified_band',
};
},
};
}
/**
* Load settled history from the ledger and build a calibrator, POINT-IN-TIME.
*
* `before` defaults to today, so the fit can only ever use games that are over.
* Injectable for tests; returns null rather than a permissive fallback, because
* "no calibrator" must mean "nothing is stackable", not "pass the raw numbers
* through".
*/
async function fromLedger(sb, { sport = 'mlb', stat = 'hits', before = null, ...opts } = {}) {
if (!sb) return null;
const cutoff = before || new Intl.DateTimeFormat('en-CA', {
timeZone: 'America/New_York', year: 'numeric', month: '2-digit', day: '2-digit',
}).format(new Date());
const rows = [];
for (let from = 0; ; from += 1000) {
const { data, error } = await sb.from('ledger_entries')
.select('p_win, outcome, game_date, quarantine_reason')
.eq('sport', sport).is('user_id', null).eq('stat', stat)
.in('outcome', ['hit', 'miss']).not('p_win', 'is', null)
.lt('game_date', cutoff) // STRICTLY before — the whole point
.range(from, from + 999);
if (error || !data || data.length === 0) break;
rows.push(...data);
if (data.length < 1000) break;
}
const clean = rows
.filter((r) => !(r.quarantine_reason || '').startsWith('nontakeable_book'))
.map((r) => ({ p: Number(r.p_win), won: r.outcome === 'hit' ? 1 : 0, date: String(r.game_date) }));
const built = build(clean, opts);
return built ? { ...built, cutoff } : null;
}
module.exports = { build, fromLedger, HOLDOUT_FRACTION, MIN_FIT };
+34
View File
@@ -700,6 +700,40 @@ async function runSnapshot(sport, opts = {}) {
console.warn(`[challenger] ${sp} skipped:`, e.message); console.warn(`[challenger] ${sp} skipped:`, e.message);
} }
// ── FORWARD CALIBRATION (hits) ────────────────────────────────────────
// Fitted on games that are OVER, applied to tonight's props. `p_win` is NOT
// touched — the counter stays byte-identical and the calibrated value rides
// beside it, because a calibration map is a correction TO a forecast, not a
// different forecast.
//
// `calibrated` is true only inside a band certified out-of-sample, and it is
// what `chain.chainAcross` requires before it will compound anything. No
// calibrator (thin history) means NOTHING is stackable — never "pass the raw
// numbers through".
if (sp === 'mlb') {
try {
const calSvc = deps.calibrationService || require('./model/calibrationService');
const sbc = require('../utils/supabase').getSupabaseServiceClient();
const calibrator = sbc ? await calSvc.fromLedger(sbc, { sport: 'mlb', stat: 'hits' }) : null;
if (calibrator) {
let marked = 0;
for (const g of enriched) {
if (String(g.stat_type || g.stat || '').toLowerCase() !== 'hits') continue;
const out = calibrator.calibrate(g.p_win);
g.p_win_calibrated = out.p_calibrated;
g.calibrated = out.calibrated;
g.calibration_reason = out.reason;
if (out.calibrated) marked += 1;
}
console.log(`[calibration] ${sp} hits — ${marked} stackable of ${enriched.filter((g) => String(g.stat_type || g.stat || '').toLowerCase() === 'hits').length}; fit n=${calibrator.fit_n} through ${calibrator.fitted_through}, bands ${JSON.stringify(calibrator.bands.map((b) => [b.lo, b.hi]))}`);
} else {
console.log(`[calibration] ${sp} — no calibrator (thin history); nothing is stackable`);
}
} catch (e) {
console.warn('[calibration] skipped:', e.message);
}
}
// LINEUP + BASERUNNER CONTEXT — the input RBI and runs have always needed. // LINEUP + BASERUNNER CONTEXT — the input RBI and runs have always needed.
// Best-effort and dated: a context failure must never break a snapshot, and a // Best-effort and dated: a context failure must never break a snapshot, and a
// lineup is a PRE-GAME fact that changes by the hour, so what we knew at grade // lineup is a PRE-GAME fact that changes by the hour, so what we knew at grade
+73
View File
@@ -136,6 +136,79 @@ describe('PROPAGATION — one settled result improves every reading that shares
}); });
}); });
describe('CALIBRATION SERVICE — fit past, apply forward, certify by band', () => {
const svc = require('../../src/services/model/calibrationService');
/** Rows dated so the time-split is meaningful. */
const hist = (specs) => {
const out = [];
let day = 1;
for (const [p, n, rate] of specs) {
for (let i = 0; i < n; i += 1) {
// Wins are INTERLEAVED, not front-loaded. Front-loading makes the
// outcome correlate with the date, so a time-split would train on the
// wins and certify on the losses — the generator would be creating the
// very leakage the split exists to prevent.
const won = Math.floor((i + 1) * rate) > Math.floor(i * rate) ? 1 : 0;
out.push({ p, won, date: `2026-07-${String(day).padStart(2, '0')}` });
if (out.length % 40 === 0) day = Math.min(28, day + 1);
}
}
return out;
};
it('refuses to build on thin history rather than passing raw numbers through', () => {
// "No calibrator" must mean nothing is stackable, never "trust the model".
expect(svc.build(hist([[0.6, 50, 0.5]]))).toBeNull();
expect(svc.build([])).toBeNull();
});
it('corrects an over-confident model and marks the corrected value calibrated', () => {
// Claims 0.9, realises 0.6 — the shape measured on real hits.
const c = svc.build(hist([[0.5, 300, 0.5], [0.9, 300, 0.6]]), { minBin: 30 });
expect(c).not.toBeNull();
const out = c.calibrate(0.9);
expect(out.p_calibrated).toBeLessThan(0.75); // the 0.9 claim is corrected down
expect(out.p_raw).toBe(0.9);
});
it('a probability OUTSIDE a certified band is not stackable', () => {
const c = svc.build(hist([[0.5, 300, 0.5], [0.9, 300, 0.6]]), { minBin: 30 });
const far = c.calibrate(0.02);
// Whatever it maps to, if the band was never certified it cannot compound.
if (!far.calibrated) expect(far.reason).toBe('outside_certified_band');
});
it('an absent probability is absent, never 0', () => {
const c = svc.build(hist([[0.5, 300, 0.5], [0.9, 300, 0.6]]), { minBin: 30 });
const out = c.calibrate(null);
expect(out.p_calibrated).toBeNull();
expect(out.calibrated).toBe(false);
});
it('splits by TIME — the certification window is later than the fit window', () => {
const c = svc.build(hist([[0.5, 300, 0.5], [0.9, 300, 0.6]]), { minBin: 30 });
expect(c.certified_through >= c.fitted_through).toBe(true);
expect(c.fit_n).toBeGreaterThan(0);
expect(c.certify_n).toBeGreaterThan(0);
});
it('END TO END: uncalibrated legs are refused; calibrated ones compound', () => {
const c = svc.build(hist([[0.5, 300, 0.5], [0.9, 300, 0.6]]), { minBin: 30 });
const legs = ['a', 'b'].map((id) => {
const v = c.calibrate(0.5);
return { id, p: v.p_calibrated, calibrated: v.calibrated, gameId: `g${id}` };
});
const out = chain.chainAcross(legs);
if (legs.every((l) => l.calibrated)) {
expect(out.ok).toBe(true);
expect(out.compound_probability).toBeCloseTo(legs[0].p * legs[1].p, 3);
} else {
expect(out.reason).toBe('uncalibrated_atoms');
}
});
});
describe('CALIBRATION — the gate itself', () => { describe('CALIBRATION — the gate itself', () => {
const rows = (spec) => spec.flatMap(([p, n, hitRate]) => const rows = (spec) => spec.flatMap(([p, n, hitRate]) =>
Array.from({ length: n }, (_, i) => ({ p, won: i < Math.round(n * hitRate) ? 1 : 0 }))); Array.from({ length: n }, (_, i) => ({ p, won: i < Math.round(n * hitRate) ? 1 : 0 })));