builtbykev
e872eff4ce
— the proven factors were never wired in
PHASE 0 — two truths recorded. The swap is a BET, not an OOS win:
isotonic beat low-param on identical held-out rows (hits +0.0028, rbi
+0.0042, TB tied) and we serve low-param anyway on an untestable prior
about shared daily structure. At 19 dates nothing here can test it. And
the MIN_SLOPE catch is preserved as standing rationale: a near-zero or
negative slope collapses toward base-rate-for-everything, which LOWERS
Brier while destroying all resolution -- a metric win that guts the
product.
PHASE 1 — the duel is now falsifiable. Both corrections computed on every
hits/TB prop; p_win_lowparam served, p_win_isotonic_shadow logged in its
own try so it can never break serving. calibrationDuel.adjudicate encodes
the rule IN CODE before any forward date exists: >=10 forward dates and
isotonic winning with a date-block CI excluding zero => REFUTED, revert;
otherwise UPHELD; under 10 dates PENDING regardless of the numbers. A
date counts as forward only if NEITHER map was fitted on it -- otherwise
we would be scoring which map memorised better. Nothing swaps now.
PHASE 2 — the ceiling, quantified via Murphy decomposition:
stat reliability RESOLUTION uncertainty variance explained
hits 0.01353 0.00252 0.24532 1.03%
TB 0.01419 0.00442 0.24329 1.82%
rbi 0.00654 0.03268 0.22531 14.51%
runs 0.00788 0.00130 0.23182 0.56%
Calibration did exactly what theory says and nothing more: hits
reliability 0.01353 -> 0.00233 (-0.0112, 83% of the error removed) while
resolution moved -0.0002. Unexpected: rbi has 13x the resolution of hits
and is the one stat we do NOT serve corrected -- it needs calibration
least and discriminates most.
PHASE 2 DIAGNOSIS — NOT-TRANSMITTED, and not weak, ABSENT. Traced in code:
sprayDefense.js and platoonSeverity.js are required by NOTHING in src/,
only by analysis scripts and their own tests. The served p_win
(intelligence/probabilityEstimator.js:54) reads exactly four inputs --
game-log frequency, opp_rank_stat +/-0.03, home_away +/-0.015, and a cv
pull -- with zero occurrences of spray, platoon, hard-hit or
contact-profile. And snapshotService grades at line 454 while computing
challenger/context at 640+, so everything proven is computed DOWNSTREAM of
the grade it would inform. The three proven hits factors have never once
moved a served number.
That reframes the recent nulls: "calibrated p_win does not separate within
archetype" was never a statement about factors. The factors were not in
the forecast.
PHASE 3 — bands rebuilt on SERVED values (hits/TB low-param, rbi/runs
raw): 28 archetype slots across four stats, ZERO show lift. No longer an
open shrug -- it is the arithmetic of resolution 0.0013-0.0327 against
uncertainty ~0.23. A forecast explaining 1% of variance cannot produce
separating bands, and no correction to its numbers will change that.
HEADLINE: calibration is complete, delivered honest numbers on two stats
and zero grade separation, because the counter has no resolution -- and
the proven factors are not wired into the forecast at all. The second is
the reason for the first, and it is plumbing rather than a modelling wall.
Per-archetype grades need proven factors that actually reach p_win. Last
calibration order.
Serving unchanged from 74cf1ce. p_win never mutated. No Bonferroni slot.
Counter and frozen clusters verified file-by-file (15 modules).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W1sivYNqY2TS5ftykmHBU9
81 lines
2.9 KiB
JavaScript
81 lines
2.9 KiB
JavaScript
'use strict';
|
|
|
|
/**
|
|
* The forward adjudication of a bet made against the measurement.
|
|
*
|
|
* We serve the map that scored WORSE in-window, on an argument the sample
|
|
* cannot test. These lock the rule that decides whether that argument survives —
|
|
* written before any forward date exists, so the bar cannot drift toward
|
|
* whichever answer arrives.
|
|
*/
|
|
|
|
const duel = require('../../src/services/model/calibrationDuel');
|
|
|
|
/** Forward rows where `edge` favours the shadow when positive. */
|
|
function rows(dates, perDate, edge, seed = 2) {
|
|
let s = seed;
|
|
const rnd = () => (s = (s * 1103515245 + 12345) % 2147483648) / 2147483648;
|
|
const out = [];
|
|
for (let d = 0; d < dates; d += 1) {
|
|
for (let i = 0; i < perDate; i += 1) {
|
|
const won = rnd() < 0.6 ? 1 : 0;
|
|
const err = 0.25 + rnd() * 0.1;
|
|
out.push({
|
|
date: `2026-09-${String(d + 1).padStart(2, '0')}`,
|
|
fitted_through: '2026-08-31',
|
|
won,
|
|
served: won ? 1 - err : err,
|
|
shadow: won ? 1 - err + edge : err - edge,
|
|
});
|
|
}
|
|
}
|
|
return out;
|
|
}
|
|
|
|
describe('the rule is pre-registered and cannot be met early', () => {
|
|
it('is PENDING below the forward-date bar, whatever the numbers say', () => {
|
|
// Even with the shadow winning decisively, 5 dates is not a verdict.
|
|
const v = duel.adjudicate(rows(5, 40, 0.15));
|
|
expect(v.verdict).toBe('PENDING');
|
|
expect(v.dates_needed).toBe(duel.MIN_FORWARD_DATES - 5);
|
|
expect(v.action).toMatch(/keep serving/);
|
|
});
|
|
|
|
it('REFUTES the bet when isotonic wins out-of-window at the bar', () => {
|
|
const v = duel.adjudicate(rows(12, 40, 0.15));
|
|
expect(v.verdict).toBe('REFUTED');
|
|
expect(v.ci[1]).toBeLessThan(0);
|
|
expect(v.action).toMatch(/REVERT/);
|
|
});
|
|
|
|
it('UPHOLDS the bet when the served map is not beaten', () => {
|
|
const v = duel.adjudicate(rows(12, 40, -0.15));
|
|
expect(v.verdict).toBe('UPHELD');
|
|
expect(v.action).toMatch(/keep serving/);
|
|
});
|
|
|
|
it('UPHOLDS on a tie — the burden is on refutation, not on us', () => {
|
|
const v = duel.adjudicate(rows(12, 40, 0));
|
|
expect(v.verdict).toBe('UPHELD');
|
|
});
|
|
});
|
|
|
|
describe('only genuinely out-of-window dates count', () => {
|
|
it('drops rows inside the fit window — that would score memorisation', () => {
|
|
const inWindow = rows(12, 40, 0.15).map((r) => ({ ...r, fitted_through: '2026-12-31' }));
|
|
const v = duel.adjudicate(inWindow);
|
|
expect(v.verdict).toBe('PENDING');
|
|
expect(v.forward_dates).toBe(0);
|
|
});
|
|
|
|
it('drops rows with no fit provenance rather than assuming they are forward', () => {
|
|
const noProv = rows(12, 40, 0.15).map(({ fitted_through, ...r }) => r);
|
|
expect(duel.adjudicate(noProv).forward_dates).toBe(0);
|
|
});
|
|
|
|
it('drops rows missing either map — a duel needs both entrants', () => {
|
|
const half = rows(12, 40, 0.15).map((r) => ({ ...r, shadow: null }));
|
|
expect(duel.adjudicate(half).forward_dates).toBe(0);
|
|
});
|
|
});
|