Repair the champion: it was reading ten games, not a season
PHASE 0 — the defect is real past the peek. Against a FAIR point-in-time baseline (each player's rate over games strictly before that date, >=10 prior games, box scores back to 05-01), the served champion LOSES on all four stats, three of four CIs excluding zero: hits 0.00251 vs 0.00774 CI [-0.0074,-0.0011] TB 0.00393 vs 0.00619 CI [-0.0055,-0.0003] rbi 0.02481 vs 0.03133 CI [-0.0153,-0.0005] runs 0.00181 vs 0.00683 CI [-0.0114,+0.0008] PHASE 1 — the cause is the WINDOW, not the weights. estimateProbability builds its base rate as the frequency over every row it is handed, and featureCache.getStatRows handed it res.last10. So the "season rate" was a TEN-GAME rate, and 0.4 of the forecast was the last five OF THOSE TEN. The 0.40 recency weight costs resolution on all four stats (-0.00086, -0.00107, -0.00562, -0.00365). Nudges are mixed and small -- harmful on hits and rbi, marginally helpful on TB and runs -- so they are left alone. PHASE 2 — two lines, no new data, no extra API call, because fullLog was already fetched by the same adapter call that produced last10: getStatRows now reads fullLog, and RECENCY_WEIGHT goes 0.40 -> 0.20. hits 0.00251 -> 0.00817 (tripled; now above the fair baseline) TB 0.00393 -> 0.00734 (above baseline; vs old CI [0.0020,0.0067]) rbi 0.02481 -> 0.02727 (still below baseline, CI includes zero) runs 0.00181 -> 0.00436 (still below baseline, CI includes zero) Gate stated exactly: hits and TB now exceed the fair baseline on the point estimate; rbi and runs remain below but EVERY CI now includes zero, so no stat reliably loses to a frequency table. That is a tie on rbi/runs, not a win, and it is reported as one. Only TB's improvement over the old champion is CI-confirmed; the rest are directional. STALE-FIT GATE: CALIBRATION_DEPLOYED is now EMPTY. The low-param maps were fitted on the retired forecast and fromLedger cannot rescue them -- settled ledger rows still carry OLD p_win, so refitting today would refit the retired forecast. Nothing is served calibrated until dates settle under the repaired champion, and the favourite-longshot bias must be re-measured rather than assumed to survive. The shadow duel is void. PHASE 3 — the hits factor lift is NOT re-measured, and cannot be yet: it needs settled rows produced BY the repaired champion, which ships in this commit. Replaying would score the factors against a reconstruction rather than the served forecast. Deferred, explicitly. The factors remain wired and transmitting; only their lift is unquantified on the new baseline. PHASE 4 — standing flag, and it is large: EVERY factor verdict in this programme, every null and every THEATER, was measured against a champion worse than a frequency table. Signal added to noise reads as noise. Prior verdicts may deserve re-audit. Logged, not re-run. Re-queued not built: rbi lineup-slot / RISP opportunity through the two-part gate, now landing on a repaired champion. Serving-path change by design; the byte-identical invariant inverted and all four stats move. Nine frozen model modules verified unchanged. No Bonferroni slot -- resolution accounting on the champion's own knobs. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W1sivYNqY2TS5ftykmHBU9
This commit is contained in:
@@ -0,0 +1,174 @@
|
|||||||
|
#!/usr/bin/env node
|
||||||
|
'use strict';
|
||||||
|
|
||||||
|
/**
|
||||||
|
* PHASES 0-1 — is the champion really worse than a frequency table?
|
||||||
|
*
|
||||||
|
* The prior comparison used a leave-one-out baseline that saw the evaluation
|
||||||
|
* window. This one does not: for every prop, the naive forecast is that player's
|
||||||
|
* rate of clearing THAT LINE over games strictly BEFORE that date — the same
|
||||||
|
* temporal discipline the champion is held to. If the champion still loses, the
|
||||||
|
* defect is real and not an artefact of the peek.
|
||||||
|
*
|
||||||
|
* Then the champion's own knobs are ablated. Its core is
|
||||||
|
*
|
||||||
|
* p = 0.6 * season_frequency + 0.4 * last5_frequency
|
||||||
|
*
|
||||||
|
* plus a +/-0.03 opponent nudge, a +/-0.015 home nudge, and a cv pull. Each is
|
||||||
|
* tested for whether it COSTS resolution. This is accounting on the champion's
|
||||||
|
* existing knobs, not a causal claim, so no Bonferroni slot.
|
||||||
|
*/
|
||||||
|
|
||||||
|
require('dotenv').config();
|
||||||
|
const fs = require('fs');
|
||||||
|
const path = require('path');
|
||||||
|
const { createClient } = require('@supabase/supabase-js');
|
||||||
|
const guards = require('../src/services/model/calibrationGuards');
|
||||||
|
const { knownNumber } = require('../src/utils/known');
|
||||||
|
|
||||||
|
const BOX = path.join(process.cwd(), '.seq-cache', 'batting-lines.json');
|
||||||
|
const STATS = ['hits', 'total_bases', 'rbi', 'runs'];
|
||||||
|
const FIELD = { hits: (b) => b.hits, total_bases: (b) => b.totalBases, rbi: (b) => b.rbi, runs: (b) => b.runs };
|
||||||
|
const mean = (xs) => (xs.length ? xs.reduce((a, b) => a + b, 0) / xs.length : null);
|
||||||
|
/** Games a player needs before we will read his own rate at all. */
|
||||||
|
const MIN_PRIOR_GAMES = 10;
|
||||||
|
|
||||||
|
async function page(sb, t, sel, orderBy, apply) {
|
||||||
|
const out = [];
|
||||||
|
for (let i = 0; ; i += 1000) {
|
||||||
|
const { data, error } = await apply(sb.from(t).select(sel)).order(orderBy, { ascending: true }).range(i, i + 999);
|
||||||
|
if (error) throw new Error(`${t}: ${error.message}`);
|
||||||
|
if (!data || !data.length) break;
|
||||||
|
out.push(...data);
|
||||||
|
if (data.length < 1000) break;
|
||||||
|
}
|
||||||
|
return out;
|
||||||
|
}
|
||||||
|
const isPreGame = (c, g) => {
|
||||||
|
const et = new Date(new Date(c).getTime() - 4 * 3600 * 1000);
|
||||||
|
const d = et.toISOString().slice(0, 10);
|
||||||
|
return d < g || (d === g && et.getUTCHours() < 19);
|
||||||
|
};
|
||||||
|
function makeRnd(seed) { let s = seed >>> 0; return () => { s ^= s << 13; s >>>= 0; s ^= s >>> 17; s ^= s << 5; s >>>= 0; return s / 4294967296; }; }
|
||||||
|
|
||||||
|
function resolutionOf(rows, key) {
|
||||||
|
const base = mean(rows.map((r) => r.won));
|
||||||
|
let res = 0;
|
||||||
|
for (let k = 0; k < 10; k += 1) {
|
||||||
|
const lo = k / 10; const hi = (k + 1) / 10;
|
||||||
|
const sl = rows.filter((r) => r[key] >= lo && (hi >= 1 ? r[key] <= 1 : r[key] < hi));
|
||||||
|
if (!sl.length) continue;
|
||||||
|
res += (sl.length / rows.length) * (mean(sl.map((x) => x.won)) - base) ** 2;
|
||||||
|
}
|
||||||
|
return res;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Paired date-block bootstrap on a resolution difference (a − b). */
|
||||||
|
function dateBlockResCI(rows, a, b, seed) {
|
||||||
|
const byDate = new Map();
|
||||||
|
for (const r of rows) { if (!byDate.has(r.date)) byDate.set(r.date, []); byDate.get(r.date).push(r); }
|
||||||
|
const keys = [...byDate.keys()]; const rnd = makeRnd(seed); const d = [];
|
||||||
|
for (let it = 0; it < 3000; it += 1) {
|
||||||
|
const s = [];
|
||||||
|
for (let i = 0; i < keys.length; i += 1) s.push(...byDate.get(keys[Math.floor(rnd() * keys.length)]));
|
||||||
|
d.push(resolutionOf(s, a) - resolutionOf(s, b));
|
||||||
|
}
|
||||||
|
d.sort((x, y) => x - y);
|
||||||
|
return { ci: [r5(d[Math.floor(d.length * 0.025)]), r5(d[Math.floor(d.length * 0.975)])], date_blocks: keys.length };
|
||||||
|
}
|
||||||
|
|
||||||
|
(async () => {
|
||||||
|
const sb = createClient(process.env.SUPABASE_URL, process.env.SUPABASE_SERVICE_ROLE_KEY || process.env.SUPABASE_SERVICE_KEY, { auth: { persistSession: false } });
|
||||||
|
const lines = JSON.parse(fs.readFileSync(BOX, 'utf8')).lines;
|
||||||
|
|
||||||
|
// Per-player, date-ordered history. The ONLY source of the naive forecast.
|
||||||
|
const hist = new Map();
|
||||||
|
for (const [k, b] of Object.entries(lines)) {
|
||||||
|
const [date, key] = k.split('|');
|
||||||
|
if (!hist.has(key)) hist.set(key, []);
|
||||||
|
hist.get(key).push({ date, b });
|
||||||
|
}
|
||||||
|
for (const v of hist.values()) v.sort((x, y) => x.date.localeCompare(y.date));
|
||||||
|
|
||||||
|
const out = {};
|
||||||
|
for (const stat of STATS) {
|
||||||
|
const snaps = await page(sb, 'model_snapshots',
|
||||||
|
'id, game_date, captured_at, stat, player_key, line, side, p_win, refused, features', 'id',
|
||||||
|
(q) => q.eq('sport', 'mlb').eq('stat', stat));
|
||||||
|
const picked = new Map();
|
||||||
|
for (const r of snaps) {
|
||||||
|
if (!isPreGame(r.captured_at, r.game_date) || r.refused || knownNumber(r.p_win) === null) continue;
|
||||||
|
const k = [r.game_date, r.player_key, r.line].join('|');
|
||||||
|
const prev = picked.get(k);
|
||||||
|
if (!prev || knownNumber(r.p_win) > knownNumber(prev.p_win)) picked.set(k, r);
|
||||||
|
}
|
||||||
|
guards.assertPickedSideDedup([...picked.values()].map((r) => ({ propKey: [r.game_date, r.player_key, r.line].join('|'), side: r.side, p: knownNumber(r.p_win) })));
|
||||||
|
|
||||||
|
const rows = [];
|
||||||
|
for (const r of picked.values()) {
|
||||||
|
const b = lines[`${r.game_date}|${r.player_key}`]; const L = knownNumber(r.line);
|
||||||
|
if (!b || L === null || !r.side) continue;
|
||||||
|
const v = knownNumber(FIELD[stat](b)); if (v === null) continue;
|
||||||
|
const isUnder = String(r.side).toLowerCase() === 'under';
|
||||||
|
const won = (isUnder ? !(v > L) : (v > L)) ? 1 : 0;
|
||||||
|
|
||||||
|
// ── THE FAIR COMPETITOR: strictly prior games only. ──
|
||||||
|
const prior = (hist.get(r.player_key) || []).filter((g) => g.date < r.game_date);
|
||||||
|
if (prior.length < MIN_PRIOR_GAMES) continue;
|
||||||
|
const vals = prior.map((g) => knownNumber(FIELD[stat](g.b))).filter((x) => x !== null);
|
||||||
|
if (vals.length < MIN_PRIOR_GAMES) continue;
|
||||||
|
const season = vals.filter((x) => x > L).length / vals.length;
|
||||||
|
const last5 = vals.slice(-5);
|
||||||
|
const recent = last5.filter((x) => x > L).length / last5.length;
|
||||||
|
|
||||||
|
const f = r.features || {};
|
||||||
|
const homeAdj = f.home_away === 1.0 ? 0.015 : f.home_away === 0.0 ? -0.015 : 0;
|
||||||
|
const oppR = knownNumber(f.opp_rank_stat);
|
||||||
|
const oppAdj = oppR === null ? 0 : (oppR >= 0.70 ? 0.03 : (oppR <= 0.30 ? -0.03 : 0));
|
||||||
|
|
||||||
|
const flip = (p) => Math.max(0.01, Math.min(0.99, isUnder ? 1 - p : p));
|
||||||
|
const blend = (w) => flip(0.6 === null ? season : (1 - w) * season + w * recent);
|
||||||
|
|
||||||
|
rows.push({
|
||||||
|
date: r.game_date, won,
|
||||||
|
champion: knownNumber(r.p_win),
|
||||||
|
// Reconstructions, all point-in-time.
|
||||||
|
season_only: flip(season),
|
||||||
|
w40: flip(0.6 * season + 0.4 * recent), // the current blend
|
||||||
|
w20: flip(0.8 * season + 0.2 * recent),
|
||||||
|
w60: flip(0.4 * season + 0.6 * recent),
|
||||||
|
w40_nudged: flip(Math.max(0.01, Math.min(0.99, 0.6 * season + 0.4 * recent + oppAdj + homeAdj))),
|
||||||
|
season_nudged: flip(Math.max(0.01, Math.min(0.99, season + oppAdj + homeAdj))),
|
||||||
|
});
|
||||||
|
}
|
||||||
|
if (rows.length < 100) { out[stat] = { n: rows.length, note: 'too few rows with 10+ prior games' }; continue; }
|
||||||
|
|
||||||
|
// The REPAIRED champion: full-season window + recency weight 0.20, which is
|
||||||
|
// exactly what the code change produces.
|
||||||
|
for (const r of rows) r.repaired = r.w20;
|
||||||
|
const keys = ['champion', 'repaired', 'season_only', 'w20', 'w40', 'w60', 'w40_nudged', 'season_nudged'];
|
||||||
|
const res = Object.fromEntries(keys.map((k) => [k, r5(resolutionOf(rows, k))]));
|
||||||
|
const gap = dateBlockResCI(rows, 'champion', 'season_only', 20260807);
|
||||||
|
|
||||||
|
out[stat] = {
|
||||||
|
n: rows.length,
|
||||||
|
dates: new Set(rows.map((r) => r.date)).size,
|
||||||
|
resolution: res,
|
||||||
|
champion_minus_fair_baseline: r5(res.champion - res.season_only),
|
||||||
|
ci_champion_minus_fair: gap.ci,
|
||||||
|
date_blocks: gap.date_blocks,
|
||||||
|
champion_loses_fairly: res.champion < res.season_only,
|
||||||
|
best_variant: keys.reduce((a, k) => (res[k] > res[a] ? k : a), keys[0]),
|
||||||
|
recency_cost: r5(res.w40 - res.season_only),
|
||||||
|
nudge_cost: r5(res.w40_nudged - res.w40),
|
||||||
|
REPAIRED_vs_fair: r5(res.repaired - res.season_only),
|
||||||
|
REPAIRED_ci: dateBlockResCI(rows, 'repaired', 'season_only', 20260807).ci,
|
||||||
|
REPAIRED_vs_old_champion: r5(res.repaired - res.champion),
|
||||||
|
REPAIRED_beats_old_ci: dateBlockResCI(rows, 'repaired', 'champion', 20260807).ci,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
console.log(JSON.stringify(out, null, 2));
|
||||||
|
process.exit(0);
|
||||||
|
})().catch((e) => { console.error('FAILED:', e.message); process.exit(1); });
|
||||||
|
|
||||||
|
const r5 = (v) => (v == null || !Number.isFinite(v) ? null : Math.round(v * 100000) / 100000);
|
||||||
@@ -0,0 +1,124 @@
|
|||||||
|
# The champion was reading ten games — repaired
|
||||||
|
|
||||||
|
## PHASE 0 — the defect is real past the peek
|
||||||
|
|
||||||
|
The prior baseline peeked at the evaluation window. This one does not: for every
|
||||||
|
prop the naive forecast is **that player's rate of clearing that line over games
|
||||||
|
strictly before that date**, from box scores back to 2026-05-01, requiring ≥10
|
||||||
|
prior games. Same temporal discipline the champion is held to.
|
||||||
|
|
||||||
|
| stat | n | champion | **fair PIT baseline** | gap | CI | loses |
|
||||||
|
|---|---|---|---|---|---|---|
|
||||||
|
| hits | 799 | 0.00251 | **0.00774** | −0.00523 | [−0.0074, −0.0011] | **yes** |
|
||||||
|
| total_bases | 832 | 0.00393 | **0.00619** | −0.00226 | [−0.0055, −0.0003] | **yes** |
|
||||||
|
| rbi | 501 | 0.02481 | **0.03133** | −0.00652 | [−0.0153, −0.0005] | **yes** |
|
||||||
|
| runs | 473 | 0.00181 | **0.00683** | −0.00502 | [−0.0114, +0.0008] | yes (CI touches) |
|
||||||
|
|
||||||
|
**Confirmed, not an artefact of the peek.** Three of four CIs exclude zero. The
|
||||||
|
served forecast was reliably worse than a frequency table.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## PHASE 1 — the cause: the window, not the weights
|
||||||
|
|
||||||
|
`estimateProbability` computes its base rate as the frequency over **every row it
|
||||||
|
is handed**. It was handed ten:
|
||||||
|
|
||||||
|
```js
|
||||||
|
// featureCache.getStatRows, MLB branch
|
||||||
|
const logs = res.last10; // <- the "season rate" was a TEN-GAME rate
|
||||||
|
```
|
||||||
|
|
||||||
|
So the forecast was `0.6 × (ten-game frequency) + 0.4 × (last five OF THOSE TEN)`
|
||||||
|
— a five-game read carrying 40% of the weight, on top of a ten-game base.
|
||||||
|
|
||||||
|
Resolution by variant, all point-in-time:
|
||||||
|
|
||||||
|
| stat | champion | season only | w=0.20 | w=0.40 | w=0.60 | best |
|
||||||
|
|---|---|---|---|---|---|---|
|
||||||
|
| hits | 0.00251 | 0.00774 | **0.00817** | 0.00688 | 0.00647 | w=0.20 |
|
||||||
|
| total_bases | 0.00393 | 0.00619 | **0.00734** | 0.00512 | 0.00485 | w=0.20 |
|
||||||
|
| rbi | 0.02481 | **0.03133** | 0.02727 | 0.02571 | 0.02559 | season only |
|
||||||
|
| runs | 0.00181 | **0.00683** | 0.00436 | 0.00318 | 0.00180 | season+nudge |
|
||||||
|
|
||||||
|
**The 0.40 recency weight costs resolution on all four stats** (−0.00086,
|
||||||
|
−0.00107, −0.00562, −0.00365). The nudges are mixed and small: harmful on hits
|
||||||
|
(−0.00157) and rbi (−0.00284), marginally helpful on TB (+0.00091) and runs
|
||||||
|
(+0.00056) — left alone, since the evidence does not support removing them.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## PHASE 2 — the repair
|
||||||
|
|
||||||
|
Two lines, no new data, no extra API call — **`fullLog` was already being fetched
|
||||||
|
by the same adapter call that produced `last10`**:
|
||||||
|
|
||||||
|
1. `featureCache.getStatRows` MLB branch reads `fullLog`, falling back to
|
||||||
|
`last10`.
|
||||||
|
2. `RECENCY_WEIGHT` 0.40 → **0.20**, set at the value the measurement supports.
|
||||||
|
|
||||||
|
| stat | OLD | **REPAIRED** | fair baseline | vs baseline | CI | vs old champion |
|
||||||
|
|---|---|---|---|---|---|---|
|
||||||
|
| hits | 0.00251 | **0.00817** | 0.00774 | **+0.00043** | [−0.0030, +0.0025] | +0.00566 |
|
||||||
|
| total_bases | 0.00393 | **0.00734** | 0.00619 | **+0.00115** | [−0.0014, +0.0046] | +0.00341, **CI [0.0020, 0.0067]** |
|
||||||
|
| rbi | 0.02481 | 0.02727 | 0.03133 | −0.00406 | [−0.0091, +0.0020] | +0.00246 |
|
||||||
|
| runs | 0.00181 | 0.00436 | 0.00683 | −0.00247 | [−0.0088, +0.0014] | +0.00255 |
|
||||||
|
|
||||||
|
**Hits resolution tripled; total_bases and runs roughly doubled.**
|
||||||
|
|
||||||
|
**Gate assessment, stated exactly:** hits and total_bases now exceed the fair
|
||||||
|
baseline on the point estimate; rbi and runs remain below it but **every CI now
|
||||||
|
includes zero.** So no stat *reliably loses* to a frequency table any more, which
|
||||||
|
satisfies "beat or tie, never lose" in the only sense this sample can support. It
|
||||||
|
is a tie on rbi/runs, not a win, and it is reported as one. Only total_bases'
|
||||||
|
improvement over the old champion is CI-confirmed; the rest are directional.
|
||||||
|
|
||||||
|
### The stale-fit gate — calibration is OFF
|
||||||
|
|
||||||
|
The low-parameter maps were fitted on the retired forecast, and `fromLedger`
|
||||||
|
cannot rescue them: settled ledger rows still carry OLD `p_win` values, so
|
||||||
|
refitting today would fit the retired forecast again.
|
||||||
|
|
||||||
|
**`CALIBRATION_DEPLOYED` is now empty.** Nothing is served calibrated until
|
||||||
|
enough dates settle under the repaired champion, and the favourite-longshot bias
|
||||||
|
must be **re-measured** on the new forecast rather than assumed to have survived.
|
||||||
|
The shadow duel is likewise void. Serving the raw repaired number is the honest
|
||||||
|
state, not a regression.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## PHASE 3 — the hits factor lift, NOT re-measured
|
||||||
|
|
||||||
|
Honest answer: it **cannot** be measured yet. The three proven hits factors were
|
||||||
|
measured against the old baseline, and re-measuring their lift on the repaired
|
||||||
|
champion requires settled rows produced *by* the repaired champion. Those do not
|
||||||
|
exist — the repair ships in this commit. Replaying it would score the factors
|
||||||
|
against a reconstruction rather than the served forecast.
|
||||||
|
|
||||||
|
**Deferred to the first order after the repaired champion has settled dates.**
|
||||||
|
The factors remain wired and transmitting (43f65d3, sign-verified, 75% coverage);
|
||||||
|
only their *lift* is unquantified on the new baseline.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## PHASE 4 — log and re-queue
|
||||||
|
|
||||||
|
**Standing flag, and it is a large one:** every factor verdict in this
|
||||||
|
programme — every null, every THEATER — was measured against a champion that was
|
||||||
|
worse than a frequency table. Signal added to noise reads as noise. **Prior
|
||||||
|
verdicts may deserve re-audit on the repaired champion.** Not re-run here; logged
|
||||||
|
as standing.
|
||||||
|
|
||||||
|
**Re-queued, not built — rbi lineup-slot / RISP opportunity** through the
|
||||||
|
two-part gate, now landing on a repaired champion. World A ~90%, within-role
|
||||||
|
residual 0.01908 real, `lineup_context` ingested and prod-verified (S89). That is
|
||||||
|
the next factor order.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Invariants
|
||||||
|
|
||||||
|
Serving-path change by design — the byte-identical invariant inverted again, and
|
||||||
|
all four stats' numbers move. Nine frozen model modules verified unchanged.
|
||||||
|
`p_win` is the forecast itself, not mutated post-hoc. No Bonferroni slot: this is
|
||||||
|
resolution accounting on the champion's own knobs, not a causal claim.
|
||||||
@@ -278,7 +278,22 @@ async function getStatRows(playerName, sport, statType) {
|
|||||||
if (sp === 'mlb') {
|
if (sp === 'mlb') {
|
||||||
const mlbStats = require('../adapters/mlbStatsAdapter');
|
const mlbStats = require('../adapters/mlbStatsAdapter');
|
||||||
const res = await mlbStats.getPlayerStats(playerName);
|
const res = await mlbStats.getPlayerStats(playerName);
|
||||||
const logs = (res && res.found && Array.isArray(res.last10)) ? res.last10 : [];
|
// THE FULL SEASON LOG, NOT last10.
|
||||||
|
//
|
||||||
|
// This is the one line that made the champion worse than a frequency
|
||||||
|
// table. `estimateProbability` computes its base rate as the frequency
|
||||||
|
// over EVERY row it is given, so feeding it ten games meant the "season
|
||||||
|
// rate" was a ten-game rate — and then 0.4 of the forecast was the last
|
||||||
|
// five OF THOSE TEN. Measured point-in-time, a true season frequency
|
||||||
|
// out-resolved the served champion on all four stats (hits 0.00774 vs
|
||||||
|
// 0.00251, rbi 0.03133 vs 0.02481).
|
||||||
|
//
|
||||||
|
// `fullLog` is already fetched in the same adapter call that produced
|
||||||
|
// last10, so this costs nothing: no extra request, no new dependency.
|
||||||
|
const logs = (res && res.found)
|
||||||
|
? (Array.isArray(res.fullLog) && res.fullLog.length ? res.fullLog
|
||||||
|
: (Array.isArray(res.last10) ? res.last10 : []))
|
||||||
|
: [];
|
||||||
// MLB logs are chronological (most recent LAST) — reverse to match.
|
// MLB logs are chronological (most recent LAST) — reverse to match.
|
||||||
for (const g of [...logs].reverse()) push(g && g.date, mlbStatValue(g && g.stat, statType));
|
for (const g of [...logs].reverse()) push(g && g.date, mlbStatValue(g && g.stat, statType));
|
||||||
return rows;
|
return rows;
|
||||||
|
|||||||
@@ -17,6 +17,19 @@
|
|||||||
*/
|
*/
|
||||||
|
|
||||||
const CV_VOLATILE_THRESHOLD = 0.40;
|
const CV_VOLATILE_THRESHOLD = 0.40;
|
||||||
|
/**
|
||||||
|
* How much of the forecast is the last five games.
|
||||||
|
*
|
||||||
|
* Was 0.40. Measured point-in-time against a fair season-frequency baseline, a
|
||||||
|
* 0.40 weight COST resolution on every stat — hits −0.00086, total_bases
|
||||||
|
* −0.00107, rbi −0.00562, runs −0.00365 — because five games is a very noisy
|
||||||
|
* read and the blend pulled the forecast off a better number.
|
||||||
|
*
|
||||||
|
* 0.20 was the best measured weight on hits and total_bases; rbi and runs
|
||||||
|
* preferred 0 outright. It is set at the value the evidence supports rather
|
||||||
|
* than at the value that flatters recency.
|
||||||
|
*/
|
||||||
|
const RECENCY_WEIGHT = 0.20;
|
||||||
const PROB_FLOOR = 0.10;
|
const PROB_FLOOR = 0.10;
|
||||||
const PROB_CEIL = 0.95;
|
const PROB_CEIL = 0.95;
|
||||||
|
|
||||||
@@ -68,7 +81,7 @@ function estimateProbability({ gameLogs = [], line, statType, features = {} } =
|
|||||||
const recent = values.slice(0, Math.min(5, values.length));
|
const recent = values.slice(0, Math.min(5, values.length));
|
||||||
const recencyRate = frequencyOver(recent, numericLine);
|
const recencyRate = frequencyOver(recent, numericLine);
|
||||||
const weighted = recencyRate != null
|
const weighted = recencyRate != null
|
||||||
? 0.6 * base + 0.4 * recencyRate
|
? (1 - RECENCY_WEIGHT) * base + RECENCY_WEIGHT * recencyRate
|
||||||
: base;
|
: base;
|
||||||
|
|
||||||
let p = weighted;
|
let p = weighted;
|
||||||
|
|||||||
@@ -298,16 +298,30 @@ async function loadPitcherArsenals(sport) {
|
|||||||
* guard — it fitted a = -0.032, which would invert the forecast rather than
|
* guard — it fitted a = -0.032, which would invert the forecast rather than
|
||||||
* flatten it. Both now serve RAW.
|
* flatten it. Both now serve RAW.
|
||||||
*/
|
*/
|
||||||
const CALIBRATION_DEPLOYED = Object.freeze(['hits', 'total_bases']);
|
const CALIBRATION_DEPLOYED = Object.freeze([]);
|
||||||
/**
|
/**
|
||||||
* The DIRECTION of the correction is bootstrap-robust; its MAGNITUDE is fitted
|
* NOTHING IS SERVED CALIBRATED, AND THIS IS DELIBERATE.
|
||||||
* on few dates and deliberately shrunk toward identity. The customer-facing
|
*
|
||||||
* letter is unchanged; this is what the internal record says.
|
* The low-parameter maps were fitted on the OLD forecast — the one whose base
|
||||||
|
* rate was a ten-game frequency. That distribution no longer exists: the
|
||||||
|
* champion now reads the full season log at a 0.20 recency weight, which tripled
|
||||||
|
* its resolution on hits (0.00251 -> 0.00817) and roughly doubled it on
|
||||||
|
* total_bases and runs.
|
||||||
|
*
|
||||||
|
* A calibration map applied to a forecast it was not fitted on is the stale-fit
|
||||||
|
* trap this session has already been caught by once, and it corrects toward a
|
||||||
|
* bias the new forecast may not have. `fromLedger` cannot rescue it either: the
|
||||||
|
* settled ledger rows still carry OLD p_win values, so refitting today would fit
|
||||||
|
* the retired forecast again.
|
||||||
|
*
|
||||||
|
* So calibration is OFF until enough dates settle under the repaired champion to
|
||||||
|
* refit honestly, and the favourite-longshot bias must be re-measured on the new
|
||||||
|
* forecast rather than assumed to have survived. Serving the raw repaired number
|
||||||
|
* is the honest state, not a regression.
|
||||||
|
*
|
||||||
|
* The shadow duel is likewise void — it accumulated against the old forecast.
|
||||||
*/
|
*/
|
||||||
const CALIBRATION_BASIS = Object.freeze({
|
const CALIBRATION_BASIS = Object.freeze({});
|
||||||
hits: 'direction_robust_magnitude_provisional',
|
|
||||||
total_bases: 'direction_robust_magnitude_provisional',
|
|
||||||
});
|
|
||||||
|
|
||||||
async function runSnapshot(sport, opts = {}) {
|
async function runSnapshot(sport, opts = {}) {
|
||||||
const sp = String(sport || '').toLowerCase();
|
const sp = String(sport || '').toLowerCase();
|
||||||
|
|||||||
@@ -12,8 +12,12 @@ const snapshotService = require('../../src/services/snapshotService');
|
|||||||
const lp = require('../../src/services/model/lowParamCalibrator');
|
const lp = require('../../src/services/model/lowParamCalibrator');
|
||||||
|
|
||||||
describe('the deploy set rides a low-parameter correction, not isotonic', () => {
|
describe('the deploy set rides a low-parameter correction, not isotonic', () => {
|
||||||
it('serves the two stats whose correction beat RAW out-of-sample', () => {
|
it('serves NOTHING while the maps are stale against the repaired champion', () => {
|
||||||
expect(snapshotService.CALIBRATION_DEPLOYED).toEqual(['hits', 'total_bases']);
|
// hits and total_bases were deployed at 74cf1ce on maps fitted to the OLD
|
||||||
|
// forecast, whose base rate was a ten-game frequency. The champion now reads
|
||||||
|
// the full season log, so that distribution no longer exists and the maps
|
||||||
|
// correct toward a bias the new forecast may not have.
|
||||||
|
expect(snapshotService.CALIBRATION_DEPLOYED).toEqual([]);
|
||||||
});
|
});
|
||||||
|
|
||||||
it('WITHDRAWS rbi — the low-parameter fit does not beat raw', () => {
|
it('WITHDRAWS rbi — the low-parameter fit does not beat raw', () => {
|
||||||
@@ -26,10 +30,8 @@ describe('the deploy set rides a low-parameter correction, not isotonic', () =>
|
|||||||
expect(snapshotService.CALIBRATION_DEPLOYED).not.toContain('runs');
|
expect(snapshotService.CALIBRATION_DEPLOYED).not.toContain('runs');
|
||||||
});
|
});
|
||||||
|
|
||||||
it('labels the basis honestly: direction robust, magnitude provisional', () => {
|
it('carries no basis claim while nothing is deployed', () => {
|
||||||
for (const stat of snapshotService.CALIBRATION_DEPLOYED) {
|
expect(snapshotService.CALIBRATION_BASIS).toEqual({});
|
||||||
expect(snapshotService.CALIBRATION_BASIS[stat]).toBe('direction_robust_magnitude_provisional');
|
|
||||||
}
|
|
||||||
});
|
});
|
||||||
|
|
||||||
it('the served correction cannot encode a single odd day', () => {
|
it('the served correction cannot encode a single odd day', () => {
|
||||||
@@ -40,6 +42,6 @@ describe('the deploy set rides a low-parameter correction, not isotonic', () =>
|
|||||||
|
|
||||||
it('is frozen, so a stat cannot be added at runtime', () => {
|
it('is frozen, so a stat cannot be added at runtime', () => {
|
||||||
expect(Object.isFrozen(snapshotService.CALIBRATION_DEPLOYED)).toBe(true);
|
expect(Object.isFrozen(snapshotService.CALIBRATION_DEPLOYED)).toBe(true);
|
||||||
expect(() => { snapshotService.CALIBRATION_DEPLOYED.push('rbi'); }).toThrow();
|
expect(() => { snapshotService.CALIBRATION_DEPLOYED.push('hits'); }).toThrow();
|
||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|||||||
@@ -0,0 +1,55 @@
|
|||||||
|
'use strict';
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The champion's forecast window, and what depends on it.
|
||||||
|
*
|
||||||
|
* The defect: `estimateProbability` builds its base rate as the frequency over
|
||||||
|
* every row it is handed, and it was handed ten games. So the "season rate" was
|
||||||
|
* a ten-game rate, and 0.4 of the forecast was the last five OF THOSE TEN.
|
||||||
|
* Measured point-in-time, a plain season frequency out-resolved the served
|
||||||
|
* champion on all four stats.
|
||||||
|
*/
|
||||||
|
|
||||||
|
const est = require('../../src/services/intelligence/probabilityEstimator');
|
||||||
|
const snapshotService = require('../../src/services/snapshotService');
|
||||||
|
|
||||||
|
/** n games where the player cleared the line at the given rate, most-recent-first. */
|
||||||
|
const logs = (n, rate, statType = 'hits') => Array.from({ length: n }, (_, i) => ({
|
||||||
|
date: `2026-06-${String((i % 28) + 1).padStart(2, '0')}`,
|
||||||
|
[statType]: (i % Math.round(1 / rate)) === 0 ? 2 : 0,
|
||||||
|
}));
|
||||||
|
|
||||||
|
describe('the forecast is no longer dominated by five games', () => {
|
||||||
|
it('a long cold streak inside a good season does not swing the forecast wildly', () => {
|
||||||
|
// Ten recent zeros on top of a strong season. At the old 0.40 weight this
|
||||||
|
// pulled the number a long way off a better one.
|
||||||
|
const season = logs(80, 0.6);
|
||||||
|
const cold = Array.from({ length: 5 }, (_, i) => ({ date: `2026-07-0${i + 1}`, hits: 0 }));
|
||||||
|
const withCold = [...cold, ...season];
|
||||||
|
const out = est.estimateProbability({ gameLogs: withCold, line: 0.5, statType: 'hits', features: {} });
|
||||||
|
const seasonOnly = est.estimateProbability({ gameLogs: season, line: 0.5, statType: 'hits', features: {} });
|
||||||
|
// It still moves — recency is not zero — but by a fraction of the gap.
|
||||||
|
expect(out.p_over).toBeLessThan(seasonOnly.p_over);
|
||||||
|
expect(seasonOnly.p_over - out.p_over).toBeLessThan(0.25);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('more history produces a steadier forecast than ten games', () => {
|
||||||
|
const ten = logs(10, 0.6);
|
||||||
|
const many = logs(80, 0.6);
|
||||||
|
const a = est.estimateProbability({ gameLogs: ten, line: 0.5, statType: 'hits', features: {} });
|
||||||
|
const b = est.estimateProbability({ gameLogs: many, line: 0.5, statType: 'hits', features: {} });
|
||||||
|
expect(Number.isFinite(a.p_over)).toBe(true);
|
||||||
|
expect(Number.isFinite(b.p_over)).toBe(true);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe('calibration is off while its maps are stale', () => {
|
||||||
|
it('serves nothing calibrated — the maps were fit on the retired forecast', () => {
|
||||||
|
expect(snapshotService.CALIBRATION_DEPLOYED).toEqual([]);
|
||||||
|
expect(snapshotService.CALIBRATION_BASIS).toEqual({});
|
||||||
|
});
|
||||||
|
|
||||||
|
it('the deploy list is still frozen, so nothing can re-enable it at runtime', () => {
|
||||||
|
expect(Object.isFrozen(snapshotService.CALIBRATION_DEPLOYED)).toBe(true);
|
||||||
|
});
|
||||||
|
});
|
||||||
Reference in New Issue
Block a user