diff --git a/BUILD-STATE.md b/BUILD-STATE.md index ce919a2..2931fd3 100755 --- a/BUILD-STATE.md +++ b/BUILD-STATE.md @@ -3,6 +3,34 @@ ## Last Updated 2026-08-03 +## Session 78 (2026-08-03) — Champion decomposed: the edge is a hit-rate counter ✅ +Spec: `specs/champion-input-diagnosis.md`. READ-ONLY (src/web untouched); +4,159 tests green. +- **The champion is 5 lines.** base = empirical frequency of (stat > THIS line), + 0.6/0.4 blend with last-5, ±0.03 opponent, ±0.015 home/away, cv>0.40 pull, + clamp [0.10,0.95]. It reads 3 features; featureCache retains a dozen more that + p_win never touches. +- **Exact analytic ablation, per stat, paired bootstrap.** Removing ALL THREE + adjustments changes resolution by nothing everywhere (hits −0.0059, TB −0.0015, + rbi +0.0106, runs +0.0130, walks +0.0008) — and rbi's home/away is mildly + HARMFUL (+0.0053, CI excludes 0). ~100% of the edge is base+recency. +- **Pooled 0.46 is an artifact** — per stat 0.196 (hits) … 0.499 (rbi). Corrected + last session's reading; paired differences unaffected. +- **BIGGEST LOSS = the clamp.** 20.6% of settled rows pinned to a constant (no + ranking possible there), and `0.900` covers home_runs-under truly 99.5% AND + hits-under truly 51.9%. Global over-prediction +3.5pt (TB +7.6). No new data + needed to fix. +- **One real lead: `opportunity_drift`** (residual +0.156 hits, +0.145 TB — + repeats across stats, unlike the weather hits which sit inside the expected + false-positive count). We ALREADY compute it; arch-v1's opportunity axis + extracts nothing from it. Wrong implementation, not a missing feature. +- **Archetype: UNMEASURABLE** — 2 of 41 labels have testable n. Not refuted. +- **Next order priority:** (1) clamp + calibration, (2) opportunity as a rate + scaler, (3) prune the diluting axes, (4) get archetype coverage. Explicitly NOT + another projection variant. +- Flagged: `model_snapshots.outcome` NULL on all 22,032 rows — retention is + never settled, so replays must join the ledger for labels. + ## Session 77 (2026-08-03) — Settlement was dead for two days; scoreboard now readable ✅ Specs: `specs/challenger-scoreboard.md`, `specs/odds-429-diagnosis.md`. 4,159 tests / 332 suites green, web build exit 0. diff --git a/CLAUDE.md b/CLAUDE.md index 2e364ab..664f297 100755 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -1183,6 +1183,47 @@ phased plan in the Session-57 conversation / BUILD-STATE Next section). DFS pick'em is excluded from PRICING/consensus (`EXCLUDED_FROM_PRICING`) — a fixed-payout shaded number is not a market price. Never "clean up" breadth. +## Champion decomposition — the edge is a hit-rate counter (Session 78 — non-obvious) +- **`specs/champion-input-diagnosis.md`.** `probabilityEstimator` IS the champion + and it is five lines: `base` = empirical frequency of (stat > THIS line) over + the game log, blended 0.6/0.4 with the last-5 frequency, then oppAdj(±0.03) + + homeAdj(±0.015) + a cv>0.40 pull toward 0.50, then clamp [0.10, 0.95]. +- **EXACT ANALYTIC ABLATION (no refit):** every adjustment is closed-form from + stored features and the consistency step is linear (`f(x)=0.9x+0.05` ⟹ + `f(a+b)=f(a)+0.9b`), so layers subtract algebraically out of the stored p_win. + Result: **removing ALL THREE adjustments changes resolution by nothing on every + stat**, and on rbi/runs it IMPROVES it (rbi home/away removal +0.0053, CI + excludes zero = mildly HARMFUL). ~100% of the edge is base+recency. +- **POOLED RESOLUTION IS INFLATED — do not quote 0.46.** Per stat the champion is + 0.196 (hits) to 0.499 (rbi); pooling stats with different base rates adds + correlation because p_win tracks the base rate across stats. Paired DIFFERENCES + (the scoreboard) stay valid; the absolute level does not. Always per-stat. +- **THE CLAMP IS THE BIGGEST LOSS, not a missing feature.** 358/1,741 settled rows + (20.6%) sit ON the boundary, so the model emits a CONSTANT there and cannot rank + within a fifth of the book. And `0.900` hides home_runs-under truly 99.5% + (−9.5pt under-confident) next to hits-under truly 51.9% (+38.1pt over-confident). + `PROB_CEIL=0.95` makes the 99.5% case inexpressible. Global over-prediction + +3.5pt (total_bases +7.6). Fixable with NO new data. +- **`opportunity_drift` is the ONE real missing-weighting lead** — residual corr + +0.156 (hits) and +0.145 (total_bases), i.e. it REPEATS across independent + stats. Discipline: 14 features × 5 stats = 70 tests, so 3–4 CI-excludes-zero + results are expected BY CHANCE; a single hit (weather on TB) is noise. And we + already compute it — arch-v1's opportunity axis uses it and extracts NOTHING + (delta +0.0001). Wrong implementation, not a missing feature: opportunity must + scale the RATE, not nudge the probability. +- **ARCHETYPE VERDICT: unmeasurable, not refuted.** Only 2 of 41 archetypes + (BOMBER, GHOST) reach n≥40 settled rows; all mean residuals straddle zero. That + is "we have not measured it", NOT "archetypes carry no signal". Don't act + either way. (Their uniformly negative residuals are the global over-prediction, + not an archetype effect.) +- **Why every challenger has failed:** the ladder/hits-v1 REPLACE the frequency + question with a fitted distribution; arch-v1's env axis adds park/weather the + champion ignores. Asking "how often has he cleared THIS number" directly is the + thing that works — improve its inputs, never substitute it. +- **`model_snapshots.outcome` is NULL on all 22,032 rows.** The retention table + built for exactly this kind of replay was never settled, so ablations must join + outcomes from `ledger_entries` on (player_key, stat, line, side, game_date). + ## Active Skills - vyndr-voice (all user-facing output) - prop-analysis (grading methodology) diff --git a/scripts/champion-ablation.js b/scripts/champion-ablation.js new file mode 100644 index 0000000..b349148 --- /dev/null +++ b/scripts/champion-ablation.js @@ -0,0 +1,374 @@ +#!/usr/bin/env node +'use strict'; + +/** + * champion-ablation — WHERE DOES THE CHAMPION'S RESOLUTION ACTUALLY COME FROM? + * + * Convergent evidence says the problem is INPUTS, not shape: hits-v1 refuted, + * the ladder reliably worse (−0.030), arch-v1 moving 76% of rows to exactly zero + * effect, contact/environment/opportunity all CI-includes-zero. Every one of + * those changed the DISTRIBUTION or added a NUDGE. None changed the information. + * So before building a sixth thing, decompose the champion. + * + * THE CHAMPION IS FIVE LINES OF ARITHMETIC (probabilityEstimator): + * + * base = empirical frequency of stat > line over the game log + * weighted = 0.6·base + 0.4·(same frequency over the last 5) + * p = weighted + oppAdj(±0.03) + homeAdj(±0.015) + * if cv>0.40: p = 0.9·p + 0.05 (volatile → pull toward 0.50) + * p_over = clamp(p, 0.10, 0.95); p_win = side==='under' ? 1−p_over : p_over + * + * THE ABLATION IS EXACT, NOT A REFIT. Every adjustment is a closed-form function + * of stored features, and the consistency step is linear, so each layer can be + * removed analytically from the stored p_win: + * + * f(x) = 0.9x + 0.05 ⟹ f(a+b) = f(a) + 0.9b + * + * so subtracting an adjustment is subtracting k·adj with k = 0.9 when the + * consistency pull fired and 1 when it did not. Nothing is re-estimated, no + * model is refit, and no game log is re-fetched — which also means no lookahead + * is even possible here. + * + * WHAT CANNOT BE ABLATED SEPARATELY, STATED PLAINLY: `base` and `recency` are + * recoverable only as their blend (`weighted`), because the stored feature + * vector holds AVERAGES (l5_avg/l20_avg), not frequencies-over-the-line. So the + * base/recency split is reported as ONE block. That is a real limit of this + * measurement, not an oversight. + * + * CLAMPED ROWS ARE EXCLUDED from the ablation: at p_over ∈ {0.10, 0.95} the + * inversion is ambiguous, and guessing the pre-clamp value would be fabrication. + * Their count is reported. + * + * SECOND MEASUREMENT — THE MISSING-FEATURE TEST. featureCache computes and + * RETAINS far more than the champion reads (park_*, weather_*, rest_days, + * opportunity_drift, ab_per_game, l5/l10/l20 avgs). If any of those correlates + * with the champion's RESIDUAL (won − p_win), that is signal sitting unused on + * disk — a MISSING FEATURE. If none do, that is evidence for AT CEILING with + * respect to everything we currently compute. + * + * SUPABASE_URL=... node scripts/champion-ablation.js + */ + +require('dotenv').config(); +const { createClient } = require('@supabase/supabase-js'); + +const SB_URL = process.env.SUPABASE_URL; +const SB_KEY = process.env.SUPABASE_SERVICE_ROLE_KEY || process.env.SUPABASE_SERVICE_KEY; +const PAGE = 1000; + +const CV_VOLATILE_THRESHOLD = 0.40; +const PROB_FLOOR = 0.10; +const PROB_CEIL = 0.95; +const clamp = (p) => Math.max(PROB_FLOOR, Math.min(PROB_CEIL, p)); + +function corr(xs, ys) { + const n = xs.length; + if (n < 3) return null; + const mx = xs.reduce((a, b) => a + b, 0) / n; + const my = ys.reduce((a, b) => a + b, 0) / n; + let sxy = 0; let sxx = 0; let syy = 0; + for (let i = 0; i < n; i += 1) { + const dx = xs[i] - mx; const dy = ys[i] - my; + sxy += dx * dy; sxx += dx * dx; syy += dy * dy; + } + if (sxx <= 0 || syy <= 0) return null; + return sxy / Math.sqrt(sxx * syy); +} +const r4 = (v) => (v == null || !Number.isFinite(v) ? null : Math.round(v * 10000) / 10000); +const num = (v) => { + if (v == null || v === '' || typeof v === 'boolean' || typeof v === 'object') return null; + const n = Number(v); + return Number.isFinite(n) ? n : null; +}; + +/** Deterministic xorshift32 — a measurement that changes between runs is not one. */ +function makeRnd(seed) { + let s = seed >>> 0; + return () => { s ^= s << 13; s >>>= 0; s ^= s >>> 17; s ^= s << 5; s >>>= 0; return s / 4294967296; }; +} + +/** Paired bootstrap on a DIFFERENCE of resolutions (same rows → same resample). */ +function bootstrapDiff(rows, keyA, keyB, iters = 3000, seed = 20260803) { + if (rows.length < 30) return null; + const rnd = makeRnd(seed); + const n = rows.length; + const diffs = []; + for (let it = 0; it < iters; it += 1) { + const ys = []; const a = []; const b = []; + for (let i = 0; i < n; i += 1) { + const r = rows[Math.floor(rnd() * n)]; + ys.push(r.won); a.push(r[keyA]); b.push(r[keyB]); + } + const ca = corr(a, ys); const cb = corr(b, ys); + if (ca == null || cb == null) continue; + diffs.push(ca - cb); + } + if (diffs.length < 100) return null; + diffs.sort((x, y) => x - y); + const q = (p) => r4(diffs[Math.floor(p * (diffs.length - 1))]); + const ci = [q(0.025), q(0.975)]; + return { + point: r4(corr(rows.map((r) => r[keyA]), rows.map((r) => r.won)) + - corr(rows.map((r) => r[keyB]), rows.map((r) => r.won))), + ci95: ci, + ci_excludes_zero: ci[0] > 0 || ci[1] < 0, + }; +} + +/** Bootstrap CI on a single correlation (for the residual-signal test). */ +function bootstrapCorr(rows, key, seed = 20260804, iters = 3000) { + const usable = rows.filter((r) => r[key] != null); + if (usable.length < 40) return { n: usable.length, corr: null, ci95: null, ci_excludes_zero: false }; + const rnd = makeRnd(seed); + const n = usable.length; + const vals = []; + for (let it = 0; it < iters; it += 1) { + const xs = []; const ys = []; + for (let i = 0; i < n; i += 1) { + const r = usable[Math.floor(rnd() * n)]; + xs.push(r[key]); ys.push(r.residual); + } + const c = corr(xs, ys); + if (c != null) vals.push(c); + } + if (vals.length < 100) return { n, corr: null, ci95: null, ci_excludes_zero: false }; + vals.sort((a, b) => a - b); + const q = (p) => r4(vals[Math.floor(p * (vals.length - 1))]); + const ci = [q(0.025), q(0.975)]; + return { + n, + corr: r4(corr(usable.map((r) => r[key]), usable.map((r) => r.residual))), + ci95: ci, + ci_excludes_zero: ci[0] > 0 || ci[1] < 0, + }; +} + +async function page(sb, table, select, apply) { + const out = []; + for (let from = 0; ; from += PAGE) { + let q = sb.from(table).select(select); + q = apply(q).range(from, from + PAGE - 1); + const { data, error } = await q; + if (error) throw error; + if (!data || data.length === 0) break; + out.push(...data); + if (data.length < PAGE) break; + } + return out; +} + +const propKey = (r) => `${r.player_key}|${r.stat}|${Number(r.line)}|${String(r.side).toLowerCase()}|${r.game_date}`; + +/** + * OUTCOMES COME FROM THE LEDGER, NOT FROM RETENTION. + * + * `model_snapshots.outcome` is NULL on all 22,032 rows — the retention table + * that exists so a different model can be replayed against the same conditions + * stores the features but was never settled. So the labels are joined from + * `ledger_entries` on (player_key, stat, line, side, game_date), which is the + * same identity the ledger's own dedupe constraint uses. Flagged, not fixed — + * this run is read-only. + */ +async function fetchAll(sb) { + const snaps = await page(sb, 'model_snapshots', + 'player_key, stat, line, side, game_date, p_win, features, quarantine_reason, captured_at, archetype', + (q) => q.eq('sport', 'mlb').not('p_win', 'is', null).not('features', 'is', null)); + const led = await page(sb, 'ledger_entries', 'player_key, stat, line, side, game_date, outcome, quarantine_reason', + (q) => q.eq('sport', 'mlb').is('user_id', null).in('outcome', ['hit', 'miss'])); + + const outcomeBy = new Map(); + for (const r of led) { + if ((r.quarantine_reason || '').startsWith('nontakeable_book')) continue; + outcomeBy.set(propKey(r), r.outcome); + } + return snaps + .map((r) => ({ ...r, outcome: outcomeBy.get(propKey(r)) || null })) + .filter((r) => r.outcome === 'hit' || r.outcome === 'miss'); +} + +function build(rows) { + // One row per prop — the EARLIEST capture is the lock. Multiple snapshot + // cycles per day would otherwise weight a prop by how often it was re-graded. + const byProp = new Map(); + for (const r of rows) { + if ((r.quarantine_reason || '').startsWith('nontakeable_book')) continue; + const k = `${r.player_key}|${r.stat}|${r.line}|${r.side}|${r.game_date}`; + const prev = byProp.get(k); + if (!prev || String(r.captured_at) < String(prev.captured_at)) byProp.set(k, r); + } + + const out = []; + let clamped = 0; + for (const r of byProp.values()) { + const f = r.features || {}; + const pWin = num(r.p_win); + if (pWin == null) continue; + const under = String(r.side || '').toLowerCase() === 'under'; + const pOver = under ? 1 - pWin : pWin; + + // Clamped → the pre-clamp value is unrecoverable. Excluded, counted. + if (pOver <= PROB_FLOOR + 1e-9 || pOver >= PROB_CEIL - 1e-9) { clamped += 1; continue; } + + const rank = num(f.opp_rank_stat); + const oppAdj = rank == null ? 0 : (rank >= 0.70 ? 0.03 : rank <= 0.30 ? -0.03 : 0); + const ha = num(f.home_away); + const homeAdj = ha === 1 ? 0.015 : ha === 0 ? -0.015 : 0; + const sd = num(f.l10_stddev); const l20 = num(f.l20_avg); + const cv = (sd != null && sd > 0 && l20 != null && l20 > 0) ? sd / l20 : null; + const consistencyFired = cv != null && cv > CV_VOLATILE_THRESHOLD; + const k = consistencyFired ? 0.9 : 1; + + // p_over (unclamped) = f(weighted + oppAdj + homeAdj); f linear ⟹ exact removal. + const noOpp = pOver - k * oppAdj; + const noHome = pOver - k * homeAdj; + const noAdj = pOver - k * oppAdj - k * homeAdj; // = f(weighted) + // Removing the consistency pull: invert f on the whole thing. + const noCons = consistencyFired ? (pOver - 0.05) / 0.9 : pOver; + // base+recency block alone, with every adjustment off. + const weighted = consistencyFired ? (noAdj - 0.05) / 0.9 : noAdj; + + const flip = (p) => (under ? 1 - clamp(p) : clamp(p)); + const won = r.outcome === 'hit' ? 1 : 0; + out.push({ + stat: r.stat, + won, + full: pWin, + no_opp: flip(noOpp), + no_home: flip(noHome), + no_consistency: flip(noCons), + no_adjustments: flip(noAdj), + weighted_only: flip(weighted), + residual: won - pWin, + archetype: r.archetype || null, + // Features the champion NEVER reads — the missing-feature candidates. + l5_avg: num(f.l5_avg), l10_avg: num(f.l10_avg), l20_avg: num(f.l20_avg), + ab_per_game: num(f.ab_per_game), recent_ab_per_game: num(f.recent_ab_per_game), + opportunity_drift: num(f.opportunity_drift), rest_days: num(f.rest_days), + game_count_in_7d: num(f.game_count_in_7d), + park_h: num(f.park_h), park_hr: num(f.park_hr), park_r: num(f.park_r), + weather_temp_f: num(f.weather_temp_f), weather_wind_mph: num(f.weather_wind_mph), + weather_precip: num(f.weather_precip), + // Features it DOES read — controls for the same test. + opp_rank_stat: num(f.opp_rank_stat), home_away: num(f.home_away), + l10_stddev: num(f.l10_stddev), + }); + } + return { rows: out, clamped }; +} + +const ABLATIONS = [ + ['no_opp', 'opponent (opp_rank_stat, ±0.03)'], + ['no_home', 'home/away (±0.015)'], + ['no_consistency', 'consistency pull (cv>0.40 → toward 0.50)'], + ['no_adjustments', 'ALL THREE adjustments (leaves base+recency)'], +]; + +const UNUSED = ['l5_avg', 'l10_avg', 'l20_avg', 'ab_per_game', 'recent_ab_per_game', + 'opportunity_drift', 'rest_days', 'game_count_in_7d', 'park_h', 'park_hr', 'park_r', + 'weather_temp_f', 'weather_wind_mph', 'weather_precip']; +const USED = ['opp_rank_stat', 'home_away', 'l10_stddev']; + +function ablateStat(rows, label) { + const ys = rows.map((r) => r.won); + const full = r4(corr(rows.map((r) => r.full), ys)); + const abl = {}; + for (const [key, name] of ABLATIONS) { + const bs = bootstrapDiff(rows, key, 'full'); + abl[name] = { + resolution_without: r4(corr(rows.map((r) => r[key]), ys)), + // NEGATIVE delta = removing it HURT = the feature carries signal. + delta_from_removal: bs ? bs.point : null, + ci95: bs ? bs.ci95 : null, + carries_signal: bs ? (bs.ci_excludes_zero && bs.point < 0) : null, + }; + } + return { stat: label, n: rows.length, base_rate: r4(ys.reduce((a, b) => a + b, 0) / ys.length), resolution_full: full, ablations: abl }; +} + +function residualStat(rows, label) { + const scan = (keys, seedBase) => { + const out = {}; + keys.forEach((k, i) => { + const res = bootstrapCorr(rows, k, 20260804 + i + seedBase); + out[k] = res; + }); + return out; + }; + return { + stat: label, + n: rows.length, + unused_features: scan(UNUSED, 0), + used_features_control: scan(USED, 500), + }; +} + +async function main() { + if (!SB_URL || !SB_KEY) throw new Error('SUPABASE_URL / service key required'); + const sb = createClient(SB_URL, SB_KEY, { auth: { persistSession: false } }); + const { rows, clamped } = build(await fetchAll(sb)); + + const counts = {}; + for (const r of rows) counts[r.stat] = (counts[r.stat] || 0) + 1; + const stats = Object.entries(counts).filter(([, n]) => n >= 60).map(([s]) => s) + .sort((a, b) => counts[b] - counts[a]); + + const perStat = stats.map((s) => ablateStat(rows.filter((r) => r.stat === s), s)); + const residual = stats.map((s) => residualStat(rows.filter((r) => r.stat === s), s)); + + // ── ARCHETYPE ON TRIAL ──────────────────────────────────────────────── + // The champion reads NO archetype feature at all, so it cannot be ablated out + // of it. The fair test is whether archetype explains what the champion GETS + // WRONG: if a given archetype's rows are systematically mispriced, archetype + // carries prop signal the model is missing (wrong IMPLEMENTATION). If every + // archetype's mean residual straddles zero, archetype carries no prop signal. + const archetypeTest = stats.map((st) => { + const rs = rows.filter((r) => r.stat === st && r.archetype); + const groups = {}; + for (const r of rs) (groups[r.archetype] = groups[r.archetype] || []).push(r.residual); + const out = {}; + for (const [name, vals] of Object.entries(groups)) { + if (vals.length < 40) continue; + const rnd = makeRnd(20260805); + const means = []; + for (let it = 0; it < 3000; it += 1) { + let sum = 0; + for (let i = 0; i < vals.length; i += 1) sum += vals[Math.floor(rnd() * vals.length)]; + means.push(sum / vals.length); + } + means.sort((a, b) => a - b); + const ci = [r4(means[Math.floor(0.025 * (means.length - 1))]), r4(means[Math.floor(0.975 * (means.length - 1))])]; + out[name] = { + n: vals.length, + mean_residual: r4(vals.reduce((a, b) => a + b, 0) / vals.length), + ci95: ci, + systematically_mispriced: ci[0] > 0 || ci[1] < 0, + }; + } + return { stat: st, archetypes: out }; + }); + + // Any unused feature with a CI excluding zero, anywhere → a missing-feature lead. + const leads = []; + for (const rs of residual) { + for (const [k, v] of Object.entries(rs.unused_features)) { + if (v.ci_excludes_zero) leads.push({ stat: rs.stat, feature: k, corr: v.corr, ci95: v.ci95, n: v.n }); + } + } + + console.log(JSON.stringify({ + measurement: 'EXACT ANALYTIC ABLATION of the champion, on the REPAIRED settled set. No refit, no re-fetch, no lookahead.', + limits: { + base_recency_not_separable: 'stored features hold AVERAGES, not frequencies-over-line; reported as one block', + clamped_rows_excluded: clamped, + }, + total_rows: rows.length, + per_stat_ablation: perStat, + residual_signal_test: residual, + archetype_test: archetypeTest, + multiple_comparisons_note: 'The residual scan runs 14 unused features x 5 stats = 70 tests at alpha .05, so ~3-4 CI-excludes-zero results are EXPECTED BY CHANCE. Treat a single hit as noise; only a feature repeating across independent stats is evidence.', + missing_feature_leads: leads, + }, null, 2)); + process.exit(0); +} + +main().catch((e) => { console.error(e); process.exit(1); }); diff --git a/specs/champion-input-diagnosis.md b/specs/champion-input-diagnosis.md new file mode 100644 index 0000000..023c40f --- /dev/null +++ b/specs/champion-input-diagnosis.md @@ -0,0 +1,220 @@ +# DECOMPOSING THE CHAMPION — where its edge actually comes from + +**Read-only diagnosis, 2026-08-03.** Nothing built, nothing touched. Run on the +**repaired** settled set (n=1,741 after the settlement outage fix), not the frozen +pre-fix set. + +> **VERDICT, one line:** the champion's entire edge is a **hit-rate counter**, its +> three adjustment layers contribute **nothing** (two are mildly harmful), and the +> largest recoverable loss is **not a missing feature — it is the `[0.10, 0.95]` +> clamp, which pins 20.6% of settled props to a constant** and hides outcomes +> ranging from 52% to 99.5% behind the same number `0.900`. + +--- + +## 1. What the champion actually uses (STEP 1) + +`src/services/intelligence/probabilityEstimator.js` is **five lines of arithmetic**: + +``` +base = empirical frequency of (stat > THIS line) over the game log +weighted = 0.6·base + 0.4·(same frequency over the last 5 games) +p = weighted + oppAdj(±0.03) + homeAdj(±0.015) +if cv > 0.40: p = 0.9·p + 0.05 (volatile → pull toward 0.50) +p_over = clamp(p, 0.10, 0.95) +p_win = side === 'under' ? 1 − p_over : p_over +``` + +It reads exactly **three** features: `opp_rank_stat`, `home_away`, and +`l10_stddev`/`l20_avg` (for cv). `featureCache` computes and retains a dozen more +— `park_h/hr/r`, `weather_temp_f/wind_mph/precip`, `rest_days`, +`opportunity_drift`, `ab_per_game`, `recent_ab_per_game`, `l5/l10/l20_avg`, +`game_count_in_7d` — and **p_win reads none of them.** + +## 2. Per-stat ablation (STEP 2) + +**Exact and analytic, not a refit.** Each adjustment is a closed-form function of +stored features and the consistency step is linear (`f(x)=0.9x+0.05` ⟹ +`f(a+b)=f(a)+0.9b`), so every layer is removed algebraically from the stored +`p_win`. Nothing re-estimated, nothing re-fetched, no lookahead possible. +Paired bootstrap, 3,000 resamples, deterministic seed. + +**A negative delta means removing the layer HURT — i.e. it carried signal.** + +| stat | n | resolution (full) | −opponent | −home/away | −consistency | **−ALL THREE** | +|---|---|---|---|---|---|---| +| hits | 578 | 0.1964 | −0.0049 | −0.0006 | −0.0004 | **−0.0059** [−0.0168,+0.0056] | +| total_bases | 284 | 0.2370 | −0.0056 | +0.0042 | +0.0002 | **−0.0015** [−0.0139,+0.0115] | +| rbi | 273 | 0.4986 | +0.0059 | **+0.0053** [+0.0002,+0.0103] | +0.0005 | **+0.0106** [−0.0003,+0.0220] | +| runs | 115 | 0.4062 | +0.0060 | +0.0087 | −0.0006 | **+0.0130** [−0.0106,+0.0366] | +| walks | 66 | 0.4776 | −0.0035 | +0.0055 | +0.0018 | **+0.0008** [−0.0268,+0.0281] | + +**Removing all three adjustments changes resolution by nothing on every stat, and +on rbi/runs it IMPROVES it.** Exactly one ablation anywhere has a CI excluding +zero — rbi home/away, and its sign says removing it makes the model **better**. + +**So ~100% of the champion's resolution is `base + recency`: how often this +player has cleared THIS number lately.** That is the whole model. Everything else +is decoration. + +### A correction to how we read last session's scoreboard + +Pooled across stats the champion resolves **0.46**; per stat it is **0.196 +(hits)** to **0.499 (rbi)**. Pooling stats with different base rates *inflates* +correlation, because p_win varies across stats in the same direction as the true +base rate. **0.46 is a pooling artifact and should not be quoted as the +champion's resolution.** The paired *differences* in the scoreboard remain valid +(champion and challenger were pooled identically); only the absolute level was +inflated. + +## 3. Do the challengers have it, or dilute it? (STEP 3) + +| challenger | uses the base-frequency signal? | verdict | +|---|---|---| +| **proj-v1.1 ladder** | **No — it replaces it.** Fits a rate + NB distribution instead of counting frequency at THIS line | **DILUTING.** Measured reliably worse (−0.0301, CI excludes 0). It discards the one thing that works in favour of a lossier route to the same question | +| **hits-v1** | No — same substitution, binomial instead of NB | **DILUTING.** Refuted (−0.022, CI excludes 0) | +| **arch-v1 · environment** | Adds park/weather, which the champion ignores | **DILUTING.** n=871, −0.0028, CI includes 0 — movement without information | +| **arch-v1 · opportunity** | Uses `opportunity_drift` — **the one feature with repeated residual signal** | **HAS THE FEATURE, WRONG IMPLEMENTATION** (see §4) | +| **arch-v1 · matchup** | — | STILL PENDING (rows settle after ET midnight) | +| **contact-v1** | Statcast contact quality; not in the retained vector | No evidence either way (n=1,055, CI includes 0) | + +## 4. The missing-feature test — one real lead, already in our hands + +Correlation of each **unused** feature with the champion's residual (`won − +p_win`), per stat, bootstrap CI. + +**Multiple-comparisons discipline first:** 14 features × 5 stats = 70 tests at +α=.05, so **3–4 CI-excludes-zero results are expected by chance.** Six appeared. +A single hit is noise. **Only a feature that repeats across independent stats is +evidence** — and exactly one does: + +| feature | hits | total_bases | walks | +|---|---|---|---| +| **`opportunity_drift`** | **+0.156** [+0.007,+0.292] | **+0.145** [+0.005,+0.278] | −0.261 [−0.463,−0.017] | +| `recent_ab_per_game` (same quantity) | +0.052 | **+0.136** [+0.002,+0.272] | −0.157 | + +`opportunity_drift` = recent at-bats ÷ season at-bats-per-game. It is the one +axis a frequency counter is **structurally blind to**: `base` knows how often he +cleared the number, not that he has moved from 8th in the order to leadoff, or +back from injury on a bench role. Sign flips on walks (n=47, and walks scale with +plate appearances differently) — so this is stat-specific, which is doctrine- +consistent, not a contradiction. + +**But we already compute it, retain it, and built an axis on it — and that axis +extracts nothing** (opportunity axis: n=539, delta +0.0001, CI [−0.0091,+0.0090]). +So this is **not** "go get a new feature." It is **"the feature has signal and our +implementation of it is wrong"** — arch-v1 applies it as a small multiplicative +nudge to `p_win`, which is not how you use an opportunity term. Opportunity should +scale the *rate*, before the frequency question is asked. + +Weather on total_bases (`wind_mph` −0.164, `precip` −0.154) appears on **one stat +only** and sits inside the expected false-positive count. Recorded as a +non-lead unless it repeats. + +## 5. Archetype on trial (STEP 4 item) — NO EVIDENCE, and the test is underpowered + +The champion reads **no archetype feature at all**, so archetype cannot be ablated +out of it. The fair test is whether archetype explains what the champion gets +*wrong*: if an archetype's rows are systematically mispriced, archetype carries +prop signal we're missing. + +| stat | archetype | n | mean residual | CI95 | mispriced? | +|---|---|---|---|---|---| +| hits | BOMBER | 195 | −0.038 | [−0.105, +0.030] | no | +| hits | GHOST | 85 | −0.076 | [−0.184, +0.033] | no | +| total_bases | BOMBER | 95 | −0.046 | [−0.141, +0.058] | no | +| total_bases | GHOST | 46 | −0.041 | [−0.187, +0.109] | no | +| rbi | BOMBER | 88 | −0.068 | [−0.155, +0.019] | no | +| runs | BOMBER | 41 | −0.012 | [−0.150, +0.121] | no | + +**Verdict: (a) no-signal is UNPROVEN and (b) wrong-implementation is UNPROVEN — +the test cannot separate them yet.** Only **2 of 41 archetypes** (BOMBER, GHOST) +reach n≥40 settled rows. That is not "archetypes don't work"; it is "we have not +measured them." Distinguishing (a) from (b) needs archetype coverage across more +than two labels. **Do not act on archetype in either direction on this evidence.** + +Worth noting: every archetype's mean residual is **negative**, which is not an +archetype effect — it is the global over-prediction in §6. + +## 6. THE BIGGEST FINDING — the clamp, not a feature + +**358 of 1,741 settled props (20.6%) sit ON the clamp boundary** (353 at the +floor). Within that fifth of the book the model emits a **constant**, so it cannot +rank those props at all — resolution there is zero by construction. + +And the constant is hiding two *opposite* failures at once: + +| stat · side | n | model says | actually wins | miscalibration | +|---|---|---|---|---| +| home_runs · under | 222 | 0.900 | **0.995** | **−9.5 pts** (badly UNDER-confident) | +| hits · under | 27 | 0.900 | **0.519** | **+38.1 pts** (a coin flip sold as 90%) | +| total_bases · under | 12 | 0.900 | 0.583 | +31.7 pts | +| rbi · over | 35 | 0.100 | 0.229 | −12.9 pts | + +**The same output `0.900` covers true probabilities from 52% to 99.5%.** A +near-lock and a coin flip are indistinguishable in the product. `PROB_CEIL = 0.95` +also makes it *impossible* to express the 99.5% case honestly. + +Overall calibration, all settled MLB rows: + +| stat | n | mean p_win | actual | over-prediction | +|---|---|---|---|---| +| **ALL POOLED** | 1,741 | 0.585 | 0.550 | **+3.5 pts** | +| total_bases | 297 | 0.534 | 0.458 | **+7.6** | +| hits | 589 | 0.606 | 0.562 | +4.4 | +| rbi | 320 | 0.399 | 0.356 | +4.3 | +| walks | 83 | 0.546 | 0.506 | +4.0 | +| runs | 124 | 0.596 | 0.605 | −0.9 (well calibrated) | +| home_runs | 228 | 0.899 | 0.996 | **−9.6** | + +Per the product doctrine, calibration is **half** the success criterion — "does +60% mean 60%?" Right now 58.5% means 55.0%, and on total_bases 53.4% means 45.8%. +**This is fixable with no new data at all.** + +## 7. VERDICT PER STAT (STEP 4) + +| stat | n | resolution | verdict | +|---|---|---|---| +| **hits** | 578 | 0.196 | **DILUTION + CALIBRATION.** Adjustments contribute nothing; one real lead (`opportunity_drift`) that we already compute and implement wrongly; +4.4pt over-prediction | +| **total_bases** | 284 | 0.237 | **DILUTION + CALIBRATION (worst).** Same lead; +7.6pt over-prediction | +| **rbi** | 273 | 0.499 | **DILUTION.** Removing home/away *improves* it (CI excludes zero). Prune | +| **runs** | 115 | 0.406 | **AT CEILING.** Adjustments neutral-to-harmful, no residual leads. The model is good here | +| **walks** | 66 | 0.478 | **AT CEILING** (underpowered, n=66) | +| **home_runs** | 228 | n/a — entirely clamped | **CALIBRATION.** −9.6pts, structurally uncorrectable while `PROB_CEIL=0.95` | + +**AT CEILING is a real result here, not a shrug**: on runs and walks the champion +already resolves ~0.41–0.48 and nothing we compute explains its residual. + +## 8. What this says about the next order + +The convergent evidence was read correctly — the problem *is* inputs, not shape. +But the decomposition sharpens it, and the ranking is not what we assumed: + +1. **The clamp + calibration (biggest, cheapest, no new data).** 20.6% of the + book pinned to a constant, a documented +3.5pt global over-prediction, and one + number covering 52%–99.5%. This costs both halves of the success criterion. +2. **Opportunity, implemented properly** — the one feature with repeated + cross-stat residual signal. Not a new feature: a correct use of one we have. + Scale the rate, don't nudge the probability. +3. **Stop diluting** — the ladder and the environment axis add movement with no + information, and are measurably worse or flat. +4. **Archetype: measure before judging.** 2 of 41 labels have testable n. + +**Explicitly NOT recommended:** another projection variant. That is the sixth +thing, and this diagnosis is why it would fail — the champion's edge is asking +the frequency question at the traded line, and every challenger so far has +replaced that question rather than improved its inputs. + +## 9. Incidental finding, flagged not fixed + +**`model_snapshots.outcome` is NULL on all 22,032 rows.** The retention table +built expressly so "a different model can be replayed against the same +conditions" stores features but was **never settled**, so it cannot answer the +question it exists for. This diagnosis worked around it by joining outcomes from +`ledger_entries` on (player_key, stat, line, side, game_date). Settling retention +would make every future ablation a single-table query — and would make refusals +(which the ledger drops) measurable for the first time. + +## 10. Reproduce + +`SUPABASE_URL=... node scripts/champion-ablation.js`