diff --git a/BUILD-STATE.md b/BUILD-STATE.md index 1cbe306..5e22828 100755 --- a/BUILD-STATE.md +++ b/BUILD-STATE.md @@ -1,7 +1,44 @@ # VYNDR — Build State ## Last Updated -2026-07-12 +2026-08-02 + +## Session 76 (2026-08-02) — hits-v1: a challenger built, measured, and REFUTED ✅ +Spec: `specs/hits-v1-binomial.md`. 4,156 tests / 332 suites green, web build exit 0. +Scope was hits only; champion, ladder, ranking, calibration, reference ruler and +the four accruing challenger verdicts are byte-identical (the diff has ZERO +deleted lines). +- **What was built.** `src/services/projection/binomialHits.js` — hits as a + bounded conversion: `N ~ the player's empirical at-bat distribution`, + `hits | N ~ Binomial(N, q)`. At the 0.5 line (84% of real hits rows) this + states `P(>=1) = 1 − E[(1−q)^N]` directly instead of inferring P(0) from a + count family. The multiplier scales `q` (conversion), never `N` (opportunity). + Wired in `projectionChallenger` as `proj_hits_p_over` / `proj_hits_meta` + (new ledger columns, migration applied). +- **STEP 0 first — inputs before model.** `scripts/hits-input-coverage.js`: + 30/30 real ledger players, 100% combined-input coverage, mean 3.518 AB/G, + mean per-AB rate 0.248. +- **FIRING, on the real board.** `scripts/verify-hits-v1.js` runs the production + `attachProjection` over the live prod snapshot: 158/159 hits props (99.4%), one + honest abstention. 94 of 159 props sit OUTSIDE the promotion band and 93 were + modelled anyway — scoping by book identity kept 59% of the board a price-shape + rule would have deleted. +- **AND IT LOST.** Point-in-time replay (log truncated strictly before each row's + game_date, real grade-time multiplier), hits-only, direction-aligned, n=242: + resolution champion **0.195** / ladder **0.048** / hits-v1 **0.026**. Paired + bootstrap: hits-v1 − ladder = −0.022, CI95 [−0.046, −0.0003]. NOT PROMOTED. +- **The finding is what it eliminates.** Family was wrong AND mean was not the + constraint (hits-v1 moved the line-0.5 mean 0.554→0.581 toward a 0.598 base + rate while resolution FELL). The hits deficit is per-prop DISCRIMINATION — the + ladder's inputs, not its distribution. +- **A pre-registered branch recorded as WRONG.** The spec's fallback ("hits may + be genuinely low-resolution for anyone") is refuted by the champion scoring + 0.276 on the identical 189 rows. Kept in the doc rather than deleted. +- **Next order is a DIAGNOSIS, not a model:** what does the champion's `p_win` + read on a hits prop that the projection ladder does not? Building another + projection variant first would repeat this session's mistake. +- Logged not fixed: local `.env` has a transposed Supabase ref — local scripts + need `SUPABASE_URL=` override; prod unaffected. ## Session S11 (a1 board, 2026-07-12) — Live Tracking: the read locked, the game watched ✅ Spec: `specs/LIVE-TRACKING.md` (+ ROW-GRAMMAR §2/§3 S11 amendment). diff --git a/CLAUDE.md b/CLAUDE.md index ce5439c..393c3f9 100755 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -1096,6 +1096,42 @@ phased plan in the Session-57 conversation / BUILD-STATE Next section). Redis runs degraded locally, so the script must `process.exit(0)` — otherwise a reconnect timer holds the process open and piped output is lost to SIGTERM. +## hits-v1 — a REFUTED challenger, and why it stays (Session 76 — non-obvious) +- **`specs/hits-v1-binomial.md` is the record.** hits-v1 models hits as a + binomial over the player's EMPIRICAL at-bat distribution (P(0) stated directly, + since 84% of hits rows trade at 0.5). It FIRES at 99.4% on the live board and + it DOES NOT WORK: point-in-time replay, hits-only, direction-aligned, n=242 — + resolution champion 0.195 / ladder 0.048 / hits-v1 0.026. Paired bootstrap + (same rows) puts hits-v1 − ladder at −0.022, CI95 excluding 0. NOT PROMOTED. +- **Two explanations are now ELIMINATED for hits, which is the useful part.** + The family was wrong (swapping it made things slightly worse) AND the mean was + not the constraint (hits-v1 moved the line-0.5 mean 0.554→0.581 against a 0.598 + base rate — closer — while resolution FELL). What remains is per-prop + DISCRIMINATION: the ladder's inputs don't separate hitters. Don't build another + projection variant for hits; diagnose what the champion's `p_win` reads first. +- **"Honest ceiling" needs its control checked before you claim it.** The spec's + own pre-registered fallback ("hits may be genuinely low-resolution for anyone") + was REFUTED by the champion scoring 0.276 on the identical 189 rows. A ceiling + claim is only honest if no instrument on the same rows beats it — check that + BEFORE writing the branch, not after. +- **Backtest ≠ verdict.** The replay truncates each player's game log strictly + BEFORE the row's `game_date` and reuses the row's stored grade-time + `combined_multiplier` (both live on real ledger rows) — without that truncation + it would be scoring predictions with the answer in hand. The verdict of record + is still the forward accrual, so hits-v1 stays wired, writing + `proj_hits_p_over`/`proj_hits_meta` only. Never served. +- **Paired bootstrap, not two independent SEs.** Challengers score the SAME rows; + comparing independent standard errors overstates uncertainty and would have + read a reliable −0.022 regression as noise. `scripts/hits-v1-holdout.js` has + the seeded implementation — reuse it for the next challenger. +- **The takeable axis paid off measurably:** 94 of 159 live hits props are + OUTSIDE the promotion band and 93 were modelled anyway. Scope by + `isTakeableMarket` (book identity); record `isWithinPromotionBand` and never + let it gate a model — a price-shape rule would have deleted 59% of the board. +- **Local `.env` has a transposed Supabase ref** (`zmdnczhtdxcddszxttub`; real is + `zmdnczhtdxcddsxzttub`), so local scripts hitting Supabase need an explicit + `SUPABASE_URL=` override. Prod + the MCP connection are fine. + ## Active Skills - vyndr-voice (all user-facing output) - prop-analysis (grading methodology) diff --git a/scripts/hits-input-coverage.js b/scripts/hits-input-coverage.js new file mode 100644 index 0000000..52bddc5 --- /dev/null +++ b/scripts/hits-input-coverage.js @@ -0,0 +1,102 @@ +#!/usr/bin/env node +'use strict'; + +/** + * hits-input-coverage — STEP 0 of hits-v1. CONFIRM THE INPUTS EXIST. + * + * hits-v1 models P(hits >= line) as a binomial over AT-BATS. That needs two + * inputs per player that the current negative-binomial ladder never asked for: + * + * 1. at-bats per game (the OPPORTUNITY count — the binomial's n) + * 2. per-AB hit rate (the CONVERSION rate — the binomial's q) + * + * A model is not "wired" until its inputs are present on real rows at real + * coverage. This probe pulls the actual players carrying hits props in the + * ledger, fetches their REAL statsapi game logs, and reports what fraction + * yields usable AB + hit-rate inputs. + * + * UNKNOWN IS NOT ZERO. Every read goes through `knownRate`. A game-log row with + * no atBats field is COUNTED AS MISSING, never as a 0-AB game — reading it as + * zero would say "this player had no opportunity", the strongest possible + * statement, from an absence of data. That is the defect this codebase has + * shipped seven times. + * + * Usage: node scripts/hits-input-coverage.js [limit] + */ + +const { knownRate } = require('../src/utils/known'); +const mlb = require('../src/services/adapters/mlbStatsAdapter'); + +// The real players carrying hits props in the public ledger (2026-08-02 pull, +// ordered by row count). Hard-coded rather than re-queried so the probe runs +// without Supabase credentials — these are REAL names off REAL rows. +const PLAYERS = [ + 'Esmerlyn Valdez', 'Trea Turner', 'Ryan Jeffers', 'Steven Kwan', 'Jake Mangum', + 'Jazz Chisholm Jr', 'JT Realmuto', 'Bo Bichette', 'Ben Rice', 'Brandon Lowe', + 'Nick Gonzales', 'Alan Roden', 'Junior Caminero', 'Bryce Harper', 'Travis Bazzana', + 'Jasson Dominguez', 'Trent Grisham', 'Chase DeLauter', 'Jorge Polanco', 'Wyatt Langford', + 'Petey Halpin', 'Alec Bohm', 'Munetaka Murakami', 'Kyle Schwarber', 'AJ Ewing', + 'Royce Lewis', 'Javier Sanoja', 'Ben Williamson', 'Bryson Stott', 'Francisco Lindor', +]; + +const MIN_GAMES = Number(process.env.HITS_MIN_GAMES || 5); + +async function main() { + const limit = Number(process.argv[2] || PLAYERS.length); + const names = PLAYERS.slice(0, limit); + + const report = []; + for (const name of names) { + const row = { player: name, resolved: false, games: 0, ab_games: 0, hit_games: 0, ab_per_game: null, hit_rate: null, usable: false }; + try { + const found = await mlb.searchPlayer(name); + if (!found || !found.id) { report.push(row); continue; } + row.resolved = true; + const log = await mlb.getPlayerGameLog(found.id); + row.games = (log || []).length; + + let abSum = 0; let hSum = 0; let abGames = 0; let hitGames = 0; + for (const g of log || []) { + const s = (g && g.stat) || {}; + const ab = knownRate(s.atBats); // absent -> null, NOT 0 + const h = knownRate(s.hits); + if (ab !== null) { abSum += ab; abGames += 1; } + if (h !== null) { hSum += h; hitGames += 1; } + } + row.ab_games = abGames; + row.hit_games = hitGames; + if (abGames >= MIN_GAMES && abSum > 0) { + row.ab_per_game = Math.round((abSum / abGames) * 1000) / 1000; + row.hit_rate = Math.round((hSum / abSum) * 1000) / 1000; + row.usable = true; + } + } catch (e) { + row.error = e.message; + } + report.push(row); + } + + const resolved = report.filter((r) => r.resolved).length; + const usable = report.filter((r) => r.usable).length; + const rates = report.filter((r) => r.usable).map((r) => r.hit_rate); + const abs = report.filter((r) => r.usable).map((r) => r.ab_per_game); + const avg = (a) => (a.length ? Math.round((a.reduce((x, y) => x + y, 0) / a.length) * 1000) / 1000 : null); + + console.log(JSON.stringify({ + probed: report.length, + resolved, + usable_combined_inputs: usable, + coverage_pct: Math.round((usable / report.length) * 1000) / 10, + min_games_required: MIN_GAMES, + mean_ab_per_game: avg(abs), + mean_hit_rate_per_ab: avg(rates), + hit_rate_range: rates.length ? [Math.min(...rates), Math.max(...rates)] : null, + rows: report, + }, null, 2)); + + // Redis runs degraded locally; a reconnect timer would hold the process open + // and piped output would be lost to SIGTERM. Same rule as verify-grade-range. + process.exit(0); +} + +main().catch((e) => { console.error(e); process.exit(1); }); diff --git a/scripts/hits-v1-holdout.js b/scripts/hits-v1-holdout.js new file mode 100644 index 0000000..3a6864d --- /dev/null +++ b/scripts/hits-v1-holdout.js @@ -0,0 +1,235 @@ +#!/usr/bin/env node +'use strict'; + +/** + * hits-v1-holdout — STEP 3. POINT-IN-TIME REPLAY, HITS ROWS ONLY. + * + * THREE guards this script exists to enforce, all of which have burned a + * measurement in this codebase before: + * + * 1. HITS ROWS ONLY. Averaging hits into the other stats would hide the + * effect entirely — hits is one stat among nine and the ladder's failure is + * specific to it. + * + * 2. DIRECTION-ALIGNED. `p_win` is P(GRADED SIDE); `proj_p_over_line` and + * `proj_hits_p_over` are P(OVER). 26% of matched hits rows are + * under-graded, and comparing a raw P(over) against an under-side outcome + * measures the model BACKWARDS. That artifact alone accounted for 41% of + * the ladder's apparent loss when it was first measured. + * + * 3. NO LOOKAHEAD. This is the guard specific to a replay. For each settled + * row, the player's game log is rebuilt STRICTLY BEFORE that row's + * game_date, and the multiplier is the REAL `combined_multiplier` recorded + * on the row at grade time. A replay that used today's full log would be + * scoring a prediction with the answer in hand — a fabricated result, and + * a worse lie than no measurement. + * + * CONTAMINATION EXCLUSION (mandatory). Rows whose price/book were stamped from a + * NON-TAKEABLE book between 2026-08-01 and the write-path fix are tagged + * `quarantine_reason LIKE 'nontakeable_book%'` and are EXCLUDED: their locked + * price describes a market you could not have bet. + * + * WHAT THIS IS AND IS NOT. It is a backtest, and it is labelled one. The verdict + * of record is the FORWARD ledger accrual, which starts at the next snapshot. + * Stated limits: statsapi is read as it stands today (retroactive stat + * corrections are invisible), and LEAGUE_HIT_RATE / PRIOR_AB are constants set + * today — at 20 at-bats against a regular's 200–400 the prior moves a settled + * hitter by thousandths, but it is not zero. + * + * node scripts/hits-v1-holdout.js + */ + +require('dotenv').config(); +const { createClient } = require('@supabase/supabase-js'); +const binomialHits = require('../src/services/projection/binomialHits'); +const mlb = require('../src/services/adapters/mlbStatsAdapter'); +const { knownNumber } = require('../src/utils/known'); +const { normalizeName } = require('../src/utils/playerName'); + +const SB_URL = process.env.SUPABASE_URL; +const SB_KEY = process.env.SUPABASE_SERVICE_ROLE_KEY || process.env.SUPABASE_SERVICE_KEY; + +/** Pearson correlation — the resolution measure the ladder is judged on. */ +function corr(xs, ys) { + const n = xs.length; + if (n < 3) return null; + const mx = xs.reduce((a, b) => a + b, 0) / n; + const my = ys.reduce((a, b) => a + b, 0) / n; + let sxy = 0; let sxx = 0; let syy = 0; + for (let i = 0; i < n; i += 1) { + const dx = xs[i] - mx; const dy = ys[i] - my; + sxy += dx * dy; sxx += dx * dx; syy += dy * dy; + } + if (sxx <= 0 || syy <= 0) return null; + return Math.round((sxy / Math.sqrt(sxx * syy)) * 10000) / 10000; +} +const mean = (a) => (a.length ? Math.round((a.reduce((x, y) => x + y, 0) / a.length) * 10000) / 10000 : null); +const sd = (a) => { + if (a.length < 2) return null; + const m = a.reduce((x, y) => x + y, 0) / a.length; + return Math.round(Math.sqrt(a.reduce((s, v) => s + (v - m) ** 2, 0) / (a.length - 1)) * 10000) / 10000; +}; +/** Brier score — lower is better. Reported beside resolution as a check. */ +const brier = (ps, ys) => (ps.length + ? Math.round((ps.reduce((s, p, i) => s + (p - ys[i]) ** 2, 0) / ps.length) * 10000) / 10000 + : null); + +/** + * Paired bootstrap CI on a DIFFERENCE of resolutions. + * + * Both models score the SAME rows, so their errors are correlated and comparing + * two independent standard errors would overstate the uncertainty. Resampling + * rows as pairs preserves that dependence. Deterministic seed — a measurement + * that changes between runs is not a measurement. + */ +function bootstrapDiff(rowsIn, keyA, keyB, iters = 4000, seed = 20260802) { + if (rowsIn.length < 20) return null; + let s = seed >>> 0; + const rnd = () => { // xorshift32 — deterministic, no Math.random + s ^= s << 13; s >>>= 0; s ^= s >>> 17; s ^= s << 5; s >>>= 0; + return s / 4294967296; + }; + const n = rowsIn.length; + const diffs = []; + for (let it = 0; it < iters; it += 1) { + const ys = []; const a = []; const bArr = []; + for (let i = 0; i < n; i += 1) { + const r = rowsIn[Math.floor(rnd() * n)]; + ys.push(r.won); a.push(r[keyA]); bArr.push(r[keyB]); + } + const ca = corr(a, ys); const cb = corr(bArr, ys); + if (ca == null || cb == null) continue; + diffs.push(ca - cb); + } + if (diffs.length < 100) return null; + diffs.sort((x, y) => x - y); + const q = (p) => Math.round(diffs[Math.floor(p * (diffs.length - 1))] * 10000) / 10000; + return { + point: Math.round(((corr(rowsIn.map((r) => r[keyA]), rowsIn.map((r) => r.won)) || 0) + - (corr(rowsIn.map((r) => r[keyB]), rowsIn.map((r) => r.won)) || 0)) * 10000) / 10000, + ci95: [q(0.025), q(0.975)], + // The question the promotion gate actually asks. + p_improves: Math.round((diffs.filter((d) => d > 0).length / diffs.length) * 1000) / 1000, + }; +} + +async function main() { + if (!SB_URL || !SB_KEY) throw new Error('SUPABASE_URL / service key required'); + const sb = createClient(SB_URL, SB_KEY, { auth: { persistSession: false } }); + + const { data, error } = await sb + .from('ledger_entries') + .select('id, player_name, player_key, stat, line, side, outcome, game_date, p_win, proj_p_over_line, proj_factors, quarantine_reason') + .eq('sport', 'mlb') + .is('user_id', null) + .eq('stat', 'hits') + .in('outcome', ['hit', 'miss']) + .not('p_win', 'is', null) + .not('proj_p_over_line', 'is', null); + if (error) throw error; + + // Contamination exclusion, applied in JS so the filter is visible here. + const rows = (data || []).filter((r) => !(r.quarantine_reason || '').startsWith('nontakeable_book')); + + // Resolve each distinct player ONCE, and cache the full game log. + const logCache = new Map(); + const names = [...new Set(rows.map((r) => r.player_name).filter(Boolean))]; + let resolved = 0; + for (const name of names) { + try { + const found = await mlb.searchPlayer(name); + if (!found || !found.id) { logCache.set(name, null); continue; } + const log = await mlb.getPlayerGameLog(found.id); + logCache.set(name, Array.isArray(log) ? log : null); + if (log && log.length) resolved += 1; + } catch { logCache.set(name, null); } + } + + const out = []; + const reasons = {}; + const bump = (k) => { reasons[k] = (reasons[k] || 0) + 1; }; + + for (const r of rows) { + const full = logCache.get(r.player_name); + if (!full) { bump('no_game_log'); continue; } + const gameDate = String(r.game_date || '').slice(0, 10); + if (!gameDate) { bump('no_game_date'); continue; } + + // ── NO LOOKAHEAD ────────────────────────────────────────────────────── + // Strictly BEFORE the graded game. A row dated the same day is the game + // being predicted; including it would hand the model the answer. + const priorLog = full.filter((g) => g && g.date && String(g.date).slice(0, 10) < gameDate); + if (priorLog.length < binomialHits.HITS_MIN_GAMES) { bump('thin_prior_log'); continue; } + + // The REAL grade-time multiplier, recorded on the row at lock. + const m = knownNumber(r.proj_factors && r.proj_factors.combined_multiplier); + const proj = binomialHits.projectHits({ + rows: priorLog, line: Number(r.line), multiplier: m == null ? 1 : m, + }); + if (!proj) { bump('inputs_underivable'); continue; } + + // ── DIRECTION-ALIGN to the graded side ──────────────────────────────── + const under = String(r.side || '').toLowerCase() === 'under'; + const won = r.outcome === 'hit' ? 1 : 0; + out.push({ + won, + under, + champ: Number(r.p_win), + ladder: under ? 1 - Number(r.proj_p_over_line) : Number(r.proj_p_over_line), + hitsv1: under ? 1 - proj.p_over_line : proj.p_over_line, + line: Number(r.line), + games_prior: priorLog.length, + }); + } + + const slice = (rowsIn, label) => { + const ys = rowsIn.map((x) => x.won); + const c = rowsIn.map((x) => x.champ); + const l = rowsIn.map((x) => x.ladder); + const h = rowsIn.map((x) => x.hitsv1); + return { + slice: label, + n: rowsIn.length, + under_rows: rowsIn.filter((x) => x.under).length, + base_rate: mean(ys), + resolution: { champion: corr(c, ys), current_ladder: corr(l, ys), hits_v1: corr(h, ys) }, + brier: { champion: brier(c, ys), current_ladder: brier(l, ys), hits_v1: brier(h, ys) }, + mean_p: { champion: mean(c), current_ladder: mean(l), hits_v1: mean(h) }, + sd_p: { champion: sd(c), current_ladder: sd(l), hits_v1: sd(h) }, + }; + }; + + console.log(JSON.stringify({ + measurement: 'POINT-IN-TIME REPLAY (backtest) — verdict of record is the forward ledger accrual', + guards: { + hits_rows_only: true, + direction_aligned: true, + no_lookahead: 'game log truncated strictly before each row game_date', + grade_time_multiplier: 'real combined_multiplier from the row', + contamination_excluded: 'nontakeable_book*', + }, + candidate_rows: rows.length, + players_resolved: `${resolved}/${names.length}`, + matched_rows: out.length, + dropped: reasons, + overall: slice(out, 'all hits rows'), + // Is the comparison a RESULT or a noise reading? Paired bootstrap, so the + // shared rows are not double-counted as independent evidence. + paired_bootstrap: { + note: 'difference in resolution, 4000 paired resamples, deterministic seed', + hits_v1_minus_ladder: bootstrapDiff(out, 'hitsv1', 'ladder'), + champion_minus_ladder: bootstrapDiff(out, 'champ', 'ladder'), + champion_minus_hits_v1: bootstrapDiff(out, 'champ', 'hitsv1'), + at_line_0_5: { + hits_v1_minus_ladder: bootstrapDiff(out.filter((x) => x.line === 0.5), 'hitsv1', 'ladder'), + champion_minus_ladder: bootstrapDiff(out.filter((x) => x.line === 0.5), 'champ', 'ladder'), + }, + }, + by_line: [0.5, 1.5, 2.5].map((ln) => slice(out.filter((x) => x.line === ln), `line ${ln}`)) + .filter((s) => s.n > 0), + }, null, 2)); + + process.exit(0); +} + +main().catch((e) => { console.error(e); process.exit(1); }); diff --git a/scripts/verify-hits-v1.js b/scripts/verify-hits-v1.js new file mode 100644 index 0000000..deaf1a0 --- /dev/null +++ b/scripts/verify-hits-v1.js @@ -0,0 +1,120 @@ +#!/usr/bin/env node +'use strict'; + +/** + * verify-hits-v1 — STEP 2. WIRED IS NOT FIRING. + * + * A challenger that exists in the source and never produces a number on a real + * row is not a challenger, it is a comment. This induces the REAL code path — + * `projectionChallenger.attachProjection`, the exact function the snapshot + * calls — over the REAL hits props on the live production snapshot, with the + * REAL statsapi game-log adapter behind it. + * + * It reports FIRING COVERAGE: of the real hits props on the board, how many + * yield a hits-v1 read, how many abstain, and why. An abstention is a valid + * answer; a silent zero is not. + * + * The current ladder value is computed on the same rows in the same call, so the + * two are compared on identical inputs. + * + * node scripts/verify-hits-v1.js [snapshotUrl] + */ + +const projection = require('../src/services/projectionChallenger'); +const mlb = require('../src/services/adapters/mlbStatsAdapter'); + +const SNAPSHOT_URL = process.argv[2] || 'https://api.vyndr.app/api/snapshot/mlb'; + +async function fetchSnapshot(url) { + const res = await fetch(url, { headers: { accept: 'application/json' } }); + if (!res.ok) throw new Error(`snapshot ${res.status}`); + return res.json(); +} + +async function main() { + const snap = await fetchSnapshot(SNAPSHOT_URL); + const grades = (snap.grades || []).filter( + (g) => String(g.stat_type || g.stat || '').toLowerCase() === 'hits', + ); + + const out = await projection.attachProjection(grades, { + // The one dep that matters here. Everything else (park/weather/platoon/ + // arsenal) is absent on the public payload and contributes a documented + // 1.0 — which is the honest behaviour, not a fabricated push. + gameLogFor: async (g) => { + if (!g.playerId) return []; + try { return (await mlb.getPlayerGameLog(g.playerId)) || []; } catch { return []; } + }, + }); + + const fired = out.filter((g) => g.proj_hits_p_over != null); + const abstained = out.filter((g) => g.proj_hits_p_over == null && g.proj_hits_meta); + const ladder = out.filter((g) => g.proj_p_over_line != null); + + // The market read — proving the model was scoped by IDENTITY, not by price. + const withMarket = out.filter((g) => g.proj_hits_meta && g.proj_hits_meta.market); + const takeableIdentity = withMarket.filter((g) => g.proj_hits_meta.market.market_takeable); + const outsidePromotion = withMarket.filter((g) => g.proj_hits_meta.market.within_promotion_band === false); + const oneSided = withMarket.filter((g) => g.proj_hits_meta.market.one_sided); + // The rows the whole disambiguation exists for: real markets that a + // price-shape rule would have thrown away, and which we modelled anyway. + const juicedModelled = fired.filter((g) => { + const m = g.proj_hits_meta.market; + return m.market_takeable && m.within_promotion_band === false; + }); + + const pct = (a, b) => (b ? Math.round((a / b) * 1000) / 10 : null); + const nums = fired.map((g) => g.proj_hits_p_over); + const avg = (a) => (a.length ? Math.round((a.reduce((x, y) => x + y, 0) / a.length) * 1000) / 1000 : null); + const sd = (a) => { + if (a.length < 2) return null; + const m = a.reduce((x, y) => x + y, 0) / a.length; + return Math.round(Math.sqrt(a.reduce((s, v) => s + (v - m) ** 2, 0) / (a.length - 1)) * 1000) / 1000; + }; + const ladderNums = ladder.map((g) => g.proj_p_over_line); + + console.log(JSON.stringify({ + snapshot: { url: SNAPSHOT_URL, updated_at: snap.updated_at, total_grades: (snap.grades || []).length }, + hits_props: grades.length, + hits_v1: { + fired: fired.length, + firing_coverage_pct: pct(fired.length, grades.length), + abstained: abstained.length, + abstain_reasons: abstained.reduce((acc, g) => { + const r = g.proj_hits_meta.reason || 'unknown'; + acc[r] = (acc[r] || 0) + 1; return acc; + }, {}), + mean_p: avg(nums), sd_p: sd(nums), + p_range: nums.length ? [Math.min(...nums), Math.max(...nums)] : null, + }, + current_ladder: { + fired: ladder.length, + mean_p: avg(ladderNums), sd_p: sd(ladderNums), + }, + takeable_axis: { + note: 'market scope = book IDENTITY; promotion band recorded, never gates the model', + rows_with_market_read: withMarket.length, + takeable_by_identity: takeableIdentity.length, + outside_promotion_band: outsidePromotion.length, + one_sided_quotes: oneSided.length, + juiced_or_longshot_MODELLED_anyway: juicedModelled.length, + any_price_filtered: withMarket.some((g) => g.proj_hits_meta.market.price_filtered), + }, + sample: fired.slice(0, 5).map((g) => ({ + player: g.player, line: g.line, side: g.direction, book: g.book, + hits_v1_p_over: g.proj_hits_p_over, + ladder_p_over: g.proj_p_over_line, + champion_p_win: g.p_win ?? null, + hit_rate: g.proj_hits_meta.hit_rate, + ab_per_game: g.proj_hits_meta.ab_per_game, + games_used: g.proj_hits_meta.games_used, + market: g.proj_hits_meta.market, + })), + }, null, 2)); + + // Redis is degraded locally; its reconnect timer would hold the process open + // and piped output would be lost to SIGTERM. Same rule as verify-grade-range. + process.exit(0); +} + +main().catch((e) => { console.error(e); process.exit(1); }); diff --git a/specs/hits-v1-binomial.md b/specs/hits-v1-binomial.md new file mode 100644 index 0000000..e56e6d1 --- /dev/null +++ b/specs/hits-v1-binomial.md @@ -0,0 +1,295 @@ +# hits-v1 — HITS AS A BINOMIAL OVER AT-BATS + +> **VERDICT: REFUTED. DO NOT PROMOTE.** The hypothesis was tested honestly and +> lost. §7 has the numbers, and §8 records where the pre-registration in §6 was +> itself wrong. hits-v1 stays wired as a challenger only, writing its own ledger +> columns, so the forward accrual can confirm or overturn the backtest. + +**Status:** built, firing (99.4% on the live board), holdout PRE-REGISTERED +(§1–§6 were written *before* the holdout was run) and RESOLVED (§7–§8). +**Scope:** `stat = 'hits'`, MLB, challenger only. Champion, current ladder, +ranking, calibration, the reference ruler and the four accruing challenger +verdicts are untouched and byte-identical. + +--- + +## 1. The diagnosis this challenger is built on + +Measured 2026-08-02 on the public ledger, hits rows only, direction-aligned, +contamination excluded: + +``` +n = 245 matched settled rows base rate 0.588 +resolution champion 0.2044 + current ladder (proj-v1.1) 0.0595 +``` + +The current ladder is not merely worse than the champion on hits — it is close to +uninformative. And the line distribution says why it matters so much: + +``` +line 0.5 842 of 1,006 hits rows (84%) +line 1.5 144 +line 2.5 20 +``` + +**84% of hits props are a single question: does he get one?** That makes the +whole stat ride on `P(0 hits)` — precisely where the choice of count family does +the most damage, because P(0) is the one point of a count distribution that is +most sensitive to its assumed shape. + +**This is the HYPOTHESIS, and it is on trial:** hits is being modelled by the +wrong family. It is not a low-rate event count. It is a *bounded conversion*. + +## 2. The structure hits actually has + +A hitter gets `N` official at-bats and converts each at some rate `q`: + +``` +N ~ the player's own EMPIRICAL at-bat distribution +hits | N ~ Binomial(N, q) +P(hits >= k) = Σ_n P(N=n) · P(Bin(n,q) >= k) +``` + +At the traded line of 0.5 this reduces to the axis that matters, stated directly +rather than inferred: + +``` +P(>=1 hit) = 1 − E_N[(1 − q)^N] +``` + +A negative binomial has **unbounded support and no notion of opportunity at +all**. It must infer from the count alone what are actually two separate things: +how many chances a hitter got, and how good he is. Two hitters with the same +mean hits and very different at-bat profiles are the same object to it. + +This is the same doctrine as tb-v1, one level deeper: model the stat by its +generative structure, not by a family that fits its name. Total bases is a +weighted **sum**; hits is a bounded **conversion**. Different wrongness, different +fix. + +### Two deliberate modelling choices + +1. **The at-bat distribution is EMPIRICAL, not a fitted family.** At-bats per + game are driven by lineup slot and how the game goes, and are strongly + *under*-dispersed (3, 4 or 5; almost never 0 or 9). Poisson would overstate + that spread badly. His own log *is* the distribution. + +2. **Zero-at-bat games are KEPT** — a conditioning argument. A statsapi game log + contains only games the player appeared in, so it is already conditioned on + appearing, which matches the settled population exactly: a prop on a player + who never appears produces no game-log row and never settles. An appearance + with 0 official at-bats (walked twice, pinch-ran) is a real outcome that + settles as 0 hits, and carrying it is a structural advantage over a count + model, which must infer that mass. + +**The multiplier moves `q`, not `N`.** Park × weather × platoon × matchup are all +effects on whether a batted ball becomes a hit; none of them changes how often a +hitter comes to the plate. Scaling the count mean, as the negative-binomial path +does, silently mixes opportunity with conversion. + +## 3. Inputs — confirmed present before the model was built (STEP 0) + +`scripts/hits-input-coverage.js`, run against the 30 real players carrying the +most hits props in the public ledger, pulling **real statsapi game logs**: + +``` +probed 30 · resolved 30 · usable combined inputs 30 · coverage 100% +mean at-bats/game 3.518 +mean per-AB hit rate 0.248 range 0.131 – 0.312 +``` + +Both binomial inputs exist at full coverage on real rows. **Unknown is not +zero:** every read goes through `knownRate`. A game-log row with no `atBats` +field is *dropped*, never counted as a 0-at-bat game — reading it as a measured +zero would assert "no opportunity", the strongest statement available, out of an +absence of data. That defect has shipped seven times in this codebase; the guard +is unit-tested here so it does not ship an eighth. + +## 4. Which takeable axis this reads, and why + +Three questions once shared one word (`src/config/takeability.js`). hits-v1 reads +two of them, for two different purposes, and never the deprecated `takeable` +mirror: + +| axis | question | used for | +|---|---|---| +| `isTakeableMarket(book)` | can a bettor place this? **book identity** | what is a real market to **MODEL** | +| `isWithinPromotionBand(odds)` | is it worth crowning? **price band** | recorded only; **never gates the model** | + +**No price-shape rule excludes a hits prop from being modelled.** Baseball hits +markets are genuinely thin, genuinely juiced and genuinely one-sided, and all +three are normal structure rather than a bad quote. A −300 hits-over from +BetRivers is `market_takeable: true, within_promotion_band: false` — both true at +once, and it gets modelled. Each row carries `market.price_filtered: false` as a +stated invariant, and the unit suite asserts the −300 one-sided case and the ++1300 longshot case are both modelled. + +## 5. Where it lives — additive, never substitutive + +- `src/services/projection/binomialHits.js` — the pure model. +- `src/services/projectionChallenger.js` — emits `proj_hits_p_over` + + `proj_hits_meta` beside `proj_p_over_line`, which is **untouched**. +- `ledger_entries.proj_hits_p_over` / `proj_hits_meta` — own columns, so it joins + to the same outcome and the same close as the champion and the ladder. +- NULL on non-hits props and on underivable inputs. Never a fabricated 0. + +## 6. THE HOLDOUT — pre-registered + +Two measurements, kept separate because they answer different questions: + +**(a) Point-in-time replay (available now, n≈245).** For each settled hits row, +rebuild the player's game log **strictly before that row's `game_date`** and +re-run hits-v1 with the row's **stored grade-time `combined_multiplier`**. No +lookahead: the log is truncated at the game being predicted, and the multiplier +is the real one recorded at lock. Hits rows only, direction-aligned to the over +basis. Reported: resolution of hits-v1 vs the current ladder vs the champion, on +matched rows, with n. + +*Stated limits of the replay:* game-log reconstruction uses statsapi as it reads +today (retroactive stat corrections would not be visible); `LEAGUE_HIT_RATE` +(0.248) and `PRIOR_AB` (20) are constants chosen today, though at 20 at-bats +against a regular's 200–400 the prior moves a settled hitter by thousandths. It +is a backtest, and it is labelled one. + +**(b) Forward ledger accrual.** hits-v1 writes at grade time from the next +snapshot onward and is judged by the same instrument as every other challenger. +This is the verdict of record. Hits is high-volume (1,006 rows, 405 settled), so +it accrues faster than tb-v1. + +### The branch, recorded before the result is known + +- **If hits resolution improves materially** → the family-mismatch mechanism + holds for hits as it did for total bases, and the per-stat structural doctrine + has a second confirmation. Promote hits-v1 for `stat='hits'` only. +- **If it does not improve** → the P(0)/family hypothesis is **wrong for hits**, + and the honest reading is that hits at a 0.5 line may be genuinely + low-resolution *for anyone*: 84% of the rows ask one near-coin-flip question at + a base rate of 0.588, and there may be little separable signal in it. That is an + **honest ceiling**, to be measured against what is achievable rather than + forced. It would NOT license rescaling, re-tuning to fit, or re-labelling. + Record it and say so. + +Promotion requires a **material** improvement in hits resolution, not a +directional one. No edge or CLV term is in this gate. + +--- + +## 7. THE RESULT (measured 2026-08-02, after §1–§6 were fixed) + +### Firing — `scripts/verify-hits-v1.js`, live production snapshot + +Real board (`api.vyndr.app/api/snapshot/mlb`, 428 grades, 159 hits props), run +through `attachProjection` — the exact function the snapshot calls — with the +real statsapi adapter behind it: + +``` +fired 158 / 159 firing coverage 99.4% +abstained 1 reason: inputs_underivable (an abstention, not a zero) +hits-v1 mean p 0.545 sd 0.124 range 0.210 – 0.728 +ladder mean p 0.523 sd 0.112 +``` + +**The takeable axis earned its keep.** Of 159 real hits props, **94 sit OUTSIDE +the promotion band** and **93 of those were modelled anyway**; `price_filtered` +is false on every row. A rule that inferred un-takeability from price shape would +have thrown away 59% of the live hits board. Every row was takeable by book +identity, which is the axis that decides what is a real market to model. + +### Holdout — `scripts/hits-v1-holdout.js`, point-in-time replay + +242 of 245 matched settled rows (3 dropped: thin prior log), 184/184 players +resolved, hits rows only, direction-aligned, game logs truncated strictly before +each row's game date, real grade-time multipliers. + +``` + RESOLUTION (corr with outcome) + overall n=242 line 0.5 n=189 +champion 0.1951 0.2760 +current ladder 0.0483 0.1439 +hits-v1 0.0259 0.0899 + base rate 0.5868 0.5979 +``` + +Paired bootstrap on the differences (4,000 resamples, same rows, deterministic +seed — the models are scored on identical rows, so treating their errors as +independent would overstate the uncertainty): + +``` +hits-v1 − ladder overall −0.0224 CI95 [−0.0460, −0.0003] P(improves) 0.023 +hits-v1 − ladder line 0.5 −0.0540 CI95 [−0.0933, −0.0197] P(improves) 0.001 +champion − ladder overall +0.1468 CI95 [+0.0423, +0.2495] P(improves) 0.997 +champion − hits-v1 overall +0.1692 CI95 [+0.0635, +0.2749] P(improves) 0.999 +``` + +**hits-v1 does not improve hits resolution. It is slightly and reliably worse** +— the confidence interval excludes zero at the traded line, where 84% of the +rows live. The promotion gate required a material improvement. It got a small +regression. **Not promoted.** + +## 8. WHAT THIS ELIMINATES — including one thing I pre-registered wrongly + +**The family-mismatch hypothesis is REFUTED for hits.** Modelling hits by its +true generative structure — a bounded conversion over at-bats, with P(0) stated +directly instead of inferred from a count family — moved resolution by nothing, +in the wrong direction. The mechanism that worked for total bases does not +transfer. The per-stat structural doctrine is not universal, and this is the case +that shows its limit. + +**The mean was not the binding constraint either, and that is now measurable.** +The working diagnosis was "it's the MEAN not the shape." hits-v1 corrected the +mean — at the 0.5 line the ladder averages 0.554 against a 0.598 base rate, and +hits-v1 averages 0.581, materially closer — **while resolution got worse**. So +both candidate explanations, the mean and the shape, are now eliminated. What is +left is per-prop DISCRIMINATION: the projection ladder's inputs do not separate +one hitter's night from another's, and no redistribution of the same inputs will +fix that. Both models are near-uninformative because they are reading nearly the +same thing. + +### The pre-registered fallback in §6 was WRONG, and I am recording it as wrong + +§6 said: if hits-v1 does not improve, "hits may be genuinely low-resolution at +0.5 — an honest ceiling for anyone." **The data refutes that fallback.** On the +identical 189 rows at the identical line, the champion resolves **0.276**, and +the paired bootstrap puts champion-over-ladder at +0.147 with a CI excluding +zero. There is real, separable signal in hits, and something in this codebase +already extracts it. Calling this an honest ceiling would have been the +comfortable reading, not the honest one. + +The error was mine and it was avoidable: the champion's number was already in +hand when I wrote the fallback, and I wrote a ceiling hypothesis that its own +control refuted. A pre-registration is only worth something if it is written +against the evidence already available. Recorded here rather than quietly +dropped, because a wrong branch that gets deleted teaches nothing. + +**The honest ceiling for hits is therefore not 0.06. It is at least 0.276 at the +traded line** — that is the achievable bar, measured, on a real instrument. The +projection ladder is not near it. + +## 9. WHAT STAYS, AND WHAT IS UNTOUCHED + +- hits-v1 **remains wired as a challenger**, writing `proj_hits_p_over` / + `proj_hits_meta` on its own columns. The replay is a backtest with stated + limits; the verdict of record is the forward accrual, and the cheapest way to + confirm a negative is to let it accrue. It is never served and never promoted. +- **Byte-identical, verified:** the diff against `projectionChallenger.js` and + `ledgerService.js` contains **zero deleted lines** — it is purely additive. The + champion (`p_win`), the current ladder (`proj_p_over_line`), the ranking, the + calibration, the reference ruler and the four accruing challenger verdicts + (arch-v1, contact-v1, proj-v1.1, tb-v1) are all untouched. +- Gates: full suite **332 suites / 4,156 tests green**; `web` build **exit 0**. + +## 10. OPEN — the next real question, NOT started here + +If the deficit is discrimination rather than distribution, the next order is +about INPUTS, and it should begin by asking what the champion's `p_win` reads on +a hits prop that the projection ladder does not. That is a diagnosis to run, not +a model to build — and building another projection variant before running it +would repeat exactly the mistake this document just recorded. + +**Unrelated finding, logged not fixed:** the local `.env` carries a transposed +Supabase project ref (`zmdnczhtdxcddszxttub`; the real ref is +`zmdnczhtdxcddsxzttub`), so any local script reading it fails DNS. Production is +unaffected — the deployed API and the MCP connection both use the correct ref. +The holdout was run with an explicit `SUPABASE_URL` override rather than editing +your credentials file. diff --git a/src/services/ledgerService.js b/src/services/ledgerService.js index dd26940..52beee0 100644 --- a/src/services/ledgerService.js +++ b/src/services/ledgerService.js @@ -335,6 +335,11 @@ function rowsFromSnapshot(sport, grades, oddsProps, nowIso) { // the components are underivable; never a fabricated 0. proj_tb_p_over: numOrNull(g.proj_tb_p_over), proj_tb_meta: g.proj_tb_meta || null, + // hits-v1 CHALLENGER — hits as a binomial over at-bats. Written alongside + // proj_p_over_line, never in place of it. NULL on non-hits props and when + // the at-bat inputs are underivable; never a fabricated 0. + proj_hits_p_over: numOrNull(g.proj_hits_p_over), + proj_hits_meta: g.proj_hits_meta || null, // Session 75 — the ENVIRONMENT that drove this projection. The FORECAST, // not the actual: this is what we knew when we projected, and it is what // the instrument measures. The actual lands in game_context and is never diff --git a/src/services/projection/binomialHits.js b/src/services/projection/binomialHits.js new file mode 100644 index 0000000..4469857 --- /dev/null +++ b/src/services/projection/binomialHits.js @@ -0,0 +1,260 @@ +'use strict'; + +/** + * binomialHits (hits-v1) — HITS MODELLED AS AN AT-BAT-BOUNDED BERNOULLI PROCESS. + * + * WHY THIS EXISTS. proj-v1.1 models every stat as a single negative binomial + * COUNT. That is right for genuine low-rate event counts (walks, runs, doubles — + * measured resolution 0.519 / 0.345 / 0.207) and wrong for hits, for a different + * reason than it was wrong for total bases. + * + * A hit is not a low-rate event drawn from an unbounded count process. It is a + * BOUNDED conversion: a hitter gets N official at-bats and converts each at some + * rate q. Hits can never exceed N. A negative binomial has unbounded support and + * no notion of opportunity at all, so it must infer from the count alone what is + * actually two separate things — how many chances he got, and how good he is. + * + * MEASURED, and this is the whole reason for the file. On 245 matched settled + * ledger rows (2026-08-02, direction-aligned, hits only): + * + * resolution — champion 0.2044 current ladder 0.0595 + * + * and 84% of all hits rows are traded at a line of 0.5. So hits is very nearly a + * single question — does he get ONE — and the answer rides almost entirely on + * P(0 hits), which is exactly where the choice of count family does the most + * damage. This is a HYPOTHESIS the ledger will judge, not a claim. + * + * THE MODEL. + * + * N ~ the player's own EMPIRICAL at-bat distribution (per appearance) + * hits | N ~ Binomial(N, q) + * P(hits >= k) = Σ_n P(N=n) · P(Bin(n, q) >= k) + * + * At the 0.5 line this reduces to the axis that matters: + * + * P(>=1 hit) = 1 − E_N[(1 − q)^N] + * + * TWO DELIBERATE CHOICES, both of which are the honest reading rather than the + * convenient one: + * + * 1. THE AT-BAT DISTRIBUTION IS EMPIRICAL, not a fitted family. A player's + * at-bats per game are driven by lineup slot and how the game goes, and they + * are strongly UNDER-dispersed (3, 4 or 5, almost never 0 or 9). Poisson would + * overstate that spread badly. His own log is the distribution; there is no + * reason to fit a shape to something we can simply count. + * + * 2. ZERO-AT-BAT GAMES ARE KEPT, and this is a conditioning argument. A statsapi + * game log contains only games the player APPEARED in, so the distribution is + * already conditioned on appearing — which matches the settled population + * exactly, since a prop on a player who never appears produces no game-log row + * and therefore never settles. An appearance with 0 official at-bats (walked + * twice, pinch-ran) is a REAL outcome that settles as 0 hits, and carrying it + * is a structural advantage over a count model, which has to infer that mass. + * + * WHAT THE MULTIPLIER MOVES. The park × weather × platoon × matchup product + * scales q, the CONVERSION rate — not N. Those four are all effects on whether a + * batted ball becomes a hit; none of them changes how many times a hitter comes + * to the plate (that is lineup slot and team offense, which we do not model + * here). Scaling the count mean, as the negative binomial path does, silently + * mixes the two. + * + * UNKNOWN IS NOT ZERO. Every read goes through `knownRate`. A game-log row with + * no `atBats` field is DROPPED, never counted as a 0-at-bat game — reading it as + * a measured zero would assert "no opportunity", the strongest statement + * available, out of an absence of data. That defect has shipped seven times in + * this codebase; it does not ship an eighth here. + * + * DOCTRINE: model the stat by its actual generative structure, not by a family + * that happens to fit its name. Same rule as tb-v1, different structure — + * total bases is a weighted SUM, hits is a BOUNDED CONVERSION. + */ + +const { knownRate } = require('../../utils/known'); + +const AB_CAP = 8; // official at-bats in one game; beyond is not real +const HITS_MIN_GAMES = 5; // below this the empirical AB shape is not a shape +const RECENT_WINDOW = 5; // mirrors the ladder's form window +const RECENT_WEIGHT = 2; // mirrors the ladder's 2x recency weight + +/** + * League per-at-bat hit rate, used ONLY as a shrinkage anchor. Measured + * 2026-08-02 across the 30 real players carrying hits props in the public + * ledger (scripts/hits-input-coverage.js): mean 0.248, range 0.131–0.312. + */ +const LEAGUE_HIT_RATE = 0.248; +/** + * Prior strength in at-bats. Deliberately weak: a regular carries 200–400 at-bats + * by midseason, so this moves a settled hitter by a few thousandths and only + * meaningfully regularises a genuinely thin sample. It exists to stop a 4-for-9 + * callup projecting as a .444 hitter, not to pull anyone toward the mean. + */ +const PRIOR_AB = 20; + +/** + * The player's own empirical at-bat distribution, recency-weighted. + * + * @param {Array} rows game-log rows ({stat:{atBats}}), MOST-RECENT-LAST + * (statsapi order — the same order `recencyWeighted` assumes). + * @returns {{pmf:number[], games:number, mean:number}|null} null when too few + * games carry a known at-bat count (honest-absent; caller falls back). + */ +function abPmfFromLog(rows, minGames = HITS_MIN_GAMES) { + const list = (rows || []).filter((r) => r && (r.stat || typeof r === 'object')); + const n = list.length; + const counts = new Array(AB_CAP + 1).fill(0); + let wTotal = 0; + let games = 0; + + list.forEach((r, i) => { + const s = (r && r.stat) || r || {}; + const ab = knownRate(s.atBats); // absent -> null, NOT a measured 0 + if (ab === null) return; // drop the row, never invent zero + const bucket = Math.min(AB_CAP, Math.round(ab)); + const w = i >= n - RECENT_WINDOW ? RECENT_WEIGHT : 1; + counts[bucket] += w; + wTotal += w; + games += 1; + }); + + if (games < minGames || wTotal === 0) return null; + const pmf = counts.map((c) => c / wTotal); + const mean = pmf.reduce((a, p, k) => a + p * k, 0); + return { pmf, games, mean }; +} + +/** + * Per-at-bat hit rate from the log, recency-weighted and shrunk toward the + * league anchor. + * + * Only rows carrying BOTH a known at-bat count and a known hit count contribute: + * a hit total without its at-bat denominator is not a rate, and pairing it with + * someone else's denominator would be a fabricated one. + * + * @returns {{q:number, at_bats:number, hits:number, games:number}|null} + */ +function hitRateFromLog(rows, opts = {}) { + const priorAb = opts.priorAb != null ? Number(opts.priorAb) : PRIOR_AB; + const priorRate = opts.priorRate != null ? Number(opts.priorRate) : LEAGUE_HIT_RATE; + const list = (rows || []); + const n = list.length; + let wAb = 0; let wHits = 0; let games = 0; let rawAb = 0; let rawHits = 0; + + list.forEach((r, i) => { + const s = (r && r.stat) || r || {}; + const ab = knownRate(s.atBats); + const h = knownRate(s.hits); + if (ab === null || h === null) return; // need the PAIR to form a rate + if (h > ab) return; // inconsistent row — skip, never clamp + const w = i >= n - RECENT_WINDOW ? RECENT_WEIGHT : 1; + wAb += w * ab; wHits += w * h; + rawAb += ab; rawHits += h; + games += 1; + }); + + if (games < (opts.minGames != null ? opts.minGames : HITS_MIN_GAMES)) return null; + // A player with games but zero weighted at-bats has no rate to measure. That + // is genuinely unknown, not a 0.000 hitter. + if (wAb <= 0) return null; + + const q = (wHits + priorAb * priorRate) / (wAb + priorAb); + return { + q, + at_bats: rawAb, + hits: rawHits, + games, + prior_ab: priorAb, + prior_rate: priorRate, + }; +} + +/** P(Bin(n, q) >= k), computed exactly. n is tiny (<= AB_CAP). */ +function binomAtLeast(n, q, k) { + const nn = Math.max(0, Math.round(Number(n))); + const kk = Math.ceil(Number(k)); + const p = Number(q); + if (!Number.isFinite(nn) || !Number.isFinite(kk) || !Number.isFinite(p)) return null; + if (kk <= 0) return 1; + if (kk > nn) return 0; // cannot get k hits in fewer at-bats + const pp = Math.min(1, Math.max(0, p)); + // pmf iteratively: P(X=0) = (1-p)^n, then the standard ratio step. + let term = (1 - pp) ** nn; + let cum = term; // P(X <= 0) + for (let x = 1; x < kk; x += 1) { + if (pp === 1) { term = 0; } else { + term = (term * (nn - x + 1) * pp) / (x * (1 - pp)); + } + cum += term; + } + return Math.min(1, Math.max(0, 1 - cum)); +} + +/** + * P(hits >= k) = Σ_n P(N=n) · P(Bin(n, q) >= k). + * + * At k = 1 (the 0.5 line, 84% of real hits rows) this is 1 − E_N[(1−q)^N] — the + * P(0) axis stated directly rather than inferred from a count family. + */ +function pAtLeastHits(abPmf, q, k) { + if (!Array.isArray(abPmf)) return null; + const kk = Math.max(0, Math.ceil(Number(k))); + if (!Number.isFinite(kk)) return null; + if (kk === 0) return 1; + let s = 0; + for (let n = 0; n < abPmf.length; n += 1) { + const pn = abPmf[n]; + if (!pn) continue; + const tail = binomAtLeast(n, q, kk); + if (tail === null) continue; + s += pn * tail; + } + return Math.min(1, Math.max(0, s)); +} + +/** + * The full read: P(hits >= line) plus the mean, or null when the inputs are not + * derivable. `multiplier` is the SAME park × weather × platoon × matchup product + * proj-v1.1 already computes, so hits-v1 and the current ladder differ only in + * STRUCTURE — it is applied to the conversion rate q, never to the at-bat count + * (see the header). + */ +function projectHits({ rows, line, multiplier = 1, minGames = HITS_MIN_GAMES } = {}) { + const ab = abPmfFromLog(rows, minGames); + if (!ab) return null; + const rate = hitRateFromLog(rows, { minGames }); + if (!rate) return null; + + const mRaw = Number(multiplier); + const m = Number.isFinite(mRaw) && mRaw > 0 ? mRaw : 1; + // q is a probability: it cannot exceed 1 however the multipliers stack. The + // ceiling is a bound on the arithmetic, not a modelling opinion. + const q = Math.min(0.999, Math.max(0, rate.q * m)); + + const target = Math.max(1, Math.ceil(Number(line))); + if (!Number.isFinite(target)) return null; + const p = pAtLeastHits(ab.pmf, q, target); + if (p === null) return null; + + return { + p_over_line: Math.round(p * 1000) / 1000, + mean: Math.round(ab.mean * q * 1000) / 1000, + hit_rate: Math.round(q * 1000) / 1000, + hit_rate_base: Math.round(rate.q * 1000) / 1000, + ab_per_game: Math.round(ab.mean * 1000) / 1000, + ab_distribution: ab.pmf.map((v) => Math.round(v * 1000) / 1000), + games_used: Math.min(ab.games, rate.games), + at_bats_observed: rate.at_bats, + hits_observed: rate.hits, + family: 'binomial_over_empirical_at_bats', + // The at-bat count is treated as independent of the conversion rate. A game + // that goes to extra innings gives a hitter both more at-bats AND, weakly, + // a different context; and a hitter who reaches keeps his own lineup turning + // over. Stated, not solved — the same class of caveat as tb-v1's + // independence approximation, and far smaller than the error it replaces. + ab_independence_caveat: true, + }; +} + +module.exports = { + abPmfFromLog, hitRateFromLog, binomAtLeast, pAtLeastHits, projectHits, + AB_CAP, HITS_MIN_GAMES, LEAGUE_HIT_RATE, PRIOR_AB, RECENT_WINDOW, RECENT_WEIGHT, +}; diff --git a/src/services/projectionChallenger.js b/src/services/projectionChallenger.js index efdefb2..c0da4bc 100644 --- a/src/services/projectionChallenger.js +++ b/src/services/projectionChallenger.js @@ -19,7 +19,17 @@ const dist = require('./projection/distribution'); const compoundTb = require('./projection/compoundTotalBases'); +const binomialHits = require('./projection/binomialHits'); const matchup = require('./projection/matchupRead'); +// THE RIGHT TAKEABLE AXIS. Three questions once shared one word; they no longer +// do (src/config/takeability.js). This module reads TWO of them, for two +// different purposes, and never the deprecated `takeable` field: +// isTakeableMarket — book IDENTITY. Decides what is a real market to +// MODEL. Thin, one-sided and juiced markets are all +// real; a -300 hits-over is a bet you can place. +// isWithinPromotionBand — a PRICE policy band. Decides what is worth +// SURFACING. Recorded here, never read by the model. +const { isTakeableMarket, isWithinPromotionBand } = require('../config/takeability'); const parkBase = require('./parkBase'); const { NAME_TO_ABBR } = require('./environmentContext'); @@ -27,6 +37,10 @@ const { NAME_TO_ABBR } = require('./environmentContext'); // projection model is byte-identical; the version bump marks the basis so pre-fix // (raw-book) and post-fix (fair) rows never silently mix in the handicapper test. const PROJ_VERSION = 'proj-v1.1'; +// hits-v1 — the per-stat structural challenger for HITS. Versioned separately +// from PROJ_VERSION so a later change to the ladder never silently re-labels +// rows that were written by this model. +const HITS_VERSION = 'hits-v1'; const PRIOR_GAMES = Number(process.env.PROJ_PRIOR_GAMES || 4); const LADDER_MAX = Number(process.env.PROJ_LADDER_MAX || 4); // Combined non-form multiplier bound (Phase B proved <0.12 stacked). A Coors + @@ -228,8 +242,66 @@ function projectProp({ } catch { tbCompound = null; } } + // ── HITS, modelled as the AT-BAT-BOUNDED CONVERSION it is (hits-v1) ─────── + // A hit is not a low-rate count. A hitter gets N official at-bats and converts + // each at rate q, so hits are BOUNDED by opportunity — something a negative + // binomial cannot express, since it has unbounded support and no notion of + // opportunity at all. Measured on 245 matched settled rows (direction-aligned, + // hits only): the current ladder resolves 0.0595 against the champion's + // 0.2044, and 84% of hits rows trade at 0.5 — so almost the whole stat is the + // single question P(0 hits), which is exactly where the count family hurts + // most. HYPOTHESIS, not a claim: the ledger decides. + // + // CHALLENGER ONLY: written alongside `proj_p_over_line`, never substituted for + // it. The current ladder and the champion are byte-identical. Inputs + // underivable (thin log, no at-bat counts) → null, and the prop keeps the + // current ladder value. Never fabricated. + // + // MARKET SCOPE READS IDENTITY, NOT PRICE. `market_takeable` comes from the + // book, and NOTHING about the price shape excludes a prop from being modelled: + // baseball hits markets are genuinely thin, genuinely juiced and genuinely + // one-sided, and all three are normal structure rather than a bad quote. The + // promotion band is recorded beside it and deliberately never consulted here — + // a -300 hits-over is takeable AND outside the band, and both are true at once. + let hitsBinom = null; + let hitsMarket = null; + if (stat === 'hits' && tradedRung != null) { + // The graded side's price. `book_odds` is the graded-side price and is the + // only one present on a one-sided quote, so it is a genuine fallback rather + // than a substitute for the other side. Absent → null, and the promotion + // band answers null (an unknown price is not an out-of-band price). + const sideOdds = num(direction === 'under' + ? (grade && grade.under_odds) + : (grade && grade.over_odds)) ?? num(grade && grade.book_odds); + hitsMarket = { + book: (grade && grade.book) || null, + market_takeable: isTakeableMarket(grade && grade.book), + within_promotion_band: isWithinPromotionBand(sideOdds), + one_sided: (grade && grade.over_odds != null) !== (grade && grade.under_odds != null), + price_filtered: false, // stated invariant: no price-shape rule gates the model + }; + try { + hitsBinom = binomialHits.projectHits({ rows: gameLog, line, multiplier: M }); + } catch { hitsBinom = null; } + } + return { proj_version: PROJ_VERSION, + proj_hits_p_over: hitsBinom ? hitsBinom.p_over_line : null, + proj_hits_meta: hitsBinom ? { + version: HITS_VERSION, + mean: hitsBinom.mean, + hit_rate: hitsBinom.hit_rate, + hit_rate_base: hitsBinom.hit_rate_base, + ab_per_game: hitsBinom.ab_per_game, + ab_distribution: hitsBinom.ab_distribution, + games_used: hitsBinom.games_used, + at_bats_observed: hitsBinom.at_bats_observed, + hits_observed: hitsBinom.hits_observed, + family: hitsBinom.family, + ab_independence_caveat: hitsBinom.ab_independence_caveat, + market: hitsMarket, + } : (hitsMarket ? { version: HITS_VERSION, market: hitsMarket, reason: 'inputs_underivable' } : null), proj_tb_p_over: tbCompound ? tbCompound.p_over_line : null, proj_tb_meta: tbCompound ? { version: 'tb-v1', mean: tbCompound.mean, rates: tbCompound.rates, @@ -297,5 +369,5 @@ async function attachProjection(grades, deps = {}) { module.exports = { projectProp, attachProjection, parkBaselineFromLogs, recencyWeighted, parkFactorFor, - PROJ_VERSION, STAT_FIELD, COMBINED_MAX, + PROJ_VERSION, HITS_VERSION, STAT_FIELD, COMBINED_MAX, }; diff --git a/tests/unit/binomialHits.test.js b/tests/unit/binomialHits.test.js new file mode 100644 index 0000000..da2f356 --- /dev/null +++ b/tests/unit/binomialHits.test.js @@ -0,0 +1,178 @@ +'use strict'; + +/** + * binomialHits (hits-v1) — hits as the at-bat-bounded conversion it is. + * + * The property that matters is the one an unbounded count model cannot express: + * hits are CAPPED by opportunity, and two hitters with the same mean hits can + * have very different P(0) once you know how they got there. Everything else + * here guards the honest-absent paths and the P(0) axis the 0.5 line rides on. + */ + +const b = require('../../src/services/projection/binomialHits'); + +const row = (ab, h) => ({ stat: { atBats: ab, hits: h } }); +const logOf = (n, ab, h) => Array.from({ length: n }, () => row(ab, h)); + +describe('the at-bat distribution is real, empirical and bounded', () => { + it('sums to 1 and its mean matches the observed at-bat average', () => { + const rows = [row(4, 1), row(3, 0), row(5, 2), row(4, 1), row(4, 0), row(3, 1)]; + const ab = b.abPmfFromLog(rows); + expect(ab.pmf.reduce((x, y) => x + y, 0)).toBeCloseTo(1, 9); + const pmfMean = ab.pmf.reduce((a, p, k) => a + p * k, 0); + expect(pmfMean).toBeCloseTo(ab.mean, 9); + expect(ab.games).toBe(6); + }); + + it('keeps a 0-at-bat APPEARANCE — it is a real game that settles as 0 hits', () => { + // A statsapi game log lists only games the player appeared in, so a 0-AB row + // is a walk/pinch-run appearance, not an absence. Dropping it would delete + // genuine P(0 hits) mass; a count model has to infer that mass instead. + const rows = [row(0, 0), row(4, 1), row(4, 2), row(3, 0), row(4, 1)]; + const ab = b.abPmfFromLog(rows); + expect(ab.pmf[0]).toBeGreaterThan(0); + expect(ab.games).toBe(5); + }); +}); + +describe('THE POINT — P(hits >= k) respects the at-bat ceiling', () => { + it('P(>=2 hits) is exactly 0 for a hitter who only ever gets one at-bat', () => { + // No count family with unbounded support can state this. Two hits in one + // at-bat is not improbable, it is impossible. + const rows = logOf(8, 1, 0); + const out = b.projectHits({ rows, line: 1.5 }); + expect(out.p_over_line).toBe(0); + }); + + it('P(>=1 hit) is 1 - E[(1-q)^N] — the P(0) axis stated directly', () => { + // Fixed 4 at-bats every game makes the expectation collapse to one term, so + // the model's answer is checkable in closed form. Read the UNROUNDED + // internals — projectHits rounds its output to 3dp for storage, and + // comparing two separately-rounded numbers is a test of the rounding. + const rows = logOf(10, 4, 1); + const ab = b.abPmfFromLog(rows); + const rate = b.hitRateFromLog(rows); + expect(b.pAtLeastHits(ab.pmf, rate.q, 1)).toBeCloseTo(1 - (1 - rate.q) ** 4, 9); + }); + + it('MORE at-bats at the same rate means a higher P(>=1) — opportunity is modelled', () => { + // Identical conversion rate (.250), different opportunity. An unbounded count + // model sees only the resulting mean and cannot separate these. + const few = b.projectHits({ rows: logOf(12, 2, 0.5), line: 0.5 }); + const many = b.projectHits({ rows: logOf(12, 6, 1.5), line: 0.5 }); + expect(many.hit_rate).toBeCloseTo(few.hit_rate, 2); + expect(many.p_over_line).toBeGreaterThan(few.p_over_line); + }); + + it('P(>=k) is monotonically non-increasing in k', () => { + const rows = [row(4, 2), row(4, 1), row(3, 0), row(5, 3), row(4, 1), row(4, 0)]; + const ab = b.abPmfFromLog(rows); + const rate = b.hitRateFromLog(rows); + let prev = 1; + for (let k = 1; k <= 6; k += 1) { + const p = b.pAtLeastHits(ab.pmf, rate.q, k); + expect(p).toBeLessThanOrEqual(prev + 1e-12); + prev = p; + } + }); +}); + +describe('the binomial tail is exact', () => { + it('matches closed form for the cases with one', () => { + expect(b.binomAtLeast(4, 0.25, 1)).toBeCloseTo(1 - 0.75 ** 4, 9); + expect(b.binomAtLeast(3, 0.5, 3)).toBeCloseTo(0.125, 9); + expect(b.binomAtLeast(5, 0.2, 0)).toBe(1); + }); + + it('cannot get more hits than at-bats', () => { + expect(b.binomAtLeast(2, 0.9, 3)).toBe(0); + expect(b.binomAtLeast(0, 0.9, 1)).toBe(0); + }); +}); + +describe('UNKNOWN IS NOT ZERO — the eighth instance does not ship', () => { + it('a row with no atBats field is DROPPED, not read as a 0-at-bat game', () => { + const rows = [row(4, 1), row(4, 2), { stat: { hits: 1 } }, row(3, 0), row(4, 1), row(4, 1)]; + const ab = b.abPmfFromLog(rows); + expect(ab.games).toBe(5); // the null-AB row contributed nothing + expect(ab.mean).toBeCloseTo(3.8, 1); // and did NOT drag the mean toward 0 + }); + + it('a null at-bat count contributes no denominator and no game', () => { + // `games` is the tell. Reading the null as 0 at-bats would count a sixth + // game asserting "he came up and got nothing" — an opportunity claim made + // out of an absence of data. It contributes nothing at all instead. + const withNull = b.hitRateFromLog([ + row(4, 1), row(4, 1), row(4, 1), row(4, 1), row(4, 1), { stat: { atBats: null, hits: 0 } }, + ]); + expect(withNull.games).toBe(5); + expect(withNull.at_bats).toBe(20); + expect(withNull.hits).toBe(5); + }); + + it('a REAL 0-at-bat game does survive — a measured zero is a fact', () => { + const ab = b.abPmfFromLog([row(0, 0), row(4, 1), row(4, 1), row(4, 1), row(4, 1)]); + expect(ab.pmf[0]).toBeCloseTo(1 / 5, 9); // uniform weights: 5 games, none recent-only + }); +}); + +describe('honest-absent — never fabricate a read', () => { + it('returns null below the minimum games rather than guessing a shape', () => { + expect(b.abPmfFromLog([row(4, 1), row(4, 1)])).toBeNull(); + expect(b.projectHits({ rows: [row(4, 1), row(4, 1)], line: 0.5 })).toBeNull(); + }); + + it('a log with games but no known at-bats yields no rate, not a .000 hitter', () => { + const rows = Array.from({ length: 8 }, () => ({ stat: { hits: 0 } })); + expect(b.hitRateFromLog(rows)).toBeNull(); + expect(b.projectHits({ rows, line: 0.5 })).toBeNull(); + }); + + it('SKIPS an impossible row (hits > at-bats) rather than clamping it', () => { + const rate = b.hitRateFromLog([row(1, 3), row(4, 1), row(4, 1), row(4, 1), row(4, 1), row(4, 1)]); + expect(rate.games).toBe(5); + expect(rate.hits).toBe(5); + }); +}); + +describe('the prior regularises a thin sample without pulling a settled one', () => { + it('a 4-for-9 callup does not project as a .444 hitter', () => { + const rows = [row(2, 1), row(2, 1), row(1, 1), row(2, 1), row(2, 0)]; + const rate = b.hitRateFromLog(rows); + expect(rate.q).toBeLessThan(0.40); + expect(rate.q).toBeGreaterThan(b.LEAGUE_HIT_RATE); + }); + + it('barely moves a hitter with a full season of at-bats', () => { + const rows = logOf(120, 4, 1); // 480 AB, exactly .250 + const rate = b.hitRateFromLog(rows); + expect(Math.abs(rate.q - 0.25)).toBeLessThan(0.005); + }); +}); + +describe('projectHits — the full read', () => { + const rows = [row(4, 1), row(3, 0), row(5, 2), row(4, 1), row(4, 0), row(3, 1), row(4, 2)]; + + it('the multiplier moves the CONVERSION RATE, not the at-bat count', () => { + const base = b.projectHits({ rows, line: 0.5, multiplier: 1 }); + const up = b.projectHits({ rows, line: 0.5, multiplier: 1.2 }); + expect(up.hit_rate).toBeGreaterThan(base.hit_rate); + expect(up.p_over_line).toBeGreaterThan(base.p_over_line); + // Park, weather, platoon and matchup do not change how often a hitter bats. + expect(up.ab_per_game).toBeCloseTo(base.ab_per_game, 9); + }); + + it('q stays a probability however the multipliers stack', () => { + const out = b.projectHits({ rows: logOf(10, 4, 3), line: 0.5, multiplier: 5 }); + expect(out.hit_rate).toBeLessThanOrEqual(1); + expect(out.p_over_line).toBeLessThanOrEqual(1); + }); + + it('labels its family and carries the stated caveat', () => { + const out = b.projectHits({ rows, line: 0.5 }); + expect(out.family).toBe('binomial_over_empirical_at_bats'); + expect(out.ab_independence_caveat).toBe(true); + expect(out.games_used).toBe(7); + expect(out.mean).toBeCloseTo(out.ab_per_game * out.hit_rate, 2); + }); +}); diff --git a/tests/unit/projectionChallenger.test.js b/tests/unit/projectionChallenger.test.js index f89d5f6..2e20c9c 100644 --- a/tests/unit/projectionChallenger.test.js +++ b/tests/unit/projectionChallenger.test.js @@ -161,3 +161,70 @@ describe('attachProjection — champion byte-identical', () => { expect(out[0].proj_point).not.toBeNull(); // projected even with no game log }); }); + +describe('hits-v1 — the challenger rides alongside, and reads the RIGHT takeable axis', () => { + // A real log: at-bat counts AND hits, which the negative-binomial path never + // asks for. Most recent LAST (statsapi order). + const abLog = (pairs) => pairs.map(([ab, h]) => ({ isHome: true, opponent: 'Boston Red Sox', stat: { atBats: ab, hits: h } })); + const LOG = abLog([[4, 1], [3, 0], [5, 2], [4, 1], [4, 0], [3, 1], [4, 2], [4, 1], [3, 1], [4, 0]]); + + it('emits proj_hits_p_over WITHOUT disturbing the current ladder or the champion', () => { + const grade = { stat_type: 'hits', line: 0.5, direction: 'over', season_avg: 0.9, fair_prob: 0.55, team: 'NYY', book: 'draftkings', over_odds: -150, under_odds: 130 }; + const withHits = pc.projectProp({ grade, gameLog: LOG }); + // The ladder value is computed from the SAME log by the SAME code path as + // before this challenger existed — hits-v1 is additive, never substitutive. + expect(withHits.proj_hits_p_over).toBeGreaterThan(0); + expect(withHits.proj_hits_p_over).toBeLessThanOrEqual(1); + expect(withHits.proj_p_over_line).not.toBeNull(); + expect(withHits.proj_distribution.family).toBe('negative_binomial'); + expect(withHits.proj_hits_meta.version).toBe('hits-v1'); + expect(withHits.proj_hits_meta.family).toBe('binomial_over_empirical_at_bats'); + }); + + it('MODELS a -300 one-sided hits market — takeable by identity, outside the promotion band', () => { + // The whole reason the three questions were disambiguated. A -300 hits-over + // from a real book is a bet you can place: steep juice is a PRICE, not a + // disqualification, and one-sidedness is normal baseball market structure. + const grade = { stat_type: 'hits', line: 0.5, direction: 'over', season_avg: 0.9, team: 'NYY', book: 'betrivers', over_odds: -300, under_odds: null }; + const p = pc.projectProp({ grade, gameLog: LOG }); + expect(p.proj_hits_p_over).not.toBeNull(); // MODELLED, not thrown away + const m = p.proj_hits_meta.market; + expect(m.market_takeable).toBe(true); // identity: yes + expect(m.within_promotion_band).toBe(false); // price band: no + expect(m.one_sided).toBe(true); // normal, not an error + expect(m.price_filtered).toBe(false); // no price-shape gate ran + }); + + it('a longshot +1300 hits market is still modelled — recording is not crowning', () => { + const grade = { stat_type: 'hits', line: 2.5, direction: 'over', season_avg: 0.9, team: 'NYY', book: 'fanduel', over_odds: 1300, under_odds: -2000 }; + const p = pc.projectProp({ grade, gameLog: LOG }); + expect(p.proj_hits_p_over).not.toBeNull(); + expect(p.proj_hits_meta.market.market_takeable).toBe(true); + expect(p.proj_hits_meta.market.within_promotion_band).toBe(false); + }); + + it('never reads the deprecated `takeable` mirror', () => { + const src = require('fs').readFileSync(require.resolve('../../src/services/projection/binomialHits'), 'utf8') + + require('fs').readFileSync(require.resolve('../../src/services/projectionChallenger'), 'utf8'); + // The disambiguated names only. `takeable:` as a bare field read would be + // the ambiguous mirror the S-prior commit deprecated. + expect(/\btakeable\s*:/.test(src.replace(/market_takeable\s*:/g, ''))).toBe(false); + }); + + it('is NULL on a non-hits prop — never a fabricated 0', () => { + const grade = { stat_type: 'total_bases', line: 1.5, direction: 'over', season_avg: 1.4, team: 'NYY', book: 'draftkings', over_odds: -110 }; + const p = pc.projectProp({ grade, gameLog: LOG }); + expect(p.proj_hits_p_over).toBeNull(); + expect(p.proj_hits_meta).toBeNull(); + }); + + it('abstains honestly when the at-bat inputs are underivable, keeping the ladder', () => { + // A log with hits but no at-bat counts — exactly what the NB path runs on. + const noAb = [1, 0, 2, 1, 1, 0, 1].map((h) => ({ isHome: true, opponent: 'Boston Red Sox', stat: { hits: h } })); + const grade = { stat_type: 'hits', line: 0.5, direction: 'over', season_avg: 0.9, team: 'NYY', book: 'draftkings', over_odds: -140 }; + const p = pc.projectProp({ grade, gameLog: noAb }); + expect(p.proj_hits_p_over).toBeNull(); // no read, not a zero + expect(p.proj_hits_meta.reason).toBe('inputs_underivable'); + expect(p.proj_p_over_line).not.toBeNull(); // the ladder still stands + }); +});