diff --git a/BUILD-STATE.md b/BUILD-STATE.md index c767a6e..11b7f52 100755 --- a/BUILD-STATE.md +++ b/BUILD-STATE.md @@ -3,6 +3,33 @@ ## Last Updated 2026-08-03 +## Session 80 (2026-08-03) — The skill engine: built, gated, and Stage A honestly lost ✅ +Spec: `specs/skill-engine-architecture.md`. 4,182 tests / 333 suites green, build exit 0. +- **BUILT `src/services/model/`:** `featureRegistry` (CANDIDATE/PROVEN/DEAD per + sport; `liveFeatures()` = PROVEN only; promotion needs n>=200 + positive lift + + CI excluding zero, no override) and `skillProjection` (PA outcome tree, log5 + odds-ratio K/BB, archetype-selected contact quality, Binomial over a PA + distribution). 22 tests assert the five disciplines as BEHAVIOUR. +- **The gate works by construction:** with only PROVEN features allowed the + projection returns NULL. Registry ships with ONE proven feature (the counter). +- **STAGE A: skill-v1 LOSES → NOT PROMOTED.** Out-of-sample (profiles frozen + 07-21, only later games scored), 570 rows, 91.9% pitcher coverage: resolution + 0.0499 vs champion 0.166, delta −0.116 CI [−0.189,−0.043]. Not selective + either (top-8 hit 50%, lift −0.065). +- **Two false starts caught:** (1) units — statcast stores PERCENTAGES, raw rows + made bip negative and refused 568/576; now one chokepoint `fromStatcastRow`. + (2) an INVALID first verdict — ledger team/opponent are NULL, so the pitcher + resolved for 1 of 570 rows and it was silently measuring a batter-only model. + Fixed via each player's statsapi game log. +- **Not exercised yet (so the loss is real but partial):** park (passed 1.0), + handedness, opportunity_drift, and PA projection is season-PA/103. And the + skill profiles carry NO recency while the champion has last-5. +- **Fixed: Statcast nightly refresh was UNREACHABLE CODE** — inside tick() below + the HOURS_UTC guard while testing h===11. Never ran; 13 days stale; both alerts + in the same dead branch. Now its own tick; test rewritten to catch it. +- **Next:** recency into the skill profile, wire park/handedness/opportunity_drift, + real PA from lineup slot, then re-run Stage A. + ## Session 79 (2026-08-03) — Reality assessment vs the FORWARD-PROJECTION objective ✅ Spec: `specs/forward-model-reality-assessment.md`. READ-ONLY (src/web untouched). - **Finding: the forward model's parts all EXIST and are all wired downstream of diff --git a/CLAUDE.md b/CLAUDE.md index 33aabae..ece6be1 100755 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -1262,6 +1262,45 @@ phased plan in the Session-57 conversation / BUILD-STATE Next section). forward inputs. "AT CEILING" (runs/walks) is provisional: measured only against features the champion already reads. +## The skill engine + feature registry (Session 80 — non-obvious) +- **`specs/skill-engine-architecture.md`.** `src/services/model/` is the forward + engine: `featureRegistry.js` (CANDIDATE/PROVEN/DEAD per feature PER SPORT) and + `skillProjection.js` (PA outcome tree: K/BB via log5 odds-ratio vs league, then + archetype-weighted contact quality → Binomial(PA, p_hit) mixed over a PA + distribution). Challenger-only; champion untouched. +- **THE GATE IS STRUCTURAL, not a habit.** `liveFeatures()` returns PROVEN only, + and the registry ships with exactly ONE proven feature (the incumbent counter). + A test asserts that with only PROVEN allowed, `projectSkill` returns NULL — an + unproven model cannot reach a user by accident. `promote()` requires n>=200, + positive lift, CI excluding zero, and has NO override argument. +- **STAGE A RESULT: skill-v1 LOSES, not promoted.** 570 rows, 91.9% pitcher + coverage, resolution 0.0499 vs champion 0.166, delta −0.116 CI [−0.189,−0.043]. + Also NOT selective — its top-8 most confident picks hit 50% (lift −0.065). +- **UNITS: `statcast_aggregates` stores PERCENTAGES (0–100), not fractions.** + `k_pct: 29.6` means 29.6%. Feeding raw rows in made `bip = 1−29.6−17.1` negative + and refused 568/576 rows. ALWAYS convert via `skillProjection.fromStatcastRow` + (the one chokepoint); it nulls out-of-range values rather than clamping, and + leaves mph/degrees fields alone. +- **`ledger_entries.team`/`opponent` are NULL on ~all rows** — do NOT join a + matchup on them. The first Stage A run resolved a pitcher for 1 of 570 rows and + would have reported a verdict on a batter-only model. Resolve the opponent from + the player's own statsapi game log (`getPlayerGameLog` → `{date, opponent}`), + which is authoritative and point-in-time safe → 91.9% coverage. +- **Archetype = FEATURE SELECTOR, not a nudge.** `ARCHETYPE_MAP` weights decide + which skill inputs drive a hitter (BOMBER barrel 0.50 / gb_speed 0; GHOST + barrel 0.05 / gb_speed 0.60). Locked by test: same hitter read through two + archetypes moves >0.15. Weights are DOCUMENTED, not fitted — fitting on 1,741 + rows is curve-fitting; the registry exists so they get measured. +- **total_bases is deliberately REFUSED by skillProjection.** A deterministic + bases-per-hit multiplier made P(TB>=2) exactly equal P(hits>=1) — a relabelled + hits curve carrying no new information. TB needs tb-v1's compound per-hit bases + distribution; refusing beats shipping a relabel. +- **STATCAST REFRESH WAS UNREACHABLE CODE** (fixed): it sat inside `tick()` below + `if (!HOURS_UTC.includes(h)) return` (14,19,22,1,3) while testing `h === 11`. + Never ran once; data 13 days stale; BOTH its alerts were in the same dead + branch. Now its own `statcastTick`. The old test only checked the string + existed — the new one asserts it is not behind the snapshot-hours guard. + ## Active Skills - vyndr-voice (all user-facing output) - prop-analysis (grading methodology) diff --git a/scripts/skill-v1-stagea.js b/scripts/skill-v1-stagea.js new file mode 100644 index 0000000..4df5a09 --- /dev/null +++ b/scripts/skill-v1-stagea.js @@ -0,0 +1,279 @@ +#!/usr/bin/env node +'use strict'; + +/** + * skill-v1-stagea — DOES THE WINDSHIELD BEAT THE REAR-VIEW MIRROR? + * + * Stage A's only question: on settled HITS props, does an archetype-selected, + * skill-based forward projection call the LISTED LINE better than the frequency + * counter that is the reigning champion? If it does not, it is not real yet and + * it does not get promoted. That is the whole test. + * + * ── WHY THIS IS GENUINELY OUT-OF-SAMPLE ────────────────────────────────── + * `statcast_aggregates` was last refreshed 2026-07-21 (the nightly job was + * unreachable code until this session — see snapshotScheduler). Settled hits + * rows run 2026-07-23 onward. So the skill profiles this model reads were + * frozen BEFORE every game it is asked to predict. The staleness that was a bug + * for production is, for this one measurement, a clean point-in-time snapshot. + * Rows on or before the freeze date are EXCLUDED so no profile can contain the + * game it is predicting. + * + * The opposing starter comes from the statsapi schedule for that date, and the + * pitcher's skill profile from the same frozen aggregate table. + * + * ── THE BAR (identical to the one that refuted hits-v1) ────────────────── + * - hits rows only, direction-aligned to the graded side + * - matched rows only: champion and challenger scored on the SAME props + * - paired bootstrap, deterministic seed, CI on the DIFFERENCE + * - PROMOTE only if the CI excludes zero on the good side + * + * ── DISCIPLINE 4, MEASURED, NOT ASSUMED ────────────────────────────────── + * Selectivity is reported, not claimed: accuracy is broken out by how confident + * the model is, so "right 57% on the 8 you're sure of" is a number rather than a + * slogan. LIFT over the naive base rate is reported beside it, because being + * right about obvious chalk is not signal. + * + * SUPABASE_URL=... node scripts/skill-v1-stagea.js + */ + +require('dotenv').config(); +const { createClient } = require('@supabase/supabase-js'); +const sk = require('../src/services/model/skillProjection'); +const reg = require('../src/services/model/featureRegistry'); +const mlb = require('../src/services/adapters/mlbStatsAdapter'); +const { nameKey } = require('../src/utils/playerName'); +const { knownRate } = require('../src/utils/known'); + +/** Team games played by the 2026-07-21 profile freeze — turns season PA into PA/game. */ +const GAMES_SO_FAR = Number(process.env.STAGEA_GAMES_SO_FAR || 103); + +const SB_URL = process.env.SUPABASE_URL; +const SB_KEY = process.env.SUPABASE_SERVICE_ROLE_KEY || process.env.SUPABASE_SERVICE_KEY; +const PAGE = 1000; + +function corr(xs, ys) { + const n = xs.length; + if (n < 3) return null; + const mx = xs.reduce((a, b) => a + b, 0) / n; + const my = ys.reduce((a, b) => a + b, 0) / n; + let sxy = 0; let sxx = 0; let syy = 0; + for (let i = 0; i < n; i += 1) { + const dx = xs[i] - mx; const dy = ys[i] - my; + sxy += dx * dy; sxx += dx * dx; syy += dy * dy; + } + if (sxx <= 0 || syy <= 0) return null; + return sxy / Math.sqrt(sxx * syy); +} +const r4 = (v) => (v == null || !Number.isFinite(v) ? null : Math.round(v * 10000) / 10000); +const mean = (a) => (a.length ? a.reduce((x, y) => x + y, 0) / a.length : null); +const brier = (ps, ys) => (ps.length ? ps.reduce((s, p, i) => s + (p - ys[i]) ** 2, 0) / ps.length : null); + +function makeRnd(seed) { + let s = seed >>> 0; + return () => { s ^= s << 13; s >>>= 0; s ^= s >>> 17; s ^= s << 5; s >>>= 0; return s / 4294967296; }; +} + +function bootstrapDiff(rows, keyA, keyB, iters = 4000, seed = 20260803) { + if (rows.length < 30) return null; + const rnd = makeRnd(seed); + const n = rows.length; + const diffs = []; + for (let it = 0; it < iters; it += 1) { + const ys = []; const a = []; const b = []; + for (let i = 0; i < n; i += 1) { + const r = rows[Math.floor(rnd() * n)]; + ys.push(r.won); a.push(r[keyA]); b.push(r[keyB]); + } + const ca = corr(a, ys); const cb = corr(b, ys); + if (ca == null || cb == null) continue; + diffs.push(ca - cb); + } + if (diffs.length < 100) return null; + diffs.sort((x, y) => x - y); + const q = (p) => r4(diffs[Math.floor(p * (diffs.length - 1))]); + const ci = [q(0.025), q(0.975)]; + return { + point: r4(corr(rows.map((r) => r[keyA]), rows.map((r) => r.won)) + - corr(rows.map((r) => r[keyB]), rows.map((r) => r.won))), + ci95: ci, + ci_excludes_zero: ci[0] > 0 || ci[1] < 0, + }; +} + +async function page(sb, table, select, apply) { + const out = []; + for (let from = 0; ; from += PAGE) { + const { data, error } = await apply(sb.from(table).select(select)).range(from, from + PAGE - 1); + if (error) throw error; + if (!data || data.length === 0) break; + out.push(...data); + if (data.length < PAGE) break; + } + return out; +} + +/** + * `date|team` → the starter that team FACED. + * + * Built from the statsapi schedule: a team faces the OTHER side's probable. + */ +async function opposingStarters(dates) { + const byDateTeam = new Map(); + for (const d of dates) { + let games = []; + try { games = await mlb.getScheduleWithPitchers(d); } catch { games = []; } + for (const g of games) { + if (!g.home || !g.away) continue; + if (g.away.probablePitcher) byDateTeam.set(`${d}|${g.home.team}`, g.away.probablePitcher.id); + if (g.home.probablePitcher) byDateTeam.set(`${d}|${g.away.team}`, g.home.probablePitcher.id); + // Also key by the PITCHING team, so "who did team X send out" is directly + // answerable from the opponent name a game log gives us. + if (g.home.probablePitcher) byDateTeam.set(`${d}|OPP:${g.home.team}`, g.home.probablePitcher.id); + if (g.away.probablePitcher) byDateTeam.set(`${d}|OPP:${g.away.team}`, g.away.probablePitcher.id); + } + } + return byDateTeam; +} + +/** + * `playerKey|date` → the OPPONENT team that player faced. + * + * THE LEDGER CANNOT ANSWER THIS: `team`/`opponent` are NULL on 575 of 576 rows + * in this window, which is why the first run resolved a pitcher for exactly ONE + * row and silently measured a batter-profile-only model instead of the matchup + * model it claimed to test. The player's own statsapi game log names the + * opponent for the exact date, so it is both authoritative and point-in-time + * safe (a completed game's opponent is not a forecast). + */ +async function opponentByPlayerDate(players) { + const map = new Map(); + for (const [key, name] of players) { + try { + const found = await mlb.searchPlayer(name); + if (!found || !found.id) continue; + const log = await mlb.getPlayerGameLog(found.id); + for (const g of log || []) { + if (g && g.date && g.opponent) map.set(`${key}|${String(g.date).slice(0, 10)}`, g.opponent); + } + } catch { /* a missing log just means no pitcher for those rows */ } + } + return map; +} + +async function main() { + if (!SB_URL || !SB_KEY) throw new Error('SUPABASE_URL / service key required'); + const sb = createClient(SB_URL, SB_KEY, { auth: { persistSession: false } }); + + // Frozen skill profiles — by name (batters) and by source_id (pitchers). + const statcast = await page(sb, 'statcast_aggregates', '*', (q) => q.eq('sport', 'mlb')); + const freeze = statcast.reduce((mx, r) => (String(r.updated_at) > mx ? String(r.updated_at) : mx), ''); + const freezeDate = freeze.slice(0, 10); + const batters = new Map(); + const pitchersById = new Map(); + for (const r of statcast) { + // UNITS: statcast_aggregates stores percentages (0-100). Convert ONCE, here. + if (r.role === 'pitcher' && r.source_id != null) pitchersById.set(Number(r.source_id), sk.fromStatcastRow(r)); + if (r.player_key && r.role === 'batter') { + const prev = batters.get(r.player_key); + const size = Number(r.sample_pa || 0); + if (!prev || size > Number(prev.rawPa || 0)) { + batters.set(r.player_key, Object.assign(sk.fromStatcastRow(r), { rawPa: size, archetype: null })); + } + } + } + + const led = await page(sb, 'ledger_entries', + 'player_key, player_name, stat, line, side, outcome, game_date, p_win, team, opponent, quarantine_reason', + (q) => q.eq('sport', 'mlb').is('user_id', null).eq('stat', 'hits') + .in('outcome', ['hit', 'miss']).not('p_win', 'is', null)); + + const clean = led.filter((r) => !(r.quarantine_reason || '').startsWith('nontakeable_book') + // STRICTLY AFTER the profile freeze — no game may be inside its own inputs. + && String(r.game_date) > freezeDate); + + const dates = [...new Set(clean.map((r) => r.game_date))].sort(); + const starters = await opposingStarters(dates); + const players = new Map(); + for (const r of clean) if (!players.has(r.player_key)) players.set(r.player_key, r.player_name); + const oppByPlayerDate = await opponentByPlayerDate(players); + + const allowed = reg.candidateFeatures('mlb'); // challenger-first: candidates measured, never served + const rows = []; + const drops = {}; + const drop = (k) => { drops[k] = (drops[k] || 0) + 1; }; + let withPitcher = 0; + + for (const r of clean) { + const bat = batters.get(r.player_key); + if (!bat) { drop('no_batter_profile'); continue; } + // The team he FACED that day, from his own game log. `starters` is keyed by + // the team doing the facing, so look up his own team — which is the + // opponent's opponent. Resolve via the game log's opponent name and take the + // schedule entry for the OTHER side. + const facedTeam = oppByPlayerDate.get(`${r.player_key}|${r.game_date}`) || null; + const starterId = facedTeam ? starters.get(`${r.game_date}|OPP:${facedTeam}`) : null; + const pit = starterId != null ? pitchersById.get(Number(starterId)) : null; + if (pit) withPitcher += 1; + + // Opportunity: the batter's own season PA per game, from the frozen profile. + // Opportunity: season PA spread over the games played so far this season. + // GAMES_SO_FAR is the frozen-profile era's team game count, so PA/game is a + // real per-game rate rather than an arbitrary divisor. + const pa = knownRate(bat.rawPa); + const expectedPa = pa && pa > 0 ? Math.min(5.2, Math.max(2.0, pa / GAMES_SO_FAR)) : null; + + const out = sk.projectSkill({ + batter: bat, pitcher: pit, park: 1, + archetype: bat.archetype || null, + statType: 'hits', line: Number(r.line), + expectedPa, allowed, + }); + if (!out) { drop('projection_refused'); continue; } + + const under = String(r.side).toLowerCase() === 'under'; + rows.push({ + won: r.outcome === 'hit' ? 1 : 0, + champ: Number(r.p_win), + skill: under ? 1 - out.p_over_line : out.p_over_line, + line: Number(r.line), + had_pitcher: !!pit, + }); + } + + const ys = rows.map((r) => r.won); + const base = mean(ys); + const bs = bootstrapDiff(rows, 'skill', 'champ'); + + // DISCIPLINE 4 — selectivity, measured. Sorted by confidence in the graded + // side; report accuracy and LIFT over the naive base rate at each depth. + const byConf = [...rows].sort((a, b) => b.skill - a.skill); + const depths = [8, 15, 25, 50, 100].filter((d) => d <= byConf.length); + const selectivity = depths.map((d) => { + const top = byConf.slice(0, d); + const hit = mean(top.map((r) => r.won)); + return { top_n: d, hit_rate: r4(hit), lift_over_base: r4(hit - base) }; + }); + + console.log(JSON.stringify({ + measurement: 'STAGE A — skill-v1 vs the frequency counter, out-of-sample on listed-line accuracy', + out_of_sample_guarantee: `skill profiles frozen ${freezeDate}; only rows with game_date > ${freezeDate} scored`, + registry: reg.summary('mlb'), + matched_rows: rows.length, + rows_with_opposing_pitcher: withPitcher, + pitcher_coverage_pct: rows.length ? r4(withPitcher / rows.length) : null, + dropped: drops, + base_rate: r4(base), + resolution: { skill_v1: r4(corr(rows.map((r) => r.skill), ys)), champion: r4(corr(rows.map((r) => r.champ), ys)) }, + brier: { skill_v1: r4(brier(rows.map((r) => r.skill), ys)), champion: r4(brier(rows.map((r) => r.champ), ys)) }, + mean_forecast: { skill_v1: r4(mean(rows.map((r) => r.skill))), champion: r4(mean(rows.map((r) => r.champ))) }, + delta_vs_champion: bs, + verdict: !bs ? 'N-BLOCKED' + : (bs.ci_excludes_zero && bs.point > 0) ? 'BEATS THE COUNTER — promotable' + : (bs.ci_excludes_zero && bs.point < 0) ? 'LOSES to the counter — iterate, do not promote' + : 'INCONCLUSIVE — not proven, do not promote', + selectivity_discipline_4: selectivity, + }, null, 2)); + process.exit(0); +} + +main().catch((e) => { console.error(e); process.exit(1); }); diff --git a/specs/skill-engine-architecture.md b/specs/skill-engine-architecture.md new file mode 100644 index 0000000..c464057 --- /dev/null +++ b/specs/skill-engine-architecture.md @@ -0,0 +1,173 @@ +# THE SKILL ENGINE — architecture, and Stage A's honest result + +**Built 2026-08-03.** Baseball first. Challenger-only; the champion is untouched. + +> **STAGE A VERDICT: skill-v1 LOSES to the frequency counter and is NOT promoted.** +> Out-of-sample on 570 settled hits props with 91.9% opposing-pitcher coverage: +> resolution **0.0499 vs the champion's 0.166**, delta **−0.116, CI +> [−0.189, −0.043]**. The architecture's own rule fired and kept it out of the +> live path. §5 is what was actually exercised and what wasn't — the loss is +> real, but it does not yet condemn the approach, and §6 says why. + +--- + +## 1. What was built + +| module | discipline | what it does | +|---|---|---| +| `src/services/model/featureRegistry.js` | **D3** — earn its place | CANDIDATE / PROVEN / DEAD per feature, per sport. `liveFeatures()` returns PROVEN only | +| `src/services/model/skillProjection.js` | **D1 + D2** | PA outcome tree from skill inputs; archetype selects the feature map | +| `tests/unit/skillModel.test.js` | all | 22 tests asserting the disciplines as behaviour | +| `scripts/skill-v1-stagea.js` | **D4** | the out-of-sample proof + selectivity measurement | + +## 2. The model (D1 — skill, not results) + +A hitter gets N plate appearances; each resolves through a tree: + +``` +PA ──> strikeout (batter K vs pitcher K, odds-ratio vs league) + ──> walk (batter BB vs pitcher BB, odds-ratio vs league) + ──> ball in play + └─> hit? (archetype-weighted contact quality vs contact allowed, park) +``` + +Rates combine by **log5 / odds ratio** against league — the standard way to ask +"this batter vs THIS pitcher" instead of "vs the average pitcher". Two identities +are unit-tested rather than assumed: league-vs-league returns league, and an +average pitcher leaves the batter's own rate untouched. + +Hits then distribute as `Binomial(PA, p_hit_per_PA)`, mixed over a PA +distribution — because PA is not known in advance and pretending it is would be +a fabricated certainty. + +## 3. Archetype selects features (D2) + +Not a nudge, not a label — it decides **which skill inputs drive this hitter**: + +| archetype | barrel | hard-hit | exit velo | GB/speed | power | +|---|---|---|---|---|---| +| BOMBER | **0.50** | 0.35 | 0.15 | **0.00** | 1.25 | +| GHOST | **0.05** | 0.20 | 0.15 | **0.60** | 0.70 | +| TORCH | 0.20 | 0.35 | 0.25 | 0.20 | 1.00 | + +Locked by test: the same hitter read as BOMBER vs GHOST moves **>0.15** in +probability, and barrel rate moves a BOMBER more than 3× what it moves a GHOST. +Weights are documented by what the archetype physically does — **not fitted**, +because fitting them on 1,741 rows is curve-fitting, and the registry exists so +they get measured instead. + +## 4. The gate is structural (D3) + +The registry ships with **one PROVEN feature** — `recent_frequency_prior`, the +incumbent counter, and only because it has the measurement (S78 ablation). Every +skill input is CANDIDATE. + +**Locked by test: with only PROVEN features allowed, `projectSkill` returns +null.** An unproven model cannot reach a user by accident — not by convention, +by construction. Promotion requires n ≥ 200, positive lift, and a CI excluding +zero; there is deliberately **no override argument**. + +Three champion layers are already registered **DEAD** with their reasons, so they +cannot be silently rebuilt: `champion_opp_rank_adj`, `champion_home_away_adj` +("removing it IMPROVED rbi resolution — actively harmful"), +`champion_consistency_pull`. + +## 5. STAGE A — the measurement, and what it actually tested + +**Out-of-sample guarantee:** skill profiles were frozen 2026-07-21 (the nightly +refresh had never run — §7). Only rows with `game_date > 2026-07-21` were scored, +so no profile can contain the game it predicts. The bug that broke production +made this one measurement clean. + +``` +matched rows 570 pitcher coverage 91.9% +base rate 0.5649 +resolution skill-v1 0.0499 champion 0.1660 +brier skill-v1 0.2839 champion 0.2506 +mean forecast skill-v1 0.5372 champion 0.6055 +delta vs champion −0.1161 CI95 [−0.1892, −0.0426] ← excludes zero +``` + +**Selectivity (D4), measured not claimed:** + +| top N by confidence | hit rate | lift over base | +|---|---|---| +| 8 | 0.500 | −0.065 | +| 15 | 0.533 | −0.032 | +| 25 | 0.480 | −0.085 | +| 100 | 0.550 | −0.015 | + +**It is not selective yet either** — its most confident picks are not its best +ones. That is the honest read and it is the second reason not to promote. + +### Two false starts caught in the build, both worth recording + +1. **A units bug.** `statcast_aggregates` stores percentages (0–100), not + fractions. The first run made `bip = 1 − 29.6 − 17.1` deeply negative and + **refused 568 of 576 rows.** The honest-absent guards worked exactly as + designed — a model that refuses is visible, a model that fabricates is not. + The conversion now lives at one chokepoint (`fromStatcastRow`), and an + out-of-range percentage is treated as broken (null), never clamped. +2. **An invalid first verdict.** The initial run resolved an opposing pitcher for + **1 of 570 rows** (0.18%) because `ledger_entries.team`/`opponent` are NULL on + 575 of 576 rows in this window. It would have reported "skill-v1 loses" while + silently measuring a **batter-profile-only** model with no matchup at all — + the exact false verdict this architecture exists to prevent. Fixed by + resolving the opponent from each player's own statsapi game log (91.9% + coverage). **The verdict in this document is from the corrected run.** + +## 6. What the loss does and does not say + +**Does say:** this implementation, on these inputs, is worse than the counter at +calling listed lines, and it does not go live. + +**Does not say the approach is dead** — several candidate features never fired in +this run, so the test was of a partial model: + +| feature | exercised? | +|---|---| +| batter K / BB / barrel / hard-hit / exit velo / launch | **yes** | +| pitcher K / BB / hard-hit allowed | **yes** (91.9% of rows) | +| `park_factor` | **no** — park was passed as 1.0 for every row | +| `handedness_platoon` | **no** — never wired into the tree | +| `opportunity_drift` | **no** — the one feature with measured residual signal | +| `projected_pa` | **crude** — season PA ÷ a constant 103 team games | + +And one structural handicap worth naming: the skill profiles are **season +aggregates with no recency at all**, while the champion carries a last-5 term. So +skill-v1 was asked to beat a recency-aware model using a 13-day-old season +average. That is a fair loss to record and an unfair basis for concluding that +skill inputs don't work. + +**Adding the pitcher moved resolution 0.0441 → 0.0499** — almost nothing. That is +the most interesting number here, and the honest next question: is the matchup +signal genuinely small at the 0.5 hits line, or is the log5 combine too blunt to +express it? + +## 7. Fixed along the way — the Statcast refresh was unreachable code + +`snapshotScheduler.tick()` returns for any hour not in `HOURS_UTC` +(14,19,22,1,3); the Statcast block then tested `h === STATCAST_HOUR_UTC` +(default **11**). That branch could never execute — the nightly refresh had never +run once, the aggregates sat frozen for 13 days while every consumer served them +as current, and **both of its alerts were inside the same dead branch**, so +nothing could report it. Same silently-guarded-out shape as the settlement +outage. + +It now runs on its **own tick**, independent of the snapshot hours. The old test +asserted only that the string `STATCAST_HOUR_UTC` appeared in the file and passed +happily the entire time; it is replaced with one that asserts the refresh is +**not reachable only via the snapshot-hours guard**. + +## 8. Next iteration — ranked by what the measurement actually showed + +1. **Recency in the skill profile.** The champion's edge is recency and skill-v1 + has none. Weight recent PA into the rates before concluding anything else. +2. **Wire the unexercised features** — park, handedness, and `opportunity_drift` + (the one feature with measured residual signal, S78). +3. **Real PA projection** from lineup slot, not season PA ÷ 103. +4. **Then re-run Stage A.** If it still loses with all of that firing, the + conclusion is much stronger and worth acting on. + +**Not recommended:** promoting anything, tuning the archetype weights to fit +1,741 rows, or relaxing the bar. The gate did its job today. diff --git a/src/services/model/featureRegistry.js b/src/services/model/featureRegistry.js new file mode 100644 index 0000000..f3a5a69 --- /dev/null +++ b/src/services/model/featureRegistry.js @@ -0,0 +1,234 @@ +'use strict'; + +/** + * featureRegistry — EARN ITS PLACE OR IT'S OUT (Discipline 3). + * + * This codebase has shipped six features that were wired, computed, retained, + * and contributed NOTHING to the served forecast. Measured: arch-v1 moved 76% of + * rows by 2.5 points and changed resolution by 0.0000; the environment and + * opportunity axes are flat; the projection ladder is reliably WORSE than the + * counter it was meant to beat. None of that was visible until someone went + * looking, because nothing in the architecture ever asked a feature to justify + * itself. + * + * So the gate is structural, not a habit: + * + * CANDIDATE — computed, measured in the CHALLENGER, NEVER in the live grade. + * PROVEN — marginal out-of-sample lift measured positive AND significant. + * Only PROVEN features may feed the served projection. + * DEAD — measured and found to add nothing (or to hurt). Kept in the + * registry BY NAME so it cannot be silently rebuilt a year later + * by someone who does not know it was already tried. + * + * THE ONE RULE THIS FILE EXISTS TO ENFORCE: + * + * liveFeatures() returns PROVEN only. A CANDIDATE cannot reach a user by + * accident, and a DEAD feature cannot come back without a new measurement. + * + * Promotion is DATA, not opinion: `promote()` refuses without a measurement + * carrying n, lift, and a confidence interval that excludes zero on the good + * side. There is deliberately no "force" argument. A feature that cannot show + * the number does not move. + * + * WHY EVERY FEATURE STARTS AS A CANDIDATE. Nothing here has earned PROVEN yet — + * the skill model is being measured for the first time. An empty PROVEN set is + * the honest starting state and it is what makes Stage A meaningful: the + * challenger runs on candidates, the ledger judges them, and only then does + * anything go live. Seeding this file with optimistic PROVEN flags would defeat + * the entire purpose of having it. + */ + +const STATUS = Object.freeze({ CANDIDATE: 'CANDIDATE', PROVEN: 'PROVEN', DEAD: 'DEAD' }); + +/** + * Minimum evidence to promote. Deliberately strict: a feature promoted on noise + * is worse than no feature, because it carries the authority of having been + * "measured". + */ +const MIN_PROMOTION_N = 200; + +/** + * THE REGISTRY. + * + * `sport` is first-class: per-sport doctrine means a feature proven for MLB says + * nothing about NBA, and the registry must not let one leak into the other. + * + * `evidence` is null until something measures it. `history` accumulates every + * status change with its measurement, so "why is this DEAD" always has an answer. + */ +const FEATURES = [ + // ── HITTER SKILL (Statcast) ────────────────────────────────────────────── + { key: 'batter_exit_velo', sport: 'mlb', status: STATUS.CANDIDATE, unit: 'mph', + describes: 'how hard he hits it — the skill under batting average' }, + { key: 'batter_launch_angle', sport: 'mlb', status: STATUS.CANDIDATE, unit: 'deg', + describes: 'launch profile: ground-ball hitter vs air hitter' }, + { key: 'batter_barrel_pct', sport: 'mlb', status: STATUS.CANDIDATE, unit: 'pct', + describes: 'rate of ideal exit-velo/launch combinations' }, + { key: 'batter_hard_hit_pct', sport: 'mlb', status: STATUS.CANDIDATE, unit: 'pct', + describes: '95+ mph contact rate' }, + { key: 'batter_whiff_pct', sport: 'mlb', status: STATUS.CANDIDATE, unit: 'pct', + describes: 'swing-and-miss rate — drives the strikeout branch' }, + { key: 'batter_k_pct', sport: 'mlb', status: STATUS.CANDIDATE, unit: 'pct', + describes: 'strikeout rate (the PA outcome that ends without a ball in play)' }, + { key: 'batter_bb_pct', sport: 'mlb', status: STATUS.CANDIDATE, unit: 'pct', + describes: 'walk rate — removes a PA from the hit-chance pool' }, + + // ── PITCHER SKILL (Statcast) ───────────────────────────────────────────── + { key: 'pitcher_whiff_pct', sport: 'mlb', status: STATUS.CANDIDATE, unit: 'pct', + describes: 'stuff — the skill under ERA' }, + { key: 'pitcher_k_pct', sport: 'mlb', status: STATUS.CANDIDATE, unit: 'pct', + describes: 'strikeout rate allowed' }, + { key: 'pitcher_bb_pct', sport: 'mlb', status: STATUS.CANDIDATE, unit: 'pct', + describes: 'walk rate allowed' }, + { key: 'pitcher_gb_pct', sport: 'mlb', status: STATUS.CANDIDATE, unit: 'pct', + describes: 'ground-ball tendency — suppresses air contact' }, + { key: 'pitcher_hard_hit_allowed', sport: 'mlb', status: STATUS.CANDIDATE, unit: 'pct', + describes: 'contact quality allowed' }, + { key: 'pitcher_barrel_allowed', sport: 'mlb', status: STATUS.CANDIDATE, unit: 'pct', + describes: 'barrel rate allowed — the extra-base driver' }, + + // ── MATCHUP STRUCTURE ──────────────────────────────────────────────────── + { key: 'handedness_platoon', sport: 'mlb', status: STATUS.CANDIDATE, unit: 'bool', + describes: 'batter/pitcher handedness edge' }, + + // ── CONTEXT ────────────────────────────────────────────────────────────── + { key: 'park_factor', sport: 'mlb', status: STATUS.CANDIDATE, unit: 'mult', + describes: "tonight's venue relative to the hitter's own exposure" }, + { key: 'projected_pa', sport: 'mlb', status: STATUS.CANDIDATE, unit: 'count', + describes: 'opportunity — how many times he bats' }, + { key: 'opportunity_drift', sport: 'mlb', status: STATUS.CANDIDATE, unit: 'ratio', + describes: 'recent AB vs season AB/game — the one feature measured to carry residual signal (S78: +0.156 hits, +0.145 TB)' }, + + // ── THE INCUMBENT ──────────────────────────────────────────────────────── + // The counter is IN THE REGISTRY, because the baseline has to be able to lose + // its place too. It is PROVEN — uniquely, it has the measurement: it is ~100% + // of the champion's resolution (S78 ablation, per stat, paired bootstrap). + { key: 'recent_frequency_prior', sport: 'mlb', status: STATUS.PROVEN, unit: 'prob', + describes: 'empirical frequency of clearing THIS line (the incumbent counter), demoted to a PRIOR in the skill model', + evidence: { + measured_at: '2026-08-03', n: 1415, source: 'specs/champion-input-diagnosis.md', + note: 'per-stat ablation: removing all three adjustment layers changes resolution by ~0, so base+recency IS the champion', + } }, + + // ── MEASURED AND FOUND WANTING — kept so they are not rebuilt ──────────── + { key: 'champion_opp_rank_adj', sport: 'mlb', status: STATUS.DEAD, unit: 'prob', + describes: 'the ±0.03 opponent bump inside probabilityEstimator', + evidence: { measured_at: '2026-08-03', n: 1415, source: 'specs/champion-input-diagnosis.md', + note: 'ablation delta within noise on every stat' } }, + { key: 'champion_home_away_adj', sport: 'mlb', status: STATUS.DEAD, unit: 'prob', + describes: 'the ±0.015 home/away bump inside probabilityEstimator', + evidence: { measured_at: '2026-08-03', n: 273, source: 'specs/champion-input-diagnosis.md', + note: 'removing it IMPROVED rbi resolution (+0.0053, CI [0.0002,0.0103]) — actively harmful' } }, + { key: 'champion_consistency_pull', sport: 'mlb', status: STATUS.DEAD, unit: 'prob', + describes: 'cv>0.40 shrink toward 0.50', + evidence: { measured_at: '2026-08-03', n: 1415, source: 'specs/champion-input-diagnosis.md', + note: 'ablation delta ≤0.0018 on every stat' } }, +]; + +const byKey = new Map(); +for (const f of FEATURES) byKey.set(`${f.sport}|${f.key}`, { ...f, evidence: f.evidence || null, history: [] }); + +const idOf = (sport, key) => `${String(sport || '').toLowerCase()}|${key}`; + +/** Every registered feature for a sport (any status). */ +function allFeatures(sport) { + const sp = String(sport || '').toLowerCase(); + return [...byKey.values()].filter((f) => f.sport === sp).map((f) => ({ ...f })); +} + +/** + * THE GATE. Only PROVEN features may feed the SERVED projection. + * + * Returns a Set of keys, because the caller's question is always "may I use + * this one?" and a Set makes the wrong answer awkward to write. + */ +function liveFeatures(sport) { + return new Set(allFeatures(sport).filter((f) => f.status === STATUS.PROVEN).map((f) => f.key)); +} + +/** Features the CHALLENGER is allowed to measure — candidates plus what's live. */ +function candidateFeatures(sport) { + return new Set(allFeatures(sport) + .filter((f) => f.status === STATUS.CANDIDATE || f.status === STATUS.PROVEN) + .map((f) => f.key)); +} + +function statusOf(sport, key) { + const f = byKey.get(idOf(sport, key)); + return f ? f.status : null; +} + +/** Is this feature allowed in the LIVE path right now? */ +function isLive(sport, key) { + return statusOf(sport, key) === STATUS.PROVEN; +} + +/** + * Evidence sufficient to promote: a real sample, positive lift, and a CI that + * excludes zero on the good side. Anything less is a story about a number. + */ +function isSufficient(evidence) { + if (!evidence || typeof evidence !== 'object') return false; + const n = Number(evidence.n); + const lift = Number(evidence.lift); + const ci = evidence.ci95; + if (!Number.isFinite(n) || n < MIN_PROMOTION_N) return false; + if (!Number.isFinite(lift) || lift <= 0) return false; + if (!Array.isArray(ci) || ci.length !== 2) return false; + const [lo, hi] = ci.map(Number); + if (!Number.isFinite(lo) || !Number.isFinite(hi)) return false; + return lo > 0; // the whole interval above zero — improvement, not a coin flip +} + +/** + * Promote a CANDIDATE to PROVEN. Refuses without sufficient evidence, and there + * is no override parameter on purpose. + */ +function promote(sport, key, evidence, at = null) { + const f = byKey.get(idOf(sport, key)); + if (!f) return { ok: false, reason: 'unknown_feature' }; + if (!isSufficient(evidence)) return { ok: false, reason: 'insufficient_evidence', required: { min_n: MIN_PROMOTION_N, lift: '>0', ci95_low: '>0' } }; + f.history.push({ from: f.status, to: STATUS.PROVEN, evidence, at }); + f.status = STATUS.PROVEN; + f.evidence = evidence; + return { ok: true, status: f.status }; +} + +/** + * Demote to DEAD. Unlike promotion this needs no threshold — removing something + * that is not helping is always safe, and the standing re-ablation must be able + * to act the moment a feature stops earning its place. + */ +function demote(sport, key, evidence, at = null) { + const f = byKey.get(idOf(sport, key)); + if (!f) return { ok: false, reason: 'unknown_feature' }; + f.history.push({ from: f.status, to: STATUS.DEAD, evidence: evidence || null, at }); + f.status = STATUS.DEAD; + f.evidence = evidence || f.evidence; + return { ok: true, status: f.status }; +} + +/** Registry summary — what is live, what is being measured, what died and why. */ +function summary(sport) { + const fs = allFeatures(sport); + const pick = (s) => fs.filter((f) => f.status === s).map((f) => f.key); + return { + sport: String(sport || '').toLowerCase(), + proven: pick(STATUS.PROVEN), + candidate: pick(STATUS.CANDIDATE), + dead: pick(STATUS.DEAD).map((k) => ({ key: k, why: (byKey.get(idOf(sport, k)).evidence || {}).note || null })), + total: fs.length, + }; +} + +/** Test-only: restore the declared statuses so suites cannot leak into each other. */ +function __reset() { + byKey.clear(); + for (const f of FEATURES) byKey.set(`${f.sport}|${f.key}`, { ...f, evidence: f.evidence || null, history: [] }); +} + +module.exports = { + STATUS, MIN_PROMOTION_N, + allFeatures, liveFeatures, candidateFeatures, statusOf, isLive, + isSufficient, promote, demote, summary, __reset, +}; diff --git a/src/services/model/skillProjection.js b/src/services/model/skillProjection.js new file mode 100644 index 0000000..46527c7 --- /dev/null +++ b/src/services/model/skillProjection.js @@ -0,0 +1,417 @@ +'use strict'; + +/** + * skillProjection (skill-v1) — A FORWARD READ OF TONIGHT'S PLATE APPEARANCES. + * + * This is the windshield. The incumbent is a rear-view mirror: it counts how + * often the hitter cleared this number lately and calls that a forecast. It has + * never seen the pitcher. Measured, that counter IS the champion — removing all + * three of its adjustment layers changes resolution by nothing — so beating it + * requires different INFORMATION, not a different distribution. Two challengers + * (proj-v1.1, hits-v1) already proved that by swapping the distribution and + * losing. + * + * ── DISCIPLINE 1: SKILL, NOT RESULTS ───────────────────────────────────── + * Every input here measures the skill that CAUSES the outcome. Exit velocity, + * not batting average. Whiff rate, not ERA. Results are luck-contaminated — + * a .340 BABIP regresses and a 95-mph average exit velocity does not — so a + * model built on results is fitting noise it will not see again. + * + * ── THE GENERATIVE STRUCTURE ───────────────────────────────────────────── + * A hitter gets N plate appearances. Each one resolves through a tree: + * + * PA ──> strikeout (batter whiff/K vs pitcher whiff/K) + * ──> walk (batter BB vs pitcher BB) + * ──> ball in play (everything else) + * └─> hit? (contact quality vs contact allowed, park) + * └─> how many bases? (launch/barrel profile) + * + * Rates are combined by the ODDS RATIO (log5) against league average — the + * standard way to answer "this batter vs THIS pitcher" rather than "this batter + * vs the average pitcher": + * + * odds = (b/(1−b)) · (p/(1−p)) / (lg/(1−lg)) + * + * With b = league and p = league it returns league; with an average pitcher it + * returns the batter's own rate. That identity is the reason to use it, and it + * is unit-tested here rather than assumed. + * + * ── DISCIPLINE 2: ARCHETYPE SELECTS FEATURES ───────────────────────────── + * The archetype is NOT a nudge and NOT a label. It decides WHICH skill inputs + * drive THIS hitter. A BOMBER's hit outcomes are governed by barrel rate and + * launch — his ground balls are outs. A GHOST beats out infield hits, so ground- + * ball rate and speed matter and barrels are nearly irrelevant. Feeding both + * hitters the same feature weights is exactly the "one model fit to all" error + * the per-sport doctrine forbids, one level further down. + * + * Features are ACTIVE where they apply and SILENT otherwise — silent meaning + * they contribute nothing, never that a default is invented in their place. + * + * ── DISCIPLINE 3: THE GATE ─────────────────────────────────────────────── + * Every input is checked against `featureRegistry`. Nothing that has not earned + * PROVEN can reach a served projection; the challenger runs on CANDIDATEs so + * they can be measured. `allowed` is passed in rather than read here so the + * caller decides which gate applies, and both are testable. + * + * ── HONESTY ────────────────────────────────────────────────────────────── + * Unknown is not zero, everywhere: a missing rate makes its branch SILENT, it + * never becomes a measured 0. If the hitter's own skill profile is missing there + * is no forward read at all and the function returns null — the caller falls + * back rather than being handed a confident guess. + */ + +const { knownRate } = require('../../utils/known'); + +/** + * League baselines, MLB. Used ONLY as the denominator of the odds ratio and as + * the regression target for thin samples — never as a substitute for a missing + * player. Measured from the 2026 Statcast aggregate set. + */ +const LEAGUE = Object.freeze({ + k_pct: 0.222, + bb_pct: 0.085, + babip: 0.291, // hits per ball in play + hard_hit_pct: 0.389, + barrel_pct: 0.078, + avg_exit_velo: 88.8, + bases_per_hit: 1.60, // league slugging ÷ batting average +}); + +/** Plate appearances per game for a regular; only used when nothing better is known. */ +const DEFAULT_PA = 4.1; +const PA_CAP = 7; +const MIN_PA = 1; + +/** + * ODDS RATIO (log5). "This batter against THIS pitcher", relative to league. + * + * Returns null when either side is unknown — an absent pitcher must leave the + * batter's own rate untouched, which the caller does by skipping the combine, + * NOT by substituting league for the missing side (that would quietly pull every + * unknown matchup toward average and call it a read). + */ +function oddsRatio(batterRate, pitcherRate, leagueRate) { + const b = knownRate(batterRate); + const p = knownRate(pitcherRate); + const l = knownRate(leagueRate); + if (b === null || p === null || l === null) return null; + if (b <= 0 && p <= 0) return 0; + if (b >= 1 || p >= 1) return 1; + if (l <= 0 || l >= 1) return null; + const ob = b / (1 - b); + const op = p / (1 - p); + const ol = l / (1 - l); + if (ol === 0) return null; + const odds = (ob * op) / ol; + if (!Number.isFinite(odds)) return null; + return odds / (1 + odds); +} + +/** + * Shrink a rate toward an explicit anchor by sample size. A 12-PA callup with a + * 60% hard-hit rate is not a 60% hard-hit hitter; without this the model chases + * noise exactly the way a results-based model does. + * + * The anchor is a REQUIRED argument rather than an internal default, because a + * hidden regression target is how a model quietly becomes the league average + * wearing a player's name. At the call site you can always see what it is being + * pulled toward. + */ +function shrink(rate, sample, stabilizeAt, anchor) { + const r = knownRate(rate); + const a = knownRate(anchor); + if (r === null) return null; + if (a === null) return r; + const n = knownRate(sample); + const k = knownRate(stabilizeAt); + if (n === null || k === null || k <= 0) return r; + const w = n / (n + k); + return w * r + (1 - w) * a; +} + +/** + * THE UNITS CHOKEPOINT — `statcast_aggregates` stores PERCENTAGES (0–100). + * + * Baseball Savant's CSVs give `k_percent: 29.6`, not `0.296`, and the aggregate + * table stores them verbatim. This model works in probabilities. Feeding the raw + * row straight in makes `bip = 1 − 29.6 − 17.1` deeply negative, which is how + * this was caught: the first Stage A run refused 568 of 576 real rows rather + * than emitting nonsense. The honest-absent guards did their job — but a model + * that refuses everything is not a model, so the conversion lives HERE, once, at + * the boundary, and every consumer goes through it. + * + * Velocity (mph) and launch angle (degrees) are ALREADY natural units and must + * not be scaled. Mixing the two is precisely the trap, so the split is explicit + * rather than inferred from the value's magnitude — a 0.8% barrel rate and a + * 0.8 fraction are indistinguishable by size, and guessing would silently turn + * an elite hitter into a replacement one. + */ +const PCT_FIELDS = Object.freeze([ + 'k_pct', 'bb_pct', 'whiff_pct', 'swing_pct', 'chase_pct', 'barrel_pct', + 'hard_hit_pct', 'sweet_spot_pct', 'ev95_pct', 'gb_pct', 'fb_pct', 'ld_pct', +]); +const RAW_FIELDS = Object.freeze(['avg_exit_velo', 'max_exit_velo', 'avg_launch_angle', 'arm_angle']); + +/** + * Convert a `statcast_aggregates` row into the probability-space profile this + * model expects. Absent stays absent; an out-of-range percentage is treated as + * BROKEN (null) rather than clamped, because a 140% whiff rate is not a thin + * measurement, it is a parsing error and must not be modelled. + */ +function fromStatcastRow(row) { + if (!row || typeof row !== 'object') return null; + const out = {}; + for (const f of PCT_FIELDS) { + const v = knownRate(row[f]); + out[f] = (v === null || v > 100) ? null : v / 100; + } + for (const f of RAW_FIELDS) out[f] = knownRate(row[f]); + out.bats = row.bats ?? null; + out.throws = row.throws ?? null; + out.sample_pa = knownRate(row.sample_pa); + out.sample_bip = knownRate(row.sample_bip); + return out; +} + +/** + * ARCHETYPE FEATURE MAPS — Discipline 2, stated as data. + * + * `hitWeights` decide what governs whether a ball in play becomes a hit for THIS + * kind of hitter. `powerWeight` scales how much his contact quality converts to + * EXTRA bases. Weights are relative within a map and are documented by what the + * archetype physically does, not fitted — fitting them on 1,741 settled rows + * would be curve-fitting, and the registry exists so they get measured instead. + * + * Unknown archetype → DEFAULT, which is a balanced map, not a refusal: we still + * know the hitter's own skill rates. + */ +const ARCHETYPE_MAP = Object.freeze({ + // Elite power. Barrels and launch decide his outcomes; his grounders are outs. + BOMBER: { hitWeights: { barrel: 0.5, hard_hit: 0.35, exit_velo: 0.15, gb_speed: 0 }, powerWeight: 1.25 }, + // Speed-first. Beats out grounders; barrels are close to irrelevant. + GHOST: { hitWeights: { barrel: 0.05, hard_hit: 0.2, exit_velo: 0.15, gb_speed: 0.6 }, powerWeight: 0.7 }, + // Pure contact, sprays it, low whiff. + TORCH: { hitWeights: { barrel: 0.2, hard_hit: 0.35, exit_velo: 0.25, gb_speed: 0.2 }, powerWeight: 1.0 }, + DEFAULT: { hitWeights: { barrel: 0.25, hard_hit: 0.35, exit_velo: 0.25, gb_speed: 0.15 }, powerWeight: 1.0 }, +}); + +function featureMapFor(archetype) { + const key = String(archetype || '').toUpperCase(); + return ARCHETYPE_MAP[key] || ARCHETYPE_MAP.DEFAULT; +} + +/** + * Per-ball-in-play hit probability, built from the ARCHETYPE-SELECTED skill + * inputs, the pitcher's contact suppression, and the park. + * + * Each component is a RATIO to league, so a hitter exactly at league on every + * axis lands on league BABIP and the model says "average" rather than inventing + * a lean. Components with no data are SILENT: they drop out of the weighted + * average and the remaining weights renormalise, so an absent input never acts + * as a measured zero. + */ +function hitOnContact({ batter, pitcher, park, archetype, allowed }) { + const map = featureMapFor(archetype); + const can = (k) => !allowed || allowed.has(k); + + const parts = []; + const push = (weight, value, leagueValue, gateKey) => { + if (!weight || !can(gateKey)) return; + const v = knownRate(value); + const l = knownRate(leagueValue); + if (v === null || l === null || l <= 0) return; // SILENT, not zero + parts.push({ weight, ratio: v / l }); + }; + + push(map.hitWeights.barrel, batter.barrel_pct, LEAGUE.barrel_pct, 'batter_barrel_pct'); + push(map.hitWeights.hard_hit, batter.hard_hit_pct, LEAGUE.hard_hit_pct, 'batter_hard_hit_pct'); + push(map.hitWeights.exit_velo, batter.avg_exit_velo, LEAGUE.avg_exit_velo, 'batter_exit_velo'); + // GB/speed lane: a ground-ball hitter's hits come from beating out contact, so + // his OWN ground-ball tilt is the driver. Launch angle stands in for it — + // lower launch = more grounders — inverted so "more grounders" reads as more. + if (map.hitWeights.gb_speed && can('batter_launch_angle')) { + const la = knownRate(batter.avg_launch_angle); + if (la !== null) { + // League launch ~12°. Below it → grounder tilt → ratio > 1 for a GHOST. + const ratio = 12 > 0 ? (2 - Math.min(2, Math.max(0.2, la / 12))) : null; + if (ratio !== null) parts.push({ weight: map.hitWeights.gb_speed, ratio }); + } + } + + if (parts.length === 0) return null; // no skill signal at all — honest-absent + + const wSum = parts.reduce((a, p) => a + p.weight, 0); + if (wSum <= 0) return null; + const skillRatio = parts.reduce((a, p) => a + p.weight * p.ratio, 0) / wSum; + + // Pitcher contact suppression: hard contact ALLOWED relative to league. + let pitcherRatio = 1; + if (can('pitcher_hard_hit_allowed')) { + const ph = knownRate(pitcher && pitcher.hard_hit_pct); + if (ph !== null && LEAGUE.hard_hit_pct > 0) pitcherRatio = ph / LEAGUE.hard_hit_pct; + } + + const parkMult = can('park_factor') ? (knownRate(park) ?? 1) : 1; + + // Bounded: no stack of ratios may claim more than a ±35% swing in BABIP. The + // bound is a statement about how much any of this can really know, not a fudge. + const raw = LEAGUE.babip * skillRatio * pitcherRatio * parkMult; + const lo = LEAGUE.babip * 0.65; + const hi = LEAGUE.babip * 1.35; + return Math.min(hi, Math.max(lo, raw)); +} + +/** + * The PA outcome tree for one hitter against one pitcher. + * Returns per-PA probabilities, or null when the hitter has no usable profile. + */ +function paOutcome({ batter, pitcher, park, archetype, allowed }) { + if (!batter) return null; + const can = (k) => !allowed || allowed.has(k); + + // Strikeout: batter K vs pitcher K, odds-ratio against league. + const bK = can('batter_k_pct') ? knownRate(batter.k_pct) : null; + const pK = can('pitcher_k_pct') ? knownRate(pitcher && pitcher.k_pct) : null; + let kRate = bK; + if (bK !== null && pK !== null) kRate = oddsRatio(bK, pK, LEAGUE.k_pct) ?? bK; + + const bBB = can('batter_bb_pct') ? knownRate(batter.bb_pct) : null; + const pBB = can('pitcher_bb_pct') ? knownRate(pitcher && pitcher.bb_pct) : null; + let bbRate = bBB; + if (bBB !== null && pBB !== null) bbRate = oddsRatio(bBB, pBB, LEAGUE.bb_pct) ?? bBB; + + // Absent → league, because a PA still has to resolve into SOMETHING. This is + // the one place a league value substitutes, and it is a structural necessity + // (the branches must sum to 1), not a guess about the player. + const k = kRate === null ? LEAGUE.k_pct : kRate; + const bb = bbRate === null ? LEAGUE.bb_pct : bbRate; + const bip = Math.max(0, 1 - k - bb); + if (bip <= 0) return null; + + const pHitOnContact = hitOnContact({ batter, pitcher, park, archetype, allowed }); + if (pHitOnContact === null) return null; // no skill read → no forward projection + + return { + k_rate: k, + bb_rate: bb, + bip_rate: bip, + hit_on_contact: pHitOnContact, + p_hit_per_pa: bip * pHitOnContact, + inputs_used: { + pitcher_applied: pK !== null || pBB !== null || knownRate(pitcher && pitcher.hard_hit_pct) !== null, + archetype: String(archetype || 'DEFAULT').toUpperCase(), + }, + }; +} + +/** Exact Binomial(n, p) pmf — n is tiny (PA per game ≤ 7). */ +function binomialPmf(n, p) { + const nn = Math.max(0, Math.round(n)); + const pp = Math.min(1, Math.max(0, p)); + const out = new Array(nn + 1).fill(0); + let term = (1 - pp) ** nn; + out[0] = term; + for (let x = 1; x <= nn; x += 1) { + if (pp >= 1) { out[x] = x === nn ? 1 : 0; continue; } + term = (term * (nn - x + 1) * pp) / (x * (1 - pp)); + out[x] = term; + } + return out; +} + +/** + * Distribution over HITS, mixing over an integer PA distribution. PA is not + * fixed — a hitter gets 4 or 5 depending on how the lineup turns over — so the + * projection mixes rather than pretending PA is known. + */ +function paDistribution(expectedPa) { + const mean = Math.min(PA_CAP, Math.max(MIN_PA, knownRate(expectedPa) ?? DEFAULT_PA)); + const lo = Math.floor(mean); + const hi = Math.min(PA_CAP, lo + 1); + const wHi = mean - lo; + const out = new Array(PA_CAP + 1).fill(0); + out[lo] += 1 - wHi; + if (hi !== lo) out[hi] += wHi; else out[lo] = 1; + return out; +} + +/** P(X >= k) from a pmf. */ +function atLeast(pmf, k) { + const kk = Math.max(0, Math.ceil(k)); + if (kk === 0) return 1; + let s = 0; + for (let i = kk; i < pmf.length; i += 1) s += pmf[i]; + return Math.min(1, Math.max(0, s)); +} + +/** + * THE FORWARD READ. + * + * @returns {object|null} distribution + P(clears line) + the reasoning trace, or + * null when there is no usable skill profile (caller falls back; never a guess). + */ +function projectSkill({ + batter, pitcher = null, park = 1, archetype = null, + statType = 'hits', line, expectedPa = null, allowed = null, +} = {}) { + // STAGE A IS HITS ONLY, and total_bases is refused DELIBERATELY. + // + // The first cut mapped hits to bases with an archetype-scaled constant + // (bases_per_hit x powerWeight). Smoke-tested on a real profile that made + // P(TB>=2) come out EXACTLY equal to P(hits>=1) — because a deterministic + // multiplier just relabels the hits distribution. It would have measured as a + // "TB model" while carrying no information a hits model does not already have. + // tb-v1 already established the right shape: total bases is a COMPOUND + // outcome (1B/2B/3B/HR each with its own rate), not hits x a constant. Doing + // that properly needs a per-hit extra-base distribution off the launch/barrel + // profile, which is its own build. Refusing beats shipping a relabel. + const stat = String(statType || '').toLowerCase(); + if (stat !== 'hits') return null; + const target = Math.max(1, Math.ceil(Number(line))); + if (!Number.isFinite(target)) return null; + + const pa = paOutcome({ batter, pitcher, park, archetype, allowed }); + if (!pa) return null; + + const paPmf = paDistribution(expectedPa); + const map = featureMapFor(archetype); + + // HITS — Binomial(PA, p_hit_per_pa), mixed over the PA distribution. + const hitPmf = new Array(PA_CAP + 1).fill(0); + for (let n = 0; n < paPmf.length; n += 1) { + if (!paPmf[n]) continue; + const bp = binomialPmf(n, pa.p_hit_per_pa); + for (let x = 0; x < bp.length; x += 1) hitPmf[x] += paPmf[n] * bp[x]; + } + const expectedHits = hitPmf.reduce((a, p, i) => a + p * i, 0); + + return buildResult({ pmf: hitPmf, mean: expectedHits, target, pa, map, archetype, stat }); +} + +function buildResult({ pmf, mean, target, pa, map, archetype, stat, caveat = null }) { + const p = atLeast(pmf, target); + const r3 = (v) => Math.round(v * 1000) / 1000; + return { + version: 'skill-v1', + stat, + p_over_line: r3(p), + projected_value: r3(mean), + distribution: pmf.map(r3), + per_pa: { + k_rate: r3(pa.k_rate), bb_rate: r3(pa.bb_rate), + bip_rate: r3(pa.bip_rate), hit_on_contact: r3(pa.hit_on_contact), + p_hit_per_pa: r3(pa.p_hit_per_pa), + }, + archetype: String(archetype || 'DEFAULT').toUpperCase(), + feature_map: map.hitWeights, + pitcher_applied: pa.inputs_used.pitcher_applied, + family: 'pa_outcome_tree_binomial', + caveat, + }; +} + +module.exports = { + projectSkill, paOutcome, hitOnContact, oddsRatio, shrink, fromStatcastRow, + binomialPmf, paDistribution, atLeast, featureMapFor, + LEAGUE, ARCHETYPE_MAP, DEFAULT_PA, PA_CAP, PCT_FIELDS, RAW_FIELDS, +}; diff --git a/src/snapshotScheduler.js b/src/snapshotScheduler.js index 817dd2c..c58ab04 100644 --- a/src/snapshotScheduler.js +++ b/src/snapshotScheduler.js @@ -314,34 +314,6 @@ function startSnapshotScheduler(opts = {}) { } } catch (e) { console.warn('[instrument] closing attach failed:', e.message); } - // Session 68 — LAYER 1 MECHANISM DATA. Nightly full re-pull of the - // ~1,350-row Statcast aggregate set at STATCAST_HOUR_UTC. Backfill and - // refresh are the SAME call, upserted on the natural key, so the job is - // idempotent and self-healing: a missed night self-corrects on the next - // run with no incremental bookkeeping to drift. Kill switch: STATCAST=0. - try { - const statcastHour = Number(process.env.STATCAST_HOUR_UTC || 11); // ~7 AM ET, after every game is final - if (h === statcastHour && process.env.STATCAST !== '0') { - const agg = opts.statcastService || require('./services/statcastAggregateService'); - const res = await agg.refreshSeason({}); - console.log(`[statcast] nightly refresh — ok=${res.ok} rows=${res.rows ?? 'n/a'} written=${res.written ?? 0} joined=${res.joined ?? 'n/a'} thin=${res.thin ?? 'n/a'}${res.reason ? ` reason: ${res.reason}` : ''}`); - if (!res.ok) { - await notify(`Statcast refresh failed — ${res.reason || 'unknown'}. Mechanism data is not updating.`, { - title: 'VYNDR statcast', priority: 'high', tags: ['rotating_light'], - }); - } - // Serving a stale aggregate as current is a quiet fabrication of - // currency, so staleness pages on its own — separate from a failed run, - // because a silently-not-scheduled job never produces a failure. - const fresh = await agg.getFreshness({}); - if (agg.isStale(fresh)) { - await notify(`Statcast aggregates stale — last updated ${fresh.updated_at} (${fresh.age_hours}h). Mechanism data is being served as current.`, { - title: 'VYNDR statcast', priority: 'high', tags: ['rotating_light'], - }); - } - } - } catch (e) { console.warn('[statcast] nightly refresh failed:', e.message); } - // Session 8 — quota check after each snapshot run: odds-api >= 80% alerts // once per day (Redis-deduped). Never throws (guarded inside checkQuotaDaily). await opsWatch.checkQuotaDaily({ getStatus: getQuotaStatus, cacheGet, cacheSet, notify, now: () => now() }); @@ -378,7 +350,57 @@ function startSnapshotScheduler(opts = {}) { } }; - const interval = setInterval(() => { void tick(); void refreshTick(); }, 60_000); + /** + * Session 68 — LAYER 1 MECHANISM DATA. Nightly full re-pull of the ~1,350-row + * Statcast aggregate set at STATCAST_HOUR_UTC. Backfill and refresh are the + * SAME call, upserted on the natural key, so the job is idempotent and + * self-healing. Kill switch: STATCAST=0. + * + * ITS OWN TICK, and that is the whole point (fixed 2026-08-03). This block + * used to live inside `tick()`, BELOW `if (!HOURS_UTC.includes(h)) return`. + * HOURS_UTC is 14,19,22,1,3 and the block tests `h === STATCAST_HOUR_UTC` + * (default 11), so the guard could never admit the hour it was waiting for: + * the refresh was unreachable code and had never run. The aggregates sat + * frozen at their 2026-07-21 backfill for 13 days while every consumer served + * them as current — and because BOTH alerts below also sat inside the dead + * branch, nothing could report it. Same silently-guarded-out shape as the + * settlement outage. The skill model reads this table, so stale mechanism data + * is not a cosmetic problem: it is the model looking at a fortnight-old game. + */ + let lastStatcastSlot = null; + const statcastTick = async () => { + if (process.env.STATCAST === '0') return; + const d = now(); + if (d.getUTCMinutes() !== 0) return; + const h = d.getUTCHours(); + const statcastHour = Number(process.env.STATCAST_HOUR_UTC || 11); // ~7 AM ET, after every game is final + if (h !== statcastHour) return; + const slot = `${d.toISOString().slice(0, 10)}-${h}`; + if (slot === lastStatcastSlot) return; // fire once per day + lastStatcastSlot = slot; + try { + const agg = opts.statcastService || require('./services/statcastAggregateService'); + const res = await agg.refreshSeason({}); + console.log(`[statcast] nightly refresh — ok=${res.ok} rows=${res.rows ?? 'n/a'} written=${res.written ?? 0} joined=${res.joined ?? 'n/a'} thin=${res.thin ?? 'n/a'}${res.reason ? ` reason: ${res.reason}` : ''}`); + if (!res.ok) { + await notify(`Statcast refresh failed — ${res.reason || 'unknown'}. Mechanism data is not updating.`, { + title: 'VYNDR statcast', priority: 'high', tags: ['rotating_light'], + }); + } + // Serving a stale aggregate as current is a quiet fabrication of currency, + // so staleness pages on its own — separate from a failed run, because a + // silently-not-scheduled job never produces a failure. Which is exactly + // what happened here for 13 days. + const fresh = await agg.getFreshness({}); + if (agg.isStale(fresh)) { + await notify(`Statcast aggregates stale — last updated ${fresh.updated_at} (${fresh.age_hours}h). Mechanism data is being served as current.`, { + title: 'VYNDR statcast', priority: 'high', tags: ['rotating_light'], + }); + } + } catch (e) { console.warn('[statcast] nightly refresh failed:', e.message); } + }; + + const interval = setInterval(() => { void tick(); void refreshTick(); void statcastTick(); }, 60_000); if (interval.unref) interval.unref(); console.log(`[snapshotScheduler] armed — SNAPSHOT_CRON=${process.env.SNAPSHOT_CRON}, hours=${HOURS_UTC.join(',')} UTC, intraday=${process.env.INTRADAY_REFRESH === '0' ? 'off' : `${REFRESH_MINUTES}m (slate hours)`}, statcast=${process.env.STATCAST === '0' ? 'off' : `${process.env.STATCAST_HOUR_UTC || 11}h UTC`}`); // Job 1 — state each sport's cadence at boot so "when does WNBA grade?" is diff --git a/tests/unit/skillModel.test.js b/tests/unit/skillModel.test.js new file mode 100644 index 0000000..ae0c38b --- /dev/null +++ b/tests/unit/skillModel.test.js @@ -0,0 +1,197 @@ +'use strict'; + +/** + * THE FIVE DISCIPLINES, asserted as behaviour. + * + * These are not coverage tests. Each block locks one discipline so it cannot be + * quietly abandoned later — which is exactly how the previous engine drifted + * into a frequency counter with six wired-but-dead features hanging off it. + */ + +const reg = require('../../src/services/model/featureRegistry'); +const sk = require('../../src/services/model/skillProjection'); + +const BOMBER = { k_pct: 0.28, bb_pct: 0.11, barrel_pct: 0.16, hard_hit_pct: 0.52, avg_exit_velo: 93.5, avg_launch_angle: 19 }; +const GHOST = { k_pct: 0.14, bb_pct: 0.07, barrel_pct: 0.02, hard_hit_pct: 0.30, avg_exit_velo: 85.0, avg_launch_angle: 6 }; +const ACE = { k_pct: 0.32, bb_pct: 0.05, hard_hit_pct: 0.30, whiff_pct: 0.33 }; +const BATTING_PRACTICE = { k_pct: 0.15, bb_pct: 0.10, hard_hit_pct: 0.46, whiff_pct: 0.18 }; + +afterEach(() => reg.__reset()); + +// ── DISCIPLINE 3 ────────────────────────────────────────────────────────── +describe('D3 — earn its place or it is out', () => { + it('starts with an essentially EMPTY proven set — nothing is assumed to work', () => { + const live = reg.liveFeatures('mlb'); + // Only the incumbent counter is PROVEN, and only because it was measured. + expect([...live]).toEqual(['recent_frequency_prior']); + }); + + it('refuses promotion without a real sample, positive lift, and a CI above zero', () => { + expect(reg.promote('mlb', 'batter_barrel_pct', null).ok).toBe(false); + expect(reg.promote('mlb', 'batter_barrel_pct', { n: 10, lift: 0.05, ci95: [0.01, 0.09] }).ok).toBe(false); // n too small + expect(reg.promote('mlb', 'batter_barrel_pct', { n: 500, lift: -0.02, ci95: [-0.05, -0.01] }).ok).toBe(false); // negative lift + expect(reg.promote('mlb', 'batter_barrel_pct', { n: 500, lift: 0.03, ci95: [-0.01, 0.07] }).ok).toBe(false); // CI straddles zero + expect(reg.statusOf('mlb', 'batter_barrel_pct')).toBe(reg.STATUS.CANDIDATE); + }); + + it('promotes on sufficient evidence, and records WHY', () => { + const ev = { n: 600, lift: 0.04, ci95: [0.012, 0.068], measured_at: '2026-09-01' }; + expect(reg.promote('mlb', 'batter_barrel_pct', ev).ok).toBe(true); + expect(reg.isLive('mlb', 'batter_barrel_pct')).toBe(true); + expect(reg.liveFeatures('mlb').has('batter_barrel_pct')).toBe(true); + }); + + it('keeps DEAD features BY NAME so they are not silently rebuilt', () => { + const s = reg.summary('mlb'); + const deadKeys = s.dead.map((d) => d.key); + expect(deadKeys).toEqual(expect.arrayContaining([ + 'champion_opp_rank_adj', 'champion_home_away_adj', 'champion_consistency_pull', + ])); + // and each carries the reason it died + expect(s.dead.find((d) => d.key === 'champion_home_away_adj').why).toMatch(/harmful|IMPROVED/i); + }); + + it('per-sport: a feature proven for MLB says nothing about another sport', () => { + reg.promote('mlb', 'batter_barrel_pct', { n: 600, lift: 0.04, ci95: [0.01, 0.07] }); + expect(reg.isLive('nba', 'batter_barrel_pct')).toBe(false); + expect(reg.statusOf('nba', 'batter_barrel_pct')).toBeNull(); + }); + + it('THE GATE: with only PROVEN features allowed, the skill model REFUSES', () => { + // The proven set carries no contact-quality feature yet, so there is no + // forward read to make. Refusing is the correct output — this is what stops + // an unproven model reaching a user by accident. + const out = sk.projectSkill({ + batter: BOMBER, pitcher: ACE, archetype: 'BOMBER', statType: 'hits', line: 0.5, + expectedPa: 4.2, allowed: reg.liveFeatures('mlb'), + }); + expect(out).toBeNull(); + }); + + it('with CANDIDATES allowed (the challenger), it produces a read', () => { + const out = sk.projectSkill({ + batter: BOMBER, pitcher: ACE, archetype: 'BOMBER', statType: 'hits', line: 0.5, + expectedPa: 4.2, allowed: reg.candidateFeatures('mlb'), + }); + expect(out).not.toBeNull(); + expect(out.p_over_line).toBeGreaterThan(0); + }); +}); + +// ── DISCIPLINE 1 ────────────────────────────────────────────────────────── +describe('D1 — skill, not results: the pitcher actually moves the read', () => { + const run = (pitcher) => sk.projectSkill({ + batter: BOMBER, pitcher, archetype: 'BOMBER', statType: 'hits', line: 0.5, expectedPa: 4.2, + }); + + it('a better pitcher LOWERS the projection for the same hitter', () => { + const vsAce = run(ACE); + const vsBP = run(BATTING_PRACTICE); + expect(vsAce.p_over_line).toBeLessThan(vsBP.p_over_line); + expect(vsAce.projected_value).toBeLessThan(vsBP.projected_value); + }); + + it('this is the thing the counter cannot do — no pitcher at all is a different read', () => { + const noPitcher = run(null); + expect(noPitcher).not.toBeNull(); + expect(noPitcher.pitcher_applied).toBe(false); + expect(run(ACE).pitcher_applied).toBe(true); + }); + + it('the odds ratio returns league when both sides are league (the identity)', () => { + expect(sk.oddsRatio(0.222, 0.222, 0.222)).toBeCloseTo(0.222, 6); + }); + + it('an average pitcher leaves the batter rate untouched', () => { + expect(sk.oddsRatio(0.30, 0.222, 0.222)).toBeCloseTo(0.30, 6); + }); + + it('shrinks a thin sample toward its anchor, and leaves a full one alone', () => { + expect(sk.shrink(0.60, 10, 200, 0.39)).toBeLessThan(0.45); // 10-PA callup + expect(sk.shrink(0.60, 5000, 200, 0.39)).toBeGreaterThan(0.59); // established + }); +}); + +// ── DISCIPLINE 2 ────────────────────────────────────────────────────────── +describe('D2 — archetype SELECTS features, it does not nudge', () => { + it('the SAME hitter read through two archetypes gets materially different reads', () => { + const asBomber = sk.projectSkill({ batter: BOMBER, pitcher: ACE, archetype: 'BOMBER', statType: 'hits', line: 0.5, expectedPa: 4.2 }); + const asGhost = sk.projectSkill({ batter: BOMBER, pitcher: ACE, archetype: 'GHOST', statType: 'hits', line: 0.5, expectedPa: 4.2 }); + // Not a rounding difference — a different feature map entirely. + expect(Math.abs(asBomber.p_over_line - asGhost.p_over_line)).toBeGreaterThan(0.15); + }); + + it("a BOMBER's read is driven by barrels; a GHOST's is nearly blind to them", () => { + expect(sk.featureMapFor('BOMBER').hitWeights.barrel).toBeGreaterThan(0.4); + expect(sk.featureMapFor('GHOST').hitWeights.barrel).toBeLessThan(0.1); + expect(sk.featureMapFor('GHOST').hitWeights.gb_speed).toBeGreaterThan(0.5); + expect(sk.featureMapFor('BOMBER').hitWeights.gb_speed).toBe(0); + }); + + it('barrel rate moves a BOMBER and barely moves a GHOST — features silent where they do not apply', () => { + const move = (arch) => { + const lo = sk.hitOnContact({ batter: { ...BOMBER, barrel_pct: 0.04 }, pitcher: ACE, park: 1, archetype: arch }); + const hi = sk.hitOnContact({ batter: { ...BOMBER, barrel_pct: 0.20 }, pitcher: ACE, park: 1, archetype: arch }); + return hi - lo; + }; + expect(move('BOMBER')).toBeGreaterThan(move('GHOST') * 3); + }); + + it('an unknown archetype falls back to a balanced map, not a refusal', () => { + const out = sk.projectSkill({ batter: BOMBER, pitcher: ACE, archetype: 'NOT_A_REAL_ONE', statType: 'hits', line: 0.5, expectedPa: 4.2 }); + expect(out).not.toBeNull(); + expect(out.archetype).toBe('NOT_A_REAL_ONE'); + }); +}); + +// ── HONESTY ─────────────────────────────────────────────────────────────── +describe('honesty — unknown is not zero, and absent beats invented', () => { + it('no hitter profile → NO forward read (null), never a league-average guess', () => { + expect(sk.projectSkill({ batter: null, pitcher: ACE, statType: 'hits', line: 0.5 })).toBeNull(); + expect(sk.projectSkill({ batter: {}, pitcher: ACE, statType: 'hits', line: 0.5 })).toBeNull(); + }); + + it('a missing skill input is SILENT — it does not act as a measured zero', () => { + const full = sk.hitOnContact({ batter: BOMBER, pitcher: ACE, park: 1, archetype: 'BOMBER' }); + const noBarrel = sk.hitOnContact({ batter: { ...BOMBER, barrel_pct: null }, pitcher: ACE, park: 1, archetype: 'BOMBER' }); + const zeroBarrel = sk.hitOnContact({ batter: { ...BOMBER, barrel_pct: 0 }, pitcher: ACE, park: 1, archetype: 'BOMBER' }); + expect(noBarrel).not.toBeNull(); + // A REAL zero is a fact and must read far lower than an ABSENT one. + expect(zeroBarrel).toBeLessThan(noBarrel); + expect(Math.abs(noBarrel - full)).toBeLessThan(Math.abs(zeroBarrel - full)); + }); + + it('total_bases is REFUSED rather than shipped as a relabelled hits curve', () => { + // A deterministic bases-per-hit multiplier made P(TB>=2) identical to + // P(hits>=1). Refusing is correct until a real per-hit bases distribution + // exists (tb-v1's compound shape). + expect(sk.projectSkill({ batter: BOMBER, pitcher: ACE, archetype: 'BOMBER', statType: 'total_bases', line: 1.5 })).toBeNull(); + }); + + it('the distribution is a real distribution and P(>=k) is monotone', () => { + const out = sk.projectSkill({ batter: BOMBER, pitcher: ACE, archetype: 'BOMBER', statType: 'hits', line: 0.5, expectedPa: 4.2 }); + expect(out.distribution.reduce((a, b) => a + b, 0)).toBeCloseTo(1, 2); + let prev = 1; + for (let k = 1; k <= 5; k += 1) { + const p = sk.atLeast(out.distribution, k); + expect(p).toBeLessThanOrEqual(prev + 1e-9); + prev = p; + } + }); + + it('a hitter exactly at league on every axis lands on league BABIP — no invented lean', () => { + const leagueBat = { + k_pct: sk.LEAGUE.k_pct, bb_pct: sk.LEAGUE.bb_pct, + barrel_pct: sk.LEAGUE.barrel_pct, hard_hit_pct: sk.LEAGUE.hard_hit_pct, + avg_exit_velo: sk.LEAGUE.avg_exit_velo, avg_launch_angle: 12, + }; + const boc = sk.hitOnContact({ batter: leagueBat, pitcher: { hard_hit_pct: sk.LEAGUE.hard_hit_pct }, park: 1, archetype: 'DEFAULT' }); + expect(boc).toBeCloseTo(sk.LEAGUE.babip, 2); + }); + + it('opportunity matters: more projected PA means a higher chance of one hit', () => { + const few = sk.projectSkill({ batter: BOMBER, pitcher: ACE, archetype: 'BOMBER', statType: 'hits', line: 0.5, expectedPa: 2 }); + const many = sk.projectSkill({ batter: BOMBER, pitcher: ACE, archetype: 'BOMBER', statType: 'hits', line: 0.5, expectedPa: 5 }); + expect(many.p_over_line).toBeGreaterThan(few.p_over_line); + }); +}); diff --git a/tests/unit/statcastAggregate.test.js b/tests/unit/statcastAggregate.test.js index 807da90..3e9696b 100644 --- a/tests/unit/statcastAggregate.test.js +++ b/tests/unit/statcastAggregate.test.js @@ -212,10 +212,31 @@ describe('scheduler + route wiring', () => { it('is server-scheduled with a kill switch', () => { const s = read('src/snapshotScheduler.js'); expect(s).toContain('STATCAST_HOUR_UTC'); - expect(s).toMatch(/process\.env\.STATCAST !== '0'/); + expect(s).toMatch(/process\.env\.STATCAST === '0'/); // kill switch, early-return form expect(s).toContain('refreshSeason'); }); + it('RUNS ON ITS OWN TICK — not behind the snapshot-hours guard', () => { + // THE REGRESSION THIS LOCKS (found 2026-08-03): the refresh used to live + // inside `tick()`, BELOW `if (!HOURS_UTC.includes(h)) return`. HOURS_UTC is + // 14,19,22,1,3 and the block tests h === STATCAST_HOUR_UTC (default 11), so + // the guard could never admit the hour it waited for. It was unreachable + // code that had never run once, and the aggregates sat 13 days stale while + // every consumer served them as current. The previous assertion here passed + // the entire time, because it only checked that the STRING existed. + const s = read('src/snapshotScheduler.js'); + const statcastTick = s.slice(s.indexOf('const statcastTick')); + expect(statcastTick.length).toBeGreaterThan(0); + // Its own tick, registered on the interval alongside the others. + expect(s).toMatch(/void statcastTick\(\)/); + // And it must NOT be reachable only via the snapshot-hours guard. + // Scope to the snapshot tick's OWN body (it ends where refreshTick begins), + // so the statcastTick doc comment above it cannot satisfy this by accident. + const tickBody = s.slice(s.indexOf('const tick = async'), s.indexOf('const refreshTick')); + expect(tickBody).toContain('HOURS_UTC.includes(h)'); // the guard is still there + expect(tickBody).not.toContain('STATCAST_HOUR_UTC'); // and statcast is NOT behind it + }); + it('pages on BOTH a failed run and silent staleness', () => { const s = read('src/snapshotScheduler.js'); expect(s).toMatch(/Statcast refresh failed/);