builtbykev/vyndr
1
0
Fork 0
Code Issues Pull Requests Actions 5 Packages Projects Releases Wiki Activity

The skill engine: built, gated by construction, and Stage A honestly lost

Browse Source 
Built src/services/model/ -- the forward, archetype-selected, skill-based
projection, as a challenger. The champion is untouched.

featureRegistry makes "earn its place or it's out" structural rather than
aspirational: CANDIDATE / PROVEN / DEAD per feature per sport, liveFeatures()
returns PROVEN only, promotion requires n>=200 with positive lift and a CI
excluding zero, and there is deliberately no override argument. It ships with
exactly ONE proven feature -- the incumbent counter, because it is the only
one with a measurement. A test asserts that with only PROVEN features allowed
the projection returns null, so an unproven model cannot reach a user by
accident. The three champion adjustment layers are registered DEAD with their
reasons so they cannot be silently rebuilt.

skillProjection is a PA outcome tree: K and BB combined by log5 odds-ratio
against league (both identities unit-tested), then archetype-weighted contact
quality against contact allowed, then Binomial(PA, p_hit) mixed over a PA
distribution. Archetype is a FEATURE SELECTOR, not a nudge -- BOMBER reads
barrels at 0.50 and ground-ball speed at 0.00, GHOST inverts it -- and a test
locks that the same hitter read two ways moves more than 0.15.

STAGE A: IT LOSES. Out-of-sample on 570 settled hits props with 91.9%
opposing-pitcher coverage, resolution 0.0499 against the champion's 0.166,
delta -0.116 with CI [-0.189, -0.043]. It is not selective either: its eight
most confident picks hit 50%, a lift of -0.065. Not promoted. The gate did its
job on its first real test, which is the point of having built it that way.

Two false starts, both recorded because they nearly produced a wrong verdict:
statcast_aggregates stores PERCENTAGES, so raw rows made bip = 1-29.6-17.1 and
refused 568 of 576 -- the honest-absent guards made a units bug loud instead of
silent, and the conversion now lives at one chokepoint. And the first run
resolved an opposing pitcher for 1 of 570 rows, because ledger team/opponent
are NULL, so it would have reported "skill-v1 loses" while measuring a
batter-only model with no matchup in it at all. The verdict above is from the
corrected run.

The loss is real but partial: park was passed as 1.0, handedness and
opportunity_drift never fired, PA is season-PA over a constant, and the skill
profiles carry no recency at all while the champion has a last-5 term.

Also fixed: the Statcast nightly refresh was unreachable code. It sat inside
tick() below "if (!HOURS_UTC.includes(h)) return" while testing h === 11, so
it had never run once; the aggregates were 13 days stale and both of its
alerts were in the same dead branch. It now runs on its own tick, and the test
that passed happily throughout -- it only checked the string existed -- is
replaced by one that asserts it is not behind the guard.

4,182 tests green (333 suites); web build exit 0.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W1sivYNqY2TS5ftykmHBU9
This commit is contained in:
Kev
2026-08-03 02:20:40 -04:00
parent c551bf0340
commit 258d8a6655
9 changed files with 1439 additions and 30 deletions
Download Patch File Download Diff File Expand all files Collapse all files
BUILD-STATE.md
+27
View File
@@ -3,6 +3,33 @@
## Last Updated ## Last Updated
2026-08-03 2026-08-03
## Session 80 (2026-08-03) — The skill engine: built, gated, and Stage A honestly lost ✅
Spec: `specs/skill-engine-architecture.md`. 4,182 tests / 333 suites green, build exit 0.
- **BUILT `src/services/model/`:** `featureRegistry` (CANDIDATE/PROVEN/DEAD per
sport; `liveFeatures()` = PROVEN only; promotion needs n>=200 + positive lift +
CI excluding zero, no override) and `skillProjection` (PA outcome tree, log5
odds-ratio K/BB, archetype-selected contact quality, Binomial over a PA
distribution). 22 tests assert the five disciplines as BEHAVIOUR.
- **The gate works by construction:** with only PROVEN features allowed the
projection returns NULL. Registry ships with ONE proven feature (the counter).
- **STAGE A: skill-v1 LOSES → NOT PROMOTED.** Out-of-sample (profiles frozen
07-21, only later games scored), 570 rows, 91.9% pitcher coverage: resolution
0.0499 vs champion 0.166, delta −0.116 CI [−0.189,−0.043]. Not selective
either (top-8 hit 50%, lift −0.065).
- **Two false starts caught:** (1) units — statcast stores PERCENTAGES, raw rows
made bip negative and refused 568/576; now one chokepoint `fromStatcastRow`.
(2) an INVALID first verdict — ledger team/opponent are NULL, so the pitcher
resolved for 1 of 570 rows and it was silently measuring a batter-only model.
Fixed via each player's statsapi game log.
- **Not exercised yet (so the loss is real but partial):** park (passed 1.0),
handedness, opportunity_drift, and PA projection is season-PA/103. And the
skill profiles carry NO recency while the champion has last-5.
- **Fixed: Statcast nightly refresh was UNREACHABLE CODE** — inside tick() below
the HOURS_UTC guard while testing h===11. Never ran; 13 days stale; both alerts
in the same dead branch. Now its own tick; test rewritten to catch it.
- **Next:** recency into the skill profile, wire park/handedness/opportunity_drift,
real PA from lineup slot, then re-run Stage A.
## Session 79 (2026-08-03) — Reality assessment vs the FORWARD-PROJECTION objective ✅ ## Session 79 (2026-08-03) — Reality assessment vs the FORWARD-PROJECTION objective ✅
Spec: `specs/forward-model-reality-assessment.md`. READ-ONLY (src/web untouched). Spec: `specs/forward-model-reality-assessment.md`. READ-ONLY (src/web untouched).
- **Finding: the forward model's parts all EXIST and are all wired downstream of - **Finding: the forward model's parts all EXIST and are all wired downstream of
CLAUDE.md
+39
View File
@@ -1262,6 +1262,45 @@ phased plan in the Session-57 conversation / BUILD-STATE Next section).
forward inputs. "AT CEILING" (runs/walks) is provisional: measured only against forward inputs. "AT CEILING" (runs/walks) is provisional: measured only against
features the champion already reads. features the champion already reads.
## The skill engine + feature registry (Session 80 — non-obvious)
- **`specs/skill-engine-architecture.md`.** `src/services/model/` is the forward
engine: `featureRegistry.js` (CANDIDATE/PROVEN/DEAD per feature PER SPORT) and
`skillProjection.js` (PA outcome tree: K/BB via log5 odds-ratio vs league, then
archetype-weighted contact quality → Binomial(PA, p_hit) mixed over a PA
distribution). Challenger-only; champion untouched.
- **THE GATE IS STRUCTURAL, not a habit.** `liveFeatures()` returns PROVEN only,
and the registry ships with exactly ONE proven feature (the incumbent counter).
A test asserts that with only PROVEN allowed, `projectSkill` returns NULL — an
unproven model cannot reach a user by accident. `promote()` requires n>=200,
positive lift, CI excluding zero, and has NO override argument.
- **STAGE A RESULT: skill-v1 LOSES, not promoted.** 570 rows, 91.9% pitcher
coverage, resolution 0.0499 vs champion 0.166, delta −0.116 CI [−0.189,−0.043].
Also NOT selective — its top-8 most confident picks hit 50% (lift −0.065).
- **UNITS: `statcast_aggregates` stores PERCENTAGES (0–100), not fractions.**
`k_pct: 29.6` means 29.6%. Feeding raw rows in made `bip = 1−29.6−17.1` negative
and refused 568/576 rows. ALWAYS convert via `skillProjection.fromStatcastRow`
(the one chokepoint); it nulls out-of-range values rather than clamping, and
leaves mph/degrees fields alone.
- **`ledger_entries.team`/`opponent` are NULL on ~all rows** — do NOT join a
matchup on them. The first Stage A run resolved a pitcher for 1 of 570 rows and
would have reported a verdict on a batter-only model. Resolve the opponent from
the player's own statsapi game log (`getPlayerGameLog` → `{date, opponent}`),
which is authoritative and point-in-time safe → 91.9% coverage.
- **Archetype = FEATURE SELECTOR, not a nudge.** `ARCHETYPE_MAP` weights decide
which skill inputs drive a hitter (BOMBER barrel 0.50 / gb_speed 0; GHOST
barrel 0.05 / gb_speed 0.60). Locked by test: same hitter read through two
archetypes moves >0.15. Weights are DOCUMENTED, not fitted — fitting on 1,741
rows is curve-fitting; the registry exists so they get measured.
- **total_bases is deliberately REFUSED by skillProjection.** A deterministic
bases-per-hit multiplier made P(TB>=2) exactly equal P(hits>=1) — a relabelled
hits curve carrying no new information. TB needs tb-v1's compound per-hit bases
distribution; refusing beats shipping a relabel.
- **STATCAST REFRESH WAS UNREACHABLE CODE** (fixed): it sat inside `tick()` below
`if (!HOURS_UTC.includes(h)) return` (14,19,22,1,3) while testing `h === 11`.
Never ran once; data 13 days stale; BOTH its alerts were in the same dead
branch. Now its own `statcastTick`. The old test only checked the string
existed — the new one asserts it is not behind the snapshot-hours guard.
## Active Skills ## Active Skills
- vyndr-voice (all user-facing output) - vyndr-voice (all user-facing output)
- prop-analysis (grading methodology) - prop-analysis (grading methodology)
scripts/skill-v1-stagea.js
+279
View File
@@ -0,0 +1,279 @@
#!/usr/bin/env node
'use strict';
/**
* skill-v1-stagea — DOES THE WINDSHIELD BEAT THE REAR-VIEW MIRROR?
*
* Stage A's only question: on settled HITS props, does an archetype-selected,
* skill-based forward projection call the LISTED LINE better than the frequency
* counter that is the reigning champion? If it does not, it is not real yet and
* it does not get promoted. That is the whole test.
*
* ── WHY THIS IS GENUINELY OUT-OF-SAMPLE ──────────────────────────────────
* `statcast_aggregates` was last refreshed 2026-07-21 (the nightly job was
* unreachable code until this session — see snapshotScheduler). Settled hits
* rows run 2026-07-23 onward. So the skill profiles this model reads were
* frozen BEFORE every game it is asked to predict. The staleness that was a bug
* for production is, for this one measurement, a clean point-in-time snapshot.
* Rows on or before the freeze date are EXCLUDED so no profile can contain the
* game it is predicting.
*
* The opposing starter comes from the statsapi schedule for that date, and the
* pitcher's skill profile from the same frozen aggregate table.
*
* ── THE BAR (identical to the one that refuted hits-v1) ──────────────────
* - hits rows only, direction-aligned to the graded side
* - matched rows only: champion and challenger scored on the SAME props
* - paired bootstrap, deterministic seed, CI on the DIFFERENCE
* - PROMOTE only if the CI excludes zero on the good side
*
* ── DISCIPLINE 4, MEASURED, NOT ASSUMED ──────────────────────────────────
* Selectivity is reported, not claimed: accuracy is broken out by how confident
* the model is, so "right 57% on the 8 you're sure of" is a number rather than a
* slogan. LIFT over the naive base rate is reported beside it, because being
* right about obvious chalk is not signal.
*
* SUPABASE_URL=... node scripts/skill-v1-stagea.js
*/
require('dotenv').config();
const { createClient } = require('@supabase/supabase-js');
const sk = require('../src/services/model/skillProjection');
const reg = require('../src/services/model/featureRegistry');
const mlb = require('../src/services/adapters/mlbStatsAdapter');
const { nameKey } = require('../src/utils/playerName');
const { knownRate } = require('../src/utils/known');
/** Team games played by the 2026-07-21 profile freeze — turns season PA into PA/game. */
const GAMES_SO_FAR = Number(process.env.STAGEA_GAMES_SO_FAR || 103);
const SB_URL = process.env.SUPABASE_URL;
const SB_KEY = process.env.SUPABASE_SERVICE_ROLE_KEY || process.env.SUPABASE_SERVICE_KEY;
const PAGE = 1000;
function corr(xs, ys) {
const n = xs.length;
if (n < 3) return null;
const mx = xs.reduce((a, b) => a + b, 0) / n;
const my = ys.reduce((a, b) => a + b, 0) / n;
let sxy = 0; let sxx = 0; let syy = 0;
for (let i = 0; i < n; i += 1) {
const dx = xs[i] - mx; const dy = ys[i] - my;
sxy += dx * dy; sxx += dx * dx; syy += dy * dy;
}
if (sxx <= 0 || syy <= 0) return null;
return sxy / Math.sqrt(sxx * syy);
}
const r4 = (v) => (v == null || !Number.isFinite(v) ? null : Math.round(v * 10000) / 10000);
const mean = (a) => (a.length ? a.reduce((x, y) => x + y, 0) / a.length : null);
const brier = (ps, ys) => (ps.length ? ps.reduce((s, p, i) => s + (p - ys[i]) ** 2, 0) / ps.length : null);
function makeRnd(seed) {
let s = seed >>> 0;
return () => { s ^= s << 13; s >>>= 0; s ^= s >>> 17; s ^= s << 5; s >>>= 0; return s / 4294967296; };
}
function bootstrapDiff(rows, keyA, keyB, iters = 4000, seed = 20260803) {
if (rows.length < 30) return null;
const rnd = makeRnd(seed);
const n = rows.length;
const diffs = [];
for (let it = 0; it < iters; it += 1) {
const ys = []; const a = []; const b = [];
for (let i = 0; i < n; i += 1) {
const r = rows[Math.floor(rnd() * n)];
ys.push(r.won); a.push(r[keyA]); b.push(r[keyB]);
}
const ca = corr(a, ys); const cb = corr(b, ys);
if (ca == null || cb == null) continue;
diffs.push(ca - cb);
}
if (diffs.length < 100) return null;
diffs.sort((x, y) => x - y);
const q = (p) => r4(diffs[Math.floor(p * (diffs.length - 1))]);
const ci = [q(0.025), q(0.975)];
return {
point: r4(corr(rows.map((r) => r[keyA]), rows.map((r) => r.won))
- corr(rows.map((r) => r[keyB]), rows.map((r) => r.won))),
ci95: ci,
ci_excludes_zero: ci[0] > 0 || ci[1] < 0,
};
}
async function page(sb, table, select, apply) {
const out = [];
for (let from = 0; ; from += PAGE) {
const { data, error } = await apply(sb.from(table).select(select)).range(from, from + PAGE - 1);
if (error) throw error;
if (!data || data.length === 0) break;
out.push(...data);
if (data.length < PAGE) break;
}
return out;
}
/**
* `date|team` → the starter that team FACED.
*
* Built from the statsapi schedule: a team faces the OTHER side's probable.
*/
async function opposingStarters(dates) {
const byDateTeam = new Map();
for (const d of dates) {
let games = [];
try { games = await mlb.getScheduleWithPitchers(d); } catch { games = []; }
for (const g of games) {
if (!g.home || !g.away) continue;
if (g.away.probablePitcher) byDateTeam.set(`${d}|${g.home.team}`, g.away.probablePitcher.id);
if (g.home.probablePitcher) byDateTeam.set(`${d}|${g.away.team}`, g.home.probablePitcher.id);
// Also key by the PITCHING team, so "who did team X send out" is directly
// answerable from the opponent name a game log gives us.
if (g.home.probablePitcher) byDateTeam.set(`${d}|OPP:${g.home.team}`, g.home.probablePitcher.id);
if (g.away.probablePitcher) byDateTeam.set(`${d}|OPP:${g.away.team}`, g.away.probablePitcher.id);
}
}
return byDateTeam;
}
/**
* `playerKey|date` → the OPPONENT team that player faced.
*
* THE LEDGER CANNOT ANSWER THIS: `team`/`opponent` are NULL on 575 of 576 rows
* in this window, which is why the first run resolved a pitcher for exactly ONE
* row and silently measured a batter-profile-only model instead of the matchup
* model it claimed to test. The player's own statsapi game log names the
* opponent for the exact date, so it is both authoritative and point-in-time
* safe (a completed game's opponent is not a forecast).
*/
async function opponentByPlayerDate(players) {
const map = new Map();
for (const [key, name] of players) {
try {
const found = await mlb.searchPlayer(name);
if (!found || !found.id) continue;
const log = await mlb.getPlayerGameLog(found.id);
for (const g of log || []) {
if (g && g.date && g.opponent) map.set(`${key}|${String(g.date).slice(0, 10)}`, g.opponent);
}
} catch { /* a missing log just means no pitcher for those rows */ }
}
return map;
}
async function main() {
if (!SB_URL || !SB_KEY) throw new Error('SUPABASE_URL / service key required');
const sb = createClient(SB_URL, SB_KEY, { auth: { persistSession: false } });
// Frozen skill profiles — by name (batters) and by source_id (pitchers).
const statcast = await page(sb, 'statcast_aggregates', '*', (q) => q.eq('sport', 'mlb'));
const freeze = statcast.reduce((mx, r) => (String(r.updated_at) > mx ? String(r.updated_at) : mx), '');
const freezeDate = freeze.slice(0, 10);
const batters = new Map();
const pitchersById = new Map();
for (const r of statcast) {
// UNITS: statcast_aggregates stores percentages (0-100). Convert ONCE, here.
if (r.role === 'pitcher' && r.source_id != null) pitchersById.set(Number(r.source_id), sk.fromStatcastRow(r));
if (r.player_key && r.role === 'batter') {
const prev = batters.get(r.player_key);
const size = Number(r.sample_pa || 0);
if (!prev || size > Number(prev.rawPa || 0)) {
batters.set(r.player_key, Object.assign(sk.fromStatcastRow(r), { rawPa: size, archetype: null }));
}
}
}
const led = await page(sb, 'ledger_entries',
'player_key, player_name, stat, line, side, outcome, game_date, p_win, team, opponent, quarantine_reason',
(q) => q.eq('sport', 'mlb').is('user_id', null).eq('stat', 'hits')
.in('outcome', ['hit', 'miss']).not('p_win', 'is', null));
const clean = led.filter((r) => !(r.quarantine_reason || '').startsWith('nontakeable_book')
// STRICTLY AFTER the profile freeze — no game may be inside its own inputs.
&& String(r.game_date) > freezeDate);
const dates = [...new Set(clean.map((r) => r.game_date))].sort();
const starters = await opposingStarters(dates);
const players = new Map();
for (const r of clean) if (!players.has(r.player_key)) players.set(r.player_key, r.player_name);
const oppByPlayerDate = await opponentByPlayerDate(players);
const allowed = reg.candidateFeatures('mlb'); // challenger-first: candidates measured, never served
const rows = [];
const drops = {};
const drop = (k) => { drops[k] = (drops[k] || 0) + 1; };
let withPitcher = 0;
for (const r of clean) {
const bat = batters.get(r.player_key);
if (!bat) { drop('no_batter_profile'); continue; }
// The team he FACED that day, from his own game log. `starters` is keyed by
// the team doing the facing, so look up his own team — which is the
// opponent's opponent. Resolve via the game log's opponent name and take the
// schedule entry for the OTHER side.
const facedTeam = oppByPlayerDate.get(`${r.player_key}|${r.game_date}`) || null;
const starterId = facedTeam ? starters.get(`${r.game_date}|OPP:${facedTeam}`) : null;
const pit = starterId != null ? pitchersById.get(Number(starterId)) : null;
if (pit) withPitcher += 1;
// Opportunity: the batter's own season PA per game, from the frozen profile.
// Opportunity: season PA spread over the games played so far this season.
// GAMES_SO_FAR is the frozen-profile era's team game count, so PA/game is a
// real per-game rate rather than an arbitrary divisor.
const pa = knownRate(bat.rawPa);
const expectedPa = pa && pa > 0 ? Math.min(5.2, Math.max(2.0, pa / GAMES_SO_FAR)) : null;
const out = sk.projectSkill({
batter: bat, pitcher: pit, park: 1,
archetype: bat.archetype || null,
statType: 'hits', line: Number(r.line),
expectedPa, allowed,
});
if (!out) { drop('projection_refused'); continue; }
const under = String(r.side).toLowerCase() === 'under';
rows.push({
won: r.outcome === 'hit' ? 1 : 0,
champ: Number(r.p_win),
skill: under ? 1 - out.p_over_line : out.p_over_line,
line: Number(r.line),
had_pitcher: !!pit,
});
}
const ys = rows.map((r) => r.won);
const base = mean(ys);
const bs = bootstrapDiff(rows, 'skill', 'champ');
// DISCIPLINE 4 — selectivity, measured. Sorted by confidence in the graded
// side; report accuracy and LIFT over the naive base rate at each depth.
const byConf = [...rows].sort((a, b) => b.skill - a.skill);
const depths = [8, 15, 25, 50, 100].filter((d) => d <= byConf.length);
const selectivity = depths.map((d) => {
const top = byConf.slice(0, d);
const hit = mean(top.map((r) => r.won));
return { top_n: d, hit_rate: r4(hit), lift_over_base: r4(hit - base) };
});
console.log(JSON.stringify({
measurement: 'STAGE A — skill-v1 vs the frequency counter, out-of-sample on listed-line accuracy',
out_of_sample_guarantee: `skill profiles frozen ${freezeDate}; only rows with game_date > ${freezeDate} scored`,
registry: reg.summary('mlb'),
matched_rows: rows.length,
rows_with_opposing_pitcher: withPitcher,
pitcher_coverage_pct: rows.length ? r4(withPitcher / rows.length) : null,
dropped: drops,
base_rate: r4(base),
resolution: { skill_v1: r4(corr(rows.map((r) => r.skill), ys)), champion: r4(corr(rows.map((r) => r.champ), ys)) },
brier: { skill_v1: r4(brier(rows.map((r) => r.skill), ys)), champion: r4(brier(rows.map((r) => r.champ), ys)) },
mean_forecast: { skill_v1: r4(mean(rows.map((r) => r.skill))), champion: r4(mean(rows.map((r) => r.champ))) },
delta_vs_champion: bs,
verdict: !bs ? 'N-BLOCKED'
: (bs.ci_excludes_zero && bs.point > 0) ? 'BEATS THE COUNTER — promotable'
: (bs.ci_excludes_zero && bs.point < 0) ? 'LOSES to the counter — iterate, do not promote'
: 'INCONCLUSIVE — not proven, do not promote',
selectivity_discipline_4: selectivity,
}, null, 2));
process.exit(0);
}
main().catch((e) => { console.error(e); process.exit(1); });
specs/skill-engine-architecture.md
+173
View File
@@ -0,0 +1,173 @@
# THE SKILL ENGINE — architecture, and Stage A's honest result
**Built 2026-08-03.** Baseball first. Challenger-only; the champion is untouched.
> **STAGE A VERDICT: skill-v1 LOSES to the frequency counter and is NOT promoted.**
> Out-of-sample on 570 settled hits props with 91.9% opposing-pitcher coverage:
> resolution **0.0499 vs the champion's 0.166**, delta **−0.116, CI
> [−0.189, −0.043]**. The architecture's own rule fired and kept it out of the
> live path. §5 is what was actually exercised and what wasn't — the loss is
> real, but it does not yet condemn the approach, and §6 says why.
---
## 1. What was built
| module | discipline | what it does |
|---|---|---|
| `src/services/model/featureRegistry.js` | **D3** — earn its place | CANDIDATE / PROVEN / DEAD per feature, per sport. `liveFeatures()` returns PROVEN only |
| `src/services/model/skillProjection.js` | **D1 + D2** | PA outcome tree from skill inputs; archetype selects the feature map |
| `tests/unit/skillModel.test.js` | all | 22 tests asserting the disciplines as behaviour |
| `scripts/skill-v1-stagea.js` | **D4** | the out-of-sample proof + selectivity measurement |
## 2. The model (D1 — skill, not results)
A hitter gets N plate appearances; each resolves through a tree:
```
PA ──> strikeout (batter K vs pitcher K, odds-ratio vs league)
──> walk (batter BB vs pitcher BB, odds-ratio vs league)
──> ball in play
└─> hit? (archetype-weighted contact quality vs contact allowed, park)
```
Rates combine by **log5 / odds ratio** against league — the standard way to ask
"this batter vs THIS pitcher" instead of "vs the average pitcher". Two identities
are unit-tested rather than assumed: league-vs-league returns league, and an
average pitcher leaves the batter's own rate untouched.
Hits then distribute as `Binomial(PA, p_hit_per_PA)`, mixed over a PA
distribution — because PA is not known in advance and pretending it is would be
a fabricated certainty.
## 3. Archetype selects features (D2)
Not a nudge, not a label — it decides **which skill inputs drive this hitter**:
| archetype | barrel | hard-hit | exit velo | GB/speed | power |
|---|---|---|---|---|---|
| BOMBER | **0.50** | 0.35 | 0.15 | **0.00** | 1.25 |
| GHOST | **0.05** | 0.20 | 0.15 | **0.60** | 0.70 |
| TORCH | 0.20 | 0.35 | 0.25 | 0.20 | 1.00 |
Locked by test: the same hitter read as BOMBER vs GHOST moves **>0.15** in
probability, and barrel rate moves a BOMBER more than 3× what it moves a GHOST.
Weights are documented by what the archetype physically does — **not fitted**,
because fitting them on 1,741 rows is curve-fitting, and the registry exists so
they get measured instead.
## 4. The gate is structural (D3)
The registry ships with **one PROVEN feature** — `recent_frequency_prior`, the
incumbent counter, and only because it has the measurement (S78 ablation). Every
skill input is CANDIDATE.
**Locked by test: with only PROVEN features allowed, `projectSkill` returns
null.** An unproven model cannot reach a user by accident — not by convention,
by construction. Promotion requires n ≥ 200, positive lift, and a CI excluding
zero; there is deliberately **no override argument**.
Three champion layers are already registered **DEAD** with their reasons, so they
cannot be silently rebuilt: `champion_opp_rank_adj`, `champion_home_away_adj`
("removing it IMPROVED rbi resolution — actively harmful"),
`champion_consistency_pull`.
## 5. STAGE A — the measurement, and what it actually tested
**Out-of-sample guarantee:** skill profiles were frozen 2026-07-21 (the nightly
refresh had never run — §7). Only rows with `game_date > 2026-07-21` were scored,
so no profile can contain the game it predicts. The bug that broke production
made this one measurement clean.
```
matched rows 570 pitcher coverage 91.9%
base rate 0.5649
resolution skill-v1 0.0499 champion 0.1660
brier skill-v1 0.2839 champion 0.2506
mean forecast skill-v1 0.5372 champion 0.6055
delta vs champion −0.1161 CI95 [−0.1892, −0.0426] ← excludes zero
```
**Selectivity (D4), measured not claimed:**
| top N by confidence | hit rate | lift over base |
|---|---|---|
| 8 | 0.500 | −0.065 |
| 15 | 0.533 | −0.032 |
| 25 | 0.480 | −0.085 |
| 100 | 0.550 | −0.015 |
**It is not selective yet either** — its most confident picks are not its best
ones. That is the honest read and it is the second reason not to promote.
### Two false starts caught in the build, both worth recording
1. **A units bug.** `statcast_aggregates` stores percentages (0–100), not
fractions. The first run made `bip = 1 − 29.6 − 17.1` deeply negative and
**refused 568 of 576 rows.** The honest-absent guards worked exactly as
designed — a model that refuses is visible, a model that fabricates is not.
The conversion now lives at one chokepoint (`fromStatcastRow`), and an
out-of-range percentage is treated as broken (null), never clamped.
2. **An invalid first verdict.** The initial run resolved an opposing pitcher for
**1 of 570 rows** (0.18%) because `ledger_entries.team`/`opponent` are NULL on
575 of 576 rows in this window. It would have reported "skill-v1 loses" while
silently measuring a **batter-profile-only** model with no matchup at all —
the exact false verdict this architecture exists to prevent. Fixed by
resolving the opponent from each player's own statsapi game log (91.9%
coverage). **The verdict in this document is from the corrected run.**
## 6. What the loss does and does not say
**Does say:** this implementation, on these inputs, is worse than the counter at
calling listed lines, and it does not go live.
**Does not say the approach is dead** — several candidate features never fired in
this run, so the test was of a partial model:
| feature | exercised? |
|---|---|
| batter K / BB / barrel / hard-hit / exit velo / launch | **yes** |
| pitcher K / BB / hard-hit allowed | **yes** (91.9% of rows) |
| `park_factor` | **no** — park was passed as 1.0 for every row |
| `handedness_platoon` | **no** — never wired into the tree |
| `opportunity_drift` | **no** — the one feature with measured residual signal |
| `projected_pa` | **crude** — season PA ÷ a constant 103 team games |
And one structural handicap worth naming: the skill profiles are **season
aggregates with no recency at all**, while the champion carries a last-5 term. So
skill-v1 was asked to beat a recency-aware model using a 13-day-old season
average. That is a fair loss to record and an unfair basis for concluding that
skill inputs don't work.
**Adding the pitcher moved resolution 0.0441 → 0.0499** — almost nothing. That is
the most interesting number here, and the honest next question: is the matchup
signal genuinely small at the 0.5 hits line, or is the log5 combine too blunt to
express it?
## 7. Fixed along the way — the Statcast refresh was unreachable code
`snapshotScheduler.tick()` returns for any hour not in `HOURS_UTC`
(14,19,22,1,3); the Statcast block then tested `h === STATCAST_HOUR_UTC`
(default **11**). That branch could never execute — the nightly refresh had never
run once, the aggregates sat frozen for 13 days while every consumer served them
as current, and **both of its alerts were inside the same dead branch**, so
nothing could report it. Same silently-guarded-out shape as the settlement
outage.
It now runs on its **own tick**, independent of the snapshot hours. The old test
asserted only that the string `STATCAST_HOUR_UTC` appeared in the file and passed
happily the entire time; it is replaced with one that asserts the refresh is
**not reachable only via the snapshot-hours guard**.
## 8. Next iteration — ranked by what the measurement actually showed
1. **Recency in the skill profile.** The champion's edge is recency and skill-v1
has none. Weight recent PA into the rates before concluding anything else.
2. **Wire the unexercised features** — park, handedness, and `opportunity_drift`
(the one feature with measured residual signal, S78).
3. **Real PA projection** from lineup slot, not season PA ÷ 103.
4. **Then re-run Stage A.** If it still loses with all of that firing, the
conclusion is much stronger and worth acting on.
**Not recommended:** promoting anything, tuning the archetype weights to fit
1,741 rows, or relaxing the bar. The gate did its job today.
src/services/model/featureRegistry.js
+234
View File
@@ -0,0 +1,234 @@
'use strict';
/**
* featureRegistry — EARN ITS PLACE OR IT'S OUT (Discipline 3).
*
* This codebase has shipped six features that were wired, computed, retained,
* and contributed NOTHING to the served forecast. Measured: arch-v1 moved 76% of
* rows by 2.5 points and changed resolution by 0.0000; the environment and
* opportunity axes are flat; the projection ladder is reliably WORSE than the
* counter it was meant to beat. None of that was visible until someone went
* looking, because nothing in the architecture ever asked a feature to justify
* itself.
*
* So the gate is structural, not a habit:
*
* CANDIDATE — computed, measured in the CHALLENGER, NEVER in the live grade.
* PROVEN — marginal out-of-sample lift measured positive AND significant.
* Only PROVEN features may feed the served projection.
* DEAD — measured and found to add nothing (or to hurt). Kept in the
* registry BY NAME so it cannot be silently rebuilt a year later
* by someone who does not know it was already tried.
*
* THE ONE RULE THIS FILE EXISTS TO ENFORCE:
*
* liveFeatures() returns PROVEN only. A CANDIDATE cannot reach a user by
* accident, and a DEAD feature cannot come back without a new measurement.
*
* Promotion is DATA, not opinion: `promote()` refuses without a measurement
* carrying n, lift, and a confidence interval that excludes zero on the good
* side. There is deliberately no "force" argument. A feature that cannot show
* the number does not move.
*
* WHY EVERY FEATURE STARTS AS A CANDIDATE. Nothing here has earned PROVEN yet —
* the skill model is being measured for the first time. An empty PROVEN set is
* the honest starting state and it is what makes Stage A meaningful: the
* challenger runs on candidates, the ledger judges them, and only then does
* anything go live. Seeding this file with optimistic PROVEN flags would defeat
* the entire purpose of having it.
*/
const STATUS = Object.freeze({ CANDIDATE: 'CANDIDATE', PROVEN: 'PROVEN', DEAD: 'DEAD' });
/**
* Minimum evidence to promote. Deliberately strict: a feature promoted on noise
* is worse than no feature, because it carries the authority of having been
* "measured".
*/
const MIN_PROMOTION_N = 200;
/**
* THE REGISTRY.
*
* `sport` is first-class: per-sport doctrine means a feature proven for MLB says
* nothing about NBA, and the registry must not let one leak into the other.
*
* `evidence` is null until something measures it. `history` accumulates every
* status change with its measurement, so "why is this DEAD" always has an answer.
*/
const FEATURES = [
// ── HITTER SKILL (Statcast) ──────────────────────────────────────────────
{ key: 'batter_exit_velo', sport: 'mlb', status: STATUS.CANDIDATE, unit: 'mph',
describes: 'how hard he hits it — the skill under batting average' },
{ key: 'batter_launch_angle', sport: 'mlb', status: STATUS.CANDIDATE, unit: 'deg',
describes: 'launch profile: ground-ball hitter vs air hitter' },
{ key: 'batter_barrel_pct', sport: 'mlb', status: STATUS.CANDIDATE, unit: 'pct',
describes: 'rate of ideal exit-velo/launch combinations' },
{ key: 'batter_hard_hit_pct', sport: 'mlb', status: STATUS.CANDIDATE, unit: 'pct',
describes: '95+ mph contact rate' },
{ key: 'batter_whiff_pct', sport: 'mlb', status: STATUS.CANDIDATE, unit: 'pct',
describes: 'swing-and-miss rate — drives the strikeout branch' },
{ key: 'batter_k_pct', sport: 'mlb', status: STATUS.CANDIDATE, unit: 'pct',
describes: 'strikeout rate (the PA outcome that ends without a ball in play)' },
{ key: 'batter_bb_pct', sport: 'mlb', status: STATUS.CANDIDATE, unit: 'pct',
describes: 'walk rate — removes a PA from the hit-chance pool' },
// ── PITCHER SKILL (Statcast) ─────────────────────────────────────────────
{ key: 'pitcher_whiff_pct', sport: 'mlb', status: STATUS.CANDIDATE, unit: 'pct',
describes: 'stuff — the skill under ERA' },
{ key: 'pitcher_k_pct', sport: 'mlb', status: STATUS.CANDIDATE, unit: 'pct',
describes: 'strikeout rate allowed' },
{ key: 'pitcher_bb_pct', sport: 'mlb', status: STATUS.CANDIDATE, unit: 'pct',
describes: 'walk rate allowed' },
{ key: 'pitcher_gb_pct', sport: 'mlb', status: STATUS.CANDIDATE, unit: 'pct',
describes: 'ground-ball tendency — suppresses air contact' },
{ key: 'pitcher_hard_hit_allowed', sport: 'mlb', status: STATUS.CANDIDATE, unit: 'pct',
describes: 'contact quality allowed' },
{ key: 'pitcher_barrel_allowed', sport: 'mlb', status: STATUS.CANDIDATE, unit: 'pct',
describes: 'barrel rate allowed — the extra-base driver' },
// ── MATCHUP STRUCTURE ────────────────────────────────────────────────────
{ key: 'handedness_platoon', sport: 'mlb', status: STATUS.CANDIDATE, unit: 'bool',
describes: 'batter/pitcher handedness edge' },
// ── CONTEXT ──────────────────────────────────────────────────────────────
{ key: 'park_factor', sport: 'mlb', status: STATUS.CANDIDATE, unit: 'mult',
describes: "tonight's venue relative to the hitter's own exposure" },
{ key: 'projected_pa', sport: 'mlb', status: STATUS.CANDIDATE, unit: 'count',
describes: 'opportunity — how many times he bats' },
{ key: 'opportunity_drift', sport: 'mlb', status: STATUS.CANDIDATE, unit: 'ratio',
describes: 'recent AB vs season AB/game — the one feature measured to carry residual signal (S78: +0.156 hits, +0.145 TB)' },
// ── THE INCUMBENT ────────────────────────────────────────────────────────
// The counter is IN THE REGISTRY, because the baseline has to be able to lose
// its place too. It is PROVEN — uniquely, it has the measurement: it is ~100%
// of the champion's resolution (S78 ablation, per stat, paired bootstrap).
{ key: 'recent_frequency_prior', sport: 'mlb', status: STATUS.PROVEN, unit: 'prob',
describes: 'empirical frequency of clearing THIS line (the incumbent counter), demoted to a PRIOR in the skill model',
evidence: {
measured_at: '2026-08-03', n: 1415, source: 'specs/champion-input-diagnosis.md',
note: 'per-stat ablation: removing all three adjustment layers changes resolution by ~0, so base+recency IS the champion',
} },
// ── MEASURED AND FOUND WANTING — kept so they are not rebuilt ────────────
{ key: 'champion_opp_rank_adj', sport: 'mlb', status: STATUS.DEAD, unit: 'prob',
describes: 'the ±0.03 opponent bump inside probabilityEstimator',
evidence: { measured_at: '2026-08-03', n: 1415, source: 'specs/champion-input-diagnosis.md',
note: 'ablation delta within noise on every stat' } },
{ key: 'champion_home_away_adj', sport: 'mlb', status: STATUS.DEAD, unit: 'prob',
describes: 'the ±0.015 home/away bump inside probabilityEstimator',
evidence: { measured_at: '2026-08-03', n: 273, source: 'specs/champion-input-diagnosis.md',
note: 'removing it IMPROVED rbi resolution (+0.0053, CI [0.0002,0.0103]) — actively harmful' } },
{ key: 'champion_consistency_pull', sport: 'mlb', status: STATUS.DEAD, unit: 'prob',
describes: 'cv>0.40 shrink toward 0.50',
evidence: { measured_at: '2026-08-03', n: 1415, source: 'specs/champion-input-diagnosis.md',
note: 'ablation delta ≤0.0018 on every stat' } },
];
const byKey = new Map();
for (const f of FEATURES) byKey.set(`${f.sport}|${f.key}`, { ...f, evidence: f.evidence || null, history: [] });
const idOf = (sport, key) => `${String(sport || '').toLowerCase()}|${key}`;
/** Every registered feature for a sport (any status). */
function allFeatures(sport) {
const sp = String(sport || '').toLowerCase();
return [...byKey.values()].filter((f) => f.sport === sp).map((f) => ({ ...f }));
}
/**
* THE GATE. Only PROVEN features may feed the SERVED projection.
*
* Returns a Set of keys, because the caller's question is always "may I use
* this one?" and a Set makes the wrong answer awkward to write.
*/
function liveFeatures(sport) {
return new Set(allFeatures(sport).filter((f) => f.status === STATUS.PROVEN).map((f) => f.key));
}
/** Features the CHALLENGER is allowed to measure — candidates plus what's live. */
function candidateFeatures(sport) {
return new Set(allFeatures(sport)
.filter((f) => f.status === STATUS.CANDIDATE || f.status === STATUS.PROVEN)
.map((f) => f.key));
}
function statusOf(sport, key) {
const f = byKey.get(idOf(sport, key));
return f ? f.status : null;
}
/** Is this feature allowed in the LIVE path right now? */
function isLive(sport, key) {
return statusOf(sport, key) === STATUS.PROVEN;
}
/**
* Evidence sufficient to promote: a real sample, positive lift, and a CI that
* excludes zero on the good side. Anything less is a story about a number.
*/
function isSufficient(evidence) {
if (!evidence || typeof evidence !== 'object') return false;
const n = Number(evidence.n);
const lift = Number(evidence.lift);
const ci = evidence.ci95;
if (!Number.isFinite(n) || n < MIN_PROMOTION_N) return false;
if (!Number.isFinite(lift) || lift <= 0) return false;
if (!Array.isArray(ci) || ci.length !== 2) return false;
const [lo, hi] = ci.map(Number);
if (!Number.isFinite(lo) || !Number.isFinite(hi)) return false;
return lo > 0; // the whole interval above zero — improvement, not a coin flip
}
/**
* Promote a CANDIDATE to PROVEN. Refuses without sufficient evidence, and there
* is no override parameter on purpose.
*/
function promote(sport, key, evidence, at = null) {
const f = byKey.get(idOf(sport, key));
if (!f) return { ok: false, reason: 'unknown_feature' };
if (!isSufficient(evidence)) return { ok: false, reason: 'insufficient_evidence', required: { min_n: MIN_PROMOTION_N, lift: '>0', ci95_low: '>0' } };
f.history.push({ from: f.status, to: STATUS.PROVEN, evidence, at });
f.status = STATUS.PROVEN;
f.evidence = evidence;
return { ok: true, status: f.status };
}
/**
* Demote to DEAD. Unlike promotion this needs no threshold — removing something
* that is not helping is always safe, and the standing re-ablation must be able
* to act the moment a feature stops earning its place.
*/
function demote(sport, key, evidence, at = null) {
const f = byKey.get(idOf(sport, key));
if (!f) return { ok: false, reason: 'unknown_feature' };
f.history.push({ from: f.status, to: STATUS.DEAD, evidence: evidence || null, at });
f.status = STATUS.DEAD;
f.evidence = evidence || f.evidence;
return { ok: true, status: f.status };
}
/** Registry summary — what is live, what is being measured, what died and why. */
function summary(sport) {
const fs = allFeatures(sport);
const pick = (s) => fs.filter((f) => f.status === s).map((f) => f.key);
return {
sport: String(sport || '').toLowerCase(),
proven: pick(STATUS.PROVEN),
candidate: pick(STATUS.CANDIDATE),
dead: pick(STATUS.DEAD).map((k) => ({ key: k, why: (byKey.get(idOf(sport, k)).evidence || {}).note || null })),
total: fs.length,
};
}
/** Test-only: restore the declared statuses so suites cannot leak into each other. */
function __reset() {
byKey.clear();
for (const f of FEATURES) byKey.set(`${f.sport}|${f.key}`, { ...f, evidence: f.evidence || null, history: [] });
}
module.exports = {
STATUS, MIN_PROMOTION_N,
allFeatures, liveFeatures, candidateFeatures, statusOf, isLive,
isSufficient, promote, demote, summary, __reset,
};
src/services/model/skillProjection.js
+417
View File
@@ -0,0 +1,417 @@
'use strict';
/**
* skillProjection (skill-v1) — A FORWARD READ OF TONIGHT'S PLATE APPEARANCES.
*
* This is the windshield. The incumbent is a rear-view mirror: it counts how
* often the hitter cleared this number lately and calls that a forecast. It has
* never seen the pitcher. Measured, that counter IS the champion — removing all
* three of its adjustment layers changes resolution by nothing — so beating it
* requires different INFORMATION, not a different distribution. Two challengers
* (proj-v1.1, hits-v1) already proved that by swapping the distribution and
* losing.
*
* ── DISCIPLINE 1: SKILL, NOT RESULTS ─────────────────────────────────────
* Every input here measures the skill that CAUSES the outcome. Exit velocity,
* not batting average. Whiff rate, not ERA. Results are luck-contaminated —
* a .340 BABIP regresses and a 95-mph average exit velocity does not — so a
* model built on results is fitting noise it will not see again.
*
* ── THE GENERATIVE STRUCTURE ─────────────────────────────────────────────
* A hitter gets N plate appearances. Each one resolves through a tree:
*
* PA ──> strikeout (batter whiff/K vs pitcher whiff/K)
* ──> walk (batter BB vs pitcher BB)
* ──> ball in play (everything else)
* └─> hit? (contact quality vs contact allowed, park)
* └─> how many bases? (launch/barrel profile)
*
* Rates are combined by the ODDS RATIO (log5) against league average — the
* standard way to answer "this batter vs THIS pitcher" rather than "this batter
* vs the average pitcher":
*
* odds = (b/(1−b)) · (p/(1−p)) / (lg/(1−lg))
*
* With b = league and p = league it returns league; with an average pitcher it
* returns the batter's own rate. That identity is the reason to use it, and it
* is unit-tested here rather than assumed.
*
* ── DISCIPLINE 2: ARCHETYPE SELECTS FEATURES ─────────────────────────────
* The archetype is NOT a nudge and NOT a label. It decides WHICH skill inputs
* drive THIS hitter. A BOMBER's hit outcomes are governed by barrel rate and
* launch — his ground balls are outs. A GHOST beats out infield hits, so ground-
* ball rate and speed matter and barrels are nearly irrelevant. Feeding both
* hitters the same feature weights is exactly the "one model fit to all" error
* the per-sport doctrine forbids, one level further down.
*
* Features are ACTIVE where they apply and SILENT otherwise — silent meaning
* they contribute nothing, never that a default is invented in their place.
*
* ── DISCIPLINE 3: THE GATE ───────────────────────────────────────────────
* Every input is checked against `featureRegistry`. Nothing that has not earned
* PROVEN can reach a served projection; the challenger runs on CANDIDATEs so
* they can be measured. `allowed` is passed in rather than read here so the
* caller decides which gate applies, and both are testable.
*
* ── HONESTY ──────────────────────────────────────────────────────────────
* Unknown is not zero, everywhere: a missing rate makes its branch SILENT, it
* never becomes a measured 0. If the hitter's own skill profile is missing there
* is no forward read at all and the function returns null — the caller falls
* back rather than being handed a confident guess.
*/
const { knownRate } = require('../../utils/known');
/**
* League baselines, MLB. Used ONLY as the denominator of the odds ratio and as
* the regression target for thin samples — never as a substitute for a missing
* player. Measured from the 2026 Statcast aggregate set.
*/
const LEAGUE = Object.freeze({
k_pct: 0.222,
bb_pct: 0.085,
babip: 0.291, // hits per ball in play
hard_hit_pct: 0.389,
barrel_pct: 0.078,
avg_exit_velo: 88.8,
bases_per_hit: 1.60, // league slugging ÷ batting average
});
/** Plate appearances per game for a regular; only used when nothing better is known. */
const DEFAULT_PA = 4.1;
const PA_CAP = 7;
const MIN_PA = 1;
/**
* ODDS RATIO (log5). "This batter against THIS pitcher", relative to league.
*
* Returns null when either side is unknown — an absent pitcher must leave the
* batter's own rate untouched, which the caller does by skipping the combine,
* NOT by substituting league for the missing side (that would quietly pull every
* unknown matchup toward average and call it a read).
*/
function oddsRatio(batterRate, pitcherRate, leagueRate) {
const b = knownRate(batterRate);
const p = knownRate(pitcherRate);
const l = knownRate(leagueRate);
if (b === null || p === null || l === null) return null;
if (b <= 0 && p <= 0) return 0;
if (b >= 1 || p >= 1) return 1;
if (l <= 0 || l >= 1) return null;
const ob = b / (1 - b);
const op = p / (1 - p);
const ol = l / (1 - l);
if (ol === 0) return null;
const odds = (ob * op) / ol;
if (!Number.isFinite(odds)) return null;
return odds / (1 + odds);
}
/**
* Shrink a rate toward an explicit anchor by sample size. A 12-PA callup with a
* 60% hard-hit rate is not a 60% hard-hit hitter; without this the model chases
* noise exactly the way a results-based model does.
*
* The anchor is a REQUIRED argument rather than an internal default, because a
* hidden regression target is how a model quietly becomes the league average
* wearing a player's name. At the call site you can always see what it is being
* pulled toward.
*/
function shrink(rate, sample, stabilizeAt, anchor) {
const r = knownRate(rate);
const a = knownRate(anchor);
if (r === null) return null;
if (a === null) return r;
const n = knownRate(sample);
const k = knownRate(stabilizeAt);
if (n === null || k === null || k <= 0) return r;
const w = n / (n + k);
return w * r + (1 - w) * a;
}
/**
* THE UNITS CHOKEPOINT — `statcast_aggregates` stores PERCENTAGES (0–100).
*
* Baseball Savant's CSVs give `k_percent: 29.6`, not `0.296`, and the aggregate
* table stores them verbatim. This model works in probabilities. Feeding the raw
* row straight in makes `bip = 1 − 29.6 − 17.1` deeply negative, which is how
* this was caught: the first Stage A run refused 568 of 576 real rows rather
* than emitting nonsense. The honest-absent guards did their job — but a model
* that refuses everything is not a model, so the conversion lives HERE, once, at
* the boundary, and every consumer goes through it.
*
* Velocity (mph) and launch angle (degrees) are ALREADY natural units and must
* not be scaled. Mixing the two is precisely the trap, so the split is explicit
* rather than inferred from the value's magnitude — a 0.8% barrel rate and a
* 0.8 fraction are indistinguishable by size, and guessing would silently turn
* an elite hitter into a replacement one.
*/
const PCT_FIELDS = Object.freeze([
'k_pct', 'bb_pct', 'whiff_pct', 'swing_pct', 'chase_pct', 'barrel_pct',
'hard_hit_pct', 'sweet_spot_pct', 'ev95_pct', 'gb_pct', 'fb_pct', 'ld_pct',
]);
const RAW_FIELDS = Object.freeze(['avg_exit_velo', 'max_exit_velo', 'avg_launch_angle', 'arm_angle']);
/**
* Convert a `statcast_aggregates` row into the probability-space profile this
* model expects. Absent stays absent; an out-of-range percentage is treated as
* BROKEN (null) rather than clamped, because a 140% whiff rate is not a thin
* measurement, it is a parsing error and must not be modelled.
*/
function fromStatcastRow(row) {
if (!row || typeof row !== 'object') return null;
const out = {};
for (const f of PCT_FIELDS) {
const v = knownRate(row[f]);
out[f] = (v === null || v > 100) ? null : v / 100;
}
for (const f of RAW_FIELDS) out[f] = knownRate(row[f]);
out.bats = row.bats ?? null;
out.throws = row.throws ?? null;
out.sample_pa = knownRate(row.sample_pa);
out.sample_bip = knownRate(row.sample_bip);
return out;
}
/**
* ARCHETYPE FEATURE MAPS — Discipline 2, stated as data.
*
* `hitWeights` decide what governs whether a ball in play becomes a hit for THIS
* kind of hitter. `powerWeight` scales how much his contact quality converts to
* EXTRA bases. Weights are relative within a map and are documented by what the
* archetype physically does, not fitted — fitting them on 1,741 settled rows
* would be curve-fitting, and the registry exists so they get measured instead.
*
* Unknown archetype → DEFAULT, which is a balanced map, not a refusal: we still
* know the hitter's own skill rates.
*/
const ARCHETYPE_MAP = Object.freeze({
// Elite power. Barrels and launch decide his outcomes; his grounders are outs.
BOMBER: { hitWeights: { barrel: 0.5, hard_hit: 0.35, exit_velo: 0.15, gb_speed: 0 }, powerWeight: 1.25 },
// Speed-first. Beats out grounders; barrels are close to irrelevant.
GHOST: { hitWeights: { barrel: 0.05, hard_hit: 0.2, exit_velo: 0.15, gb_speed: 0.6 }, powerWeight: 0.7 },
// Pure contact, sprays it, low whiff.
TORCH: { hitWeights: { barrel: 0.2, hard_hit: 0.35, exit_velo: 0.25, gb_speed: 0.2 }, powerWeight: 1.0 },
DEFAULT: { hitWeights: { barrel: 0.25, hard_hit: 0.35, exit_velo: 0.25, gb_speed: 0.15 }, powerWeight: 1.0 },
});
function featureMapFor(archetype) {
const key = String(archetype || '').toUpperCase();
return ARCHETYPE_MAP[key] || ARCHETYPE_MAP.DEFAULT;
}
/**
* Per-ball-in-play hit probability, built from the ARCHETYPE-SELECTED skill
* inputs, the pitcher's contact suppression, and the park.
*
* Each component is a RATIO to league, so a hitter exactly at league on every
* axis lands on league BABIP and the model says "average" rather than inventing
* a lean. Components with no data are SILENT: they drop out of the weighted
* average and the remaining weights renormalise, so an absent input never acts
* as a measured zero.
*/
function hitOnContact({ batter, pitcher, park, archetype, allowed }) {
const map = featureMapFor(archetype);
const can = (k) => !allowed || allowed.has(k);
const parts = [];
const push = (weight, value, leagueValue, gateKey) => {
if (!weight || !can(gateKey)) return;
const v = knownRate(value);
const l = knownRate(leagueValue);
if (v === null || l === null || l <= 0) return; // SILENT, not zero
parts.push({ weight, ratio: v / l });
};
push(map.hitWeights.barrel, batter.barrel_pct, LEAGUE.barrel_pct, 'batter_barrel_pct');
push(map.hitWeights.hard_hit, batter.hard_hit_pct, LEAGUE.hard_hit_pct, 'batter_hard_hit_pct');
push(map.hitWeights.exit_velo, batter.avg_exit_velo, LEAGUE.avg_exit_velo, 'batter_exit_velo');
// GB/speed lane: a ground-ball hitter's hits come from beating out contact, so
// his OWN ground-ball tilt is the driver. Launch angle stands in for it —
// lower launch = more grounders — inverted so "more grounders" reads as more.
if (map.hitWeights.gb_speed && can('batter_launch_angle')) {
const la = knownRate(batter.avg_launch_angle);
if (la !== null) {
// League launch ~12°. Below it → grounder tilt → ratio > 1 for a GHOST.
const ratio = 12 > 0 ? (2 - Math.min(2, Math.max(0.2, la / 12))) : null;
if (ratio !== null) parts.push({ weight: map.hitWeights.gb_speed, ratio });
}
}
if (parts.length === 0) return null; // no skill signal at all — honest-absent
const wSum = parts.reduce((a, p) => a + p.weight, 0);
if (wSum <= 0) return null;
const skillRatio = parts.reduce((a, p) => a + p.weight * p.ratio, 0) / wSum;
// Pitcher contact suppression: hard contact ALLOWED relative to league.
let pitcherRatio = 1;
if (can('pitcher_hard_hit_allowed')) {
const ph = knownRate(pitcher && pitcher.hard_hit_pct);
if (ph !== null && LEAGUE.hard_hit_pct > 0) pitcherRatio = ph / LEAGUE.hard_hit_pct;
}
const parkMult = can('park_factor') ? (knownRate(park) ?? 1) : 1;
// Bounded: no stack of ratios may claim more than a ±35% swing in BABIP. The
// bound is a statement about how much any of this can really know, not a fudge.
const raw = LEAGUE.babip * skillRatio * pitcherRatio * parkMult;
const lo = LEAGUE.babip * 0.65;
const hi = LEAGUE.babip * 1.35;
return Math.min(hi, Math.max(lo, raw));
}
/**
* The PA outcome tree for one hitter against one pitcher.
* Returns per-PA probabilities, or null when the hitter has no usable profile.
*/
function paOutcome({ batter, pitcher, park, archetype, allowed }) {
if (!batter) return null;
const can = (k) => !allowed || allowed.has(k);
// Strikeout: batter K vs pitcher K, odds-ratio against league.
const bK = can('batter_k_pct') ? knownRate(batter.k_pct) : null;
const pK = can('pitcher_k_pct') ? knownRate(pitcher && pitcher.k_pct) : null;
let kRate = bK;
if (bK !== null && pK !== null) kRate = oddsRatio(bK, pK, LEAGUE.k_pct) ?? bK;
const bBB = can('batter_bb_pct') ? knownRate(batter.bb_pct) : null;
const pBB = can('pitcher_bb_pct') ? knownRate(pitcher && pitcher.bb_pct) : null;
let bbRate = bBB;
if (bBB !== null && pBB !== null) bbRate = oddsRatio(bBB, pBB, LEAGUE.bb_pct) ?? bBB;
// Absent → league, because a PA still has to resolve into SOMETHING. This is
// the one place a league value substitutes, and it is a structural necessity
// (the branches must sum to 1), not a guess about the player.
const k = kRate === null ? LEAGUE.k_pct : kRate;
const bb = bbRate === null ? LEAGUE.bb_pct : bbRate;
const bip = Math.max(0, 1 - k - bb);
if (bip <= 0) return null;
const pHitOnContact = hitOnContact({ batter, pitcher, park, archetype, allowed });
if (pHitOnContact === null) return null; // no skill read → no forward projection
return {
k_rate: k,
bb_rate: bb,
bip_rate: bip,
hit_on_contact: pHitOnContact,
p_hit_per_pa: bip * pHitOnContact,
inputs_used: {
pitcher_applied: pK !== null || pBB !== null || knownRate(pitcher && pitcher.hard_hit_pct) !== null,
archetype: String(archetype || 'DEFAULT').toUpperCase(),
},
};
}
/** Exact Binomial(n, p) pmf — n is tiny (PA per game ≤ 7). */
function binomialPmf(n, p) {
const nn = Math.max(0, Math.round(n));
const pp = Math.min(1, Math.max(0, p));
const out = new Array(nn + 1).fill(0);
let term = (1 - pp) ** nn;
out[0] = term;
for (let x = 1; x <= nn; x += 1) {
if (pp >= 1) { out[x] = x === nn ? 1 : 0; continue; }
term = (term * (nn - x + 1) * pp) / (x * (1 - pp));
out[x] = term;
}
return out;
}
/**
* Distribution over HITS, mixing over an integer PA distribution. PA is not
* fixed — a hitter gets 4 or 5 depending on how the lineup turns over — so the
* projection mixes rather than pretending PA is known.
*/
function paDistribution(expectedPa) {
const mean = Math.min(PA_CAP, Math.max(MIN_PA, knownRate(expectedPa) ?? DEFAULT_PA));
const lo = Math.floor(mean);
const hi = Math.min(PA_CAP, lo + 1);
const wHi = mean - lo;
const out = new Array(PA_CAP + 1).fill(0);
out[lo] += 1 - wHi;
if (hi !== lo) out[hi] += wHi; else out[lo] = 1;
return out;
}
/** P(X >= k) from a pmf. */
function atLeast(pmf, k) {
const kk = Math.max(0, Math.ceil(k));
if (kk === 0) return 1;
let s = 0;
for (let i = kk; i < pmf.length; i += 1) s += pmf[i];
return Math.min(1, Math.max(0, s));
}
/**
* THE FORWARD READ.
*
* @returns {object|null} distribution + P(clears line) + the reasoning trace, or
* null when there is no usable skill profile (caller falls back; never a guess).
*/
function projectSkill({
batter, pitcher = null, park = 1, archetype = null,
statType = 'hits', line, expectedPa = null, allowed = null,
} = {}) {
// STAGE A IS HITS ONLY, and total_bases is refused DELIBERATELY.
//
// The first cut mapped hits to bases with an archetype-scaled constant
// (bases_per_hit x powerWeight). Smoke-tested on a real profile that made
// P(TB>=2) come out EXACTLY equal to P(hits>=1) — because a deterministic
// multiplier just relabels the hits distribution. It would have measured as a
// "TB model" while carrying no information a hits model does not already have.
// tb-v1 already established the right shape: total bases is a COMPOUND
// outcome (1B/2B/3B/HR each with its own rate), not hits x a constant. Doing
// that properly needs a per-hit extra-base distribution off the launch/barrel
// profile, which is its own build. Refusing beats shipping a relabel.
const stat = String(statType || '').toLowerCase();
if (stat !== 'hits') return null;
const target = Math.max(1, Math.ceil(Number(line)));
if (!Number.isFinite(target)) return null;
const pa = paOutcome({ batter, pitcher, park, archetype, allowed });
if (!pa) return null;
const paPmf = paDistribution(expectedPa);
const map = featureMapFor(archetype);
// HITS — Binomial(PA, p_hit_per_pa), mixed over the PA distribution.
const hitPmf = new Array(PA_CAP + 1).fill(0);
for (let n = 0; n < paPmf.length; n += 1) {
if (!paPmf[n]) continue;
const bp = binomialPmf(n, pa.p_hit_per_pa);
for (let x = 0; x < bp.length; x += 1) hitPmf[x] += paPmf[n] * bp[x];
}
const expectedHits = hitPmf.reduce((a, p, i) => a + p * i, 0);
return buildResult({ pmf: hitPmf, mean: expectedHits, target, pa, map, archetype, stat });
}
function buildResult({ pmf, mean, target, pa, map, archetype, stat, caveat = null }) {
const p = atLeast(pmf, target);
const r3 = (v) => Math.round(v * 1000) / 1000;
return {
version: 'skill-v1',
stat,
p_over_line: r3(p),
projected_value: r3(mean),
distribution: pmf.map(r3),
per_pa: {
k_rate: r3(pa.k_rate), bb_rate: r3(pa.bb_rate),
bip_rate: r3(pa.bip_rate), hit_on_contact: r3(pa.hit_on_contact),
p_hit_per_pa: r3(pa.p_hit_per_pa),
},
archetype: String(archetype || 'DEFAULT').toUpperCase(),
feature_map: map.hitWeights,
pitcher_applied: pa.inputs_used.pitcher_applied,
family: 'pa_outcome_tree_binomial',
caveat,
};
}
module.exports = {
projectSkill, paOutcome, hitOnContact, oddsRatio, shrink, fromStatcastRow,
binomialPmf, paDistribution, atLeast, featureMapFor,
LEAGUE, ARCHETYPE_MAP, DEFAULT_PA, PA_CAP, PCT_FIELDS, RAW_FIELDS,
};
src/snapshotScheduler.js
+51 -29
View File
@@ -314,34 +314,6 @@ function startSnapshotScheduler(opts = {}) {
} }
} catch (e) { console.warn('[instrument] closing attach failed:', e.message); } } catch (e) { console.warn('[instrument] closing attach failed:', e.message); }
// Session 68 — LAYER 1 MECHANISM DATA. Nightly full re-pull of the
// ~1,350-row Statcast aggregate set at STATCAST_HOUR_UTC. Backfill and
// refresh are the SAME call, upserted on the natural key, so the job is
// idempotent and self-healing: a missed night self-corrects on the next
// run with no incremental bookkeeping to drift. Kill switch: STATCAST=0.
try {
const statcastHour = Number(process.env.STATCAST_HOUR_UTC || 11); // ~7 AM ET, after every game is final
if (h === statcastHour && process.env.STATCAST !== '0') {
const agg = opts.statcastService || require('./services/statcastAggregateService');
const res = await agg.refreshSeason({});
console.log(`[statcast] nightly refresh — ok=${res.ok} rows=${res.rows ?? 'n/a'} written=${res.written ?? 0} joined=${res.joined ?? 'n/a'} thin=${res.thin ?? 'n/a'}${res.reason ? ` reason: ${res.reason}` : ''}`);
if (!res.ok) {
await notify(`Statcast refresh failed — ${res.reason || 'unknown'}. Mechanism data is not updating.`, {
title: 'VYNDR statcast', priority: 'high', tags: ['rotating_light'],
});
}
// Serving a stale aggregate as current is a quiet fabrication of
// currency, so staleness pages on its own — separate from a failed run,
// because a silently-not-scheduled job never produces a failure.
const fresh = await agg.getFreshness({});
if (agg.isStale(fresh)) {
await notify(`Statcast aggregates stale — last updated ${fresh.updated_at} (${fresh.age_hours}h). Mechanism data is being served as current.`, {
title: 'VYNDR statcast', priority: 'high', tags: ['rotating_light'],
});
}
}
} catch (e) { console.warn('[statcast] nightly refresh failed:', e.message); }
// Session 8 — quota check after each snapshot run: odds-api >= 80% alerts // Session 8 — quota check after each snapshot run: odds-api >= 80% alerts
// once per day (Redis-deduped). Never throws (guarded inside checkQuotaDaily). // once per day (Redis-deduped). Never throws (guarded inside checkQuotaDaily).
await opsWatch.checkQuotaDaily({ getStatus: getQuotaStatus, cacheGet, cacheSet, notify, now: () => now() }); await opsWatch.checkQuotaDaily({ getStatus: getQuotaStatus, cacheGet, cacheSet, notify, now: () => now() });
@@ -378,7 +350,57 @@ function startSnapshotScheduler(opts = {}) {
} }
}; };
const interval = setInterval(() => { void tick(); void refreshTick(); }, 60_000); /**
* Session 68 — LAYER 1 MECHANISM DATA. Nightly full re-pull of the ~1,350-row
* Statcast aggregate set at STATCAST_HOUR_UTC. Backfill and refresh are the
* SAME call, upserted on the natural key, so the job is idempotent and
* self-healing. Kill switch: STATCAST=0.
*
* ITS OWN TICK, and that is the whole point (fixed 2026-08-03). This block
* used to live inside `tick()`, BELOW `if (!HOURS_UTC.includes(h)) return`.
* HOURS_UTC is 14,19,22,1,3 and the block tests `h === STATCAST_HOUR_UTC`
* (default 11), so the guard could never admit the hour it was waiting for:
* the refresh was unreachable code and had never run. The aggregates sat
* frozen at their 2026-07-21 backfill for 13 days while every consumer served
* them as current — and because BOTH alerts below also sat inside the dead
* branch, nothing could report it. Same silently-guarded-out shape as the
* settlement outage. The skill model reads this table, so stale mechanism data
* is not a cosmetic problem: it is the model looking at a fortnight-old game.
*/
let lastStatcastSlot = null;
const statcastTick = async () => {
if (process.env.STATCAST === '0') return;
const d = now();
if (d.getUTCMinutes() !== 0) return;
const h = d.getUTCHours();
const statcastHour = Number(process.env.STATCAST_HOUR_UTC || 11); // ~7 AM ET, after every game is final
if (h !== statcastHour) return;
const slot = `${d.toISOString().slice(0, 10)}-${h}`;
if (slot === lastStatcastSlot) return; // fire once per day
lastStatcastSlot = slot;
try {
const agg = opts.statcastService || require('./services/statcastAggregateService');
const res = await agg.refreshSeason({});
console.log(`[statcast] nightly refresh — ok=${res.ok} rows=${res.rows ?? 'n/a'} written=${res.written ?? 0} joined=${res.joined ?? 'n/a'} thin=${res.thin ?? 'n/a'}${res.reason ? ` reason: ${res.reason}` : ''}`);
if (!res.ok) {
await notify(`Statcast refresh failed — ${res.reason || 'unknown'}. Mechanism data is not updating.`, {
title: 'VYNDR statcast', priority: 'high', tags: ['rotating_light'],
});
}
// Serving a stale aggregate as current is a quiet fabrication of currency,
// so staleness pages on its own — separate from a failed run, because a
// silently-not-scheduled job never produces a failure. Which is exactly
// what happened here for 13 days.
const fresh = await agg.getFreshness({});
if (agg.isStale(fresh)) {
await notify(`Statcast aggregates stale — last updated ${fresh.updated_at} (${fresh.age_hours}h). Mechanism data is being served as current.`, {
title: 'VYNDR statcast', priority: 'high', tags: ['rotating_light'],
});
}
} catch (e) { console.warn('[statcast] nightly refresh failed:', e.message); }
};
const interval = setInterval(() => { void tick(); void refreshTick(); void statcastTick(); }, 60_000);
if (interval.unref) interval.unref(); if (interval.unref) interval.unref();
console.log(`[snapshotScheduler] armed — SNAPSHOT_CRON=${process.env.SNAPSHOT_CRON}, hours=${HOURS_UTC.join(',')} UTC, intraday=${process.env.INTRADAY_REFRESH === '0' ? 'off' : `${REFRESH_MINUTES}m (slate hours)`}, statcast=${process.env.STATCAST === '0' ? 'off' : `${process.env.STATCAST_HOUR_UTC || 11}h UTC`}`); console.log(`[snapshotScheduler] armed — SNAPSHOT_CRON=${process.env.SNAPSHOT_CRON}, hours=${HOURS_UTC.join(',')} UTC, intraday=${process.env.INTRADAY_REFRESH === '0' ? 'off' : `${REFRESH_MINUTES}m (slate hours)`}, statcast=${process.env.STATCAST === '0' ? 'off' : `${process.env.STATCAST_HOUR_UTC || 11}h UTC`}`);
// Job 1 — state each sport's cadence at boot so "when does WNBA grade?" is // Job 1 — state each sport's cadence at boot so "when does WNBA grade?" is
tests/unit/skillModel.test.js
+197
View File
@@ -0,0 +1,197 @@
'use strict';
/**
* THE FIVE DISCIPLINES, asserted as behaviour.
*
* These are not coverage tests. Each block locks one discipline so it cannot be
* quietly abandoned later — which is exactly how the previous engine drifted
* into a frequency counter with six wired-but-dead features hanging off it.
*/
const reg = require('../../src/services/model/featureRegistry');
const sk = require('../../src/services/model/skillProjection');
const BOMBER = { k_pct: 0.28, bb_pct: 0.11, barrel_pct: 0.16, hard_hit_pct: 0.52, avg_exit_velo: 93.5, avg_launch_angle: 19 };
const GHOST = { k_pct: 0.14, bb_pct: 0.07, barrel_pct: 0.02, hard_hit_pct: 0.30, avg_exit_velo: 85.0, avg_launch_angle: 6 };
const ACE = { k_pct: 0.32, bb_pct: 0.05, hard_hit_pct: 0.30, whiff_pct: 0.33 };
const BATTING_PRACTICE = { k_pct: 0.15, bb_pct: 0.10, hard_hit_pct: 0.46, whiff_pct: 0.18 };
afterEach(() => reg.__reset());
// ── DISCIPLINE 3 ──────────────────────────────────────────────────────────
describe('D3 — earn its place or it is out', () => {
it('starts with an essentially EMPTY proven set — nothing is assumed to work', () => {
const live = reg.liveFeatures('mlb');
// Only the incumbent counter is PROVEN, and only because it was measured.
expect([...live]).toEqual(['recent_frequency_prior']);
});
it('refuses promotion without a real sample, positive lift, and a CI above zero', () => {
expect(reg.promote('mlb', 'batter_barrel_pct', null).ok).toBe(false);
expect(reg.promote('mlb', 'batter_barrel_pct', { n: 10, lift: 0.05, ci95: [0.01, 0.09] }).ok).toBe(false); // n too small
expect(reg.promote('mlb', 'batter_barrel_pct', { n: 500, lift: -0.02, ci95: [-0.05, -0.01] }).ok).toBe(false); // negative lift
expect(reg.promote('mlb', 'batter_barrel_pct', { n: 500, lift: 0.03, ci95: [-0.01, 0.07] }).ok).toBe(false); // CI straddles zero
expect(reg.statusOf('mlb', 'batter_barrel_pct')).toBe(reg.STATUS.CANDIDATE);
});
it('promotes on sufficient evidence, and records WHY', () => {
const ev = { n: 600, lift: 0.04, ci95: [0.012, 0.068], measured_at: '2026-09-01' };
expect(reg.promote('mlb', 'batter_barrel_pct', ev).ok).toBe(true);
expect(reg.isLive('mlb', 'batter_barrel_pct')).toBe(true);
expect(reg.liveFeatures('mlb').has('batter_barrel_pct')).toBe(true);
});
it('keeps DEAD features BY NAME so they are not silently rebuilt', () => {
const s = reg.summary('mlb');
const deadKeys = s.dead.map((d) => d.key);
expect(deadKeys).toEqual(expect.arrayContaining([
'champion_opp_rank_adj', 'champion_home_away_adj', 'champion_consistency_pull',
]));
// and each carries the reason it died
expect(s.dead.find((d) => d.key === 'champion_home_away_adj').why).toMatch(/harmful|IMPROVED/i);
});
it('per-sport: a feature proven for MLB says nothing about another sport', () => {
reg.promote('mlb', 'batter_barrel_pct', { n: 600, lift: 0.04, ci95: [0.01, 0.07] });
expect(reg.isLive('nba', 'batter_barrel_pct')).toBe(false);
expect(reg.statusOf('nba', 'batter_barrel_pct')).toBeNull();
});
it('THE GATE: with only PROVEN features allowed, the skill model REFUSES', () => {
// The proven set carries no contact-quality feature yet, so there is no
// forward read to make. Refusing is the correct output — this is what stops
// an unproven model reaching a user by accident.
const out = sk.projectSkill({
batter: BOMBER, pitcher: ACE, archetype: 'BOMBER', statType: 'hits', line: 0.5,
expectedPa: 4.2, allowed: reg.liveFeatures('mlb'),
});
expect(out).toBeNull();
});
it('with CANDIDATES allowed (the challenger), it produces a read', () => {
const out = sk.projectSkill({
batter: BOMBER, pitcher: ACE, archetype: 'BOMBER', statType: 'hits', line: 0.5,
expectedPa: 4.2, allowed: reg.candidateFeatures('mlb'),
});
expect(out).not.toBeNull();
expect(out.p_over_line).toBeGreaterThan(0);
});
});
// ── DISCIPLINE 1 ──────────────────────────────────────────────────────────
describe('D1 — skill, not results: the pitcher actually moves the read', () => {
const run = (pitcher) => sk.projectSkill({
batter: BOMBER, pitcher, archetype: 'BOMBER', statType: 'hits', line: 0.5, expectedPa: 4.2,
});
it('a better pitcher LOWERS the projection for the same hitter', () => {
const vsAce = run(ACE);
const vsBP = run(BATTING_PRACTICE);
expect(vsAce.p_over_line).toBeLessThan(vsBP.p_over_line);
expect(vsAce.projected_value).toBeLessThan(vsBP.projected_value);
});
it('this is the thing the counter cannot do — no pitcher at all is a different read', () => {
const noPitcher = run(null);
expect(noPitcher).not.toBeNull();
expect(noPitcher.pitcher_applied).toBe(false);
expect(run(ACE).pitcher_applied).toBe(true);
});
it('the odds ratio returns league when both sides are league (the identity)', () => {
expect(sk.oddsRatio(0.222, 0.222, 0.222)).toBeCloseTo(0.222, 6);
});
it('an average pitcher leaves the batter rate untouched', () => {
expect(sk.oddsRatio(0.30, 0.222, 0.222)).toBeCloseTo(0.30, 6);
});
it('shrinks a thin sample toward its anchor, and leaves a full one alone', () => {
expect(sk.shrink(0.60, 10, 200, 0.39)).toBeLessThan(0.45); // 10-PA callup
expect(sk.shrink(0.60, 5000, 200, 0.39)).toBeGreaterThan(0.59); // established
});
});
// ── DISCIPLINE 2 ──────────────────────────────────────────────────────────
describe('D2 — archetype SELECTS features, it does not nudge', () => {
it('the SAME hitter read through two archetypes gets materially different reads', () => {
const asBomber = sk.projectSkill({ batter: BOMBER, pitcher: ACE, archetype: 'BOMBER', statType: 'hits', line: 0.5, expectedPa: 4.2 });
const asGhost = sk.projectSkill({ batter: BOMBER, pitcher: ACE, archetype: 'GHOST', statType: 'hits', line: 0.5, expectedPa: 4.2 });
// Not a rounding difference — a different feature map entirely.
expect(Math.abs(asBomber.p_over_line - asGhost.p_over_line)).toBeGreaterThan(0.15);
});
it("a BOMBER's read is driven by barrels; a GHOST's is nearly blind to them", () => {
expect(sk.featureMapFor('BOMBER').hitWeights.barrel).toBeGreaterThan(0.4);
expect(sk.featureMapFor('GHOST').hitWeights.barrel).toBeLessThan(0.1);
expect(sk.featureMapFor('GHOST').hitWeights.gb_speed).toBeGreaterThan(0.5);
expect(sk.featureMapFor('BOMBER').hitWeights.gb_speed).toBe(0);
});
it('barrel rate moves a BOMBER and barely moves a GHOST — features silent where they do not apply', () => {
const move = (arch) => {
const lo = sk.hitOnContact({ batter: { ...BOMBER, barrel_pct: 0.04 }, pitcher: ACE, park: 1, archetype: arch });
const hi = sk.hitOnContact({ batter: { ...BOMBER, barrel_pct: 0.20 }, pitcher: ACE, park: 1, archetype: arch });
return hi - lo;
};
expect(move('BOMBER')).toBeGreaterThan(move('GHOST') * 3);
});
it('an unknown archetype falls back to a balanced map, not a refusal', () => {
const out = sk.projectSkill({ batter: BOMBER, pitcher: ACE, archetype: 'NOT_A_REAL_ONE', statType: 'hits', line: 0.5, expectedPa: 4.2 });
expect(out).not.toBeNull();
expect(out.archetype).toBe('NOT_A_REAL_ONE');
});
});
// ── HONESTY ───────────────────────────────────────────────────────────────
describe('honesty — unknown is not zero, and absent beats invented', () => {
it('no hitter profile → NO forward read (null), never a league-average guess', () => {
expect(sk.projectSkill({ batter: null, pitcher: ACE, statType: 'hits', line: 0.5 })).toBeNull();
expect(sk.projectSkill({ batter: {}, pitcher: ACE, statType: 'hits', line: 0.5 })).toBeNull();
});
it('a missing skill input is SILENT — it does not act as a measured zero', () => {
const full = sk.hitOnContact({ batter: BOMBER, pitcher: ACE, park: 1, archetype: 'BOMBER' });
const noBarrel = sk.hitOnContact({ batter: { ...BOMBER, barrel_pct: null }, pitcher: ACE, park: 1, archetype: 'BOMBER' });
const zeroBarrel = sk.hitOnContact({ batter: { ...BOMBER, barrel_pct: 0 }, pitcher: ACE, park: 1, archetype: 'BOMBER' });
expect(noBarrel).not.toBeNull();
// A REAL zero is a fact and must read far lower than an ABSENT one.
expect(zeroBarrel).toBeLessThan(noBarrel);
expect(Math.abs(noBarrel - full)).toBeLessThan(Math.abs(zeroBarrel - full));
});
it('total_bases is REFUSED rather than shipped as a relabelled hits curve', () => {
// A deterministic bases-per-hit multiplier made P(TB>=2) identical to
// P(hits>=1). Refusing is correct until a real per-hit bases distribution
// exists (tb-v1's compound shape).
expect(sk.projectSkill({ batter: BOMBER, pitcher: ACE, archetype: 'BOMBER', statType: 'total_bases', line: 1.5 })).toBeNull();
});
it('the distribution is a real distribution and P(>=k) is monotone', () => {
const out = sk.projectSkill({ batter: BOMBER, pitcher: ACE, archetype: 'BOMBER', statType: 'hits', line: 0.5, expectedPa: 4.2 });
expect(out.distribution.reduce((a, b) => a + b, 0)).toBeCloseTo(1, 2);
let prev = 1;
for (let k = 1; k <= 5; k += 1) {
const p = sk.atLeast(out.distribution, k);
expect(p).toBeLessThanOrEqual(prev + 1e-9);
prev = p;
}
});
it('a hitter exactly at league on every axis lands on league BABIP — no invented lean', () => {
const leagueBat = {
k_pct: sk.LEAGUE.k_pct, bb_pct: sk.LEAGUE.bb_pct,
barrel_pct: sk.LEAGUE.barrel_pct, hard_hit_pct: sk.LEAGUE.hard_hit_pct,
avg_exit_velo: sk.LEAGUE.avg_exit_velo, avg_launch_angle: 12,
};
const boc = sk.hitOnContact({ batter: leagueBat, pitcher: { hard_hit_pct: sk.LEAGUE.hard_hit_pct }, park: 1, archetype: 'DEFAULT' });
expect(boc).toBeCloseTo(sk.LEAGUE.babip, 2);
});
it('opportunity matters: more projected PA means a higher chance of one hit', () => {
const few = sk.projectSkill({ batter: BOMBER, pitcher: ACE, archetype: 'BOMBER', statType: 'hits', line: 0.5, expectedPa: 2 });
const many = sk.projectSkill({ batter: BOMBER, pitcher: ACE, archetype: 'BOMBER', statType: 'hits', line: 0.5, expectedPa: 5 });
expect(many.p_over_line).toBeGreaterThan(few.p_over_line);
});
});
tests/unit/statcastAggregate.test.js
+22 -1
View File
@@ -212,10 +212,31 @@ describe('scheduler + route wiring', () => {
it('is server-scheduled with a kill switch', () => { it('is server-scheduled with a kill switch', () => {
const s = read('src/snapshotScheduler.js'); const s = read('src/snapshotScheduler.js');
expect(s).toContain('STATCAST_HOUR_UTC'); expect(s).toContain('STATCAST_HOUR_UTC');
expect(s).toMatch(/process\.env\.STATCAST !== '0'/); expect(s).toMatch(/process\.env\.STATCAST === '0'/); // kill switch, early-return form
expect(s).toContain('refreshSeason'); expect(s).toContain('refreshSeason');
}); });
it('RUNS ON ITS OWN TICK — not behind the snapshot-hours guard', () => {
// THE REGRESSION THIS LOCKS (found 2026-08-03): the refresh used to live
// inside `tick()`, BELOW `if (!HOURS_UTC.includes(h)) return`. HOURS_UTC is
// 14,19,22,1,3 and the block tests h === STATCAST_HOUR_UTC (default 11), so
// the guard could never admit the hour it waited for. It was unreachable
// code that had never run once, and the aggregates sat 13 days stale while
// every consumer served them as current. The previous assertion here passed
// the entire time, because it only checked that the STRING existed.
const s = read('src/snapshotScheduler.js');
const statcastTick = s.slice(s.indexOf('const statcastTick'));
expect(statcastTick.length).toBeGreaterThan(0);
// Its own tick, registered on the interval alongside the others.
expect(s).toMatch(/void statcastTick\(\)/);
// And it must NOT be reachable only via the snapshot-hours guard.
// Scope to the snapshot tick's OWN body (it ends where refreshTick begins),
// so the statcastTick doc comment above it cannot satisfy this by accident.
const tickBody = s.slice(s.indexOf('const tick = async'), s.indexOf('const refreshTick'));
expect(tickBody).toContain('HOURS_UTC.includes(h)'); // the guard is still there
expect(tickBody).not.toContain('STATCAST_HOUR_UTC'); // and statcast is NOT behind it
});
it('pages on BOTH a failed run and silent staleness', () => { it('pages on BOTH a failed run and silent staleness', () => {
const s = read('src/snapshotScheduler.js'); const s = read('src/snapshotScheduler.js');
expect(s).toMatch(/Statcast refresh failed/); expect(s).toMatch(/Statcast refresh failed/);