Ingest defence, and make Bonferroni cumulative across the programme

Two things shipped that stand regardless of sample.

DEFENCE. Statcast Outs Above Average is free on the host we already pull six
feeds from, so there was nothing to decide. 514 fielders, aggregated to team
level -- the unit a batter's prop actually needs, the defence behind the
pitcher he faces -- and persisted as 31 team rows. Verified in production.
Cubs +56 best, Mariners -29 worst.

Unknown is not zero, and it bites unusually hard here: an OAA of 0 is a REAL
reading meaning exactly average, so coercing absence to 0 would assert that
every unmeasured fielder is league-average, which is the commonest defensive
profile there is. team_defense also carries as_of_date in its primary key from
the first row -- statcast_aggregates was built upsert-in-place and that
silently made every backtest leak the games it predicted, so point-in-time is
available here before it is needed rather than after a wrong answer.

A bug worth recording as a class: BASE already ends in /leaderboard, so the
new feed built a doubled path and 404'd. Because a failing feed degrades to an
empty index by design -- correct, so one broken source cannot fail the whole
pull -- it surfaced as "fielding_oaa: 0 rows", which reads exactly like
"Statcast has no fielding data". Graceful degradation makes a wiring bug look
like an honest absence.

CUMULATIVE CORRECTION. Bonferroni had been applied per session throughout: a
run testing eight features corrected by eight. Across a programme's lifetime
that is wrong in the dangerous direction, because every order gets a fresh
generous alpha and the false-positive rate compounds quietly. Correcting by 8
when sixty have been tried is how a noise result eventually gets recorded as
PROVEN with a p-value to point at. The denominator is now distinct hypotheses
ever tested, persisted, and it moved 19 -> 38 within this session alone, alpha
0.0026 -> 0.0013. Re-tests deliberately do not inflate it: re-asking the same
question on more data is not a new shot on goal, and counting it would punish
the discipline of waiting for sample.

THE MEASUREMENT. The differential the theory predicted is present: defence
correlates with the counter's residual at +0.130 for GHOST, the contact and
speed archetype, and -0.018 for BOMBER, the power archetype. A GHOST's hits
depend on whether anyone can range to the ball; a BOMBER's barrels clear the
defence entirely. So a flat BOMBER result is the theory working rather than
the test failing.

It is not a result. GHOST is n=104 against a 500 bar, with p=0.188 against a
corrected alpha of 0.0013 -- three orders of magnitude short. Both are
recorded as CANDIDATE with their measured lift, tagged contact-skill, so the
re-run at full sample compares against a recorded baseline.

Nothing proved, so nothing was recalibrated and nothing shipped.

4,228 tests green (336 suites); web build exit 0.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W1sivYNqY2TS5ftykmHBU9
This commit is contained in:
Kev
2026-08-03 22:20:30 -04:00
parent 25e36c0257
commit ece2b9f5f9
6 changed files with 435 additions and 6 deletions
+50 -6
View File
@@ -199,7 +199,7 @@ async function opponentByPlayerDate(players) {
const SOLO = ['batter_barrel_pct', 'batter_hard_hit_pct', 'batter_exit_velo',
'batter_launch_angle', 'batter_k_pct', 'batter_bb_pct',
'pitcher_k_pct', 'pitcher_hard_hit_allowed',
'pitcher_gb_pct', 'pitcher_fb_pct', 'pitcher_breaking_share'];
'pitcher_gb_pct', 'pitcher_fb_pct', 'pitcher_breaking_share', 'team_defense'];
/**
* PER-STAT INTERACTION SETS — the total_bases conditioning map RE-WEIGHTED, not
@@ -209,8 +209,8 @@ const SOLO = ['batter_barrel_pct', 'batter_hard_hit_pct', 'batter_exit_velo',
* and RBI through another (does anyone happen to be on base when it does).
*/
const STAT_INTERACTIONS = {
total_bases: ['launch_x_exit_velo', 'exitvelo_x_pitcher_suppression', 'barrel_x_power_archetype', 'batterK_x_pitcherK', 'launch_x_pitcher_gb', 'barrel_x_breaking_share'],
hits: ['launch_x_exit_velo', 'exitvelo_x_pitcher_suppression', 'batterK_x_pitcherK', 'launch_x_pitcher_gb', 'barrel_x_breaking_share'],
total_bases: ['launch_x_exit_velo', 'exitvelo_x_pitcher_suppression', 'barrel_x_power_archetype', 'batterK_x_pitcherK', 'launch_x_pitcher_gb', 'barrel_x_breaking_share', 'defense_x_contact', 'defense_x_speed_profile'],
hits: ['launch_x_exit_velo', 'exitvelo_x_pitcher_suppression', 'batterK_x_pitcherK', 'launch_x_pitcher_gb', 'barrel_x_breaking_share', 'defense_x_contact', 'defense_x_speed_profile'],
// HOME RUNS are the purest barrel stat: the ball must be hit hard AND at the
// right angle, and the pitcher must be the kind who allows that combination.
home_runs: ['launch_x_exit_velo', 'barrel_x_power_archetype', 'exitvelo_x_pitcher_suppression'],
@@ -243,6 +243,18 @@ const INTERACTIONS = [
mechanism: 'ARCHETYPE-CONDITIONAL. Barrels convert to extra bases for hitters whose lane is power; for a speed/contact profile the same barrel rate is a rarer event on a swing built for something else. This is Discipline 2 stated as a testable interaction. NOTE: it is currently UNTESTABLE — statcast rows carry no archetype label, and the barrel-relative proxy is an exact linear function of barrel_pct, so controlling for both components is rank-deficient. It needs a real archetype classification joined in.',
build: (r) => r.batter_barrel_pct * r.archetype_power,
},
{
key: 'defense_x_contact',
components: ['team_defense', 'batter_hard_hit_pct'],
mechanism: 'DEFENCE. A ball in play becomes a hit or an out partly by who is standing behind the pitcher. This should matter MOST for hitters whose value is contact that stays in the park, and LEAST for power hitters whose barrels clear the defence entirely — so a DEAD result for BOMBER is not a failure, it is the differential the theory predicts.',
build: (r) => r.team_defense * r.batter_hard_hit_pct,
},
{
key: 'defense_x_speed_profile',
components: ['team_defense', 'batter_launch_angle'],
mechanism: 'DEFENCE x BATTED-BALL PROFILE. A low-launch (ground-ball) hitter puts the ball where fielders range; a high-launch hitter does not. Launch angle stands in for the profile, so defence should condition the ground-ball hitter far more.',
build: (r) => r.team_defense * r.batter_launch_angle,
},
{
key: 'launch_x_pitcher_gb',
components: ['batter_launch_angle', 'pitcher_gb_pct'],
@@ -356,6 +368,14 @@ async function main() {
console.error(`[debug] sample statcast keys=${JSON.stringify([...batters.keys()].slice(0, 5))}`);
console.error(`[debug] matches in sample=${sampleKeys.filter((k) => batters.has(k)).length}/5`);
}
// TEAM DEFENCE — the newly-ingested Statcast OAA, per team, dated.
const defRows = await page(sb, 'team_defense', '*', (q) => q.eq('sport', 'mlb'));
const defByTeam = new Map();
for (const d of defRows) {
const prev = defByTeam.get(d.team);
if (!prev || String(d.as_of_date) > String(prev.as_of_date)) defByTeam.set(d.team, d);
}
const allowed = reg.candidateFeaturesForStat('mlb', STAT);
const rowsAll = [];
@@ -405,7 +425,17 @@ async function main() {
pitcher_breaking_share: pit && pit.pitch_mix ? breakingShare(pit.pitch_mix) : null,
// DEFENSE: NOT DERIVABLE from what we ingest — see the report. Recorded as
// null rather than proxied by something that is really pitching quality.
team_defense: null,
// DEFENCE behind the pitcher he faces. knownRate: an unmeasured team is
// ABSENT, never league-average — OAA 0 is a real "exactly average" reading
// and the two must stay distinguishable.
team_defense: (() => {
if (!faced) return null;
// team_defense keys on Savant's display name (a nickname, "Cubs"),
// while the game log gives the full name ("Chicago Cubs"). Try both.
const nick = String(faced).split(' ').pop();
const d = defByTeam.get(faced) || defByTeam.get(nick);
return d ? knownNumber(d.oaa_sum) : null;
})(),
});
}
@@ -413,8 +443,21 @@ async function main() {
const archRows = ARCH ? rowsAll.filter((r) => String(r.archetype || '').toUpperCase() === ARCH) : rowsAll;
rows.length = 0; rows.push(...archRows);
// Bonferroni denominator = every test in this family (solo + interaction).
const TESTS = SOLO.length + INTERACTIONS.length;
// ── CUMULATIVE BONFERRONI ─────────────────────────────────────────────
// The denominator is every DISTINCT hypothesis this programme has tested,
// not just this run's. Correcting by 8 in a session that tries 8, forever,
// while the programme as a whole has tried sixty, is how a noise result
// eventually gets recorded as PROVEN with a p-value to point at.
const tl = require('../src/services/model/testLedger');
const store = tl.supabaseStore(sb);
const chosenKeys = new Set(STAT_INTERACTIONS[STAT] || []);
const entries = [
...SOLO.map((f) => ({ sport: 'mlb', stat: STAT, archetype: ARCH, interaction: `solo:${f}`, target: 'counter_residual' })),
...INTERACTIONS.filter((x) => chosenKeys.has(x.key))
.map((x) => ({ sport: 'mlb', stat: STAT, archetype: ARCH, interaction: x.key, target: 'counter_residual' })),
];
const mc = await tl.recordAndCount(store, entries);
const TESTS = mc.cumulative_tests;
// ── STEP 1 — SOLO PASS (the control) ────────────────────────────────────
const solo = {};
@@ -482,6 +525,7 @@ async function main() {
rows_scored: rows.length,
gate_spec: cv.VALIDATION_REQUIREMENTS,
bonferroni_tests: TESTS,
multiple_comparisons: { ...mc, note: 'denominator is DISTINCT hypotheses across the programme lifetime, not this session' },
n_gap_note: `the gate needs ${cv.VALIDATION_REQUIREMENTS.min_historical_instances} rows; this run has ${rows.length}`,
archetype_coverage: {
labelled: rows.filter((r) => r.archetype).length,