builtbykev
ece2b9f5f9
Two things shipped that stand regardless of sample. DEFENCE. Statcast Outs Above Average is free on the host we already pull six feeds from, so there was nothing to decide. 514 fielders, aggregated to team level -- the unit a batter's prop actually needs, the defence behind the pitcher he faces -- and persisted as 31 team rows. Verified in production. Cubs +56 best, Mariners -29 worst. Unknown is not zero, and it bites unusually hard here: an OAA of 0 is a REAL reading meaning exactly average, so coercing absence to 0 would assert that every unmeasured fielder is league-average, which is the commonest defensive profile there is. team_defense also carries as_of_date in its primary key from the first row -- statcast_aggregates was built upsert-in-place and that silently made every backtest leak the games it predicted, so point-in-time is available here before it is needed rather than after a wrong answer. A bug worth recording as a class: BASE already ends in /leaderboard, so the new feed built a doubled path and 404'd. Because a failing feed degrades to an empty index by design -- correct, so one broken source cannot fail the whole pull -- it surfaced as "fielding_oaa: 0 rows", which reads exactly like "Statcast has no fielding data". Graceful degradation makes a wiring bug look like an honest absence. CUMULATIVE CORRECTION. Bonferroni had been applied per session throughout: a run testing eight features corrected by eight. Across a programme's lifetime that is wrong in the dangerous direction, because every order gets a fresh generous alpha and the false-positive rate compounds quietly. Correcting by 8 when sixty have been tried is how a noise result eventually gets recorded as PROVEN with a p-value to point at. The denominator is now distinct hypotheses ever tested, persisted, and it moved 19 -> 38 within this session alone, alpha 0.0026 -> 0.0013. Re-tests deliberately do not inflate it: re-asking the same question on more data is not a new shot on goal, and counting it would punish the discipline of waiting for sample. THE MEASUREMENT. The differential the theory predicted is present: defence correlates with the counter's residual at +0.130 for GHOST, the contact and speed archetype, and -0.018 for BOMBER, the power archetype. A GHOST's hits depend on whether anyone can range to the ball; a BOMBER's barrels clear the defence entirely. So a flat BOMBER result is the theory working rather than the test failing. It is not a result. GHOST is n=104 against a 500 bar, with p=0.188 against a corrected alpha of 0.0013 -- three orders of magnitude short. Both are recorded as CANDIDATE with their measured lift, tagged contact-skill, so the re-run at full sample compares against a recorded baseline. Nothing proved, so nothing was recalibrated and nothing shipped. 4,228 tests green (336 suites); web build exit 0. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W1sivYNqY2TS5ftykmHBU9
6.6 KiB
DEFENCE INGESTED + CUMULATIVE CORRECTION LOCKED
2026-08-03. Challenger-only. Counter, batter model and pitcher engine byte-identical (verified by diff).
Two durable things shipped, and they stand regardless of sample: the free Statcast fielding feed is ingested and persisted (514 fielders, 31 teams, verified in production), and Bonferroni is now corrected against the programme's lifetime test count, not the session's.
The differential the theory predicted actually appears: defence correlates with the counter's residual for GHOST (contact/speed, +0.130) and is flat for BOMBER (power, −0.018). That is "defence matters, and for whom" showing up in the data — at n=104 and n=245, so it is a signal shape, not a result.
Nothing proved. Nothing recalibrated. Nothing shipped.
1. Defence ingested (STEP 1)
Statcast Outs Above Average, free, same host as the six feeds already pulled.
feed rows 514 fielders
team defence 31 teams
best Cubs oaa_sum +56
worst Mariners oaa_sum −29
prod verified fielding rows 514 · team_defense_written 31
Stored per fielder (oaa, runs_prevented, success_diff, position, team) and
aggregated to team level, which is the unit a batter's prop needs: the
defence behind the pitcher he faces. Summed OAA is the team's outs converted
above average; the mean rides along because a team with more measured fielders
would otherwise look better merely for being measured more. Under three measured
fielders → absent, not thin.
Unknown is not zero, and it bites unusually hard here. An OAA of 0 is a REAL
reading meaning exactly average. Coercing absence to 0 would assert that every
unmeasured fielder is league-average — the most common defensive profile there
is — which is a fabricated fact wearing the costume of a neutral default. Every
read goes through knownRate/knownNumber.
team_defense carries as_of_date in its primary key from the first row.
statcast_aggregates was built upsert-in-place with a single as-of date, which
silently made every backtest leak the games it was predicting and cost a full
session to discover. Point-in-time is available here before it is needed.
A bug worth recording as a class
The first prod run reported fielding_oaa: 0 rows. BASE already ends in
/leaderboard, so the new feed built .../leaderboard/leaderboard/... and 404'd.
Because a failing feed degrades to an empty index by design — correct, so one
broken source cannot fail the whole mechanism pull — it surfaced as zero
fielders, which reads exactly like "Statcast has no fielding data."
Graceful degradation makes a wiring bug look like an honest absence. Any feed reporting 0 should be treated as suspect until the URL is fetched by hand.
2. Defence conditioning per archetype (STEP 2)
Cumulative Bonferroni (see §3): denominator 38, corrected α = 0.0013.
| archetype | n | defence solo r | p | defence × contact (incr) | defence × launch profile (incr) |
|---|---|---|---|---|---|
| GHOST (contact/speed) | 104 | +0.130 | 0.188 | −0.148 | +0.029 |
| BOMBER (power) | 245 | −0.018 | 0.782 | +0.030 | +0.025 |
The differential is the point, and it is present. Defence carries a signal against the counter's residual for the contact/speed archetype and essentially nothing for the power archetype — which is the causal story: a GHOST's hits depend on whether anyone can range to the ball, while a BOMBER's barrels clear the defence entirely. A flat BOMBER result is the theory working, not the test failing.
But neither is a result. GHOST is n=104 against a 500 bar with p=0.188 against a corrected α of 0.0013 — three orders of magnitude short. The direction matches the theory, which is worth carrying forward; it is not worth acting on.
Both recorded in the registry as CANDIDATE with measured lift, tagged
CONTACT-skill, so re-running at n≥500 compares against a recorded baseline.
3. Cumulative multiple-comparisons correction (STEP 4) — locked
src/services/model/testLedger.js + mc_test_ledger.
Bonferroni had been applied per session throughout: a run testing 8 features corrected by 8. Across a programme's lifetime that is wrong in the dangerous direction — every order gets a fresh, generous alpha, so the false-positive rate compounds quietly. Correcting by 8 when sixty have been tried is exactly how a noise result eventually gets recorded as PROVEN, with a p-value to point at.
The denominator is now the count of distinct hypotheses ever tested, persisted. Demonstrated live this session:
GHOST run → cumulative 19 (19 new)
BOMBER run → cumulative 38 (19 new) corrected α: 0.0026 → 0.0013
Re-tests do not inflate it. Re-running the same hypothesis on more data is
the same question asked again, not a new shot on goal — counting it again would
punish the discipline of waiting for sample, which is the behaviour this
programme depends on. times_tested increments; the denominator does not.
The alpha only ever shrinks, which is the correct ordering: an interaction proved late has cleared a genuinely higher bar than one proved on day one, because by then we have had far more chances to get lucky. Seven tests lock this.
4. Ship + registry (STEP 5)
Nothing proved → nothing recalibrated, nothing shipped. The counter continues
to grade everything. validatedSkills() returns {} for every archetype.
5. Programme state
| status | |
|---|---|
| proven set | EMPTY (node scripts/proven-status.js) |
| gate-ready archetype × stat | none — BOMBER×hits 287 is closest, short by 213 |
| defence | ingested, testable, underpowered |
| cumulative correction | locked, α now 0.0013 and falling |
| point-in-time window | statcast_history 1 day; team_defense dated from row one |
6. Next
- Sample is still the only constraint. The cap fix (907 grades/snapshot vs 334) is compounding it; GHOST × hits needs ~396 more rows, BOMBER × hits ~213.
- Re-run
scripts/cluster-prove.jsper archetype at n≥500. The GHOST defence differential is the single most theory-consistent signal the programme has produced — it deserves a fair test, and it will get one. - Note the moving bar: every new hypothesis tightens α for everything that follows. Prefer re-testing the standing candidates over inventing new ones — that is now mathematically, not just methodologically, the disciplined choice.
Not recommended: reading GHOST's +0.130 as evidence, recalibrating anything, or adding new hypotheses while the standing ones are unresolved.