6452926732741ae75f8f61fe62ae1e063db1ea6c
3 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 builtbykev
|
6452926732 |
Retain raw weather, and record platoon severity 48 rows short
Two fixes in the weather path, and the second was hiding behind the first. The scalar weather_mod cannot express a hit-TYPE conversion at all -- wind out and warm turning fly balls into extra bases, and cold heavy air turning them into outs, collapse to the same number once multiplied -- so the raw temperature, wind speed and wind direction are now retained alongside it. And the old guard only kept the environment when the multiplier was not 1, which silently discarded the forecast for every ordinary night. That is the majority of games, and precisely the rows a hit-type model would need in order to learn what ordinary looks like. Platoon severity is built and measured at n=452, which is 48 rows short of the gate: CANDIDATE_PENDING, neither proven nor theatre. It moves less than flat platoon (0.021 against 0.026), consistent with the pattern, and its Brier point estimate is favourable but the corrected interval still spans zero. Worth naming: the refusal costs sample, and that is the design working. Flat platoon scores 741 rows because it will happily apply a boost to anyone; severity scores 452 because the other 289 are hitters whose split we cannot actually read at 60 plate appearances on the short side. Buying those rows back by shrinking instead of refusing would have produced a number indistinguishable from a measured league-average split, which is a different claim from the one the data supports. Park dimensions are ingested and verified in production across fifteen venues, joined by the venue the game is actually at rather than inferred from the home team -- neutral-site and international games break that assumption without surfacing an error. The park-and-weather-to-hit-type atom is NOT built. Its inputs landed this session and carry a single as_of date, so testing it on total_bases would be scoring games with inputs that postdate them. Building it now would produce something plausible rather than something proven. Proven factors for hits remain pitcher_contact_profile and defense_by_direction. 4,307 tests green (344 suites); web build exit 0. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W1sivYNqY2TS5ftykmHBU9 |
||
|
builtbykev
|
20c45cbcd1 |
The causally-correct defence atom proves where the crude one did not
Kev's insight holds, and the data says so cleanly. defense_by_direction PROVES on hits -- n=528, Brier -0.0034, interval [-0.0059, -0.0009] at the 99.9% level the cumulative correction now demands -- while team-average defence remains not proven, its interval still spanning zero. Same signal, same rows, different unit. The detail worth keeping is that the causally-correct atom moves the number LESS THAN HALF as much as the crude one, 0.013 against 0.030, and is the one that is reliably right. The team average was moving more and knowing less. Big movement is not evidence of a good factor; it is frequently the tell. Both halves turned out to be free, as the order expected. Savant's batted-ball leaderboard carries pull/straight/oppo crossed with ground/air for 609 hitters -- the statcast leaderboard we already pull does not, it has nineteen columns and no direction at all -- and the OAA feed already carries each fielder's position, so per-position defence is a regrouping of last week's ingest rather than a new source. Verified in production: 609 spray profiles, 31 teams. Handedness is what joins them and getting it backwards would have been invisible. Pull for a right-handed hitter is the left side; for a left-handed hitter it is the right side. A model that ignored `bats` would send half the league's grounders to the wrong infielders and still look like it was reading defence, and nothing downstream would have caught it. Switch hitters bat opposite the pitcher, which this does not resolve, so they are unreadable rather than guessed. Unmeasured zones are renormalised away rather than contributing a zero, since a zero asserts an exactly-average fielder standing there, and coverage states honestly what share of a hitter's contact we could actually read. ATOM 2 is input-blocked rather than sample-blocked, and the distinction matters because waiting will not fix it. The weather free-source check passes -- Open-Meteo is already wired and exposes temperature, wind speed, wind direction and precipitation -- but those raw fields are collapsed into a single scalar modifier and wx_forecast is empty on all 1,119 settled rows. Park DIMENSIONS are not ingested at all; parkFactors holds coefficients, not wall heights or fence distances. A park-and-weather-to-hit-type conversion needs both, so it is scoped rather than half-built: retaining the raw weather fields is the cheap half, dimensions are the missing one. Proven factors for hits are now pitcher_contact_profile and defense_by_direction, both pooled; every per-archetype slot remains sample-blocked. 4,297 tests green (342 suites); web build exit 0. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W1sivYNqY2TS5ftykmHBU9 |
||
|
builtbykev
|
a9ee55550b |
Build the two-part factor gate: one factor proves, and zero are theatre
The question was whether the hit grade reads tonight's game or just says he is due. Answering it needed a gate that correlation cannot provide, because correlation cannot separate the two ways a factor looks alive: it reads the game, or it moves the number and reads nothing. The second is what a product ships by accident -- arch-v1 moved 76% of rows by 2.5 points, changed resolution by 0.0000, and was live for months, and no user could have told. So a factor must now clear both conditions: move the prediction off the player's own leave-one-out base rate, AND improve out-of-sample Brier. Brier rather than correlation, because correlation asks whether the ordering improved and this asks whether the NUMBER got closer to what happened -- and for a graded probability the number is the product. The correction applies to the interval itself, which turned out to matter more than expected. A plain 95% CI is the right bar for one test; at fifty cumulative tests roughly two or three intervals exclude zero by chance alone. Widening to 1 - 0.05/tests, currently 99.9%, flipped both defence and platoon out of "proves". A 95% interval would have shipped two unproven factors into the grade, with reasoning text explaining them to users. That forced a distinction I had initially collapsed. Defence and platoon have FAVOURABLE point estimates whose corrected intervals merely span zero, and calling that THEATER would repeat the error this codebase keeps correcting: insufficient evidence is not evidence of absence. THEATER is now reserved for its one real meaning -- moves the number, reads nothing -- and NOT_PROVEN_AT_CORRECTED_BAR names a real candidate held to a bar that rises with every hypothesis the programme tests. Result on 741 settled hits rows: pitcher_contact_profile PROVES, improving Brier by 0.0066 with a 99.9% interval of [-0.0114, -0.0016]. Defence (-0.0043) and platoon (-0.0039) are not proven at the corrected bar. Park is sample-blocked at n=405. Zero factors are theatre, which is the genuinely good news: nothing decorative is being wired. Per-archetype every slot is sample-blocked (BOMBER 252-294, GHOST 67-125). Two spec gaps worth recording. The approach identities the order names -- SPRAY, DAMAGE-DEALER, COUNT-WORKER -- do not exist in the registry; the MLB batter archetypes are BOMBER, GHOST, TORCH, BRUSH, DRIVER, FLEX, ALPHA, HYBRID and CATALYST. And parkFactors maps hits to run_base, so there is no hits-specific park factor at all: a park that turns outs into hits without producing runs is invisible to the input we have. The grade rescale is NOT run. It was explicitly gated on the factor proving, and one pooled factor worth 0.0066 of Brier is not a factor-informed distribution -- rescaling on it would dress a base-rate model as a matchup model, which is the exact thing this gate was built to prevent. 4,286 tests green (340 suites); web build exit 0. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W1sivYNqY2TS5ftykmHBU9 |