# Were we out of data, or not using what we had? **Both — and which one it is depends entirely on what unit a factor varies over.** That distinction turned out to matter more than the sample counts themselves. --- ## 1. Player-level factors: we were under-querying The platoon test reported n=452 and "48 short of the gate." That number described how much of the JOIN survived, not how much data exists. | | used | actually available | |---|---|---| | clean settled `hits` rows | 452 | **1,266** | | clean settled `total_bases` rows | 383 | **928** | | quarantined `hits` rows | — | **0** | | hitters with platoon splits | 298 | 380 needed | `platoon_splits` had been ingested from *tonight's lineups only* — 315 players — so any hitter who settled a prop but was not in a lineup on an ingest day was silently absent from every test. Backfilling all 380 (`scripts/backfill-context.js`) took one pass and no waiting: **81 hitters fetched, 0 unresolved, coverage now 380/380.** Re-run on the full clean history (`scripts/prove-hit-factors.js`, rows 452 → **1,059**): | factor | n | mean shift | Brier Δ | CI (corrected, 55 tests) | verdict | |---|---|---|---|---|---| | `defense_by_direction` | 782 | 0.0127 | −0.0031 | [−0.0050, −0.0013] | **PROVES** | | `platoon` | 1,056 | 0.0268 | −0.0033 | [−0.0062, −0.0001] | PROVES\* | | `platoon_severity` | 700 | 0.0218 | −0.0038 | [−0.0076, −0.0001] | PROVES\* | | `defense` | 912 | 0.0289 | −0.0038 | [−0.0079, +0.0002] | NOT_PROVEN | | `pitcher_contact_profile` | 1,059 | 0.0259 | −0.0034 | [−0.0078, +0.0006] | **NOT_PROVEN — demoted** | | `park_hits` | 619 | 0.0190 | −0.0037 | [−0.0076, +0.0005] | NOT_PROVEN | ### 1a. The demotion is the real headline `pitcher_contact_profile` was the strongest proven factor in the programme (Brier −0.0064, CI [−0.0113, −0.0014]). On more than double the sample its point estimate **roughly halved to −0.0034** and the corrected interval now spans zero. Two things moved at once and honesty requires naming both: the cumulative Bonferroni denominator also rose to 55, which widens every interval. But the denominator cannot touch a *point estimate*, and that halved on its own. This is the standing second line doing exactly what it exists for — more data demoting a favourite rather than confirming it. ### 1b. \*The two platoon passes are NOT promoted Both clear the bar with an upper bound of **−0.0001**. That is as marginal as a pass can be, and they ride a reconstructed input: `platoon_splits` are **season-to-date**, so applying today's split to a game from 2026-07-15 means the split contains that game. Measured, not assumed: - median contamination **4.5%** of the split's plate appearances - p90 **12.4%** - worst **137%** (call-ups whose scored games outnumber their split sample) I had originally estimated ~1%. It is four and a half times that, and it runs in the flattering direction on a result whose margin is one ten-thousandth. These stay **CANDIDATE — pending point-in-time splits**. Promoting a 4.5%-contaminated input on a −0.0001 bound would be exactly the kind of pass this programme keeps having to retract. --- ## 2. Game-level factors: genuinely short, and no backfill fixes it `game_context` held **zero** weather readings, ever. The fetcher was correct and already pointed at Open-Meteo's **archive** endpoint. The failure was that the two tables had never been introduced: ``` ledger_entries.game_id = mlb:2026-08-03:WashingtonNationals@PhiladelphiaPhillies game_context.game_id = mlb:823437 ``` Every lookup missed, and NULL weather columns read exactly like "the weather was unavailable." **Same class as the doubled `/leaderboard` path: graceful degradation wearing the mask of honest absence.** That is now three occurrences; it is the failure mode this codebase produces most reliably. Fixed in `scripts/reconstruct-game-environment.js` — resolves each ledger slug to its real statsapi game and venue, writes `game_context` keyed by the *ledger's* key, then pulls actual archived weather: - 101 settled games → **96 matched**, 30 venues - **96/96 venue-days returned real archived weather** (`open_meteo_archive`) - park dimensions backfilled 15 → **30 venues**, zero dimension changes observed ### 2a. Park dimensions: what I verified and what I did not statsapi serves only **current** venue geometry — it has no historical record. My capture window is 2026-08-04 to 08-05, so "no mid-season change" is verified across *two days*, which is nearly no verification at all. Applying current dimensions to July games is the order's stated allowance and it is almost certainly fine, but I did not verify it and will not claim to have. ### 2b. Why 928 rows are 47 readings Park and weather assign **one value per game**. The 928 clean settled `total_bases` rows sit on **47 distinct games — median 17.6 rows per game.** Eighteen hitters in one ballpark on one night are one reading of that ballpark, not eighteen. Resampling rows would treat them as independent and return an interval far tighter than the evidence supports. `factorGate.improvement` now resamples **clusters** when rows carry one, and `adjudicate` judges sample against `effective_n`. Rows without a cluster keep the original path byte-for-byte. `scripts/prove-park-weather.js`: ``` rows_built 828 | distinct_games 47 | rows_per_game 17.6 brier_delta +0.0011 (WORSE, not merely unproven) effective_n 47 VERDICT: CANDIDATE_PENDING_SAMPLE — 47 independent clusters < 500 ``` --- ## 3. The verdict: tested-now vs real-wait | factor | unit it varies over | units held | ceiling | real wait | |---|---|---|---|---| | platoon, defence-by-direction | **hitter-game** | 1,059 | none | **none — answered now** | | weather | **game** | 47 | none | **~57 days** at 7 games/settled-day | | park dimensions | **venue** | 30 | **30, permanently** | **never** | The last row is arithmetic, not pessimism. **There are 30 ballparks in MLB.** A factor constant per venue can never accumulate 500 independent units no matter how long the ledger runs. A park-geometry effect is only ever validatable as a fixed effect with many games per park under a hierarchical model — never under a bar expressed in independent units. The n≥500 bar was designed for player-level factors and quietly does not transfer. **So: we were under-querying at the player level, and genuinely short at the game level — and for park geometry specifically, "wait for more data" was never going to be the answer.** --- ## 4. Wind is refused `parkWeather` reads temperature, elevation and geometry. It does **not** read wind, and says so on every read (`wind_readable: false`). We have the wind — Open-Meteo returns speed and bearing for all 96 games. What we lack is **park orientation**: which compass direction each stadium's centre field faces. A 15 mph wind from 220° is blowing out to right at one park and straight in at another, and those are opposite predictions. The tempting move is to use wind *speed* alone as a magnitude of disruption. That asserts an effect while discarding the sign that determines what the effect is. Wind stays unreadable until orientation is a real column.