inconclusive THE BUG THIS NEARLY SHIPPED AS A FINDING. The first audit reported 0 factors fired on all 1,140 rows. Not a result -- my paging helper ordered by `id`, and batter_spray, team_defense, platoon_splits and statcast_aggregates have composite primary keys with NO id column. The query errored, the loop broke on error, and four fully-populated tables read as empty. hitsFactorContext.js -- the PRODUCTION loader -- had the identical defect, so live wiring would have loaded nothing and served unadjusted while logging success. Third occurrence of this class in one session. Both loaders now order by a real column and THROW rather than degrade. The Phase 2 gate is what caught it: no resolution number was quoted until transmission was proved. PHASE 1 — pipeline is now base -> FACTORS -> CALIBRATE -> GRADE. Context built in snapshotService BEFORE gradeAndCacheSlate (was line 640+, grade at 454), threaded per prop, applied to p_over before p_win is set with p_win_prefactor and a full trace retained. Hits only. Coverage 859/1140 rows (75%): 474 with all three factors, 256 two, 129 one, 281 none. PHASE 2 — TRANSMISSION PROVEN, 12/12 sign-correct, 4/4 per factor, each applied IN ISOLATION. My first table compared each factor's expected sign against the COMPOSITE change and showed 3 false failures -- with three factors firing the net can oppose any single member; that was a flaw in the test, not the wiring. Two under-side rows confirm the flip is handled: a factor raising p(over) correctly lowers p_win. Switch hitters (Bailey, Bell, Rocchio) took no spray adjustment while their other factors fired normally -- the refusal is selective, not a blanket skip. PHASE 3/4 — both maps refit on the factor-adjusted forecast; the shadow-duel baseline is VOID and restarts, since it accumulated against a different forecast. Point-in-time, 765 held-out rows: reliability 0.00795 -> 0.00828 RESOLUTION 0.00229 -> 0.00345 (variance explained 0.93% -> 1.39%) Brier 0.25398 -> 0.25305 delta -0.00093 CI [-0.00225,+0.00002] Resolution rose 51% relative. The CI TOUCHES ZERO on 4 eval dates, so the composition does NOT earn a proven keep -- three isolated passes did not grant a composed pass. INCONCLUSIVE, reported as such. The gain is far below the sum of the isolated effects, which is expected: all three run through the same pitcher-batter confrontation and share signal. PHASE 5 — 1.39% of variance is still far below what band separation needs. The pivot was correct and incomplete: the plumbing defect was real and is fixed, three proven factors reach the served number for the first time, and transmission alone did not buy grade separation. Next arc is factor STRENGTH and BREADTH, not more plumbing. PHASE 6 — rbi anomaly logged, not chased: 14.51% variance explained vs hits 1.03%, on the stat we do not serve corrected and which has no proven factors. Either the biggest lever on the board or a mirage; it deserves its own order. The byte-identical invariant INVERTED for hits by design. All 13 frozen non-hits modules verified unchanged, probabilityEstimator included -- the factors ride outside it. No new Bonferroni slot; the composed OOS claim is reported with its CI and not claimed as a pass. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W1sivYNqY2TS5ftykmHBU9
6.6 KiB
Wiring the three proven hits factors — transmission proven, gain inconclusive
The bug this order nearly shipped as a finding
The first audit run reported 0 factors fired on all 1,140 rows. Not a
modelling result — my paging helper ordered by id, and batter_spray,
team_defense, platoon_splits and statcast_aggregates have composite
primary keys with no id column. The query errored, the loop broke on error,
and four fully-populated tables read as empty.
hitsFactorContext.js — the production loader — had the identical defect, so
the live wiring would have loaded nothing and served unadjusted forecasts while
logging success.
Third occurrence of this class in one session (doubled /leaderboard, the
silent settlement outage, this). Both loaders now order by a real column and
throw rather than degrade, because a wiring fault must not be able to wear
the costume of an honest absence.
The Phase 2 transmission gate is what caught it: no resolution number was quoted until transmission was proved mechanically.
PHASE 1 — the pipeline, in order
base rate -> FACTORS (pre-grade) -> CALIBRATE -> GRADE
model/hitsFactors.js— the three proven factors composed, each applying only where it proved, bounded at ±0.25 combined.model/hitsFactorContext.js— loads the inputs once per slate, indexed.snapshotServicebuilds the context beforegradeAndCacheSlate; it was previously computed at line 640+, downstream of the grade at 454.gradeSlateServicethreads it per prop;analyzeViaEngine1applies it top_overbeforep_winis set, recordingp_win_prefactorand a fullfactor_adjustmenttrace.
Hits only. TB/rbi/runs have no proven factors and are untouched.
Coverage: 859 of 1,140 rows (75%) have at least one factor fire — 474 with all three, 256 with two, 129 with one, 281 with none.
PHASE 2 — transmission, proven mechanically
Each factor applied in isolation to the same base. (My first table compared each factor's expected sign against the composite change and showed 3 false failures — with three factors firing, the net can oppose any single member. That was a flaw in the test, not the wiring.)
| factor | player | side | expected | mult | p before | p after (solo) | sign |
|---|---|---|---|---|---|---|---|
| pitcher_contact | Travis Bazzana | over | raise | 1.0396 | 0.604 | 0.6279 | ✓ |
| platoon_severity | Travis Bazzana | over | raise | 1.0860 | 0.604 | 0.6559 | ✓ |
| pitcher_contact | Patrick Bailey | over | raise | 1.0396 | 0.604 | 0.6279 | ✓ |
| pitcher_contact | Kyle Manzardo | under | raise p(over) | 1.0396 | 0.684 | 0.6715 | ✓ |
| platoon_severity | Kyle Manzardo | under | raise p(over) | 1.0350 | 0.684 | 0.6729 | ✓ |
| defense_by_direction | Royce Lewis | over | lower | 0.9690 | 0.577 | 0.5591 | ✓ |
| platoon_severity | Royce Lewis | over | lower | 0.9650 | 0.577 | 0.5568 | ✓ |
| pitcher_contact | Petey Halpin | over | raise | 1.0396 | 0.662 | 0.6882 | ✓ |
| platoon_severity | Chase DeLauter | over | lower | 0.9620 | 0.838 | 0.8062 | ✓ |
| defense_by_direction | Gabriel Arias | over | raise | 1.0400 | 0.536 | 0.5574 | ✓ |
| defense_by_direction | Austin Hedges | over | raise | 1.0390 | 0.685 | 0.7117 | ✓ |
| defense_by_direction | Ryan Kreidler | over | lower | 0.9560 | 0.523 | 0.5000 | ✓ |
12/12 sign-correct — 4/4 for each of the three factors. The two under rows
confirm the flip is handled: a factor raising p(over) correctly lowers p_win.
Unreadables static: Patrick Bailey, Josh Bell and Brayan Rocchio are switch hitters — spray applied to none of them, while their other factors fired normally. The refusal is selective, not a blanket skip.
TRANSMISSION PASSES. Resolution may now be quoted.
PHASE 3/4 — refit and OOS measurement
Both calibration maps refit on the factor-adjusted forecast (the un-factored distribution no longer exists). The shadow-duel baseline is VOID and restarts — it accumulated against a different forecast.
Point-in-time, fit on dates < 2026-08-02, evaluated on 765 held-out rows:
| reliability | resolution | uncertainty | variance explained | |
|---|---|---|---|---|
| before (four-input) | 0.00795 | 0.00229 | 0.2476 | 0.93% |
| after (factor-adjusted) | 0.00828 | 0.00345 | 0.2476 | 1.39% |
Resolution rose 51% relative (+0.00116). Held-out Brier 0.25398 → 0.25305, delta −0.00093, date-block CI [−0.00225, +0.00002].
The CI touches zero on 4 eval dates. The composition does NOT earn a proven keep. The point estimate favours the factors and the resolution gain is real in sample, but the honest verdict is INCONCLUSIVE — three isolated passes did not grant a composed pass, exactly as the order anticipated.
Double-counting note: the gain is far below the sum of the isolated factor effects. Expected — defence, pitcher contact and platoon all run through the same pitcher-batter confrontation and share signal.
PHASE 5 — bands and the honest headline
Resolution moved from 0.93% to 1.39% of variance. Both are far below what band separation requires — a forecast explaining 1.4% of an outcome's variance cannot produce archetype bands that clear their own base rate.
The headline, landed as it fell: the pivot was correct and incomplete. The plumbing defect was real and is fixed — three proven factors now reach the served number for the first time, verified sign-by-sign. But transmission alone did not buy grade separation. The factors are real and too weak in combination at current strength.
So the next arc is factor STRENGTH and BREADTH, not more plumbing. The wiring is now a working conduit with three things flowing through it; it needs more, and stronger.
PHASE 6 — the rbi anomaly, logged only
rbi shows 14.51% variance explained vs hits 1.03% — 13×, on the stat we do
not serve corrected and which has no proven factors. Open question for the
next order: real counter structure (its four inputs happen to discriminate on a
stat where opportunity is lumpier), or an artefact of line placement and
base-rate spread? Not investigated here. It is either the biggest lever on
the board or a mirage, and it deserves its own order.
Invariants
The byte-identical invariant inverted for hits by design — the served hits
number should move, and does. TB/rbi/runs paths and all frozen non-hits modules
verified unchanged. p_win_prefactor preserves the un-factored forecast in the
trace. No new Bonferroni slot (the factors were already proven); the composed
OOS claim is reported with its CI and is not claimed as a pass.