Files
vyndr/specs/factor-wiring-hits.md
builtbykev 43f65d30cb Wire the three proven hits factors pre-grade: transmission proven, gain
inconclusive

THE BUG THIS NEARLY SHIPPED AS A FINDING. The first audit reported 0
factors fired on all 1,140 rows. Not a result -- my paging helper ordered
by `id`, and batter_spray, team_defense, platoon_splits and
statcast_aggregates have composite primary keys with NO id column. The
query errored, the loop broke on error, and four fully-populated tables
read as empty. hitsFactorContext.js -- the PRODUCTION loader -- had the
identical defect, so live wiring would have loaded nothing and served
unadjusted while logging success. Third occurrence of this class in one
session. Both loaders now order by a real column and THROW rather than
degrade. The Phase 2 gate is what caught it: no resolution number was
quoted until transmission was proved.

PHASE 1 — pipeline is now base -> FACTORS -> CALIBRATE -> GRADE. Context
built in snapshotService BEFORE gradeAndCacheSlate (was line 640+, grade
at 454), threaded per prop, applied to p_over before p_win is set with
p_win_prefactor and a full trace retained. Hits only. Coverage 859/1140
rows (75%): 474 with all three factors, 256 two, 129 one, 281 none.

PHASE 2 — TRANSMISSION PROVEN, 12/12 sign-correct, 4/4 per factor, each
applied IN ISOLATION. My first table compared each factor's expected sign
against the COMPOSITE change and showed 3 false failures -- with three
factors firing the net can oppose any single member; that was a flaw in
the test, not the wiring. Two under-side rows confirm the flip is handled:
a factor raising p(over) correctly lowers p_win. Switch hitters (Bailey,
Bell, Rocchio) took no spray adjustment while their other factors fired
normally -- the refusal is selective, not a blanket skip.

PHASE 3/4 — both maps refit on the factor-adjusted forecast; the
shadow-duel baseline is VOID and restarts, since it accumulated against a
different forecast. Point-in-time, 765 held-out rows:

  reliability 0.00795 -> 0.00828
  RESOLUTION  0.00229 -> 0.00345   (variance explained 0.93% -> 1.39%)
  Brier       0.25398 -> 0.25305   delta -0.00093  CI [-0.00225,+0.00002]

Resolution rose 51% relative. The CI TOUCHES ZERO on 4 eval dates, so the
composition does NOT earn a proven keep -- three isolated passes did not
grant a composed pass. INCONCLUSIVE, reported as such. The gain is far
below the sum of the isolated effects, which is expected: all three run
through the same pitcher-batter confrontation and share signal.

PHASE 5 — 1.39% of variance is still far below what band separation
needs. The pivot was correct and incomplete: the plumbing defect was real
and is fixed, three proven factors reach the served number for the first
time, and transmission alone did not buy grade separation. Next arc is
factor STRENGTH and BREADTH, not more plumbing.

PHASE 6 — rbi anomaly logged, not chased: 14.51% variance explained vs
hits 1.03%, on the stat we do not serve corrected and which has no proven
factors. Either the biggest lever on the board or a mirage; it deserves
its own order.

The byte-identical invariant INVERTED for hits by design. All 13 frozen
non-hits modules verified unchanged, probabilityEstimator included -- the
factors ride outside it. No new Bonferroni slot; the composed OOS claim is
reported with its CI and not claimed as a pass.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W1sivYNqY2TS5ftykmHBU9
2026-08-07 02:53:48 -04:00

6.6 KiB
Raw Permalink Blame History

Wiring the three proven hits factors — transmission proven, gain inconclusive

The bug this order nearly shipped as a finding

The first audit run reported 0 factors fired on all 1,140 rows. Not a modelling result — my paging helper ordered by id, and batter_spray, team_defense, platoon_splits and statcast_aggregates have composite primary keys with no id column. The query errored, the loop broke on error, and four fully-populated tables read as empty.

hitsFactorContext.js — the production loader — had the identical defect, so the live wiring would have loaded nothing and served unadjusted forecasts while logging success.

Third occurrence of this class in one session (doubled /leaderboard, the silent settlement outage, this). Both loaders now order by a real column and throw rather than degrade, because a wiring fault must not be able to wear the costume of an honest absence.

The Phase 2 transmission gate is what caught it: no resolution number was quoted until transmission was proved mechanically.


PHASE 1 — the pipeline, in order

base rate  ->  FACTORS (pre-grade)  ->  CALIBRATE  ->  GRADE
  • model/hitsFactors.js — the three proven factors composed, each applying only where it proved, bounded at ±0.25 combined.
  • model/hitsFactorContext.js — loads the inputs once per slate, indexed.
  • snapshotService builds the context before gradeAndCacheSlate; it was previously computed at line 640+, downstream of the grade at 454.
  • gradeSlateService threads it per prop; analyzeViaEngine1 applies it to p_over before p_win is set, recording p_win_prefactor and a full factor_adjustment trace.

Hits only. TB/rbi/runs have no proven factors and are untouched.

Coverage: 859 of 1,140 rows (75%) have at least one factor fire — 474 with all three, 256 with two, 129 with one, 281 with none.


PHASE 2 — transmission, proven mechanically

Each factor applied in isolation to the same base. (My first table compared each factor's expected sign against the composite change and showed 3 false failures — with three factors firing, the net can oppose any single member. That was a flaw in the test, not the wiring.)

factor player side expected mult p before p after (solo) sign
pitcher_contact Travis Bazzana over raise 1.0396 0.604 0.6279
platoon_severity Travis Bazzana over raise 1.0860 0.604 0.6559
pitcher_contact Patrick Bailey over raise 1.0396 0.604 0.6279
pitcher_contact Kyle Manzardo under raise p(over) 1.0396 0.684 0.6715
platoon_severity Kyle Manzardo under raise p(over) 1.0350 0.684 0.6729
defense_by_direction Royce Lewis over lower 0.9690 0.577 0.5591
platoon_severity Royce Lewis over lower 0.9650 0.577 0.5568
pitcher_contact Petey Halpin over raise 1.0396 0.662 0.6882
platoon_severity Chase DeLauter over lower 0.9620 0.838 0.8062
defense_by_direction Gabriel Arias over raise 1.0400 0.536 0.5574
defense_by_direction Austin Hedges over raise 1.0390 0.685 0.7117
defense_by_direction Ryan Kreidler over lower 0.9560 0.523 0.5000

12/12 sign-correct — 4/4 for each of the three factors. The two under rows confirm the flip is handled: a factor raising p(over) correctly lowers p_win.

Unreadables static: Patrick Bailey, Josh Bell and Brayan Rocchio are switch hitters — spray applied to none of them, while their other factors fired normally. The refusal is selective, not a blanket skip.

TRANSMISSION PASSES. Resolution may now be quoted.


PHASE 3/4 — refit and OOS measurement

Both calibration maps refit on the factor-adjusted forecast (the un-factored distribution no longer exists). The shadow-duel baseline is VOID and restarts — it accumulated against a different forecast.

Point-in-time, fit on dates < 2026-08-02, evaluated on 765 held-out rows:

reliability resolution uncertainty variance explained
before (four-input) 0.00795 0.00229 0.2476 0.93%
after (factor-adjusted) 0.00828 0.00345 0.2476 1.39%

Resolution rose 51% relative (+0.00116). Held-out Brier 0.25398 → 0.25305, delta 0.00093, date-block CI [0.00225, +0.00002].

The CI touches zero on 4 eval dates. The composition does NOT earn a proven keep. The point estimate favours the factors and the resolution gain is real in sample, but the honest verdict is INCONCLUSIVE — three isolated passes did not grant a composed pass, exactly as the order anticipated.

Double-counting note: the gain is far below the sum of the isolated factor effects. Expected — defence, pitcher contact and platoon all run through the same pitcher-batter confrontation and share signal.


PHASE 5 — bands and the honest headline

Resolution moved from 0.93% to 1.39% of variance. Both are far below what band separation requires — a forecast explaining 1.4% of an outcome's variance cannot produce archetype bands that clear their own base rate.

The headline, landed as it fell: the pivot was correct and incomplete. The plumbing defect was real and is fixed — three proven factors now reach the served number for the first time, verified sign-by-sign. But transmission alone did not buy grade separation. The factors are real and too weak in combination at current strength.

So the next arc is factor STRENGTH and BREADTH, not more plumbing. The wiring is now a working conduit with three things flowing through it; it needs more, and stronger.


PHASE 6 — the rbi anomaly, logged only

rbi shows 14.51% variance explained vs hits 1.03% — 13×, on the stat we do not serve corrected and which has no proven factors. Open question for the next order: real counter structure (its four inputs happen to discriminate on a stat where opportunity is lumpier), or an artefact of line placement and base-rate spread? Not investigated here. It is either the biggest lever on the board or a mirage, and it deserves its own order.


Invariants

The byte-identical invariant inverted for hits by design — the served hits number should move, and does. TB/rbi/runs paths and all frozen non-hits modules verified unchanged. p_win_prefactor preserves the un-factored forecast in the trace. No new Bonferroni slot (the factors were already proven); the composed OOS claim is reported with its CI and is not claimed as a pass.