Files
vyndr/specs/reliever-chain.md
builtbykev e4dae0e6b0 Reliever chain: Link 1 proves, Link 2 does not, and the premise inverts
The causal insight is right -- the game is a sequence and the matchup does
shift mid-game. The direction is backwards, measured on 93,663 plate
appearances from 1,238 games pulled free from statsapi.

LINK 1 PROVES. Starter batters-faced, point-in-time from his own prior
starts only, clustered on the pitcher: MAE 3.2226 -> 2.7990, delta
-0.4236, CI [-0.6006,-0.2731] at 0.9995 corrected for 107 tests, 1,706
starts across 204 pitchers. It finds the tail the chain needed -- early
exits are a 23.2% base rate, model-flagged starts are 34.0% early, lift
+10.8pp.

Scope correction inside Link 1: the order specifies fatigue x GAME
SCRIPT, but game script is not available at grade time -- whether he gets
hit tonight is the thing being projected, not an input to it. Only the
workload half is measured; the in-game half is recorded as a live feature,
out of scope, rather than quietly folded in.

LINK 2 DOES NOT PROVE, twice over. Model accuracy 17.2% vs an 8.6%
baseline -- doubling it sounds good and is not, since naming a specific
arm is wrong five times in six. And structurally the entity is the
BULLPEN: 39,629 post-starter plate appearances across 30 clubs is 30
readings, below the 40-cluster floor, the same permanent ceiling as park
geometry and team defence. LINK 3 NOT RUN, per the order's own rule.

THE PREMISE IS REFUTED, and this chains on nothing so it was safe to
measure:

  vs STARTER  n=48,492  hit rate 0.2444 +/-0.0038
  vs BULLPEN  n=35,760  hit rate 0.2373 +/-0.0044

The pen is 0.7pp HARDER. The specific effect the chain exists to exploit
-- early exit making later at-bats softer -- is +0.0010 on 35,760 PAs. A
well-powered null, not a sample problem.

What IS real is times through the order: TTO1 0.2351 -> TTO2 0.2515 ->
TTO3 0.2518. A starter does decay as the lineup sees him again, but that
advantage is SURRENDERED when he leaves, not extended -- the pen is
harder than his second and third time through. A modern bullpen is a
queue of fresh specialists throwing one inning each; there is no tiring
arm to punish.

So the insight survives inverted, and Link 1 stays valuable for the
opposite reason it was built: a likely early hook predicts the hitter
LOSES his third-time-through look (0.2518 -> 0.2373 on that PA). The
mispricing is on hitters who get an EXTRA look at a starter going deep.

BUILT: predictionGate.js + tests -- the two-part gate for a continuous
prediction. factorGate binarises outcomes for Brier, which would destroy
a target like batters faced. Same discipline, same THEATER verdict, real
scale.

PRE-REGISTERED NOT RUN: Link 2' using a PA-weighted bullpen AGGREGATE
rather than a named arm. Recorded rather than substituted in -- running
Link 3 on a swapped-in Link 2 is the assumed-link failure the order
forbids. Given the premise result its expected value is now low.

PARALLEL TRACK logged: total_bases n=948 pooled, BOMBER x TB 340, short
by 160. Sample-readiness only, not a verdict.

Counter and frozen clusters byte-identical.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W1sivYNqY2TS5ftykmHBU9
2026-08-06 01:58:38 -04:00

6.2 KiB
Raw Permalink Blame History

The reliever chain — Link 1 proves, Link 2 does not, and the premise inverts

The causal insight is right: the game is a sequence and the matchup does shift mid-game. The direction is backwards. Measured on 93,663 plate appearances from 1,238 games, the bullpen is harder than the starter, not softer.


Target is batters faced, because that is what decides how many of a hitter's plate appearances come against the starter rather than the pen. Point-in-time: each start predicted only from that pitcher's starts strictly before it, shrunk toward the league mean by prior-start count. Baseline is the league mean — the naive "a starter goes about six."

1,706 starts · 204 pitchers · clustered on the pitcher · 107 cumulative tests
MAE  3.2226 (baseline)  ->  2.7990 (model)   delta -0.4236
CI [-0.6006, -0.2731] at 0.9995        VERDICT: PROVES

It finds the tail, which is what the chain needed: early exits (≤20 batters faced) occur at a 23.2% base rate, and among model-flagged starts they occur at 34.0% — lift +10.8pp.

Scope correction made here: the order specifies fatigue profile × game script ("getting hit → pulled early"). Game script is not available when a prop is graded — whether he gets hit tonight is the thing being projected, not an input to it. Using it would be reading the answer. Only the fatigue/workload half is measured above; the in-game half is a LIVE feature, recorded as out of scope rather than quietly folded in.

Predict which arm throws a given post-starter plate appearance. Baseline: the team's most-used reliever to date. Model: the arm that team has most often used in that inning to date — the cheapest expression of bullpen role.

39,629 post-starter plate appearances · 30 bullpens
baseline accuracy 8.6%  ->  model accuracy 17.2%
VERDICT: PENDING_SAMPLE — 30 independent clusters < 40
  1. On merit. Doubling the baseline sounds good and is not: naming a specific arm is wrong five times out of six.
  2. Structurally. Bullpen usage is a team-level process — same manager, same arms, same roles all season — so the entity this prediction rides on is the club, and there are 30. Row count cannot create replication that does not exist. The same permanent ceiling as park geometry (30 venues) and team defence (26 teams).

Per the order's own discipline, a link that does not prove does not feed the next. Link 3 needs the reliever's profile, and Link 2 cannot say whose profile it is.


The premise, tested directly — because that chains on nothing

This required no unproven link, so it was safe to measure, and it is the finding that matters most:

n hit rate per PA ±95%
vs STARTER 48,492 0.2444 0.0038
vs BULLPEN 35,760 0.2373 0.0044
bullpen | starter exited early 14,672 0.2379 0.0069
bullpen | starter went normal 21,088 0.2369 0.0057

The bullpen is 0.7pp HARDER than the starter, and the intervals barely overlap. The specific effect the chain was built to exploit — an early exit making later at-bats softer — is +0.0010, indistinguishable from zero on 35,760 plate appearances. That is a well-powered null, not a sample problem.

What IS real: times through the order

n hit rate
TTO 1 21,596 0.2351
TTO 2 18,426 0.2515
TTO 3 8,278 0.2518

A starter does decay as the lineup sees him again: +1.6pp from first look to second. But that advantage is surrendered when he leaves, not extended — the pen (0.2373) is harder than the starter's second and third time through (0.2515).

The mechanism is a modern bullpen: a queue of specialists throwing max effort for one inning each, fresh, often handedness-matched. There is no tiring arm to punish.

The insight survives, inverted

The sequence framing is correct and the edge is real — it just points the other way. A hitter's soft spot is a starter still in the game on the third time through, and an early hook takes it away. So Link 1 remains valuable, for the opposite reason it was built: flagging a likely early exit predicts that a hitter loses his third-time-through look (0.2518 → 0.2373, a 1.45pp shift on that plate appearance) — a downgrade signal, not an upgrade.

That is also market-relevant in the way the order wanted, with the sign flipped: if a line is set on the starter's matchup, the mispricing is on hitters who will get an extra look at a starter going deep.


Built

  • src/services/model/predictionGate.js (+ tests) — the two-part gate for a CONTINUOUS prediction. factorGate binarises outcomes for Brier, which would destroy a target like batters faced. Same discipline: movement AND out-of-sample improvement, paired bootstrap, clustered, cumulative-corrected. Names THEATER the same way.
  • scripts/ingest-game-sequences.js — per-PA batter/pitcher/hand/inning/result plus boxscore exit lines, free from statsapi. 1,238 games cached.
  • scripts/link1-pull-timing.js, scripts/link2-reliever-identity.js.

Pre-registered, NOT run

Link 2 — bullpen AGGREGATE instead of a named arm. Naming the arm fails, but a PA-weighted aggregate of the pen's contact-allowed and handedness profile may be knowable, and the hitter×bullpen unit would have real replication where the bullpen alone has 30. This is recorded rather than substituted in, because running Link 3 on a swapped-in Link 2 is precisely the assumed-link failure the order forbids. Given the premise result above, its expected value is now low.

Parallel track — total_bases per-archetype (logged, not run)

Sample audit only: total_bases settled n=948 pooled; BOMBER × TB 340, short by 160 against the gate. No archetype slot is testable yet. Per the S88 lesson, this is a sample-readiness note and not a verdict — and per specs/per-archetype-grade-bands.md, the grade does not yet separate within any archetype on hits, so a TB rescale would face the same second blocker.

Counter and frozen clusters byte-identical.