Files
vyndr/specs/lineup-k-rate-rung1.md
T
builtbykev 9538e11198 Derive the lineup K-rate free, and fingerprint the cap fix
Two premise corrections first. Pitcher stuff features have NOT proven solo
through the gate -- every one was refused on sample (n=57 against 500). Four
exceed the effect-size bar (arm angle -0.250, whiff +0.213, k rate +0.206,
chase +0.195), which is why they are worth pursuing, but clearing one of three
thresholds is not passing. And the carrier was not blocked only on the lineup
input: that input was built and measured last session at 94.7% coverage. What
blocks it is n, and n was being throttled by the grading cap.

RUNG 1 IS DERIVED AND COSTS NOTHING. Opposing-team K-rate comes from joining
the opposing roster to the batter k_pct values already in statcast_aggregates
-- no new feed. The improvement this session is that it is PA-WEIGHTED: an
unweighted roster mean counts a 12-PA callup the same as an everyday starter,
which is not the lineup a pitcher faces.

That change alone reversed the term's sign. Unweighted, the lineup term HURT
the model (0.1738 -> 0.1285). PA-weighted, it HELPS (0.1738 -> 0.1953). Same
hypothesis, same data -- the derivation was the problem, not the signal, which
is the entire argument for deriving the best honest version before sourcing
anything. Head-to-head is now +0.2592 with a CI of [-0.0167, +0.5645], very
nearly excluding zero, at n=57.

Within archetype, the two strata come out with OPPOSITE signs -- FLAME
incremental -0.152, non-FLAME +0.145 -- and the pooled value (+0.077) sits
between them, which is the shape a conditional effect makes and is invisible
when pooled. That is what stratifying was for. But n is 20 and 24, the
standard error on a correlation there is about 0.22, and the direction
contradicts the theory that predicted a stronger effect for finesse arms. It
is recorded as a structure to re-test, not as a finding.

Rungs 2 and 3 are NOT triggered. A rung fails only once it has been fairly
tested, and Rung 1 is n-blocked rather than failed. Sourcing confirmed lineups
now would be paying for precision on top of a proxy we have not yet measured.

THE RESULT THAT DECIDES THE TIMELINE: yesterday's cap raise is fingerprinted
in production at 907 grades per snapshot, up from 334, with strikeouts going 6
to 17. That puts n>=500 for pitcher Ks about a week out instead of three
months. Operational note: the manual internal snapshot endpoint now 524s at
the Cloudflare edge because grading the full board exceeds 100s -- the run
still completes server-side (this very snapshot was written by a 524'd
request) and the cron is in-process, so a 524 there is not a failure.

Nothing proven, nothing calibrated, nothing shipped. The counter remains
anti-predictive on strikeouts at -0.064 and the skill model leads it by 0.26.

4,221 tests green (335 suites); web build exit 0.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W1sivYNqY2TS5ftykmHBU9
2026-08-03 19:47:01 -04:00

6.8 KiB
Raw Blame History

RUNG 1 — opposing-lineup K-rate, derived from data we already hold

2026-08-03. Challenger-only. Counter, batter cluster and pitcher engine all byte-identical (verified by diff).

VERDICT: Rung 1 is DERIVED and needs ZERO new sourcing — but it does not clear, and neither does anything else, because every pitcher result is still n=57 against a 500 bar. Rung 2 and Rung 3 are NOT triggered: a rung only fails when it has been fairly tested, and Rung 1 has not been.

The real result is that the cap fix landed: 907 grades per snapshot, up from 334, with strikeouts 6 → 17. That is the constraint that decides when any of this becomes answerable.


0. Two premise corrections

"whiff / put-away / stuff prove SOLO through the gate." They do not. Every pitcher feature was refused on insufficient_data (n=57 < 500). Four exceeded the effect-size bar (arm_angle 0.250, whiff +0.213, k_pct +0.206, chase +0.195) — which is why they are worth pursuing — but exceeding one of three thresholds is not passing the gate.

"The carrier was blocked ONLY on the lineup input." The lineup input was built and measured last session (roster join, 94.7% coverage). It was not missing: it was tested and did not add. What blocks it is n, and n was being throttled by the grading cap — which is what this session actually fixed.

1. The derivation (Rung 1) — free, from batter data we already ingest

Opposing-team K-rate is built by joining the opposing team's roster to the batter k_pct values already in statcast_aggregates. No new feed, no new cost. Coverage 94.7% of settled strikeout rows.

Improved this session: PA-WEIGHTED, not a flat roster average. An unweighted mean counts a 12-PA September call-up the same as an everyday starter, which is not the lineup a pitcher faces. Weighting each batter's K-rate by his own sample_pa is the closest honest approximation of who actually bats from data we already hold.

That change alone reversed the term's contribution:

head-to-head on strikeouts resolution delta vs counter
pitcher skill alone 0.1738 +0.2377, CI [0.020, +0.528]
+ PA-weighted lineup K-rate 0.1953 +0.2592, CI [0.0167, +0.5645]
(last session's unweighted version) 0.1285 +0.192

The unweighted proxy hurt (0.174 → 0.129). The PA-weighted one helps (0.174 → 0.195). Same hypothesis, same data — the derivation was the problem, not the signal. That is the whole point of "derive the best honest version before sourcing anything."

The CI is now [0.0167, +0.5645] — very nearly excluding zero, at n=57.

2. The gate (STEP 2)

Bonferroni across 9 tests. Everything is refused on sample size; r is shown because "not enough data yet" and "nothing here" require opposite decisions.

Lineup K-rate, solo: r vs residual = +0.004 (n=54). On its own the team K-rate says nothing — which is expected and is not an argument against it: it is hypothesised as a conditioner, not a standalone predictor.

The carrier, pooled: raw r 0.175, best component 0.198, incremental +0.077. Still explained largely by whiff, but positive now (it was 0.102 unweighted).

Within pitcher archetype — the order's sharper hypothesis

stratum n lineup solo r best component incremental
FLAME (power) 20 +0.117 0.117 0.152
non-FLAME (finesse/contact) 24 0.161 0.183 +0.145

The two strata have opposite signs — the shape a genuinely conditional effect makes, and invisible when pooled (the pooled incremental, +0.077, sits between them). That is exactly why the order asked for stratification.

But it is not evidence yet. At n=20 and n=24 the standard error on a correlation is roughly 0.22, so both estimates are inside noise, and the direction does not match the theory: the order predicted the interaction should matter more for finesse arms, and the magnitudes here are near-identical with flipped signs. Recorded as a structure to re-test, not a finding.

3. Head-to-head, and the number that keeps standing out

n 57 · base rate 0.597
resolution   pitch-v1 (with lineup)  0.1953      counter  0.0639
brier        pitch-v1                0.2412      counter   0.2703
delta        +0.2592   CI95 [0.0167, +0.5645]   → INCONCLUSIVE

The counter's resolution on strikeouts is NEGATIVE. Counting a pitcher's recent strikeouts is anti-predictive of whether he clears his line, because recent K totals track which lineups he drew and how long he was left in rather than his skill. It remains the one stat where the incumbent has no defensible edge, and the skill model leads it by 0.26 with a CI that now barely spans zero.

4. THE CAP FIX LANDED — which is what decides the timeline

Verified in production this session:

before   334 grades/snapshot · strikeouts 6
after    907 grades/snapshot · strikeouts 17 · outs 16 · ER 16 · hits_allowed 16

2.7× across the board, 2.8× on strikeouts. Operational note: the manual POST /api/internal/snapshot/:sport now 524s at the Cloudflare gateway — grading the full board exceeds the 100s edge timeout. The run still completes server-side (this snapshot was written by a 524'd request), and the cron is in-process so it is unaffected. Do not read a 524 there as a failure; check /api/internal/snapshot/status.

5. Rung verdict

rung status
Rung 1 — team-season K-rate DERIVED, zero new sourcing, 94.7% coverage, PA-weighted. Contribution is positive but n=57 — not fairly tested yet
Rung 2 — projected lineup NOT TRIGGERED. Rung 1 has not failed; it has not been tested
Rung 3 — confirmed-lineup sourcing NOT TRIGGERED. Do not source

Ship now? No. Nothing passed the gate, so nothing is calibrated and nothing ships. Source confirmed lineups? No — that would be paying for precision on top of a proxy we have not yet measured properly.

6. Next

  1. Wait for n. At 17 strikeout props/snapshot × 5 snapshots/day, strikeouts should reach n≥500 in roughly a week, not the three months it was before the cap fix.
  2. Re-run scripts/pitcher-prove-k.js at n≥500 with a point-in-time window from statcast_history. Both arrive on a similar timescale.
  3. Re-test the within-archetype strata then — with ~200 per stratum instead of 20, the opposite-sign structure either survives or dissolves.
  4. Give arm_angle a registry entry with a stated mechanism before it is tested for real. It is the largest |r| in the programme and currently has no declared hypothesis, which is how a fishing result gets promoted.

Not recommended: sourcing confirmed lineups, moving to Rung 2, calibrating, or reading the opposite-sign strata as a conditional effect. All four would be acting on n=20.