vyndr/specs/link2-pen-quality.md

Link 2 at the coarse grain — pen QUALITY proves, archetype does not

The refinement was right. Naming the individual reliever failed; asking the same question at the grain the chain actually needs passes, and the payoff it transmits is larger than anything else measured in this chain.

---

Why this was worth re-asking

Last session closed Link 2 on the individual grain and separately measured that the bullpen is, on average, no softer than the starter (+0.0010 on 35,760 plate appearances). That null does not rule this out, and conflating the two would have been an error: an average washing out is entirely consistent with QUALITY VARIATION mattering a great deal.

It does. Measured on 18,809 post-starter plate appearances against arms with ≥40 prior appearances:

arm faced (prior allowed-hit-rate quartile)	n	realized hit rate
Q1 — best arms	4,702	0.2244 ±0.0119
Q2	4,702	0.2293
Q3	4,702	0.2410
Q4 — worst arms	4,702	0.2501 ±0.0124

Monotone, spread +2.57pp — larger than the whole times-through-the-order effect (+1.6pp) and far larger than the pen-vs-starter difference (−0.7pp).

---

The cluster unit, corrected — and then checked rather than argued

Last session refused Link 2 partly as a team-borne prediction: 30 bullpens, the park-geometry ceiling. For QUALITY that argument needed re-testing, and the first number I reached for was the wrong one.

• Treatment variance: 76% of a game's pen-quality variance is WITHIN team. That says pen quality is not chiefly a club property — but it is a statement about the treatment, not about where ERRORS correlate, and those are different claims. Stopping there would have been picking the convenient answer.
• Measured directly: ICC of the prediction ERROR by team = 0.0261. With 16.7 rows per club that is a design effect of 1.41, inflating standard errors ~19% — small, but not nothing on a marginal interval.

So the verdict was checked under all three treatments rather than resting on the most favourable:

inference treatment	CI on loss delta
unclustered	[−0.0067, −0.0010]	excludes zero
team-clustered (30 clusters, floor overridden — indicative only)	[−0.0086, −0.0003]	excludes zero
design-effect adjusted (deff 1.41)	[−0.0072, −0.0005]	excludes zero

It survives all three. Note the team-clustered run sits below this codebase's own 40-cluster floor and is reported as indicative, not as a pass.

---

The gate

Concentrated subset as instructed — team-games where Link 1's point-in-time early-exit signal is elevated (predicted ≤22 batters faced), i.e. where the pen actually enters for the later plate appearances.

QUALITY grain — PROVES

n=501 team-games · 426 game clusters · 110 cumulative tests
MAE 0.0294 (league-average baseline) -> 0.0260   delta -0.0034
CI [-0.0063,-0.0005] at 0.9995                   VERDICT: PROVES

Pooled across all games it also proves (n=1,305, delta −0.0030, CI [−0.0049, −0.0015]), so the result is not an artefact of the subset.

ARCHETYPE grain — NOT PROVEN

n=501 · modal-guess baseline 0.5309 -> model 0.5669
corrected interval [-0.1073, +0.0268] spans zero
VERDICT: NOT_PROVEN_AT_CORRECTED_BAR

Two grains were tested; one earned a place. penQuality.js deliberately exposes no archetype, and a test asserts it.

---

What Link 3 actually receives

The number that matters is not the MAE gain but how much real outcome separation the prediction buys — measured on realized outcomes, prediction strictly point-in-time:

our prediction	games	PAs	realized hit rate
predicted BEST pen (bottom tercile)	167	2,044	0.2231 ±0.0180
predicted WORST pen (top tercile)	167	1,799	0.2501 ±0.0200

2.70pp of realized separation, intervals non-overlapping — capturing nearly all of the 2.57pp available at the quartile grain. corr(predicted, actual pen quality) = 0.393.

Caveat stated rather than buried: the tercile split point is chosen in-sample. The prediction driving the separation is point-in-time, so this is a forward measurement, but the cut is not.

---

Built

src/services/model/penQuality.js (+ 9 tests) — the proven half, ready for Link 3. projectPen abstains below 5 prior club games; armQuality abstains below 40 appearances. A league-average stand-in would assert "this is an ordinary bullpen", which is a claim, and usually the wrong one for exactly the clubs whose pens have just turned over.

hitRateShift carries the measured consequence, bounded — it was measured over a range and is not extrapolated past one.

Link 3 is now unblocked on a proven Link 2 at the quality grain only. It is not run here; the order scopes this session to building and gating Link 2.

Parallel track — total_bases per-archetype (logged, not run)

Unchanged from last session: total_bases settled n=948 pooled, BOMBER × TB 340, short by 160. Sample-readiness only, not a verdict. The second blocker from specs/per-archetype-grade-bands.md still applies — the grade does not yet separate within any archetype.

Counter and frozen clusters byte-identical.