# Link 2 at the coarse grain — pen QUALITY proves, archetype does not **The refinement was right.** Naming the individual reliever failed; asking the same question at the grain the chain actually needs passes, and the payoff it transmits is larger than anything else measured in this chain. --- ## Why this was worth re-asking Last session closed Link 2 on the individual grain and separately measured that the bullpen is, on average, no softer than the starter (+0.0010 on 35,760 plate appearances). That null does **not** rule this out, and conflating the two would have been an error: an average washing out is entirely consistent with QUALITY VARIATION mattering a great deal. It does. Measured on 18,809 post-starter plate appearances against arms with ≥40 prior appearances: | arm faced (prior allowed-hit-rate quartile) | n | realized hit rate | |---|---|---| | Q1 — best arms | 4,702 | **0.2244** ±0.0119 | | Q2 | 4,702 | 0.2293 | | Q3 | 4,702 | 0.2410 | | Q4 — worst arms | 4,702 | **0.2501** ±0.0124 | Monotone, spread **+2.57pp** — larger than the whole times-through-the-order effect (+1.6pp) and far larger than the pen-vs-starter difference (−0.7pp). --- ## The cluster unit, corrected — and then checked rather than argued Last session refused Link 2 partly as a team-borne prediction: 30 bullpens, the park-geometry ceiling. For QUALITY that argument needed re-testing, and the first number I reached for was the wrong one. - **Treatment variance:** 76% of a game's pen-quality variance is WITHIN team. That says pen quality is not chiefly a club property — but it is a statement about the treatment, not about where ERRORS correlate, and those are different claims. Stopping there would have been picking the convenient answer. - **Measured directly:** ICC of the prediction ERROR by team = **0.0261**. With 16.7 rows per club that is a design effect of 1.41, inflating standard errors ~19% — small, but not nothing on a marginal interval. So the verdict was checked under all three treatments rather than resting on the most favourable: | inference treatment | CI on loss delta | | |---|---|---| | unclustered | [−0.0067, −0.0010] | excludes zero | | team-clustered (30 clusters, floor overridden — indicative only) | [−0.0086, −0.0003] | excludes zero | | design-effect adjusted (deff 1.41) | [−0.0072, −0.0005] | excludes zero | It survives all three. Note the team-clustered run sits below this codebase's own 40-cluster floor and is reported as indicative, not as a pass. --- ## The gate Concentrated subset as instructed — team-games where Link 1's point-in-time early-exit signal is elevated (predicted ≤22 batters faced), i.e. where the pen actually enters for the later plate appearances. ### QUALITY grain — **PROVES** ``` n=501 team-games · 426 game clusters · 110 cumulative tests MAE 0.0294 (league-average baseline) -> 0.0260 delta -0.0034 CI [-0.0063,-0.0005] at 0.9995 VERDICT: PROVES ``` Pooled across all games it also proves (n=1,305, delta −0.0030, CI [−0.0049, −0.0015]), so the result is not an artefact of the subset. ### ARCHETYPE grain — **NOT PROVEN** ``` n=501 · modal-guess baseline 0.5309 -> model 0.5669 corrected interval [-0.1073, +0.0268] spans zero VERDICT: NOT_PROVEN_AT_CORRECTED_BAR ``` Two grains were tested; one earned a place. `penQuality.js` deliberately exposes no archetype, and a test asserts it. --- ## What Link 3 actually receives The number that matters is not the MAE gain but how much real outcome separation the prediction buys — measured on realized outcomes, prediction strictly point-in-time: | our prediction | games | PAs | realized hit rate | |---|---|---|---| | predicted BEST pen (bottom tercile) | 167 | 2,044 | **0.2231** ±0.0180 | | predicted WORST pen (top tercile) | 167 | 1,799 | **0.2501** ±0.0200 | **2.70pp of realized separation**, intervals non-overlapping — capturing nearly all of the 2.57pp available at the quartile grain. corr(predicted, actual pen quality) = 0.393. Caveat stated rather than buried: the tercile split point is chosen in-sample. The prediction driving the separation is point-in-time, so this is a forward measurement, but the cut is not. --- ## Built `src/services/model/penQuality.js` (+ 9 tests) — the proven half, ready for Link 3. `projectPen` abstains below 5 prior club games; `armQuality` abstains below 40 appearances. A league-average stand-in would assert "this is an ordinary bullpen", which is a claim, and usually the wrong one for exactly the clubs whose pens have just turned over. `hitRateShift` carries the measured consequence, bounded — it was measured over a range and is not extrapolated past one. **Link 3 is now unblocked** on a proven Link 2 at the quality grain only. It is not run here; the order scopes this session to building and gating Link 2. ## Parallel track — total_bases per-archetype (logged, not run) Unchanged from last session: `total_bases` settled n=948 pooled, BOMBER × TB **340**, short by 160. Sample-readiness only, not a verdict. The second blocker from `specs/per-archetype-grade-bands.md` still applies — the grade does not yet separate within any archetype. Counter and frozen clusters byte-identical.