vyndr/specs/tb-solo-and-interactions.md

TOTAL BASES — solo baseline, interactions, and the validity blocker

2026-08-03. Challenger-only. Counter byte-identical (zero diff on probabilityEstimator.js / analyzeViaEngine1.js).

HEADLINE 1 — nothing can be gate-validated today, and the reason is my own fix. statcast_aggregates is upserted in place and keeps one as-of date. The first skill backtest was honest only by accident: the nightly refresh was unreachable code, so profiles sat frozen at 2026-07-21 — before the settled window. Repairing that cron (correct for production) refreshed them to today and destroyed every prior version. Point-in-time validation is now structurally impossible from that table, so every number below is contaminated / directional and none is a gate verdict.

HEADLINE 2 — on total bases the skill model is at PARITY with the counter (0.2718 vs 0.2647, CI includes zero). On hits it lost by 0.116 with a CI excluding zero. Same engine, same day. The stat choice was the whole story.

---

0. Premise correction

The order states "barrel rate PASSED, exit velo/launch/matchup FAILED solo." Nothing passed. Yesterday's run refused every TB feature on insufficient_data (n=295 < 500); barrel rate's marginal r was 0.088 and the strongest was hard-hit at 0.153. Today at n=383, hard-hit has fallen to 0.135 — see §3, because that drift is itself the most informative number here.

1. The validity blocker, and the fix

select updated_at::date, count(*) from statcast_aggregates group by 1;
→ 2026-08-03 | 1387        ← ONE row. No history. 2026-07-21 is gone.

A season aggregate dated 2026-08-03 used to score a 2026-07-25 game contains that game. That is leakage, and it is not repairable after the fact.

Fixed forward: statcast_history (migration applied) retains a dated snapshot on every refresh, keyed (as_of_date, sport, season, source_id, role). Point-in-time becomes where as_of_date < game_date order by as_of_date desc limit 1 — the profile as it stood before the game, by construction. Retention is best-effort and cannot fail the refresh; both properties are unit-tested. It has one day of data, which is not yet a window.

2. Feature → conditioning variable → mechanism (STEP 2)

interaction	conditioning on	mechanism
launch × exit_velo	each other	Extra bases need both: hit hard AND in the air. A 105-mph ground ball is an out; a 25° popup is an out. Neither predicts alone — which is exactly why each can fail solo and the product might not.
exit_velo × pitcher_hard_hit_allowed	pitcher suppression	A hitter only realises contact quality against a pitcher who permits it. Elite suppression attenuates a power bat; a contact-permitting arm amplifies it. Conditional by construction.
barrel × power archetype	real archetype label	Barrels convert to extra bases for hitters whose lane is power. This is Discipline 2 as a testable claim.
batter_K × pitcher_K	each other	Strikeout risk compounds (log5 is this shape). A PA lost to a strikeout is a base opportunity that never happens — suppression through opportunity, not contact.
pull_rate × park	park	UNTESTABLE — park is passed as 1.0 everywhere and pull rate is not ingested.

3. STEP 1 — solo baseline (the control)

n=383, Bonferroni across 12 tests → corrected α = 0.00417. The decider is correlation with the counter's residual (won − p_win) — a feature correlating with the raw outcome may simply know what the counter already knows.

feature	r vs outcome	r vs residual	p	would it pass at n≥500?
batter_hard_hit_pct	0.172	0.135	0.0080	No —
batter_exit_velo	0.120	0.070	0.173	No
batter_barrel_pct	0.109	0.078	0.127	No
pitcher_k_pct	−0.094	−0.087	0.093	No
batter_launch_angle	0.032	0.003	0.948	No
batter_bb_pct	−0.019	−0.026	0.607	No
pitcher_hard_hit_allowed	0.024	0.035	0.501	No
batter_k_pct	−0.009	−0.003	0.952	No

No feature passes solo, and hard-hit is drifting the wrong way. Its marginal r went 0.153 (n=295) → 0.135 (n=383) as sample grew. That is the signature of an estimate regressing toward its true value as noise averages out, not of a real effect firming up. Yesterday I called it "the most encouraging number this work has produced"; on 88 more rows it is fading. Worth stating plainly, because the temptation is to keep quoting the older, better number.

4. STEP 3 — interactions, each against its own parts

Incremental signal = partial correlation of the interaction with the residual, controlling for both components. If an interaction is barrel rate in a hat, regressing out barrel rate collapses it.

interaction	n	raw r	best component r	incremental r	verdict
barrel × power archetype	260	−0.046	0.019	−0.101	UNDERPOWERED — n below gate
launch × exit_velo	383	0.007	0.070	−0.053	UNDERPOWERED
batter_K × pitcher_K	378	−0.050	0.087	−0.024	UNDERPOWERED
exit_velo × pitcher_suppression	378	0.048	0.073	0.013	UNDERPOWERED

Only the archetype interaction has an incremental exceeding its parts (0.101 vs 0.019) — the shape Discipline 2 predicts — but at n=260, with a negative sign, and nowhere near the gate. It is a lead, not a finding.

A methodological catch worth recording

The archetype conditioning variable was first built as barrel_pct / LEAGUE.barrel_pct — a monotone transform of one of its own components. That interaction was therefore barrel², measuring nonlinearity in barrel rate, not an archetype effect at all, and it produced the run's only "interaction adds" result (−0.132). Two fixes: a scale-free collinearity check on the control columns (the Gauss-Jordan pivot test missed it, because the columns differ by a scale factor), and real archetype labels joined from model_snapshots (260 labelled rows: 140 BOMBER, 120 other) so the conditioning variable is categorical and independent of barrel by construction.

Without that catch this document would have reported a fabricated interaction as the session's finding.

5. STEP 4 — combined skill model vs the counter

The compound TB projection was built for this (see §6). Paired bootstrap is valid at this n even though the gate is not.

n 383 · pitcher coverage 100% · base rate 0.483
resolution   skill TB 0.2718     counter 0.2647
brier        skill TB 0.2439     counter 0.2426
delta        +0.0071   CI95 [−0.0648, +0.0789]   → INCONCLUSIVE

Not a win — but the first time a challenger has not LOST. For contrast, the same engine on hits: −0.1161, CI [−0.1892, −0.0426], excluding zero. The difference between those two lines is the entire argument for moving to total bases, and it is what the physics said: contact quality governs extra bases, not whether a grounder finds a hole.

6. Built this session: the compound TB projection

skillProjection no longer refuses total bases. A deterministic bases-per-hit multiplier made P(TB≥2) exactly P(hits≥1) — a relabelled hits curve — so it was refused. It is now a real convolution: each PA yields 0/1/2/3/4 bases with hit-type shares shifted by skill (barrel → home-run share, exit velo → doubles and triples), convolved over a PA distribution. A BOMBER gets a 37.5% home-run share against a slap hitter's 4.5%. Non-degeneracy is locked by test.

7. Verdict

• Solo-proven set for total_bases: EMPTY. Nothing passes; hard-hit is the closest and is drifting down.
• Interaction-proven set: EMPTY. One lead (archetype × barrel), underpowered.
• Combined vs counter: INCONCLUSIVE at parity — genuinely better news than hits, and still not promotable.
• Nothing promoted. Counter untouched.

8. Next, in order

1. Let statcast_history accrue. Nothing here can be validated honestly until there is a point-in-time window — a week gives a real one. This is now the binding constraint on the entire programme, and it is a waiting problem, not a building problem.
2. Re-run this exact script once history exists and TB clears n≥500 (383 today, ~117 short). Both arrive on roughly the same timescale.
3. Archetype × barrel is the one interaction worth carrying forward, with real labels and adequate n.
4. Do not re-run hits. §5 settles it.

Not recommended: quoting the 0.153, lowering n≥500, or promoting on a parity result. Parity with contamination is not a win.