diff --git a/BUILD-STATE.md b/BUILD-STATE.md index 9661ba0..05fc70b 100755 --- a/BUILD-STATE.md +++ b/BUILD-STATE.md @@ -3,6 +3,27 @@ ## Last Updated 2026-08-03 +## Session 85 (2026-08-03) — Rung 1 derived free; the cap fix fingerprinted ✅ +Spec: `specs/lineup-k-rate-rung1.md`. 4,221 tests / 335 suites green, build exit +0. Counter + batter cluster + pitcher engine byte-identical. +- **CAP FIX VERIFIED IN PROD: 334 -> 907 grades/snapshot; strikeouts 6 -> 17.** + n>=500 for Ks is ~a week out instead of ~3 months. NOTE: the manual internal + snapshot endpoint now 524s at Cloudflare (>100s) but COMPLETES server-side. +- **RUNG 1 DERIVED, zero new sourcing:** opposing-team K-rate from the roster + joined to batter k_pct we already ingest, 94.7% coverage — now PA-WEIGHTED. + That change flipped its contribution: unweighted HURT (0.174->0.129), + PA-weighted HELPS (0.174->0.195). Head-to-head delta +0.259, CI + [−0.0167,+0.5645] — nearly excluding zero, still INCONCLUSIVE at n=57. +- **Within-archetype:** FLAME incremental −0.152, non-FLAME +0.145 — opposite + signs, invisible when pooled (+0.077). But n=20/24 and the direction + contradicts the theory. Structure to re-test, not a finding. +- **Rungs 2/3 NOT triggered** — Rung 1 is n-blocked, not failed. Do not source + confirmed lineups. +- **Nothing proven, nothing calibrated, nothing shipped.** Counter is still + anti-predictive on Ks (−0.064); skill model leads by 0.26. +- **Next:** wait ~1 week for n>=500 + a statcast_history window, re-run, re-test + the strata at ~200/stratum, and give arm_angle a registry entry + mechanism. + ## Session 84 (2026-08-03) — Pitcher engine built; the cap was eating the board ✅ Spec: `specs/pitcher-engine-strikeouts.md`. 4,221 tests / 335 suites green, build exit 0. Batter model + counter byte-identical (verified by diff). diff --git a/CLAUDE.md b/CLAUDE.md index eab7936..91514b7 100755 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -1437,6 +1437,35 @@ phased plan in the Session-57 conversation / BUILD-STATE Next section). is explained by whiff alone), and adding the lineup term LOWERED head-to-head resolution (0.174 -> 0.1285). n=54, so not a verdict — but recorded, not dropped. +## Lineup K-rate (Rung 1) + the cap fingerprint (Session 85 — non-obvious) +- **THE CAP FIX LANDED: 334 -> 907 grades/snapshot, strikeouts 6 -> 17** (2.7x + across the board). n>=500 for pitcher Ks is now ~a week away, not 3 months. +- **OPERATIONAL: `POST /api/internal/snapshot/:sport` now 524s at Cloudflare** — + grading the full board exceeds the 100s edge timeout. **The run still COMPLETES + server-side** (the 907-grade snapshot was written by a 524'd request), and the + cron is in-process so it is unaffected. Never read that 524 as a failure; check + `/api/internal/snapshot/status`. +- **PA-WEIGHT the team K-rate.** Opposing-lineup K-rate is derived free by joining + the opposing roster to batter `k_pct` we already ingest (94.7% coverage, zero + new sourcing). An UNWEIGHTED roster mean counts a 12-PA callup like an everyday + starter and it HURT the model (0.174 -> 0.129); PA-weighted it HELPS + (0.174 -> 0.195). Same hypothesis, same data — the derivation was the problem. + Always weight a team aggregate by playing time. +- **A conditioner can have ~zero solo signal and still matter.** Lineup K-rate + solo r = +0.004. That is not evidence against it — it is hypothesised as a + CONDITIONER, not a standalone predictor. Judge it by its incremental partial, + stratified. +- **Within-archetype strata have OPPOSITE signs** (FLAME incremental −0.152, + non-FLAME +0.145) and the pooled value (+0.077) sits between them — the shape a + conditional effect makes, and invisible when pooled. But n=20/24 (SE≈0.22) and + the DIRECTION contradicts the theory (predicted stronger for finesse; magnitudes + are near-equal with flipped signs). Structure to re-test, NOT a finding. +- **Rungs 2 and 3 are NOT triggered.** A rung only fails once fairly tested, and + Rung 1 is n-blocked, not failed. Do not source confirmed lineups yet. +- **Pitcher features still have NOT passed the gate** — all refused at n=57. + Four exceed the |r|>=0.15 effect bar (arm_angle −0.250, whiff +0.213, k_pct + +0.206, chase +0.195) but exceeding one of three thresholds is not passing. + ## Active Skills - vyndr-voice (all user-facing output) - prop-analysis (grading methodology) diff --git a/scripts/pitcher-prove-k.js b/scripts/pitcher-prove-k.js index 87cdee5..a7f7231 100644 --- a/scripts/pitcher-prove-k.js +++ b/scripts/pitcher-prove-k.js @@ -161,13 +161,23 @@ async function main() { if (!abbr) { teamKRate.set(teamName, null); return null; } const team = await mlb.resolveTeam(abbr); const roster = team && team.id ? await mlb.getTeamRoster(team.id) : null; - const rates = []; + // PA-WEIGHTED, not a flat roster average. An unweighted mean counts a + // 12-PA September call-up the same as an everyday starter, which is not + // the lineup a pitcher faces. Weighting by each batter's own sample_pa is + // the closest honest approximation of "who actually bats" from data we + // already hold — and it needs no new sourcing at all. + let wSum = 0; let wK = 0; let counted = 0; for (const p of roster || []) { const prof = batterByKey.get(nameKey(p.name || p.fullName || '')); - const k = prof ? knownRate(prof.k_pct) : null; - if (k !== null) rates.push(k); + if (!prof) continue; + const k = knownRate(prof.k_pct); + const pa = knownRate(prof.sample_pa); + if (k === null) continue; + const w = pa === null ? 0 : pa; // no PA read -> contributes nothing + if (w <= 0) continue; + wSum += w; wK += w * k; counted += 1; } - if (rates.length >= 5) val = rates.reduce((a, b) => a + b, 0) / rates.length; + if (counted >= 5 && wSum > 0) val = wK / wSum; } catch { val = null; } teamKRate.set(teamName, val); return val; @@ -201,9 +211,16 @@ async function main() { pitcher: prof, lineupKRate: lineupK, archetype: arch, role: 'starter', line: Number(r.line), }); + // The SAME model with the lineup term switched off, so the term's + // contribution is isolated rather than inferred. + const projNo = pe.projectStrikeouts({ + pitcher: prof, lineupKRate: null, archetype: arch, + role: 'starter', line: Number(r.line), + }); rows.push({ won, champ, residual: won - champ, pitch: proj ? (under ? 1 - proj.p_over_line : proj.p_over_line) : null, + pitch_nolineup: projNo ? (under ? 1 - projNo.p_over_line : projNo.p_over_line) : null, lineup_applied: !!lineupK, archetype: arch, archetype_flame: arch == null ? null : (arch === 'FLAME' ? 1 : 0), @@ -253,9 +270,49 @@ async function main() { }; } + // ── WITHIN-ARCHETYPE: the order's sharper hypothesis ──────────────────── + // The interaction should matter MORE for finesse arms (SCALPEL/SINKER), whose + // strikeouts need a lineup that will chase or can be beaten, than for power + // arms (FLAME) whose stuff whiffs regardless of who is standing there. Pooling + // the two would average a real conditional effect toward zero — which is + // exactly the failure mode "test within archetype" exists to prevent. + const strata = {}; + for (const [label, pred] of [ + ['FLAME (power)', (r) => r.archetype === 'FLAME'], + ['non-FLAME (finesse/contact)', (r) => r.archetype && r.archetype !== 'FLAME'], + ]) { + const rs = rows.filter((r) => pred(r) + && knownNumber(r.pitcher_whiff_pct) !== null + && knownNumber(r.opposing_lineup_k_rate) !== null); + if (rs.length < 15) { strata[label] = { n: rs.length, verdict: 'UNTESTABLE — stratum too thin' }; continue; } + const I = rs.map((r) => r.pitcher_whiff_pct * r.opposing_lineup_k_rate); + const Y = rs.map((r) => r.residual); + const ctrl = [rs.map((r) => r.pitcher_whiff_pct), rs.map((r) => r.opposing_lineup_k_rate)]; + const incr = partialCorr(I, Y, ctrl); + const parts = [ + { feature: 'pitcher_whiff_pct', r: r4(cv.pearson(rs.map((r) => r.pitcher_whiff_pct), Y).r) }, + { feature: 'opposing_lineup_k_rate', r: r4(cv.pearson(rs.map((r) => r.opposing_lineup_k_rate), Y).r) }, + ]; + const best = Math.max(...parts.map((p) => Math.abs(p.r ?? 0))); + strata[label] = { + n: rs.length, + raw_r_vs_residual: r4(cv.pearson(I, Y).r), + lineup_solo_r: parts[1].r, + component_solo_r: parts, + best_component_abs_r: r4(best), + INCREMENTAL_partial_r: r4(incr), + adds_over_components: incr !== null && Math.abs(incr) > best, + gate: cv.validateFactor(I, Y, TESTS), + verdict: incr === null ? 'UNTESTABLE — collinear' + : rs.length < 500 ? 'UNDERPOWERED — n below the gate' + : (Math.abs(incr) >= 0.15 ? 'ADDS' : 'REDUNDANT'), + }; + } + const h2h = rows.filter((r) => r.pitch != null); const ys = h2h.map((r) => r.won); const bs = bootstrapDiff(h2h, 'pitch', 'champ'); + const bsNoLineup = bootstrapDiff(h2h.filter((r) => r.pitch_nolineup != null), 'pitch_nolineup', 'champ'); console.log(JSON.stringify({ stat: 'strikeouts', @@ -267,12 +324,19 @@ async function main() { bonferroni_tests: TESTS, step1_solo_baseline: solo, step3_interactions: interactions, + step3b_within_archetype_carrier: strata, step4_vs_counter: { n: h2h.length, base_rate: r4(mean(ys)), resolution: { pitch_v1: r4(cv.pearson(h2h.map((r) => r.pitch), ys).r), counter: r4(cv.pearson(h2h.map((r) => r.champ), ys).r) }, brier: { pitch_v1: r4(brier(h2h.map((r) => r.pitch), ys)), counter: r4(brier(h2h.map((r) => r.champ), ys)) }, delta: bs, + // Isolating the lineup term: does including it help or hurt? + without_lineup_term: { + resolution: r4(cv.pearson(h2h.filter((r) => r.pitch_nolineup != null).map((r) => r.pitch_nolineup), + h2h.filter((r) => r.pitch_nolineup != null).map((r) => r.won)).r), + delta_vs_counter: bsNoLineup, + }, verdict: !bs ? 'N-BLOCKED — too few rows to bootstrap' : (bs.ci_excludes_zero && bs.point > 0) ? 'BEATS THE COUNTER' : (bs.ci_excludes_zero && bs.point < 0) ? 'LOSES to the counter' : 'INCONCLUSIVE', diff --git a/specs/lineup-k-rate-rung1.md b/specs/lineup-k-rate-rung1.md new file mode 100644 index 0000000..d202821 --- /dev/null +++ b/specs/lineup-k-rate-rung1.md @@ -0,0 +1,145 @@ +# RUNG 1 — opposing-lineup K-rate, derived from data we already hold + +**2026-08-03.** Challenger-only. Counter, batter cluster and pitcher engine all +byte-identical (verified by diff). + +> **VERDICT: Rung 1 is DERIVED and needs ZERO new sourcing — but it does not +> clear, and neither does anything else, because every pitcher result is still +> n=57 against a 500 bar.** Rung 2 and Rung 3 are NOT triggered: a rung only +> fails when it has been fairly tested, and Rung 1 has not been. +> +> **The real result is that the cap fix landed: 907 grades per snapshot, up from +> 334, with strikeouts 6 → 17.** That is the constraint that decides when any of +> this becomes answerable. + +--- + +## 0. Two premise corrections + +**"whiff / put-away / stuff prove SOLO through the gate."** They do not. Every +pitcher feature was **refused on `insufficient_data`** (n=57 < 500). Four +*exceeded the effect-size bar* (arm_angle −0.250, whiff +0.213, k_pct +0.206, +chase +0.195) — which is why they are worth pursuing — but exceeding one of three +thresholds is not passing the gate. + +**"The carrier was blocked ONLY on the lineup input."** The lineup input was +built and measured last session (roster join, 94.7% coverage). It was not +missing: it was tested and did not add. What blocks it is **n**, and n was being +throttled by the grading cap — which is what this session actually fixed. + +## 1. The derivation (Rung 1) — free, from batter data we already ingest + +Opposing-team K-rate is built by joining the opposing team's roster to the +**batter `k_pct` values already in `statcast_aggregates`**. No new feed, no new +cost. Coverage **94.7%** of settled strikeout rows. + +**Improved this session: PA-WEIGHTED, not a flat roster average.** An unweighted +mean counts a 12-PA September call-up the same as an everyday starter, which is +not the lineup a pitcher faces. Weighting each batter's K-rate by his own +`sample_pa` is the closest honest approximation of *who actually bats* from data +we already hold. + +**That change alone reversed the term's contribution:** + +| head-to-head on strikeouts | resolution | delta vs counter | +|---|---|---| +| pitcher skill alone | 0.1738 | +0.2377, CI [−0.020, +0.528] | +| **+ PA-weighted lineup K-rate** | **0.1953** | **+0.2592, CI [−0.0167, +0.5645]** | +| *(last session's unweighted version)* | *0.1285* | *+0.192* | + +The unweighted proxy **hurt** (0.174 → 0.129). The PA-weighted one **helps** +(0.174 → 0.195). Same hypothesis, same data — the derivation was the problem, not +the signal. That is the whole point of "derive the best honest version before +sourcing anything." + +The CI is now **[−0.0167, +0.5645]** — very nearly excluding zero, at n=57. + +## 2. The gate (STEP 2) + +Bonferroni across 9 tests. **Everything is refused on sample size**; r is shown +because "not enough data yet" and "nothing here" require opposite decisions. + +**Lineup K-rate, solo:** r vs residual = **+0.004** (n=54). On its own the team +K-rate says nothing — which is expected and is *not* an argument against it: it +is hypothesised as a **conditioner**, not a standalone predictor. + +**The carrier, pooled:** raw r 0.175, best component 0.198, **incremental ++0.077**. Still explained largely by whiff, but positive now (it was −0.102 +unweighted). + +### Within pitcher archetype — the order's sharper hypothesis + +| stratum | n | lineup solo r | best component | **incremental** | +|---|---|---|---|---| +| FLAME (power) | 20 | +0.117 | 0.117 | **−0.152** | +| non-FLAME (finesse/contact) | 24 | −0.161 | 0.183 | **+0.145** | + +**The two strata have opposite signs** — the shape a genuinely conditional effect +makes, and invisible when pooled (the pooled incremental, +0.077, sits between +them). That is exactly why the order asked for stratification. + +**But it is not evidence yet.** At n=20 and n=24 the standard error on a +correlation is roughly 0.22, so both estimates are inside noise, and the +*direction* does not match the theory: the order predicted the interaction should +matter **more** for finesse arms, and the magnitudes here are near-identical with +flipped signs. **Recorded as a structure to re-test, not a finding.** + +## 3. Head-to-head, and the number that keeps standing out + +``` +n 57 · base rate 0.597 +resolution pitch-v1 (with lineup) 0.1953 counter −0.0639 +brier pitch-v1 0.2412 counter 0.2703 +delta +0.2592 CI95 [−0.0167, +0.5645] → INCONCLUSIVE +``` + +**The counter's resolution on strikeouts is NEGATIVE.** Counting a pitcher's +recent strikeouts is *anti-predictive* of whether he clears his line, because +recent K totals track which lineups he drew and how long he was left in rather +than his skill. It remains the one stat where the incumbent has no defensible +edge, and the skill model leads it by 0.26 with a CI that now barely spans zero. + +## 4. THE CAP FIX LANDED — which is what decides the timeline + +Verified in production this session: + +``` +before 334 grades/snapshot · strikeouts 6 +after 907 grades/snapshot · strikeouts 17 · outs 16 · ER 16 · hits_allowed 16 +``` + +**2.7× across the board, 2.8× on strikeouts.** Operational note: the manual +`POST /api/internal/snapshot/:sport` now **524s at the Cloudflare gateway** — +grading the full board exceeds the 100s edge timeout. **The run still completes +server-side** (this snapshot was written by a 524'd request), and the cron is +in-process so it is unaffected. Do not read a 524 there as a failure; check +`/api/internal/snapshot/status`. + +## 5. Rung verdict + +| rung | status | +|---|---| +| **Rung 1 — team-season K-rate** | **DERIVED, zero new sourcing, 94.7% coverage, PA-weighted.** Contribution is positive but n=57 — **not fairly tested yet** | +| Rung 2 — projected lineup | **NOT TRIGGERED.** Rung 1 has not failed; it has not been tested | +| Rung 3 — confirmed-lineup sourcing | **NOT TRIGGERED.** Do not source | + +**Ship now? No.** Nothing passed the gate, so nothing is calibrated and nothing +ships. **Source confirmed lineups? No** — that would be paying for precision on +top of a proxy we have not yet measured properly. + +## 6. Next + +1. **Wait for n.** At 17 strikeout props/snapshot × 5 snapshots/day, strikeouts + should reach n≥500 in roughly **a week**, not the three months it was before + the cap fix. +2. **Re-run `scripts/pitcher-prove-k.js`** at n≥500 with a point-in-time window + from `statcast_history`. Both arrive on a similar timescale. +3. **Re-test the within-archetype strata then** — with ~200 per stratum instead + of 20, the opposite-sign structure either survives or dissolves. +4. **Give `arm_angle` a registry entry with a stated mechanism** before it is + tested for real. It is the largest |r| in the programme and currently has no + declared hypothesis, which is how a fishing result gets promoted. + +**Not recommended:** sourcing confirmed lineups, moving to Rung 2, calibrating, +or reading the opposite-sign strata as a conditional effect. All four would be +acting on n=20.