builtbykev
c0621e7aa2
PREMISE CORRECTION FIRST, because it defines the bar. total_bases has not
passed BAR 1. Its head-to-head is inconclusive at parity -- delta +0.004 to
+0.007 with a CI spanning zero -- and it is contaminated, and no feature of
its passed the gate. It was described last session as the first challenger
that did not LOSE, which is not the same as proven. If it is installed as the
frozen proven reference and every other stat is held to "the identical bar
total_bases cleared", the bar becomes "be inconclusive at parity" and the
whole cluster passes on a null result. The proven set is EMPTY.
HITS IS NOW A FINAL ANSWER. At n=803 it clears the gate's sample requirement,
so its features were properly TESTED rather than refused: every one fails on
effect size (max marginal |r| 0.053 against a 0.15 bar), every interaction's
incremental contribution collapses to about zero, and the model loses
head-to-head by 0.096 with a CI excluding zero. That is a well-powered
negative and hits should be closed rather than retried.
The rest are n-blocked: total_bases 383, rbi 391, home_runs 228, runs 188,
against a bar of 500. Two leads are worth carrying. home_runs barrel rate has
a marginal r of -0.135, and the sign matters -- higher barrel rate goes with
the counter OVER-predicting, which would be a correction rather than a new
predictor. And runs batterK x pitcherK has the largest incremental in the
cluster at +0.132, with a clean mechanism: strikeouts destroy plate
appearances, and a PA that never happens cannot score.
RBI deserves a caveat rather than a verdict. It is power times OPPORTUNITY,
and we ingest no baserunner state at all, so half its mechanism is missing. A
weak RBI result is evidence that we are modelling half the stat.
total_bases was held frozen: git diff on skillProjection against the prior
commit is empty. The counter is untouched.
Also fixed and verified in production: the point-in-time retention shipped
after yesterday's refresh had already run, so statcast_history was empty, and
its first run then failed on a hand-enumerated schema that had already drifted
from its source ("could not find the 'swing_pct' column"). The refresh itself
still succeeded and wrote all 1,387 aggregate rows, which confirmed the
best-effort guard in prod. The table now mirrors the source via LIKE and the
writer passes rows through whole. Verified live: 1,387 rows retained at as_of
2026-08-03. A usable point-in-time window starts 2026-08-04.
Stage B has nothing to calibrate. Everything now waits on a point-in-time
window and on sample -- both waiting problems, not building problems.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W1sivYNqY2TS5ftykmHBU9
127 lines
6.4 KiB
Markdown
127 lines
6.4 KiB
Markdown
# THE BATTER CLUSTER — both-ways results, and what the bar actually is
|
||
|
||
**2026-08-03.** Challenger-only. Counter byte-identical. `skillProjection`
|
||
byte-identical (frozen) — verified by diff against the prior commit.
|
||
|
||
> **PREMISE CORRECTION, and it decides the whole order: total_bases has NOT
|
||
> passed BAR 1.** Yesterday's result was **delta +0.0071, CI95
|
||
> [−0.0648, +0.0789] — inconclusive, at parity, and contaminated**. No feature
|
||
> passed the gate. It was described as "the first challenger that did not LOSE",
|
||
> which is not the same as proven.
|
||
>
|
||
> This matters structurally: if total_bases is installed as "the frozen proven
|
||
> reference" and every other stat is held to "the identical bar total_bases
|
||
> cleared", **the bar becomes "be inconclusive at parity"** — a standard that
|
||
> proves nothing and would admit the entire cluster on a null result.
|
||
>
|
||
> **The proven set is EMPTY.** That is the honest Stage-B input.
|
||
|
||
---
|
||
|
||
## 0. Two orders running, one pattern
|
||
|
||
This is the second consecutive order whose premise promoted a null result to a
|
||
pass (the previous one had "barrel rate PASSED solo" when every TB feature was
|
||
refused on sample size). Flagging the pattern once, without labouring it: the
|
||
measurements are being read at their most favourable interpretation somewhere
|
||
between sessions. The numbers below are stated so that cannot happen again.
|
||
|
||
## 1. Validity: still contaminated today, but the clock has started
|
||
|
||
`statcast_history` was **empty** — the retention shipped *after* yesterday's
|
||
refresh had already run. Two things fixed this session:
|
||
|
||
1. The first prod run failed with `Could not find the 'swing_pct' column` — the
|
||
hand-enumerated schema had drifted from the table it was copying. **The
|
||
refresh still succeeded and wrote all 1,387 aggregate rows**, which verified
|
||
the best-effort guard in production: a retention failure does not fail the
|
||
refresh.
|
||
2. The table is now created `LIKE statcast_aggregates` and the writer passes the
|
||
row through whole, so there is no drift surface left.
|
||
|
||
**Verified live: `history_retained: 1387, as_of 2026-08-03`.** The point-in-time
|
||
clock is running. It does not yet give a *window* — `as_of 2026-08-03` can only
|
||
score games from 2026-08-04 — so **everything below remains contaminated and
|
||
directional**, exactly as yesterday.
|
||
|
||
## 2. STEP 1 — both-ways results, per stat
|
||
|
||
Bonferroni across each stat's own sweep. The decider is correlation with the
|
||
counter's **residual** (`won − p_win`): only what the counter misses is new.
|
||
|
||
| stat | n | strongest solo (marginal r) | p | gate | best interaction (incremental) | head-to-head vs counter |
|
||
|---|---|---|---|---|---|---|
|
||
| **hits** | **803** | exit_velo **−0.053** | 0.130 | **FAILS** (properly tested) | all ≈0, all **FAIL** | **LOSES** −0.0961, CI [−0.165, −0.029] |
|
||
| total_bases | 383 | hard_hit **+0.135** | 0.008 | insufficient n | barrel×archetype −0.101 | **INCONCLUSIVE** +0.0038, CI [−0.068, +0.075] |
|
||
| rbi | 391 | exit_velo −0.090 | 0.074 | insufficient n | barrel×archetype +0.073 | no projection built |
|
||
| home_runs | 228 | barrel **−0.135** | 0.041 | insufficient n | barrel×archetype −0.116 | no projection built |
|
||
| runs | 188 | launch −0.088 | 0.230 | insufficient n | **K×K +0.132** | no projection built |
|
||
|
||
### hits is now a FINAL answer, not a pending one
|
||
|
||
At **n=803** hits clears the gate's sample requirement, so its features were
|
||
**properly tested rather than refused**. Every one fails on effect size (max
|
||
|r| 0.053 against a 0.15 bar), every interaction's incremental collapses to
|
||
≈0, and the model loses head-to-head with a CI excluding zero. **This is a
|
||
well-powered negative and hits should be closed.**
|
||
|
||
### The rest are all n-blocked
|
||
|
||
TB 383, rbi 391, HR 228, runs 188 — none reaches 500. Their numbers are
|
||
directional only. Two are worth carrying:
|
||
|
||
- **home_runs · barrel_pct, marginal r = −0.135 (p=0.041).** Note the **sign**:
|
||
higher barrel rate goes with the counter *over*-predicting. If that survives
|
||
more data it is a real correction, not a new predictor.
|
||
- **runs · batter_K × pitcher_K, incremental +0.132** — the largest incremental
|
||
anywhere in the cluster, with a clean mechanism (strikeouts destroy plate
|
||
appearances, and a PA that never happens cannot score). At n=188 it is a lead.
|
||
|
||
### RBI has a missing half, stated rather than papered over
|
||
|
||
RBI is power × **opportunity** — the same swing drives in one run or three
|
||
depending on who is on base. **We do not ingest baserunner state at all**, so
|
||
half the mechanism is absent. A weak RBI result here is not evidence that skill
|
||
inputs fail for RBI; it is evidence we are modelling half the stat.
|
||
|
||
## 3. STEP 2 — total_bases held frozen
|
||
|
||
`git diff` on `src/services/model/skillProjection.js` against the prior commit
|
||
is **empty**. The projection is byte-identical; nothing was re-opened or re-fit.
|
||
It is frozen — but as §0 records, it is frozen as an **inconclusive** model, not
|
||
a proven reference.
|
||
|
||
## 4. STEP 3 — the proven set for Stage B
|
||
|
||
| stat | BAR 1 (proven) | BAR 2 (calibrated) |
|
||
|---|---|---|
|
||
| total_bases | **NO** — inconclusive at parity, contaminated | not attempted |
|
||
| hits | **NO** — loses, CI excludes zero, well-powered | n/a |
|
||
| home_runs | not testable (n=228) | n/a |
|
||
| rbi | not testable (n=391, half the mechanism missing) | n/a |
|
||
| runs | not testable (n=188) | n/a |
|
||
|
||
**Proven set: EMPTY. Stage B has nothing to calibrate.** Calibrating an
|
||
inconclusive model into grade bands would produce an "A" backed by a model not
|
||
shown to beat counting — which is exactly what BAR 2 exists to prevent.
|
||
|
||
## 5. What actually unblocks this
|
||
|
||
Everything now waits on the same two things, and both are **waiting problems,
|
||
not building problems**:
|
||
|
||
1. **A point-in-time window.** `statcast_history` starts accruing usable
|
||
comparisons from 2026-08-04. A week gives a real one. Until then no skill
|
||
result can be honest, whatever its n.
|
||
2. **Sample.** TB needs ~117 more settled rows, rbi ~109, HR ~272, runs ~312.
|
||
At current volume TB and rbi arrive within days; HR and runs are weeks away.
|
||
|
||
Re-run `scripts/cluster-prove.js` (per stat via `CLUSTER_STAT=`) once both hold.
|
||
|
||
**Ranked, when the data arrives:** total_bases (closest to both bars) → rbi
|
||
(needs baserunner state to be a fair test) → home_runs → runs. **Close hits.**
|
||
|
||
**Not recommended:** treating parity as proven, calibrating anything yet,
|
||
lowering n≥500, or letting a stat inherit a pass from the shared conditioning
|
||
map. Reuse sped the search; it granted nothing, and nothing has been granted.
|