hits-v1: built on the right structure, measured honestly, REFUTED

Hits was diagnosed as a family mismatch: 84% of hits rows trade at 0.5, so
the stat rides on P(0), and a negative binomial has unbounded support and no
notion of opportunity at all. hits-v1 models it as the bounded conversion it
is -- N ~ the player's empirical at-bat distribution, hits|N ~ Binomial(N,q),
with the multiplier scaling q (conversion) and never N (opportunity).

STEP 0 confirmed the inputs before the model existed: 30/30 real ledger
players, 100% combined-input coverage. Every read goes through knownRate --
a row with no atBats is dropped, never counted as a 0-at-bat game.

It FIRES: 158/159 hits props (99.4%) on the live production snapshot, through
the real attachProjection path. Scoping by book IDENTITY rather than price
shape kept 94 out-of-promotion-band props on the board, 93 of them modelled --
59% that a price rule would have deleted.

And it LOST. Point-in-time replay (game log truncated strictly before each
row's game_date, real grade-time multiplier), hits-only, direction-aligned,
n=242: resolution champion 0.195 / ladder 0.048 / hits-v1 0.026. Paired
bootstrap on the same rows: hits-v1 - ladder = -0.022, CI95 excluding zero.
Not promoted.

The value is in what it eliminates. The family was wrong AND the mean was not
the constraint -- hits-v1 moved the line-0.5 mean 0.554 -> 0.581 toward a
0.598 base rate while resolution fell. What is left is per-prop
discrimination: the ladder's inputs, not its distribution.

The pre-registered fallback is recorded as WRONG rather than deleted. It said
hits might be genuinely low-resolution for anyone; the champion scores 0.276
on the identical 189 rows, so there is real signal and the ceiling claim was
the comfortable reading, not the honest one. Its own control refuted it, and
that control was already in hand when the branch was written.

hits-v1 stays wired as a challenger writing its own ledger columns so the
forward accrual can confirm the backtest. Champion, ladder, ranking,
calibration, reference ruler and the four accruing verdicts are byte-identical
-- the diff has zero deleted lines.

Tests 4,156 green (332 suites); web build exit 0.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W1sivYNqY2TS5ftykmHBU9
This commit is contained in:
Kev
2026-08-02 19:04:08 -04:00
parent d103ecf4c3
commit 07626de3de
11 changed files with 1409 additions and 2 deletions
+36
View File
@@ -1096,6 +1096,42 @@ phased plan in the Session-57 conversation / BUILD-STATE Next section).
Redis runs degraded locally, so the script must `process.exit(0)` — otherwise a
reconnect timer holds the process open and piped output is lost to SIGTERM.
## hits-v1 — a REFUTED challenger, and why it stays (Session 76 — non-obvious)
- **`specs/hits-v1-binomial.md` is the record.** hits-v1 models hits as a
binomial over the player's EMPIRICAL at-bat distribution (P(0) stated directly,
since 84% of hits rows trade at 0.5). It FIRES at 99.4% on the live board and
it DOES NOT WORK: point-in-time replay, hits-only, direction-aligned, n=242 —
resolution champion 0.195 / ladder 0.048 / hits-v1 0.026. Paired bootstrap
(same rows) puts hits-v1 ladder at 0.022, CI95 excluding 0. NOT PROMOTED.
- **Two explanations are now ELIMINATED for hits, which is the useful part.**
The family was wrong (swapping it made things slightly worse) AND the mean was
not the constraint (hits-v1 moved the line-0.5 mean 0.554→0.581 against a 0.598
base rate — closer — while resolution FELL). What remains is per-prop
DISCRIMINATION: the ladder's inputs don't separate hitters. Don't build another
projection variant for hits; diagnose what the champion's `p_win` reads first.
- **"Honest ceiling" needs its control checked before you claim it.** The spec's
own pre-registered fallback ("hits may be genuinely low-resolution for anyone")
was REFUTED by the champion scoring 0.276 on the identical 189 rows. A ceiling
claim is only honest if no instrument on the same rows beats it — check that
BEFORE writing the branch, not after.
- **Backtest ≠ verdict.** The replay truncates each player's game log strictly
BEFORE the row's `game_date` and reuses the row's stored grade-time
`combined_multiplier` (both live on real ledger rows) — without that truncation
it would be scoring predictions with the answer in hand. The verdict of record
is still the forward accrual, so hits-v1 stays wired, writing
`proj_hits_p_over`/`proj_hits_meta` only. Never served.
- **Paired bootstrap, not two independent SEs.** Challengers score the SAME rows;
comparing independent standard errors overstates uncertainty and would have
read a reliable 0.022 regression as noise. `scripts/hits-v1-holdout.js` has
the seeded implementation — reuse it for the next challenger.
- **The takeable axis paid off measurably:** 94 of 159 live hits props are
OUTSIDE the promotion band and 93 were modelled anyway. Scope by
`isTakeableMarket` (book identity); record `isWithinPromotionBand` and never
let it gate a model — a price-shape rule would have deleted 59% of the board.
- **Local `.env` has a transposed Supabase ref** (`zmdnczhtdxcddszxttub`; real is
`zmdnczhtdxcddsxzttub`), so local scripts hitting Supabase need an explicit
`SUPABASE_URL=` override. Prod + the MCP connection are fine.
## Active Skills
- vyndr-voice (all user-facing output)
- prop-analysis (grading methodology)