tb-v1 report + plan: diagnosis on trial, branch pre-registered
Firing verified on a real prod snapshot: 10/10 total_bases props carry proj_tb_p_over. The snapshot HTTP call returned 524 (Cloudflare's 100s origin timeout vs a ~115s snapshot) but the work completed server-side -- confirmed from the ledger rather than assumed. Face validity is good and diagnostic: means agree almost exactly with the ladder (1.813 vs 1.833), so this is a SHAPE-ONLY intervention, which is what was intended. Component rates are plausible, and Carroll's triples rate (0.112, far above his peers) is a clean check -- he is a speed player and the model sees it. AN OBSERVATION I AM NOT RESOLVING BY EYE: tb-v1 reads systematically LOWER than the ladder (0.424 vs 0.540 at the same mean). That is the expected DIRECTION, since the NB overstates P(>=2) by treating a home run as four accumulating events -- but whether 0.424 is right or an overcorrection is not knowable from face validity. A ~1.8-TB hitter clearing 1.5 empirically sits nearer 45-50%, between the two. I am not claiming tb-v1 is better; the holdout decides. BRANCH PRE-REGISTERED, before the result, so the verdict cannot be reinterpreted afterward: improves -> family-mismatch HOLDS, similarity stays off the critical path, hits is next; does not improve -> hypothesis WRONG and the mean-weakness/similarity branch REOPENS. Also recorded: I hit Number(null)===0 in my own new module -- a null component rate treated as a measured zero, the difference between "never triples" and "we don't know his triple rate". A test caught it. Sixth appearance of this trap in this codebase, and it caught the person writing the warnings about it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QJs13VsyiSKYQP6rj3NNmc
This commit is contained in:
+17
-22
@@ -29,33 +29,27 @@ stay provisional until re-run** · documented ≠ verified.
|
||||
|
||||
## ▶ NEXT EXECUTABLE ORDER
|
||||
|
||||
**TARGETED PER-STAT FIX for `hits` and `total_bases` in proj-v1.1.** Diagnosis:
|
||||
`specs/proj-v11-diagnosis.md`.
|
||||
**AWAIT the tb-v1 verdict, then `hits`.** `specs/tb-v1-compound.md`.
|
||||
|
||||
The ladder's loss is **concentrated, not systemic**. On matched, direction-aligned
|
||||
rows (n=437) the real gap is **0.252 vs 0.352** — not the 0.108 vs 0.331 previously
|
||||
reported, which was **unaligned on direction across 31.4% under-graded rows**.
|
||||
**tb-v1 is LIVE as a challenger** — total_bases modelled as the compound outcome
|
||||
it is (per-component Poisson rates convolved exactly) instead of one negative
|
||||
binomial that treats a home run as four events. Firing on **10/10 TB props**;
|
||||
means agree with the ladder, **only the shape differs** — the intended
|
||||
intervention.
|
||||
|
||||
**`hits` (n=245) and `total_bases` (n=49) are 67% of the sample and carry
|
||||
essentially no signal** (0.060 and 0.009). Everything else is fine or better:
|
||||
**`walks` 0.519 vs champion 0.544**, and on **`doubles` the ladder's MEAN beats the
|
||||
champion's** (0.207 vs −0.062). Priors and plumbing are clean; compression was
|
||||
tested and **disproved** (spread ratio 0.80–0.94).
|
||||
**The diagnosis is on trial, and the branch is pre-registered:**
|
||||
- **tb-v1 improves TB resolution** → family-mismatch **HOLDS**, similarity stays
|
||||
**off** the critical path, **`hits` is next** (AB-bounded, mostly 0.5 lines, so
|
||||
it rides on P(0)).
|
||||
- **tb-v1 does not improve** → hypothesis **WRONG**, the **mean-weakness /
|
||||
MLB-similarity branch REOPENS**.
|
||||
|
||||
**Hypothesis to test in that order:** the negative binomial mismatches those two
|
||||
stats' structure — `total_bases` is a **weighted sum**, not an event count, and
|
||||
`hits` is **bounded by at-bats**, so both violate the unbounded-count assumption
|
||||
that `walks`/`runs`/`doubles` satisfy.
|
||||
**Verdict n-blocked:** 0 settled TB rows carry tb-v1 yet; first settle is
|
||||
tomorrow. `scripts/tb-compound-holdout.sql` is TB-rows-only and direction-aligned.
|
||||
|
||||
> **This diagnosis REMOVES the MLB similarity build from the critical path.** That
|
||||
> branch assumed a *global* mean weakness; the mean is fine or better on three of
|
||||
> six stat families. Similarity may be worth building later — **on evidence, not
|
||||
> on this.**
|
||||
**Then:** `archetype_x_archetype` → retire/rewrite `bayesianEngine`.
|
||||
|
||||
**Then:** `archetype_x_archetype` (matchup's upper rung) → retire/rewrite
|
||||
`bayesianEngine` (keys on 7 non-live stat names; defaults count stats to Gaussian).
|
||||
|
||||
### FOUR challengers accruing in parallel — do NOT re-run early
|
||||
### FIVE challengers accruing in parallel — do NOT re-run early
|
||||
Verified firing on a real prod snapshot (293 grades), not inferred:
|
||||
|
||||
| axis | coverage | mean \|nudge\| | holdout query |
|
||||
@@ -63,6 +57,7 @@ Verified firing on a real prod snapshot (293 grades), not inferred:
|
||||
| **environment** | **84.6%** | 0.057 | *(shares the arch-v1 pattern)* |
|
||||
| **matchup** (`batter_own_split`) | **82.9%** | 0.015 | `scripts/matchup-axis-holdout.sql` |
|
||||
| **opportunity** | 30.0% | 0.142 | `scripts/opportunity-axis-holdout.sql` |
|
||||
| **tb-v1** (compound total_bases) | **10/10 TB props** | *(shape, not a nudge)* | pre-registered: improves → `hits` next; doesn't → similarity reopens |
|
||||
| **proj-v1.1** (distribution ladder) | **94.2%** | *(forms the projection, not a nudge)* | 437 settled — aligned gap **0.252 vs 0.352**, concentrated in `hits`+`total_bases` |
|
||||
|
||||
All three orthogonal (r ≈ 0 vs projection, `p_win`, line and each other). Each
|
||||
|
||||
Reference in New Issue
Block a user