Diagnose proj-v1.1: concentrated mean failure, NOT a similarity problem

READ-ONLY. Nothing built or fixed; the four challengers untouched.

41% OF THE REPORTED GAP WAS A MEASUREMENT ARTIFACT. p_win is P(graded
side); proj_p_over_line is P(over); 31.4% of settled rows are UNDER-graded,
so comparing them raw measures the ladder backwards on a third of the
sample. Matched + direction-aligned (n=437): 0.252 vs champion 0.352, not
0.108 vs 0.331. The PRODUCT is not making this mistake -- I checked;
projectionChallenger normalises both to the over basis deliberately. The
error was in the measurement.

THE LOSS IS CONCENTRATED. hits (n=245, res 0.060) and total_bases (n=49,
res 0.009) are 67% of rows and carry essentially no signal. Everything else
is fine or better: walks 0.519 vs champion 0.544, runs mean 0.345 vs 0.392,
and on DOUBLES the ladder's mean BEATS the champion's (0.207 vs -0.062).

IT IS THE MEAN, NOT THE SHAPE. On the two failing families the mean itself
carries no signal (0.052, -0.019) against the champion's 0.158 and 0.085.
Where the mean is good the probability is good -- shape follows mean.

A HYPOTHESIS I TESTED AND DISPROVED: prediction compression. I expected
P(>=1 hit) to sit in a narrow band and fail to rank. It does not -- spread
ratio 0.94 overall, 0.80 for hits, 0.94 for total_bases. The ladder has
comparable spread; it is spread in a direction uncorrelated with outcomes.
Recorded because it was a plausible story the data refused.

PRIORS AND PLUMBING CLEAN. proj_factors carries form_rate,
combined_multiplier and breakdown on every row; proj_point 100% populated
with sane centres (hits 0.830 vs line 0.578). Not the environment-style
silent-null failure.

NAMED CAUSE (structural, flagged as hypothesis not finding): the count
model mismatches those two stats. total_bases is a WEIGHTED SUM (1B..HR =
1..4), so an NB treats one home run as four events and mis-states variance
-- and TB has the worst result in the table. hits is BOUNDED BY AT-BATS and
mostly traded at 0.5, so almost everything rides on P(0), the region where
the wrong family hurts most. walks/runs/doubles ARE genuine low-rate counts
and are exactly the ones that work.

FIX BRANCH: targeted per-stat fix for hits and total_bases. THIS REMOVES
THE MLB SIMILARITY BUILD FROM THE CRITICAL PATH -- that branch assumed a
GLOBAL mean weakness, and the mean is fine or better on three of six stat
families. Similarity may be worth building later, on evidence, not on this.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QJs13VsyiSKYQP6rj3NNmc

specs/MASTER-PLAN.md

@@ -29,24 +29,31 @@ stay provisional until re-run** · documented ≠ verified.
 
 ## ▶ NEXT EXECUTABLE ORDER
 
-**DIAGNOSE WHY proj-v1.1 LOSES.** The distribution ladder is **not dormant** — it
+**TARGETED PER-STAT FIX for `hits` and `total_bases` in proj-v1.1.** Diagnosis:
-is a **fourth accruing challenger**, live at **94.2%** coverage with **437 settled
+`specs/proj-v11-diagnosis.md`.
-rows**, and it is **losing**: resolution **0.108** vs the champion's **0.331**.
 
-It is the only one of the candidate layers already carrying real settled evidence,
+The ladder's loss is **concentrated, not systemic**. On matched, direction-aligned
-it is the **per-stat distribution §10.3 called the biggest modelling gap**, and its
+rows (n=437) the real gap is **0.252 vs 0.352** — not the 0.108 vs 0.331 previously
-diagnosis decides whether an MLB similarity layer is worth building at all.
+reported, which was **unaligned on direction across 31.4% under-graded rows**.
 
-**Then, in order:** `archetype_x_archetype` (matchup's upper rung — ready, small
+**`hits` (n=245) and `total_bases` (n=49) are 67% of the sample and carry
-radius) → **build** an MLB similarity layer *only if* the diagnosis says a better
+essentially no signal** (0.060 and 0.009). Everything else is fine or better:
-mean is what proj-v1.1 needs → retire/rewrite `bayesianEngine`.
+**`walks` 0.519 vs champion 0.544**, and on **`doubles` the ladder's MEAN beats the
+champion's** (0.207 vs −0.062). Priors and plumbing are clean; compression was
+tested and **disproved** (spread ratio 0.80–0.94).
 
-**Nothing is left to "connect"** — see `specs/dormant-layer-audit.md`:
+**Hypothesis to test in that order:** the negative binomial mismatches those two
-- **similarity** — BUILT but **NBA-shaped** (pace, referees, score state). For MLB
+stats' structure — `total_bases` is a **weighted sum**, not an event count, and
-  it is **CONSTRUCT, not connect**; wiring it would be the sport-stubbed-in breach.
+`hits` is **bounded by at-bats**, so both violate the unbounded-count assumption
-- **bayesian** — BUILT but keys on **7 stat names that are not live** and defaults
+that `walks`/`runs`/`doubles` satisfy.
-  to `'normal'`, so it would silently model count stats as Gaussian. Also
+
-  **superseded** by `distribution.js`. **Do not connect.**
+> **This diagnosis REMOVES the MLB similarity build from the critical path.** That
+> branch assumed a *global* mean weakness; the mean is fine or better on three of
+> six stat families. Similarity may be worth building later — **on evidence, not
+> on this.**
+
+**Then:** `archetype_x_archetype` (matchup's upper rung) → retire/rewrite
+`bayesianEngine` (keys on 7 non-live stat names; defaults count stats to Gaussian).
 
 ### FOUR challengers accruing in parallel — do NOT re-run early
 Verified firing on a real prod snapshot (293 grades), not inferred:
@@ -56,7 +63,7 @@ Verified firing on a real prod snapshot (293 grades), not inferred:
 | **environment** | **84.6%** | 0.057 | *(shares the arch-v1 pattern)* |
 | **matchup** (`batter_own_split`) | **82.9%** | 0.015 | `scripts/matchup-axis-holdout.sql` |
 | **opportunity** | 30.0% | 0.142 | `scripts/opportunity-axis-holdout.sql` |
-| **proj-v1.1** (distribution ladder) | **94.2%** | *(forms the projection, not a nudge)* | 437 settled — **currently LOSING 0.108 vs 0.331** |
+| **proj-v1.1** (distribution ladder) | **94.2%** | *(forms the projection, not a nudge)* | 437 settled — aligned gap **0.252 vs 0.352**, concentrated in `hits`+`total_bases` |
 
 All three orthogonal (r ≈ 0 vs projection, `p_win`, line and each other). Each
 promotes ONLY on its own axis-filtered holdout, and ONLY if **reliability AND

specs/proj-v11-diagnosis.md

@@ -0,0 +1,159 @@
+# WHY proj-v1.1 LOSES — DIAGNOSIS
+
+**Date:** 2026-08-02 · **READ-ONLY** — nothing built, fixed or wired. The four
+accruing challengers were not touched.
+
+---
+
+## HEADLINE: THE GAP IS REAL BUT WAS OVERSTATED, AND IT IS NOT A SIMILARITY PROBLEM
+
+> **CAUSE: a bad MEAN on TWO stat families — `hits` and `total_bases` — which are
+> 67% of the sample. Everything else works.**
+>
+> **FIX BRANCH: targeted per-stat model fix. NOT an MLB similarity build.**
+
+The diagnosis removes the biggest remaining build, which is what it was for.
+
+---
+
+## STEP 1 — MATCHED ROWS: 41% OF THE "GAP" WAS A MEASUREMENT ARTIFACT
+
+`p_win` is P(**graded side**). `proj_p_over_line` is P(**over**). **31.4% of settled
+rows are UNDER-graded**, so comparing raw P(over) against a graded-under outcome
+measures the ladder backwards on a third of the sample.
+
+Matched rows, n=437:
+
+| | resolution |
+|---|---:|
+| champion `p_win` | **0.3523** |
+| proj-v1.1, **unaligned** (as previously reported) | 0.1494 |
+| proj-v1.1, **direction-aligned** | **0.2521** |
+
+**Aligning direction recovers 41% of the apparent gap.** The previously reported
+*0.108 vs 0.331* substantially overstated the loss.
+
+**The product itself is NOT making this mistake** — I checked. `projectionChallenger`
+deliberately normalises both quantities to the over basis (`proj_book_implied` =
+`fairOver`, converting the graded-side fair when direction is under). **The error
+was in the measurement, not the model.** The real gap is **0.252 vs 0.352**.
+
+---
+
+## STEP 2 — PER-STAT: THE LOSS IS CONCENTRATED, NOT SYSTEMIC
+
+| stat | n | base | **res champ** | **res proj** | champ MEAN | proj MEAN |
+|---|---:|---:|---:|---:|---:|---:|
+| **hits** | **245** | .588 | 0.204 | **0.060** | 0.158 | **0.052** |
+| **total_bases** | **49** | .612 | 0.273 | **0.009** | 0.085 | **−0.019** |
+| rbi | 42 | .333 | 0.412 | 0.218 | 0.349 | 0.159 |
+| runs | 38 | .579 | 0.467 | 0.270 | 0.392 | **0.345** |
+| **walks** | 29 | .517 | 0.544 | **0.519** | 0.297 | **0.519** |
+| **doubles** | 28 | .179 | 0.315 | 0.213 | **−0.062** | **0.207** |
+| **ALL** | 437 | .533 | 0.352 | 0.252 | 0.199 | 0.166 |
+
+**`hits` + `total_bases` = 294 of 437 rows (67%), and on both the ladder has
+essentially NO signal.** They drag the aggregate on their own.
+
+**Where the ladder works, it works well:**
+- **`walks`: 0.519 vs the champion's 0.544** — competitive.
+- **`doubles`: the ladder's MEAN (0.207) BEATS the champion's (−0.062).**
+- `runs`: mean 0.345 vs 0.392 — close.
+
+**The machinery is not broken. Two stat families are.**
+
+---
+
+## STEP 3 — MEAN vs SHAPE: IT IS THE MEAN
+
+For the two failing families the **mean itself carries no signal** — `hits` 0.052,
+`total_bases` **−0.019** — while the champion's mean on the same rows carries 0.158
+and 0.085. The distribution cannot rescue a mean that does not rank.
+
+Conversely, where the mean is good (`walks` 0.519, `runs` 0.345, `doubles` 0.207)
+the ladder's probability is good. **Shape follows mean, cleanly.**
+
+### A hypothesis I tested and DISPROVED
+
+I expected **prediction compression** — that P(≥1 hit) would land in a narrow band
+across players and so could not rank. **Wrong:**
+
+| stat | sd champion | sd proj | ratio |
+|---|---:|---:|---:|
+| ALL | 0.2048 | 0.1932 | **0.94** |
+| hits | 0.1703 | 0.1369 | 0.80 |
+| total_bases | 0.1876 | 0.1758 | **0.94** |
+
+**The ladder has comparable spread. It is not compressed — it is spread in a
+direction uncorrelated with outcomes.** Recording this because it was a plausible
+story that the data refused.
+
+---
+
+## STEP 4 — PRIOR / PLUMBING INTEGRITY: CLEAN
+
+No silent default found. On a real snapshot, `proj_factors` carries `form_rate`,
+`combined_multiplier`, `breakdown`, `book_implied_basis` on every row, and
+`proj_point` is populated 100%. The central values are sane:
+
+| stat | mean `proj_point` | mean line |
+|---|---:|---:|
+| hits | 0.830 | 0.578 |
+| total_bases | 1.639 | 1.500 |
+| strikeouts | 4.972 | 5.204 |
+
+**The priors are reaching the posterior. This is not the environment-style
+silent-null failure.**
+
+---
+
+## THE CAUSE, AND WHY THOSE TWO STATS
+
+**Named cause: the count model does not match the generative structure of `hits`
+and `total_bases`.**
+
+Stated as a **hypothesis** — it follows from the structure, and this diagnosis did
+not test it directly:
+
+- **`total_bases` is not a count of events — it is a WEIGHTED SUM** (1B=1, 2B=2,
+  3B=3, HR=4). A negative binomial fitted to TB treats one home run as "four
+  events", which mis-states the variance badly. This is a structural mismatch, not
+  a tuning error — and TB has the worst result in the table (**−0.019**).
+- **`hits` is bounded by at-bats** (~4/game). It is closer to
+  binomial(AB, avg) than to an unbounded Poisson/NB, and most lines are 0.5
+  (mean line 0.578), so almost everything rides on P(0) — the exact region where
+  the wrong family hurts most.
+- **`walks`, `runs`, `doubles` ARE genuine low-rate event counts** — and they are
+  precisely the ones that work.
+
+---
+
+## FIX BRANCH — and what it rules OUT
+
+**RECOMMENDED NEXT ORDER: a targeted per-stat model fix for `hits` and
+`total_bases`.** Small, contained, and aimed at 67% of the sample.
+
+**This diagnosis REMOVES the MLB similarity build from the critical path.** The
+order's "bad MEAN → build similarity (large)" branch assumed a *global* mean
+weakness. It is not global: the mean is fine or better than the champion's on
+`walks`, `runs` and `doubles`. **A similarity layer would not fix a
+family-mismatched count model, and building one now would be a large project aimed
+at the wrong defect.**
+
+Similarity may still be worth building later — but **on evidence, not on this.**
+
+## TAGS
+
+**VERIFIED:** matched-row aligned gap 0.252 vs 0.352 (n=437) · 31.4% under-graded ·
+`projectionChallenger` normalises direction correctly, so the misalignment was
+measurement-only · per-stat table above · mean-vs-shape isolation · priors and
+plumbing clean.
+
+**DISPROVED:** prediction compression (spread ratio 0.80–0.94).
+
+**HYPOTHESIS, NOT VERIFIED:** that the NB family mismatch is *why* hits and TB fail.
+Structurally motivated; the fix order should test it before committing to a family
+change.
+
+**CORRECTED:** "proj-v1.1 loses 0.108 vs 0.331" — that comparison was unaligned on
+direction and across different row sets.