vyndr/specs/champion-input-diagnosis.md

# DECOMPOSING THE CHAMPION — where its edge actually comes from

**Read-only diagnosis, 2026-08-03.** Nothing built, nothing touched. Run on the
**repaired** settled set (n=1,741 after the settlement outage fix), not the frozen
pre-fix set.

> **VERDICT, one line:** the champion's entire edge is a **hit-rate counter**, its
> three adjustment layers contribute **nothing** (two are mildly harmful), and the
> largest recoverable loss is **not a missing feature — it is the `[0.10, 0.95]`
> clamp, which pins 20.6% of settled props to a constant** and hides outcomes
> ranging from 52% to 99.5% behind the same number `0.900`.

---

## 1. What the champion actually uses (STEP 1)

`src/services/intelligence/probabilityEstimator.js` is **five lines of arithmetic**:

```
base      = empirical frequency of (stat > THIS line) over the game log
weighted  = 0.6·base + 0.4·(same frequency over the last 5 games)
p         = weighted + oppAdj(±0.03) + homeAdj(±0.015)
if cv > 0.40:   p = 0.9·p + 0.05                  (volatile → pull toward 0.50)
p_over    = clamp(p, 0.10, 0.95)
p_win     = side === 'under' ? 1 − p_over : p_over
```

It reads exactly **three** features: `opp_rank_stat`, `home_away`, and
`l10_stddev`/`l20_avg` (for cv). `featureCache` computes and retains a dozen more
— `park_h/hr/r`, `weather_temp_f/wind_mph/precip`, `rest_days`,
`opportunity_drift`, `ab_per_game`, `recent_ab_per_game`, `l5/l10/l20_avg`,
`game_count_in_7d` — and **p_win reads none of them.**

## 2. Per-stat ablation (STEP 2)

**Exact and analytic, not a refit.** Each adjustment is a closed-form function of
stored features and the consistency step is linear (`f(x)=0.9x+0.05` ⟹
`f(a+b)=f(a)+0.9b`), so every layer is removed algebraically from the stored
`p_win`. Nothing re-estimated, nothing re-fetched, no lookahead possible.
Paired bootstrap, 3,000 resamples, deterministic seed.

**A negative delta means removing the layer HURT — i.e. it carried signal.**

| stat | n | resolution (full) | −opponent | −home/away | −consistency | **−ALL THREE** |
|---|---|---|---|---|---|---|
| hits | 578 | 0.1964 | −0.0049 | −0.0006 | −0.0004 | **−0.0059** [−0.0168,+0.0056] |
| total_bases | 284 | 0.2370 | −0.0056 | +0.0042 | +0.0002 | **−0.0015** [−0.0139,+0.0115] |
| rbi | 273 | 0.4986 | +0.0059 | **+0.0053** [+0.0002,+0.0103] | +0.0005 | **+0.0106** [−0.0003,+0.0220] |
| runs | 115 | 0.4062 | +0.0060 | +0.0087 | −0.0006 | **+0.0130** [−0.0106,+0.0366] |
| walks | 66 | 0.4776 | −0.0035 | +0.0055 | +0.0018 | **+0.0008** [−0.0268,+0.0281] |

**Removing all three adjustments changes resolution by nothing on every stat, and
on rbi/runs it IMPROVES it.** Exactly one ablation anywhere has a CI excluding
zero — rbi home/away, and its sign says removing it makes the model **better**.

**So ~100% of the champion's resolution is `base + recency`: how often this
player has cleared THIS number lately.** That is the whole model. Everything else
is decoration.

### A correction to how we read last session's scoreboard

Pooled across stats the champion resolves **0.46**; per stat it is **0.196
(hits)** to **0.499 (rbi)**. Pooling stats with different base rates *inflates*
correlation, because p_win varies across stats in the same direction as the true
base rate. **0.46 is a pooling artifact and should not be quoted as the
champion's resolution.** The paired *differences* in the scoreboard remain valid
(champion and challenger were pooled identically); only the absolute level was
inflated.

## 3. Do the challengers have it, or dilute it? (STEP 3)

| challenger | uses the base-frequency signal? | verdict |
|---|---|---|
| **proj-v1.1 ladder** | **No — it replaces it.** Fits a rate + NB distribution instead of counting frequency at THIS line | **DILUTING.** Measured reliably worse (−0.0301, CI excludes 0). It discards the one thing that works in favour of a lossier route to the same question |
| **hits-v1** | No — same substitution, binomial instead of NB | **DILUTING.** Refuted (−0.022, CI excludes 0) |
| **arch-v1 · environment** | Adds park/weather, which the champion ignores | **DILUTING.** n=871, −0.0028, CI includes 0 — movement without information |
| **arch-v1 · opportunity** | Uses `opportunity_drift` — **the one feature with repeated residual signal** | **HAS THE FEATURE, WRONG IMPLEMENTATION** (see §4) |
| **arch-v1 · matchup** | — | STILL PENDING (rows settle after ET midnight) |
| **contact-v1** | Statcast contact quality; not in the retained vector | No evidence either way (n=1,055, CI includes 0) |

## 4. The missing-feature test — one real lead, already in our hands

Correlation of each **unused** feature with the champion's residual (`won −
p_win`), per stat, bootstrap CI.

**Multiple-comparisons discipline first:** 14 features × 5 stats = 70 tests at
α=.05, so **3–4 CI-excludes-zero results are expected by chance.** Six appeared.
A single hit is noise. **Only a feature that repeats across independent stats is
evidence** — and exactly one does:

| feature | hits | total_bases | walks |
|---|---|---|---|
| **`opportunity_drift`** | **+0.156** [+0.007,+0.292] | **+0.145** [+0.005,+0.278] | −0.261 [−0.463,−0.017] |
| `recent_ab_per_game` (same quantity) | +0.052 | **+0.136** [+0.002,+0.272] | −0.157 |

`opportunity_drift` = recent at-bats ÷ season at-bats-per-game. It is the one
axis a frequency counter is **structurally blind to**: `base` knows how often he
cleared the number, not that he has moved from 8th in the order to leadoff, or
back from injury on a bench role. Sign flips on walks (n=47, and walks scale with
plate appearances differently) — so this is stat-specific, which is doctrine-
consistent, not a contradiction.

**But we already compute it, retain it, and built an axis on it — and that axis
extracts nothing** (opportunity axis: n=539, delta +0.0001, CI [−0.0091,+0.0090]).
So this is **not** "go get a new feature." It is **"the feature has signal and our
implementation of it is wrong"** — arch-v1 applies it as a small multiplicative
nudge to `p_win`, which is not how you use an opportunity term. Opportunity should
scale the *rate*, before the frequency question is asked.

Weather on total_bases (`wind_mph` −0.164, `precip` −0.154) appears on **one stat
only** and sits inside the expected false-positive count. Recorded as a
non-lead unless it repeats.

## 5. Archetype on trial (STEP 4 item) — NO EVIDENCE, and the test is underpowered

The champion reads **no archetype feature at all**, so archetype cannot be ablated
out of it. The fair test is whether archetype explains what the champion gets
*wrong*: if an archetype's rows are systematically mispriced, archetype carries
prop signal we're missing.

| stat | archetype | n | mean residual | CI95 | mispriced? |
|---|---|---|---|---|---|
| hits | BOMBER | 195 | −0.038 | [−0.105, +0.030] | no |
| hits | GHOST | 85 | −0.076 | [−0.184, +0.033] | no |
| total_bases | BOMBER | 95 | −0.046 | [−0.141, +0.058] | no |
| total_bases | GHOST | 46 | −0.041 | [−0.187, +0.109] | no |
| rbi | BOMBER | 88 | −0.068 | [−0.155, +0.019] | no |
| runs | BOMBER | 41 | −0.012 | [−0.150, +0.121] | no |

**Verdict: (a) no-signal is UNPROVEN and (b) wrong-implementation is UNPROVEN —
the test cannot separate them yet.** Only **2 of 41 archetypes** (BOMBER, GHOST)
reach n≥40 settled rows. That is not "archetypes don't work"; it is "we have not
measured them." Distinguishing (a) from (b) needs archetype coverage across more
than two labels. **Do not act on archetype in either direction on this evidence.**

Worth noting: every archetype's mean residual is **negative**, which is not an
archetype effect — it is the global over-prediction in §6.

## 6. THE BIGGEST FINDING — the clamp, not a feature

**358 of 1,741 settled props (20.6%) sit ON the clamp boundary** (353 at the
floor). Within that fifth of the book the model emits a **constant**, so it cannot
rank those props at all — resolution there is zero by construction.

And the constant is hiding two *opposite* failures at once:

| stat · side | n | model says | actually wins | miscalibration |
|---|---|---|---|---|
| home_runs · under | 222 | 0.900 | **0.995** | **−9.5 pts** (badly UNDER-confident) |
| hits · under | 27 | 0.900 | **0.519** | **+38.1 pts** (a coin flip sold as 90%) |
| total_bases · under | 12 | 0.900 | 0.583 | +31.7 pts |
| rbi · over | 35 | 0.100 | 0.229 | −12.9 pts |

**The same output `0.900` covers true probabilities from 52% to 99.5%.** A
near-lock and a coin flip are indistinguishable in the product. `PROB_CEIL = 0.95`
also makes it *impossible* to express the 99.5% case honestly.

Overall calibration, all settled MLB rows:

| stat | n | mean p_win | actual | over-prediction |
|---|---|---|---|---|
| **ALL POOLED** | 1,741 | 0.585 | 0.550 | **+3.5 pts** |
| total_bases | 297 | 0.534 | 0.458 | **+7.6** |
| hits | 589 | 0.606 | 0.562 | +4.4 |
| rbi | 320 | 0.399 | 0.356 | +4.3 |
| walks | 83 | 0.546 | 0.506 | +4.0 |
| runs | 124 | 0.596 | 0.605 | −0.9 (well calibrated) |
| home_runs | 228 | 0.899 | 0.996 | **−9.6** |

Per the product doctrine, calibration is **half** the success criterion — "does
60% mean 60%?" Right now 58.5% means 55.0%, and on total_bases 53.4% means 45.8%.
**This is fixable with no new data at all.**

## 7. VERDICT PER STAT (STEP 4)

| stat | n | resolution | verdict |
|---|---|---|---|
| **hits** | 578 | 0.196 | **DILUTION + CALIBRATION.** Adjustments contribute nothing; one real lead (`opportunity_drift`) that we already compute and implement wrongly; +4.4pt over-prediction |
| **total_bases** | 284 | 0.237 | **DILUTION + CALIBRATION (worst).** Same lead; +7.6pt over-prediction |
| **rbi** | 273 | 0.499 | **DILUTION.** Removing home/away *improves* it (CI excludes zero). Prune |
| **runs** | 115 | 0.406 | **AT CEILING.** Adjustments neutral-to-harmful, no residual leads. The model is good here |
| **walks** | 66 | 0.478 | **AT CEILING** (underpowered, n=66) |
| **home_runs** | 228 | n/a — entirely clamped | **CALIBRATION.** −9.6pts, structurally uncorrectable while `PROB_CEIL=0.95` |

**AT CEILING is a real result here, not a shrug**: on runs and walks the champion
already resolves ~0.41–0.48 and nothing we compute explains its residual.

## 8. What this says about the next order

The convergent evidence was read correctly — the problem *is* inputs, not shape.
But the decomposition sharpens it, and the ranking is not what we assumed:

1. **The clamp + calibration (biggest, cheapest, no new data).** 20.6% of the
   book pinned to a constant, a documented +3.5pt global over-prediction, and one
   number covering 52%–99.5%. This costs both halves of the success criterion.
2. **Opportunity, implemented properly** — the one feature with repeated
   cross-stat residual signal. Not a new feature: a correct use of one we have.
   Scale the rate, don't nudge the probability.
3. **Stop diluting** — the ladder and the environment axis add movement with no
   information, and are measurably worse or flat.
4. **Archetype: measure before judging.** 2 of 41 labels have testable n.

**Explicitly NOT recommended:** another projection variant. That is the sixth
thing, and this diagnosis is why it would fail — the champion's edge is asking
the frequency question at the traded line, and every challenger so far has
replaced that question rather than improved its inputs.

## 9. Incidental finding, flagged not fixed

**`model_snapshots.outcome` is NULL on all 22,032 rows.** The retention table
built expressly so "a different model can be replayed against the same
conditions" stores features but was **never settled**, so it cannot answer the
question it exists for. This diagnosis worked around it by joining outcomes from
`ledger_entries` on (player_key, stat, line, side, game_date). Settling retention
would make every future ablation a single-table query — and would make refusals
(which the ledger drops) measurable for the first time.

## 10. Reproduce

`SUPABASE_URL=... node scripts/champion-ablation.js`