vyndr/specs/champion-input-diagnosis.md

DECOMPOSING THE CHAMPION — where its edge actually comes from

Read-only diagnosis, 2026-08-03. Nothing built, nothing touched. Run on the repaired settled set (n=1,741 after the settlement outage fix), not the frozen pre-fix set.

VERDICT, one line: the champion's entire edge is a hit-rate counter, its three adjustment layers contribute nothing (two are mildly harmful), and the largest recoverable loss is not a missing feature — it is the [0.10, 0.95] clamp, which pins 20.6% of settled props to a constant and hides outcomes ranging from 52% to 99.5% behind the same number 0.900.

---

1. What the champion actually uses (STEP 1)

src/services/intelligence/probabilityEstimator.js is five lines of arithmetic:

base      = empirical frequency of (stat > THIS line) over the game log
weighted  = 0.6·base + 0.4·(same frequency over the last 5 games)
p         = weighted + oppAdj(±0.03) + homeAdj(±0.015)
if cv > 0.40:   p = 0.9·p + 0.05                  (volatile → pull toward 0.50)
p_over    = clamp(p, 0.10, 0.95)
p_win     = side === 'under' ? 1 − p_over : p_over

It reads exactly three features: opp_rank_stat, home_away, and l10_stddev/l20_avg (for cv). featureCache computes and retains a dozen more — park_h/hr/r, weather_temp_f/wind_mph/precip, rest_days, opportunity_drift, ab_per_game, recent_ab_per_game, l5/l10/l20_avg, game_count_in_7d — and p_win reads none of them.

2. Per-stat ablation (STEP 2)

Exact and analytic, not a refit. Each adjustment is a closed-form function of stored features and the consistency step is linear (f(x)=0.9x+0.05 ⟹ f(a+b)=f(a)+0.9b), so every layer is removed algebraically from the stored p_win. Nothing re-estimated, nothing re-fetched, no lookahead possible. Paired bootstrap, 3,000 resamples, deterministic seed.

A negative delta means removing the layer HURT — i.e. it carried signal.

stat	n	resolution (full)	−opponent	−home/away	−consistency	−ALL THREE
hits	578	0.1964	−0.0049	−0.0006	−0.0004	−0.0059 [−0.0168,+0.0056]
total_bases	284	0.2370	−0.0056	+0.0042	+0.0002	−0.0015 [−0.0139,+0.0115]
rbi	273	0.4986	+0.0059	+0.0053 [+0.0002,+0.0103]	+0.0005	+0.0106 [−0.0003,+0.0220]
runs	115	0.4062	+0.0060	+0.0087	−0.0006	+0.0130 [−0.0106,+0.0366]
walks	66	0.4776	−0.0035	+0.0055	+0.0018	+0.0008 [−0.0268,+0.0281]

Removing all three adjustments changes resolution by nothing on every stat, and on rbi/runs it IMPROVES it. Exactly one ablation anywhere has a CI excluding zero — rbi home/away, and its sign says removing it makes the model better.

So ~100% of the champion's resolution is base + recency: how often this player has cleared THIS number lately. That is the whole model. Everything else is decoration.

A correction to how we read last session's scoreboard

Pooled across stats the champion resolves 0.46; per stat it is 0.196 (hits) to 0.499 (rbi). Pooling stats with different base rates inflates correlation, because p_win varies across stats in the same direction as the true base rate. 0.46 is a pooling artifact and should not be quoted as the champion's resolution. The paired differences in the scoreboard remain valid (champion and challenger were pooled identically); only the absolute level was inflated.

3. Do the challengers have it, or dilute it? (STEP 3)

challenger	uses the base-frequency signal?	verdict
proj-v1.1 ladder	No — it replaces it. Fits a rate + NB distribution instead of counting frequency at THIS line	DILUTING. Measured reliably worse (−0.0301, CI excludes 0). It discards the one thing that works in favour of a lossier route to the same question
hits-v1	No — same substitution, binomial instead of NB	DILUTING. Refuted (−0.022, CI excludes 0)
arch-v1 · environment	Adds park/weather, which the champion ignores	DILUTING. n=871, −0.0028, CI includes 0 — movement without information
arch-v1 · opportunity	Uses opportunity_drift — the one feature with repeated residual signal	HAS THE FEATURE, WRONG IMPLEMENTATION (see §4)
arch-v1 · matchup	—	STILL PENDING (rows settle after ET midnight)
contact-v1	Statcast contact quality; not in the retained vector	No evidence either way (n=1,055, CI includes 0)

4. The missing-feature test — one real lead, already in our hands

Correlation of each unused feature with the champion's residual (won − p_win), per stat, bootstrap CI.

Multiple-comparisons discipline first: 14 features × 5 stats = 70 tests at α=.05, so 3–4 CI-excludes-zero results are expected by chance. Six appeared. A single hit is noise. Only a feature that repeats across independent stats is evidence — and exactly one does:

feature	hits	total_bases	walks
opportunity_drift	+0.156 [+0.007,+0.292]	+0.145 [+0.005,+0.278]	−0.261 [−0.463,−0.017]
recent_ab_per_game (same quantity)	+0.052	+0.136 [+0.002,+0.272]	−0.157

opportunity_drift = recent at-bats ÷ season at-bats-per-game. It is the one axis a frequency counter is structurally blind to: base knows how often he cleared the number, not that he has moved from 8th in the order to leadoff, or back from injury on a bench role. Sign flips on walks (n=47, and walks scale with plate appearances differently) — so this is stat-specific, which is doctrine- consistent, not a contradiction.

But we already compute it, retain it, and built an axis on it — and that axis extracts nothing (opportunity axis: n=539, delta +0.0001, CI [−0.0091,+0.0090]). So this is not "go get a new feature." It is "the feature has signal and our implementation of it is wrong" — arch-v1 applies it as a small multiplicative nudge to p_win, which is not how you use an opportunity term. Opportunity should scale the rate, before the frequency question is asked.

Weather on total_bases (wind_mph −0.164, precip −0.154) appears on one stat only and sits inside the expected false-positive count. Recorded as a non-lead unless it repeats.

5. Archetype on trial (STEP 4 item) — NO EVIDENCE, and the test is underpowered

The champion reads no archetype feature at all, so archetype cannot be ablated out of it. The fair test is whether archetype explains what the champion gets wrong: if an archetype's rows are systematically mispriced, archetype carries prop signal we're missing.

stat	archetype	n	mean residual	CI95	mispriced?
hits	BOMBER	195	−0.038	[−0.105, +0.030]	no
hits	GHOST	85	−0.076	[−0.184, +0.033]	no
total_bases	BOMBER	95	−0.046	[−0.141, +0.058]	no
total_bases	GHOST	46	−0.041	[−0.187, +0.109]	no
rbi	BOMBER	88	−0.068	[−0.155, +0.019]	no
runs	BOMBER	41	−0.012	[−0.150, +0.121]	no

Verdict: (a) no-signal is UNPROVEN and (b) wrong-implementation is UNPROVEN — the test cannot separate them yet. Only 2 of 41 archetypes (BOMBER, GHOST) reach n≥40 settled rows. That is not "archetypes don't work"; it is "we have not measured them." Distinguishing (a) from (b) needs archetype coverage across more than two labels. Do not act on archetype in either direction on this evidence.

Worth noting: every archetype's mean residual is negative, which is not an archetype effect — it is the global over-prediction in §6.

6. THE BIGGEST FINDING — the clamp, not a feature

358 of 1,741 settled props (20.6%) sit ON the clamp boundary (353 at the floor). Within that fifth of the book the model emits a constant, so it cannot rank those props at all — resolution there is zero by construction.

And the constant is hiding two opposite failures at once:

stat · side	n	model says	actually wins	miscalibration
home_runs · under	222	0.900	0.995	−9.5 pts (badly UNDER-confident)
hits · under	27	0.900	0.519	+38.1 pts (a coin flip sold as 90%)
total_bases · under	12	0.900	0.583	+31.7 pts
rbi · over	35	0.100	0.229	−12.9 pts

The same output 0.900 covers true probabilities from 52% to 99.5%. A near-lock and a coin flip are indistinguishable in the product. PROB_CEIL = 0.95 also makes it impossible to express the 99.5% case honestly.

Overall calibration, all settled MLB rows:

stat	n	mean p_win	actual	over-prediction
ALL POOLED	1,741	0.585	0.550	+3.5 pts
total_bases	297	0.534	0.458	+7.6
hits	589	0.606	0.562	+4.4
rbi	320	0.399	0.356	+4.3
walks	83	0.546	0.506	+4.0
runs	124	0.596	0.605	−0.9 (well calibrated)
home_runs	228	0.899	0.996	−9.6

Per the product doctrine, calibration is half the success criterion — "does 60% mean 60%?" Right now 58.5% means 55.0%, and on total_bases 53.4% means 45.8%. This is fixable with no new data at all.

7. VERDICT PER STAT (STEP 4)

stat	n	resolution	verdict
hits	578	0.196	DILUTION + CALIBRATION. Adjustments contribute nothing; one real lead (opportunity_drift) that we already compute and implement wrongly; +4.4pt over-prediction
total_bases	284	0.237	DILUTION + CALIBRATION (worst). Same lead; +7.6pt over-prediction
rbi	273	0.499	DILUTION. Removing home/away improves it (CI excludes zero). Prune
runs	115	0.406	AT CEILING. Adjustments neutral-to-harmful, no residual leads. The model is good here
walks	66	0.478	AT CEILING (underpowered, n=66)
home_runs	228	n/a — entirely clamped	CALIBRATION. −9.6pts, structurally uncorrectable while PROB_CEIL=0.95

AT CEILING is a real result here, not a shrug: on runs and walks the champion already resolves ~0.41–0.48 and nothing we compute explains its residual.

8. What this says about the next order

The convergent evidence was read correctly — the problem is inputs, not shape. But the decomposition sharpens it, and the ranking is not what we assumed:

1. The clamp + calibration (biggest, cheapest, no new data). 20.6% of the book pinned to a constant, a documented +3.5pt global over-prediction, and one number covering 52%–99.5%. This costs both halves of the success criterion.
2. Opportunity, implemented properly — the one feature with repeated cross-stat residual signal. Not a new feature: a correct use of one we have. Scale the rate, don't nudge the probability.
3. Stop diluting — the ladder and the environment axis add movement with no information, and are measurably worse or flat.
4. Archetype: measure before judging. 2 of 41 labels have testable n.

Explicitly NOT recommended: another projection variant. That is the sixth thing, and this diagnosis is why it would fail — the champion's edge is asking the frequency question at the traded line, and every challenger so far has replaced that question rather than improved its inputs.

9. Incidental finding, flagged not fixed

model_snapshots.outcome is NULL on all 22,032 rows. The retention table built expressly so "a different model can be replayed against the same conditions" stores features but was never settled, so it cannot answer the question it exists for. This diagnosis worked around it by joining outcomes from ledger_entries on (player_key, stat, line, side, game_date). Settling retention would make every future ablation a single-table query — and would make refusals (which the ledger drops) measurable for the first time.

10. Reproduce

SUPABASE_URL=... node scripts/champion-ablation.js