vyndr/specs/audit-data/grade-collapse.md

GRADE COLLAPSE + DEAD PROBABILITY LAYER — diagnosis

REPORT ONLY. No grade logic, thresholds, or engine code changed (Kev's instruction). Two findings. The second one is bigger than the question I was asked.

Data: live Supabase ledger_entries (604 rows, all users) + live prod API api.vyndr.app, 2026-07-19 ~22:05 UTC.

---

FINDING 1 — THE COLLAPSE IS REAL, LIVE, AND STRUCTURAL

Not a thin-slate artifact. Across 604 ledger rows and both sports:

• 2 distinct grades ever emitted: B and C. Zero A+, A, A−, B+, B−, C−, D, F.
• 9 distinct confidence values ever emitted: 63, 57, 55, 52, 47, 45, 35, 25, 20.
• Confidence ceiling = 63. It has never exceeded 63 in the recorded era.

Still true today (Jul 18 + 19, post every fix, both sports): 4 confidence values (63/57/52/47), 2 grades.

Sport	Grade	n	conf min	conf max
mlb	B	276	45	63
mlb	C	107	20	52
wnba	B	130	45	63
wnba	C	91	35	52

Confidence does NOT determine the letter

conf	grade	n
63	B	54
57	B	175
55	B	29
52	C	100
47	C	14
45	B	148
35	C	80

conf 45 → B, but conf 47 and 52 → C. The mapping is non-monotonic, so the surfaced confidence is not the quantity the letter was derived from. This confirms the mlb-grade-degradation.md "grade↔confidence mismatch" as a display artifact: two different quantities are being shown as if one explains the other.

(At conf 45→B the avg edge is 103; at conf 52→C it is 49 — so the letter tracks the engine composite/edge, not the displayed confidence.)

Collateral: the edge scale is still broken and still live

• 311 of 604 rows (51.5 %) have |edge| > 40 — the frontend's EDGE_BOARD_SANE_MAX, i.e. over half the board's edge is nulled at render.
• 39 rows have |edge| > 100 — impossible as a percentage. Worst: 620.
• Live today: edges of 140, 180, 220 on Jul 18–19 rows.

U-deg's projection == 0 leak IS closed (0 since 07-18). The edge_pct scale is not — it remains open and is now quantified.

---

FINDING 2 — 🔴 THE ENTIRE PROBABILITY LAYER IS DEAD IN PRODUCTION

Found while fingerprinting Arc 1 (U-fp). This is the headline.

Live fingerprint, GET /api/snapshot/mlb, 8 graded props

Field	Present
projection, confidence, book_odds, fair_odds, takeable, devig_method, alt_lines	8 / 8
p_win	0 / 8
kelly	0 / 8
ev_pct	0 / 8
model_odds	0 / 8
value	0 / 8

Control: alt_lines is present 8/8 and is Desk-gated in tierGating.js:55, which proves the payload is not being tier-stripped. These fields are genuinely never computed — not hidden.

Root cause — a one-line sport gate, and an S46 fix that was only half-applied

analyzeViaEngine1.js:509 feeds the estimator from meta.gameLogs:

const est = estimateProbability({ gameLogs: meta.gameLogs, line: prop.line, ... });

meta.gameLogs comes from computeFeatures.js:173-181 → gameLogService.getGameLogs. And gameLogService.js:21-26:

function pythonPath(sport) {
  switch (sport) {
    case 'nba':  return '/stats/last-n';
    case 'wnba': return '/wnba/stats/last-n';
    default:     return null;      // ← MLB exits here
  }
}

with getGameLogs line 31: if (!path) return null;

So:

• MLB — returns null by construction. Never had game logs on this path.
• NBA/WNBA — hits the Python stats service, which is offline in prod (documented in CLAUDE.md; degrades to null).

⇒ meta.gameLogs is [] for every sport in production ⇒ estimateProbability returns {p_over: null, reason:'insufficient_data'} (probabilityEstimator.js:55-57) ⇒ pWin is null ⇒ every field guarded by if (pWin != null) is skipped: p_win, kelly, model_odds, ev_pct, value.

This is the S46 bug, second location, never fixed. CLAUDE.md records that gameLogService.getGameLogs being NBA/WNBA-only starved MLB, and that the fix was an MLB branch in featureCache.gameLogFeatures. That fixed the feature path — which is why projection, confidence, and grades still work. The estimator path was never given the same branch, so it has been silently dead the whole time.

What this actually breaks

1. EV — the Model Train's entire ranking signal — does not exist in production. Arc 1 shipped ev_pct and it has never once been computed on a live prop.
2. Hero v2 is non-functional. pickHeroProp requires a finite ev_pct (heroPropService.js:84), so the EV loop matches nothing and always falls through to the "most recent graded read" fallback. Live proof: /api/hero-prop returns "is_recent": true — the fallback path, every time. The hero has not been an EV pick since the day it shipped.
3. Quarter-Kelly is dead — same pWin dependency (analyzeViaEngine1.js:516-520). This is a promise-audit issue: Kelly sizing is sold on the pricing page and PROMISE-AUDIT.md lists it as BUILT. It is built and never runs.
4. The "value triplet" is a duet live — book_odds + fair_odds render; model_odds is always absent.
5. value is never true, so the VALUE marker can never light up.

Why this reframes the whole train

• C-led would persist a column of nulls. Do not build EV persistence until EV exists.
• G-a's EV_FLEX_THRESHOLD would gate on a permanently-null value. With EV_FLEX_ENFORCE=0 (Kev's ruling) this is harmless today — but had we enforced it, the flex band would have been cut to zero, because ev_pct >= 4 can never be true. The ruling to ship it disabled accidentally prevented an outage.
• S-b (rank board on EV) would rank on nulls.

---

RELATIONSHIP BETWEEN THE TWO FINDINGS

They are adjacent, not identical, and both trace to the same missing input:

• The dead estimator explains why no probability-derived output exists (EV, Kelly, model_odds, p_win).
• It does not by itself explain the B/C letter collapse, because the letter comes from engine1's rule-based composite over the feature vector, which is alive.
• But they share a root: the model is running on a partial input set. One of its two probability inputs (the empirical quantile distribution over real game logs) is absent for 100 % of props, so whatever spread the composite was designed to produce is being generated from the surviving features only.

The 9-discrete-confidence-values pattern is a small set of additive rule hits — a scorer landing on a lattice rather than a continuum. Mechanism now traced in full below.

---

FINDING 3 — THE MECHANISM: A IS MATHEMATICALLY UNREACHABLE

Every claim here was verified directly against the source.

The grade is an integer index, not a score

engine1.js:16-17, 158-163:

const GRADE_SCALE = ['F','D','C-','C','C+','B-','B','B+','A-','A','A+'];
const NEUTRAL_INDEX = 3;  // 'C'
...
let idx = NEUTRAL_INDEX;
for (const f of factors) idx += f.delta;      // flat sum of ±1.0 / ±0.5
idx = clampIndex(Math.round(idx));

grade_thresholds.json is not an input mapper in the JS path. Nothing compares a probability to those cutoffs. engine1.js:29-36 reads the table backwards — it takes the letter the index already produced and looks up that band's midpoint to manufacture a confidence number.

So confidence is a cosmetic re-encoding of the letter. It carries zero information beyond the letter and by construction can never disagree with it. There is no data-sufficiency penalty in the live path — the one CLAUDE.md describes lives in mlbGrader.js:50-69, which is DEAD CODE. (computeFeatures.js:21 still carries a stale comment claiming the adapter downgrades confidence; it does not.)

Six of thirteen factors are wired to features nothing populates

Dead factor	Δ	Why it never fires
weak/top_opponent_defense	±1.0	needs opp_rank_stat ← team_stats:{sport}:{abbr} ← refreshTeamStats has ZERO production callers (verified: only its own export + tests)
consistency_elite/boom_bust	±1.0	MLB consistency logs come from the same dead gameLogService path as Finding 2
opp_starters_out	+1.0/+0.5	featureCache.js:280: if (!teamId) return out; — computeFeatures never passes teamId
playoff factors	±0.5	season_type never set
heavy_workload_7d	−0.5	game_count_in_7d never set
ref_* / coach_*	±0.5	NBA-flavored caches, absent for MLB

computeFeatures.js:234-236 builds gameContext as { home_away } and nothing else.

The arithmetic

idx = clamp(round(3 + Σδ)). Live-firing factors for MLB reduce to: l5_* (±1.0), l20_* (+1.0 only — verified, BOTH branches are delta: 1.0, there is no negative L20 contribution), home_game (+0.5), rest (±0.5), trap_composite_high (−1.0).

4-letter	needs Σδ	live reachable?
A (A−/A/A+)	≥ +4.5	NO — live max is +3.0 (+2.0 on a back-to-back, and MLB rest_days is 0 most days)
B	+1.5 … +4.49	yes
C	−1.5 … +1.49	yes
D	≤ −1.51	NO — live min is −1.5, and Math.round(1.5) = 2 → C−. Misses by one rounding tick.
F	≤ −2.51	NO

An A is short by at least 1.5 index steps — and the ≥1.5 of deltas that would close the gap (opp_rank_stat ±1.0, consistency ±1.0, injury +1.0) are exactly the permanently- null features. The reachable index band is 2…6 = {C−, C, C+, B−, B}, which gradeAdapter.FOUR_LETTER_MAP (gradeAdapter.js:31-37, a 3→1 collapse) renders as exactly {C, B}. That is the observed output, derived from first principles.

Confidence corroborates exactly: reachable letters carry {42, 47, 52, 57, 63}. Live today we observe precisely {47, 52, 57, 63} — C− (42) is absent because gradeSlateService.js:76 keeps the higher-confidence side of each prop, truncating the bottom. The older values in the ledger (55, 45, 35, 25, 20) are from the pre-888d103 hand-rolled table {10,15,20,25,35,45,55,65,80,90,100} — the ledger is append-only, so it contains both eras.

Relationship to mlb-grade-degradation.md

Shared table, different bug — and its "fix" made this collapse invisible. That audit redefined confidence as the band midpoint so the letter round-trips through the table. The resulting "25/25 agreement" is a tautology, not a validation: confidence is derived from the letter, so it would report 25/25 even if every grade were wrong. That audit only examined the output encoding. This collapse is one layer upstream, on the input side — whether computeFactors has enough live features to move the index at all.

---

RECOMMENDATION (no code changed pending Kev's call)

Re-sequence: fix the dead estimator FIRST — before G-a, before C-led.

Rationale: it is the cheapest fix on the board (an MLB branch in the estimator's log source, mirroring the one already written for featureCache), and it simultaneously restores EV, Kelly, model_odds, the VALUE flag, and hero v2. Every other Arc 2-5 item is downstream of it. Building the gate, the persistence layer, or the board ranking on a null signal is building on nothing.

Suggested order:

1. Revive the probability layer (MLB branch + a real NBA/WNBA fallback, since Python is offline). Fingerprint that p_win/ev_pct appear live.
2. Then C-led — persist EV that now has values.
3. Then G-a — with the flex band still disabled per the standing ruling.
4. Then the grade range — now diagnosed (Finding 3), and it is NOT primarily a consequence of step 1. It needs its own decision, because there are two very different fixes and picking wrong bakes in a lie:
   • (a) Feed the starving factors. Call refreshTeamStats (nothing does), pass teamId/season_type/game_count_in_7d through gameContext, give safeGetConsistency the same MLB branch as step 1. This restores ±3.0 of range and makes A/D reachable on merit.
   • (b) Re-scale the index/thresholds so the current narrow spread spans more letters. This is the tempting one and it is the wrong one — it would mint A's without adding a single bit of information, and every "A" would be a relabelled B. It converts a visible limitation into an invisible lie. Recommend (a), explicitly reject (b). If (a) proves infeasible, the honest fallback is to keep the two-letter output and stop advertising a scale we don't produce — not to stretch the scale.

Copy consequence, either way: "A-RATED" appears on public surfaces and AccuracyBadge for a grade the engine has never emitted. Until (a) lands, that copy is unsupported.

Open question for Kev: NBA/WNBA have no free game-log source on this path with Python down. espnStatsAdapter.getPlayerGameLog (Wave 0) already solves exactly this for featureCache — reusing it here is the obvious candidate, and costs no quota.

---

RESOLUTION — Session 63 (shipped)

Kev's ruling: (a) fix on merit, never (b) rescale. Rescaling would mint A's without adding information — a relabelled B marketed as an A, corrupting an append-only ledger permanently. That option is permanently rejected.

What shipped

Fix	File	Effect
Normalized per-game rows for ALL sports	featureCache.getStatRows	Revives p_win → ev_pct, kelly, model_odds, value, hero v2. Also feeds consistency.
Rows wired into the grade path	computeFeatures.safeGetConsistency	One fetch per prop, shared by 3 starving consumers
refreshTeamStats called in production	snapshotService.runSnapshot	opp_rank_stat populated → the ±1.0 opponent factor can fire (it had ZERO callers)
game_count_in_7d derived from real logs	computeFeatures gameContext	heavy_workload_7d (−0.5) can fire
L20 symmetry	engine1.computeFactors	NEW l20_contradicts_* −1.0. There was no negative L20 path at all — a structural reason D was unreachable
Consistency CV floor	consistencyScore	See calibration finding below
confidence_basis: 'grade_band'	gradeAdapter.toLegacyShape	Confidence labelled as derived, not a probability
Dead mlbGrader.js removed	—	Referenced only by its own test. Described-but-dead penalty eliminated

Deliberately NOT wired (would have been dead code dressed as a fix, documented inline): teamId (no team_id column exists; getFeatures reads it top-level not off gameContext; and the factor needs a starter-id list that doesn't exist) and season_type (engine1 gates playoff factors on season_type >= 2, but ESPN's 2 means REGULAR season — threading it raw would fire "veteran_in_playoffs" in July).

🔶 CALIBRATION FINDING — consistency was NBA-tuned and would have flooded boom_bust

Reviving consistency exposed a latent bug. The CV thresholds (cv >= 0.5 → boom_bust) were calibrated for NBA points (mean ~20). For a Poisson-ish counting stat, cv ≈ 1/√mean, so any stat with mean < 4 forces cv > 0.5 — it classifies boom_bust regardless of actual behaviour. Verified on real logs:

• Alonso hits [0,0,0,1,2,1,0,1,1,0] → mean 0.60, cv 1.17 → boom_bust
• Henderson hits [1,0,0,3,1,1,0,0,1,0] → mean 0.70, cv 1.36 → boom_bust

First verification run confirmed it: 8/8 MLB props classified boom_bust, a blanket −1.0 that dropped the whole board to C. That is a systematic downgrade masquerading as a signal — the mirror image of the "flooding A's" failure Kev warned about.

Guard shipped: MIN_MEAN_FOR_CV = 4 (env CONSISTENCY_MIN_MEAN). Below it, consistency returns unknown (no factor) with reason: 'low_mean_cv_unreliable'. Absent beats wrong. Consequence: MLB low-count stats still get no consistency factor — honest, not fixed. The correct long-term fix is an index-of-dispersion (variance/mean vs the Poisson baseline) classifier, which is scale-free. Tracked as an open item; it is a modelling change needing its own validation.

VERIFICATION ON MERIT — real props, real logs, real engine

scripts/verify-grade-range.js replays live-board props through the repaired engine using free feeds (statsapi/ESPN). Caveat stated up front: opp_rank_stat needs the Redis team-stats cache that only production populates, so these local runs OMIT a ±1.0 factor and therefore UNDERSTATE the restored range.

WNBA — 25 real props

	BEFORE (live board)	AFTER (repaired)
A	0	0
B	17 (68 %)	8 (32 %)
C	8 (32 %)	16 (64 %)
D	0	1 (4 %)

11-step spread: C 6 · C+ 10 · B− 8 · D 1 — five distinct steps where there were two. Revived signals: p_win 25/25 (was 0), rows 25/25, consistency known 15/25 (the floor correctly abstains on low-mean assists/rebounds).

The D is earned, not manufactured: Angel Reese assists over 2.5, p_win 0.365 — the model gives it 36.5 % and says so.

MLB — 8 real props: B 5 / C 3, p_win 8/8 (was 0). No A or D on a thin 8-prop late-night board of near-identical 0.5-hits props.

Reading it honestly:

• D emits on merit. ✅
• A did not emit locally — expected: A needs Σδ ≥ +4.5 and the local ceiling is +3.0 without opp_rank_stat. Structural reachability is proven arithmetically and locked in tests/unit/gradeRangeRestore.test.js; empirical A emission requires production and is the outstanding fingerprint.
• Nothing flooded. Grades got harder, not easier — B fell 68 % → 32 %. The B→C movers are driven by the new L20 negative branch: props whose season baseline contradicts the graded side no longer get a free pass. That is the intended correction.

🔴 MARKETING HOLD — A-rated copy is UNSUPPORTED until A verifiably emits

Confirmed the honest fallbacks are what render today:

• /api/ledger/accuracy returns buckets B and C only — no A bucket. So AccuracyBadge's aRated sample is 0, below minSample, and it falls through to "MODEL · 63% HIT". No fabricated A-RATED is displayed.
• TopSignals self-hides when there are no A-rated grades.

Nothing fabricated is shipping — but the copy describes a grade the engine has never emitted. Do not promote "A-RATED" in marketing, and do not build new surfaces on an A bucket, until a production fingerprint shows real A grades. Lift this hold only against live data.

---

✅ PRODUCTION FINGERPRINT — 2026-07-19 ~22:50 UTC

POST /api/analyze/prop (grades on demand, so it exercises the repaired path immediately rather than waiting for the snapshot cron):

Gunnar Henderson hits o0.5 @ -140 (fanduel)
  grade            : B
  confidence       : 57
  confidence_basis : grade_band     <- NEW (truth label)
  p_win            : 0.523          <- WAS ABSENT on 100% of grades
  ev_pct           : -10.4          <- WAS ABSENT
  model_odds       : -109           <- WAS ABSENT (triplet was a duet)
  value            : false          <- WAS ABSENT
  takeable         : true
  book_odds        : -140
  fair_odds        : -125

The value triplet is finally whole: book −140 · vig-free −125 · model −109. And it tells the truth — the model gives 52.3 % where the de-vigged market says 55.6 %, so EV is −10.4 % and value is correctly FALSE. The engine now refuses to call a bad price good, which is the entire point of the train.

Still broken, unchanged by this work: edge_pct: 100 on that same response — the edge scale remains on a bad scale (open item, U-deg part 2).

Outstanding: A-emission in production

opp_rank_stat only populates when refreshTeamStats runs inside a snapshot, and the next cron slot is 01:00 UTC. Until then the live ceiling is still +3.0, so A cannot yet emit in prod. Re-measure the ledger grade distribution after that slot — that is the remaining proof, and the MARKETING HOLD stays until it passes.

---

Diagnosed + resolved 2026-07-19 (Session 63). Verified on real props; production A-emission fingerprint outstanding.