vyndr/specs/audit-data/grade-collapse.md

# GRADE COLLAPSE + DEAD PROBABILITY LAYER — diagnosis

**REPORT ONLY. No grade logic, thresholds, or engine code changed** (Kev's instruction).
Two findings. The second one is bigger than the question I was asked.

Data: live Supabase `ledger_entries` (604 rows, all users) + live prod API
`api.vyndr.app`, 2026-07-19 ~22:05 UTC.

---

## FINDING 1 — THE COLLAPSE IS REAL, LIVE, AND STRUCTURAL

Not a thin-slate artifact. Across **604 ledger rows and both sports**:

- **2 distinct grades ever emitted: B and C.** Zero A+, A, A−, B+, B−, C−, D, F.
- **9 distinct confidence values ever emitted:** 63, 57, 55, 52, 47, 45, 35, 25, 20.
- **Confidence ceiling = 63.** It has never exceeded 63 in the recorded era.

Still true **today** (Jul 18 + 19, post every fix, both sports): 4 confidence values
(63/57/52/47), 2 grades.

| Sport | Grade | n | conf min | conf max |
|---|---|---|---|---|
| mlb | B | 276 | 45 | 63 |
| mlb | C | 107 | 20 | 52 |
| wnba | B | 130 | 45 | 63 |
| wnba | C | 91 | 35 | 52 |

### Confidence does NOT determine the letter

| conf | grade | n |
|---|---|---|
| 63 | B | 54 |
| 57 | B | 175 |
| 55 | B | 29 |
| 52 | **C** | 100 |
| 47 | **C** | 14 |
| **45** | **B** | **148** |
| 35 | C | 80 |

**conf 45 → B, but conf 47 and 52 → C.** The mapping is non-monotonic, so the surfaced
`confidence` is not the quantity the letter was derived from. This confirms the
`mlb-grade-degradation.md` "grade↔confidence mismatch" as a *display* artifact: two
different quantities are being shown as if one explains the other.

(At conf 45→B the avg edge is 103; at conf 52→C it is 49 — so the letter tracks the
engine composite/edge, not the displayed confidence.)

### Collateral: the edge scale is still broken and still live

- **311 of 604 rows (51.5 %) have |edge| > 40** — the frontend's `EDGE_BOARD_SANE_MAX`,
  i.e. over half the board's edge is nulled at render.
- **39 rows have |edge| > 100** — impossible as a percentage. Worst: **620**.
- Live today: edges of 140, 180, 220 on Jul 18–19 rows.

`U-deg`'s `projection == 0` leak IS closed (0 since 07-18). The **edge_pct scale is
not** — it remains open and is now quantified.

---

## FINDING 2 — 🔴 THE ENTIRE PROBABILITY LAYER IS DEAD IN PRODUCTION

Found while fingerprinting Arc 1 (U-fp). This is the headline.

### Live fingerprint, `GET /api/snapshot/mlb`, 8 graded props

| Field | Present |
|---|---|
| `projection`, `confidence`, `book_odds`, `fair_odds`, `takeable`, `devig_method`, `alt_lines` | **8 / 8** |
| **`p_win`** | **0 / 8** |
| **`kelly`** | **0 / 8** |
| **`ev_pct`** | **0 / 8** |
| **`model_odds`** | **0 / 8** |
| **`value`** | **0 / 8** |

**Control:** `alt_lines` is present 8/8 and is Desk-gated in `tierGating.js:55`, which
proves the payload is **not** being tier-stripped. These fields are genuinely never
computed — not hidden.

### Root cause — a one-line sport gate, and an S46 fix that was only half-applied

`analyzeViaEngine1.js:509` feeds the estimator from `meta.gameLogs`:

```js
const est = estimateProbability({ gameLogs: meta.gameLogs, line: prop.line, ... });
```

`meta.gameLogs` comes from `computeFeatures.js:173-181` → `gameLogService.getGameLogs`.
And `gameLogService.js:21-26`:

```js
function pythonPath(sport) {
  switch (sport) {
    case 'nba':  return '/stats/last-n';
    case 'wnba': return '/wnba/stats/last-n';
    default:     return null;      // ← MLB exits here
  }
}
```
with `getGameLogs` line 31: `if (!path) return null;`

So:
- **MLB** — returns `null` by construction. Never had game logs on this path.
- **NBA/WNBA** — hits the Python stats service, which is **offline in prod** (documented
  in CLAUDE.md; degrades to null).

⇒ `meta.gameLogs` is `[]` for **every sport in production** ⇒
`estimateProbability` returns `{p_over: null, reason:'insufficient_data'}`
(`probabilityEstimator.js:55-57`) ⇒ `pWin` is null ⇒ **every field guarded by
`if (pWin != null)` is skipped**: `p_win`, `kelly`, `model_odds`, `ev_pct`, `value`.

**This is the S46 bug, second location, never fixed.** CLAUDE.md records that
`gameLogService.getGameLogs` being NBA/WNBA-only starved MLB, and that the fix was an
MLB branch in **`featureCache.gameLogFeatures`**. That fixed the *feature* path — which
is why `projection`, `confidence`, and grades still work. The **estimator path was never
given the same branch**, so it has been silently dead the whole time.

### What this actually breaks

1. **EV — the Model Train's entire ranking signal — does not exist in production.**
   Arc 1 shipped `ev_pct` and it has never once been computed on a live prop.
2. **Hero v2 is non-functional.** `pickHeroProp` requires a finite `ev_pct`
   (`heroPropService.js:84`), so the EV loop matches **nothing** and always falls through
   to the "most recent graded read" fallback. Live proof: `/api/hero-prop` returns
   `"is_recent": true` — the fallback path, every time. The hero has not been an EV pick
   since the day it shipped.
3. **Quarter-Kelly is dead** — same `pWin` dependency (`analyzeViaEngine1.js:516-520`).
   This is a **promise-audit issue**: Kelly sizing is sold on the pricing page and
   `PROMISE-AUDIT.md` lists it as BUILT. It is built and never runs.
4. **The "value triplet" is a duet live** — `book_odds` + `fair_odds` render;
   `model_odds` is always absent.
5. **`value` is never true**, so the VALUE marker can never light up.

### Why this reframes the whole train

- **C-led would persist a column of nulls.** Do not build EV persistence until EV exists.
- **G-a's `EV_FLEX_THRESHOLD` would gate on a permanently-null value.** With
  `EV_FLEX_ENFORCE=0` (Kev's ruling) this is harmless today — but had we enforced it,
  the flex band would have been cut to **zero**, because `ev_pct >= 4` can never be true.
  The ruling to ship it disabled accidentally prevented an outage.
- **S-b (rank board on EV)** would rank on nulls.

---

## RELATIONSHIP BETWEEN THE TWO FINDINGS

They are **adjacent, not identical**, and both trace to the same missing input:

- The dead estimator explains **why no probability-derived output exists** (EV, Kelly,
  model_odds, p_win).
- It does **not by itself** explain the B/C letter collapse, because the letter comes
  from engine1's rule-based composite over the *feature vector*, which is alive.
- But they share a root: **the model is running on a partial input set.** One of its two
  probability inputs (the empirical quantile distribution over real game logs) is absent
  for 100 % of props, so whatever spread the composite was designed to produce is being
  generated from the surviving features only.

**The 9-discrete-confidence-values pattern is a small set of additive rule hits** — a
scorer landing on a lattice rather than a continuum. Mechanism now traced in full below.

---

## FINDING 3 — THE MECHANISM: **`A` IS MATHEMATICALLY UNREACHABLE**

Every claim here was verified directly against the source.

### The grade is an integer index, not a score

`engine1.js:16-17, 158-163`:
```js
const GRADE_SCALE = ['F','D','C-','C','C+','B-','B','B+','A-','A','A+'];
const NEUTRAL_INDEX = 3;  // 'C'
...
let idx = NEUTRAL_INDEX;
for (const f of factors) idx += f.delta;      // flat sum of ±1.0 / ±0.5
idx = clampIndex(Math.round(idx));
```

`grade_thresholds.json` is **not an input mapper in the JS path.** Nothing compares a
probability to those cutoffs. `engine1.js:29-36` reads the table *backwards* — it takes
the letter the index already produced and looks up that band's **midpoint** to
manufacture a confidence number.

**So `confidence` is a cosmetic re-encoding of the letter.** It carries zero information
beyond the letter and by construction can never disagree with it. There is **no
data-sufficiency penalty in the live path** — the one CLAUDE.md describes lives in
`mlbGrader.js:50-69`, which is DEAD CODE. (`computeFeatures.js:21` still carries a stale
comment claiming the adapter downgrades confidence; it does not.)

### Six of thirteen factors are wired to features nothing populates

| Dead factor | Δ | Why it never fires |
|---|---|---|
| `weak/top_opponent_defense` | **±1.0** | needs `opp_rank_stat` ← `team_stats:{sport}:{abbr}` ← **`refreshTeamStats` has ZERO production callers** (verified: only its own export + tests) |
| `consistency_elite/boom_bust` | **±1.0** | MLB consistency logs come from the same dead `gameLogService` path as Finding 2 |
| `opp_starters_out` | +1.0/+0.5 | `featureCache.js:280`: `if (!teamId) return out;` — `computeFeatures` never passes `teamId` |
| playoff factors | ±0.5 | `season_type` never set |
| `heavy_workload_7d` | −0.5 | `game_count_in_7d` never set |
| `ref_*` / `coach_*` | ±0.5 | NBA-flavored caches, absent for MLB |

`computeFeatures.js:234-236` builds `gameContext` as **`{ home_away }` and nothing else.**

### The arithmetic

`idx = clamp(round(3 + Σδ))`. Live-firing factors for MLB reduce to: `l5_*` (±1.0),
`l20_*` (**+1.0 only — verified, BOTH branches are `delta: 1.0`, there is no negative L20
contribution**), `home_game` (+0.5), rest (±0.5), `trap_composite_high` (−1.0).

| 4-letter | needs Σδ | live reachable? |
|---|---|---|
| **A** (A−/A/A+) | **≥ +4.5** | **NO — live max is +3.0** (+2.0 on a back-to-back, and MLB `rest_days` is 0 most days) |
| B | +1.5 … +4.49 | yes |
| C | −1.5 … +1.49 | yes |
| **D** | ≤ −1.51 | **NO — live min is −1.5**, and `Math.round(1.5) = 2` → `C−`. Misses by one rounding tick. |
| **F** | ≤ −2.51 | **NO** |

**An A is short by at least 1.5 index steps — and the ≥1.5 of deltas that would close the
gap (`opp_rank_stat` ±1.0, `consistency` ±1.0, `injury` +1.0) are exactly the permanently-
null features.** The reachable index band is **2…6 = {C−, C, C+, B−, B}**, which
`gradeAdapter.FOUR_LETTER_MAP` (`gradeAdapter.js:31-37`, a 3→1 collapse) renders as
exactly **{C, B}**. That is the observed output, derived from first principles.

Confidence corroborates exactly: reachable letters carry `{42, 47, 52, 57, 63}`. **Live
today we observe precisely `{47, 52, 57, 63}`** — C− (42) is absent because
`gradeSlateService.js:76` keeps the higher-confidence side of each prop, truncating the
bottom. The older values in the ledger (`55, 45, 35, 25, 20`) are from the pre-`888d103`
hand-rolled table `{10,15,20,25,35,45,55,65,80,90,100}` — the ledger is append-only, so
it contains both eras.

### Relationship to `mlb-grade-degradation.md`

**Shared table, different bug — and its "fix" made this collapse invisible.** That audit
redefined confidence as the band midpoint so the letter round-trips through the table.
The resulting "25/25 agreement" is **a tautology, not a validation**: confidence is
derived *from* the letter, so it would report 25/25 even if every grade were wrong. That
audit only examined the output encoding. This collapse is one layer upstream, on the
input side — whether `computeFactors` has enough live features to move the index at all.

---

## RECOMMENDATION (no code changed pending Kev's call)

**Re-sequence: fix the dead estimator FIRST — before G-a, before C-led.**

Rationale: it is the cheapest fix on the board (an MLB branch in the estimator's log
source, mirroring the one already written for `featureCache`), and it simultaneously
restores EV, Kelly, `model_odds`, the VALUE flag, and hero v2. Every other Arc 2-5 item
is downstream of it. Building the gate, the persistence layer, or the board ranking on a
null signal is building on nothing.

Suggested order:
1. **Revive the probability layer** (MLB branch + a real NBA/WNBA fallback, since Python
   is offline). Fingerprint that `p_win`/`ev_pct` appear live.
2. **Then C-led** — persist EV that now has values.
3. **Then G-a** — with the flex band still disabled per the standing ruling.
4. **Then the grade range** — now diagnosed (Finding 3), and it is NOT primarily a
   consequence of step 1. It needs its own decision, because there are two very different
   fixes and picking wrong bakes in a lie:
   - **(a) Feed the starving factors.** Call `refreshTeamStats` (nothing does), pass
     `teamId`/`season_type`/`game_count_in_7d` through `gameContext`, give
     `safeGetConsistency` the same MLB branch as step 1. This restores ±3.0 of range and
     makes A/D reachable **on merit**.
   - **(b) Re-scale the index/thresholds** so the current narrow spread spans more
     letters. **This is the tempting one and it is the wrong one** — it would mint A's
     without adding a single bit of information, and every "A" would be a relabelled B.
     It converts a visible limitation into an invisible lie.
   **Recommend (a), explicitly reject (b).** If (a) proves infeasible, the honest fallback
   is to keep the two-letter output and stop advertising a scale we don't produce — not to
   stretch the scale.

**Copy consequence, either way:** "A-RATED" appears on public surfaces and `AccuracyBadge`
for a grade the engine has never emitted. Until (a) lands, that copy is unsupported.

Open question for Kev: NBA/WNBA have no free game-log source on this path with Python
down. `espnStatsAdapter.getPlayerGameLog` (Wave 0) already solves exactly this for
`featureCache` — reusing it here is the obvious candidate, and costs no quota.

---

# RESOLUTION — Session 63 (shipped)

Kev's ruling: **(a) fix on merit, never (b) rescale.** Rescaling would mint A's
without adding information — a relabelled B marketed as an A, corrupting an
append-only ledger permanently. That option is permanently rejected.

## What shipped

| Fix | File | Effect |
|---|---|---|
| Normalized per-game rows for ALL sports | `featureCache.getStatRows` | Revives `p_win` → `ev_pct`, `kelly`, `model_odds`, `value`, hero v2. Also feeds consistency. |
| Rows wired into the grade path | `computeFeatures.safeGetConsistency` | One fetch per prop, shared by 3 starving consumers |
| `refreshTeamStats` called in production | `snapshotService.runSnapshot` | `opp_rank_stat` populated → the ±1.0 opponent factor can fire (it had ZERO callers) |
| `game_count_in_7d` derived from real logs | `computeFeatures` gameContext | `heavy_workload_7d` (−0.5) can fire |
| **L20 symmetry** | `engine1.computeFactors` | NEW `l20_contradicts_*` −1.0. There was no negative L20 path at all — a structural reason D was unreachable |
| Consistency CV floor | `consistencyScore` | See calibration finding below |
| `confidence_basis: 'grade_band'` | `gradeAdapter.toLegacyShape` | Confidence labelled as derived, not a probability |
| Dead `mlbGrader.js` **removed** | — | Referenced only by its own test. Described-but-dead penalty eliminated |

**Deliberately NOT wired** (would have been dead code dressed as a fix, documented
inline): `teamId` (no `team_id` column exists; `getFeatures` reads it top-level not
off gameContext; and the factor needs a starter-id list that doesn't exist) and
`season_type` (engine1 gates playoff factors on `season_type >= 2`, but ESPN's 2
means REGULAR season — threading it raw would fire "veteran_in_playoffs" in July).

## 🔶 CALIBRATION FINDING — consistency was NBA-tuned and would have flooded `boom_bust`

Reviving consistency exposed a latent bug. The CV thresholds (`cv >= 0.5` →
`boom_bust`) were calibrated for NBA points (mean ~20). For a Poisson-ish counting
stat, **cv ≈ 1/√mean**, so any stat with mean < 4 forces `cv > 0.5` — it classifies
`boom_bust` regardless of actual behaviour. Verified on real logs:

- Alonso hits `[0,0,0,1,2,1,0,1,1,0]` → mean 0.60, **cv 1.17** → boom_bust
- Henderson hits `[1,0,0,3,1,1,0,0,1,0]` → mean 0.70, **cv 1.36** → boom_bust

First verification run confirmed it: **8/8 MLB props classified boom_bust**, a
blanket −1.0 that dropped the whole board to C. That is a systematic downgrade
masquerading as a signal — the mirror image of the "flooding A's" failure Kev
warned about.

**Guard shipped:** `MIN_MEAN_FOR_CV = 4` (env `CONSISTENCY_MIN_MEAN`). Below it,
consistency returns `unknown` (no factor) with `reason: 'low_mean_cv_unreliable'`.
Absent beats wrong. **Consequence: MLB low-count stats still get no consistency
factor** — honest, not fixed. The correct long-term fix is an index-of-dispersion
(variance/mean vs the Poisson baseline) classifier, which is scale-free. Tracked
as an open item; it is a modelling change needing its own validation.

## VERIFICATION ON MERIT — real props, real logs, real engine

`scripts/verify-grade-range.js` replays live-board props through the repaired
engine using free feeds (statsapi/ESPN). **Caveat stated up front: `opp_rank_stat`
needs the Redis team-stats cache that only production populates, so these local
runs OMIT a ±1.0 factor and therefore UNDERSTATE the restored range.**

**WNBA — 25 real props**

| | BEFORE (live board) | AFTER (repaired) |
|---|---|---|
| A | 0 | 0 |
| B | 17 (68 %) | 8 (32 %) |
| C | 8 (32 %) | 16 (64 %) |
| **D** | **0** | **1 (4 %)** |

11-step spread: `C 6 · C+ 10 · B− 8 · D 1` — five distinct steps where there were
two. Revived signals: **`p_win` 25/25 (was 0)**, rows 25/25, consistency known
15/25 (the floor correctly abstains on low-mean assists/rebounds).

The D is earned, not manufactured: *Angel Reese assists over 2.5, p_win 0.365* —
the model gives it 36.5 % and says so.

**MLB — 8 real props:** B 5 / C 3, `p_win` 8/8 (was 0). No A or D on a thin
8-prop late-night board of near-identical 0.5-hits props.

**Reading it honestly:**
- **D emits on merit. ✅**
- **A did not emit locally** — expected: A needs Σδ ≥ +4.5 and the local ceiling is
  +3.0 without `opp_rank_stat`. Structural reachability is proven arithmetically
  and locked in `tests/unit/gradeRangeRestore.test.js`; **empirical A emission
  requires production and is the outstanding fingerprint.**
- **Nothing flooded.** Grades got *harder*, not easier — B fell 68 % → 32 %. The
  B→C movers are driven by the new L20 negative branch: props whose season
  baseline contradicts the graded side no longer get a free pass. That is the
  intended correction.

## 🔴 MARKETING HOLD — A-rated copy is UNSUPPORTED until A verifiably emits

Confirmed the honest fallbacks are what render today:
- `/api/ledger/accuracy` returns buckets **B and C only** — no A bucket. So
  `AccuracyBadge`'s `aRated` sample is 0, below `minSample`, and it falls through
  to **"MODEL · 63% HIT"**. No fabricated A-RATED is displayed.
- `TopSignals` self-hides when there are no A-rated grades.

**Nothing fabricated is shipping — but the copy describes a grade the engine has
never emitted.** Do not promote "A-RATED" in marketing, and do not build new
surfaces on an A bucket, until a production fingerprint shows real A grades. Lift
this hold only against live data.

---

## ✅ PRODUCTION FINGERPRINT — 2026-07-19 ~22:50 UTC

`POST /api/analyze/prop` (grades on demand, so it exercises the repaired path
immediately rather than waiting for the snapshot cron):

```
Gunnar Henderson hits o0.5 @ -140 (fanduel)
  grade            : B
  confidence       : 57
  confidence_basis : grade_band     <- NEW (truth label)
  p_win            : 0.523          <- WAS ABSENT on 100% of grades
  ev_pct           : -10.4          <- WAS ABSENT
  model_odds       : -109           <- WAS ABSENT (triplet was a duet)
  value            : false          <- WAS ABSENT
  takeable         : true
  book_odds        : -140
  fair_odds        : -125
```

**The value triplet is finally whole: book −140 · vig-free −125 · model −109.**
And it tells the truth — the model gives 52.3 % where the de-vigged market says
55.6 %, so EV is −10.4 % and `value` is correctly FALSE. The engine now refuses to
call a bad price good, which is the entire point of the train.

Still broken, unchanged by this work: `edge_pct: 100` on that same response — the
edge scale remains on a bad scale (open item, `U-deg` part 2).

### Outstanding: A-emission in production
`opp_rank_stat` only populates when `refreshTeamStats` runs inside a snapshot, and
the next cron slot is 01:00 UTC. Until then the live ceiling is still +3.0, so **A
cannot yet emit in prod**. Re-measure the ledger grade distribution after that slot
— that is the remaining proof, and the MARKETING HOLD stays until it passes.

---

*Diagnosed + resolved 2026-07-19 (Session 63). Verified on real props; production
A-emission fingerprint outstanding.*