builtbykev
3fe840ab83
Kev's call: investigate the B/C grade collapse before building. Report only — no grade logic, thresholds, or engine code touched. FINDING 1 — the collapse is real, live and structural. Across 604 ledger rows and both sports the engine has emitted exactly TWO grades (B, C) and NINE confidence values (63/57/55/52/47/45/35/25/20), ceiling 63. Still true today on both sports. Confidence does NOT determine the letter: conf 45 -> B while 47 and 52 -> C (non-monotonic), so the surfaced confidence is not the quantity the letter came from. Edge scale still broken: 311/604 rows exceed the frontend's sane cap of 40, 39 exceed 100, worst 620. FINDING 2 (bigger) — the entire probability layer is DEAD in production. Live /api/snapshot/mlb: p_win, kelly, ev_pct, model_odds and value are absent on 0/8 grades, while alt_lines (Desk-gated) IS present 8/8 — proving nothing is tier-stripped, they are simply never computed. Root cause: gameLogService.pythonPath returns null for MLB by construction and the Python service is offline for NBA/WNBA, so meta.gameLogs is [] for every sport; estimateProbability returns p_over null; every field guarded by `if (pWin != null)` is skipped. This is the S46 bug in a second location — that fix added an MLB branch to featureCache.gameLogFeatures (which is why grades/projections still work) but never to the estimator path. Consequences: EV — the Model Train's whole ranking signal — has never been computed on a live prop. Hero v2 matches nothing and always falls through to the recent-read fallback (live /api/hero-prop returns is_recent:true). Quarter-Kelly, sold on the pricing page and listed BUILT in PROMISE-AUDIT.md, never runs. The value triplet is a duet live. Recommend re-sequencing: revive the probability layer BEFORE G-a and C-led (C-led would persist a column of nulls; G-a's EV_FLEX_THRESHOLD would gate on a permanently-null value — Kev's EV_FLEX_ENFORCE=0 ruling accidentally prevented an outage). featureCache:206-226 already has both adapter branches and is the template. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SmNjJAwEnqHPtXbvSZR8kA
190 lines
8.1 KiB
Markdown
190 lines
8.1 KiB
Markdown
# GRADE COLLAPSE + DEAD PROBABILITY LAYER — diagnosis
|
||
|
||
**REPORT ONLY. No grade logic, thresholds, or engine code changed** (Kev's instruction).
|
||
Two findings. The second one is bigger than the question I was asked.
|
||
|
||
Data: live Supabase `ledger_entries` (604 rows, all users) + live prod API
|
||
`api.vyndr.app`, 2026-07-19 ~22:05 UTC.
|
||
|
||
---
|
||
|
||
## FINDING 1 — THE COLLAPSE IS REAL, LIVE, AND STRUCTURAL
|
||
|
||
Not a thin-slate artifact. Across **604 ledger rows and both sports**:
|
||
|
||
- **2 distinct grades ever emitted: B and C.** Zero A+, A, A−, B+, B−, C−, D, F.
|
||
- **9 distinct confidence values ever emitted:** 63, 57, 55, 52, 47, 45, 35, 25, 20.
|
||
- **Confidence ceiling = 63.** It has never exceeded 63 in the recorded era.
|
||
|
||
Still true **today** (Jul 18 + 19, post every fix, both sports): 4 confidence values
|
||
(63/57/52/47), 2 grades.
|
||
|
||
| Sport | Grade | n | conf min | conf max |
|
||
|---|---|---|---|---|
|
||
| mlb | B | 276 | 45 | 63 |
|
||
| mlb | C | 107 | 20 | 52 |
|
||
| wnba | B | 130 | 45 | 63 |
|
||
| wnba | C | 91 | 35 | 52 |
|
||
|
||
### Confidence does NOT determine the letter
|
||
|
||
| conf | grade | n |
|
||
|---|---|---|
|
||
| 63 | B | 54 |
|
||
| 57 | B | 175 |
|
||
| 55 | B | 29 |
|
||
| 52 | **C** | 100 |
|
||
| 47 | **C** | 14 |
|
||
| **45** | **B** | **148** |
|
||
| 35 | C | 80 |
|
||
|
||
**conf 45 → B, but conf 47 and 52 → C.** The mapping is non-monotonic, so the surfaced
|
||
`confidence` is not the quantity the letter was derived from. This confirms the
|
||
`mlb-grade-degradation.md` "grade↔confidence mismatch" as a *display* artifact: two
|
||
different quantities are being shown as if one explains the other.
|
||
|
||
(At conf 45→B the avg edge is 103; at conf 52→C it is 49 — so the letter tracks the
|
||
engine composite/edge, not the displayed confidence.)
|
||
|
||
### Collateral: the edge scale is still broken and still live
|
||
|
||
- **311 of 604 rows (51.5 %) have |edge| > 40** — the frontend's `EDGE_BOARD_SANE_MAX`,
|
||
i.e. over half the board's edge is nulled at render.
|
||
- **39 rows have |edge| > 100** — impossible as a percentage. Worst: **620**.
|
||
- Live today: edges of 140, 180, 220 on Jul 18–19 rows.
|
||
|
||
`U-deg`'s `projection == 0` leak IS closed (0 since 07-18). The **edge_pct scale is
|
||
not** — it remains open and is now quantified.
|
||
|
||
---
|
||
|
||
## FINDING 2 — 🔴 THE ENTIRE PROBABILITY LAYER IS DEAD IN PRODUCTION
|
||
|
||
Found while fingerprinting Arc 1 (U-fp). This is the headline.
|
||
|
||
### Live fingerprint, `GET /api/snapshot/mlb`, 8 graded props
|
||
|
||
| Field | Present |
|
||
|---|---|
|
||
| `projection`, `confidence`, `book_odds`, `fair_odds`, `takeable`, `devig_method`, `alt_lines` | **8 / 8** |
|
||
| **`p_win`** | **0 / 8** |
|
||
| **`kelly`** | **0 / 8** |
|
||
| **`ev_pct`** | **0 / 8** |
|
||
| **`model_odds`** | **0 / 8** |
|
||
| **`value`** | **0 / 8** |
|
||
|
||
**Control:** `alt_lines` is present 8/8 and is Desk-gated in `tierGating.js:55`, which
|
||
proves the payload is **not** being tier-stripped. These fields are genuinely never
|
||
computed — not hidden.
|
||
|
||
### Root cause — a one-line sport gate, and an S46 fix that was only half-applied
|
||
|
||
`analyzeViaEngine1.js:509` feeds the estimator from `meta.gameLogs`:
|
||
|
||
```js
|
||
const est = estimateProbability({ gameLogs: meta.gameLogs, line: prop.line, ... });
|
||
```
|
||
|
||
`meta.gameLogs` comes from `computeFeatures.js:173-181` → `gameLogService.getGameLogs`.
|
||
And `gameLogService.js:21-26`:
|
||
|
||
```js
|
||
function pythonPath(sport) {
|
||
switch (sport) {
|
||
case 'nba': return '/stats/last-n';
|
||
case 'wnba': return '/wnba/stats/last-n';
|
||
default: return null; // ← MLB exits here
|
||
}
|
||
}
|
||
```
|
||
with `getGameLogs` line 31: `if (!path) return null;`
|
||
|
||
So:
|
||
- **MLB** — returns `null` by construction. Never had game logs on this path.
|
||
- **NBA/WNBA** — hits the Python stats service, which is **offline in prod** (documented
|
||
in CLAUDE.md; degrades to null).
|
||
|
||
⇒ `meta.gameLogs` is `[]` for **every sport in production** ⇒
|
||
`estimateProbability` returns `{p_over: null, reason:'insufficient_data'}`
|
||
(`probabilityEstimator.js:55-57`) ⇒ `pWin` is null ⇒ **every field guarded by
|
||
`if (pWin != null)` is skipped**: `p_win`, `kelly`, `model_odds`, `ev_pct`, `value`.
|
||
|
||
**This is the S46 bug, second location, never fixed.** CLAUDE.md records that
|
||
`gameLogService.getGameLogs` being NBA/WNBA-only starved MLB, and that the fix was an
|
||
MLB branch in **`featureCache.gameLogFeatures`**. That fixed the *feature* path — which
|
||
is why `projection`, `confidence`, and grades still work. The **estimator path was never
|
||
given the same branch**, so it has been silently dead the whole time.
|
||
|
||
### What this actually breaks
|
||
|
||
1. **EV — the Model Train's entire ranking signal — does not exist in production.**
|
||
Arc 1 shipped `ev_pct` and it has never once been computed on a live prop.
|
||
2. **Hero v2 is non-functional.** `pickHeroProp` requires a finite `ev_pct`
|
||
(`heroPropService.js:84`), so the EV loop matches **nothing** and always falls through
|
||
to the "most recent graded read" fallback. Live proof: `/api/hero-prop` returns
|
||
`"is_recent": true` — the fallback path, every time. The hero has not been an EV pick
|
||
since the day it shipped.
|
||
3. **Quarter-Kelly is dead** — same `pWin` dependency (`analyzeViaEngine1.js:516-520`).
|
||
This is a **promise-audit issue**: Kelly sizing is sold on the pricing page and
|
||
`PROMISE-AUDIT.md` lists it as BUILT. It is built and never runs.
|
||
4. **The "value triplet" is a duet live** — `book_odds` + `fair_odds` render;
|
||
`model_odds` is always absent.
|
||
5. **`value` is never true**, so the VALUE marker can never light up.
|
||
|
||
### Why this reframes the whole train
|
||
|
||
- **C-led would persist a column of nulls.** Do not build EV persistence until EV exists.
|
||
- **G-a's `EV_FLEX_THRESHOLD` would gate on a permanently-null value.** With
|
||
`EV_FLEX_ENFORCE=0` (Kev's ruling) this is harmless today — but had we enforced it,
|
||
the flex band would have been cut to **zero**, because `ev_pct >= 4` can never be true.
|
||
The ruling to ship it disabled accidentally prevented an outage.
|
||
- **S-b (rank board on EV)** would rank on nulls.
|
||
|
||
---
|
||
|
||
## RELATIONSHIP BETWEEN THE TWO FINDINGS
|
||
|
||
They are **adjacent, not identical**, and both trace to the same missing input:
|
||
|
||
- The dead estimator explains **why no probability-derived output exists** (EV, Kelly,
|
||
model_odds, p_win).
|
||
- It does **not by itself** explain the B/C letter collapse, because the letter comes
|
||
from engine1's rule-based composite over the *feature vector*, which is alive.
|
||
- But they share a root: **the model is running on a partial input set.** One of its two
|
||
probability inputs (the empirical quantile distribution over real game logs) is absent
|
||
for 100 % of props, so whatever spread the composite was designed to produce is being
|
||
generated from the surviving features only.
|
||
|
||
**The 9-discrete-confidence-values pattern is consistent with a small set of additive
|
||
rule hits** — a scorer landing on a lattice rather than a continuum. Confirming exactly
|
||
where the letter range compresses requires the engine1/threshold trace, which is the one
|
||
open thread in this report.
|
||
|
||
---
|
||
|
||
## RECOMMENDATION (no code changed pending Kev's call)
|
||
|
||
**Re-sequence: fix the dead estimator FIRST — before G-a, before C-led.**
|
||
|
||
Rationale: it is the cheapest fix on the board (an MLB branch in the estimator's log
|
||
source, mirroring the one already written for `featureCache`), and it simultaneously
|
||
restores EV, Kelly, `model_odds`, the VALUE flag, and hero v2. Every other Arc 2-5 item
|
||
is downstream of it. Building the gate, the persistence layer, or the board ranking on a
|
||
null signal is building on nothing.
|
||
|
||
Suggested order:
|
||
1. **Revive the probability layer** (MLB branch + a real NBA/WNBA fallback, since Python
|
||
is offline). Fingerprint that `p_win`/`ev_pct` appear live.
|
||
2. **Then C-led** — persist EV that now has values.
|
||
3. **Then G-a** — with the flex band still disabled per the standing ruling.
|
||
4. **Separately: the grade-range investigation** (engine1 composite + thresholds), which
|
||
may be partly a consequence of step 1 and should be re-measured after it.
|
||
|
||
Open question for Kev: NBA/WNBA have no free game-log source on this path with Python
|
||
down. `espnStatsAdapter.getPlayerGameLog` (Wave 0) already solves exactly this for
|
||
`featureCache` — reusing it here is the obvious candidate, and costs no quota.
|
||
|
||
---
|
||
|
||
*Diagnosis 2026-07-19. Data + live API. No engine code changed.*
|