From 416639efe4ef057c97e221e2e42a25d885219643 Mon Sep 17 00:00:00 2001 From: Kev Date: Sun, 19 Jul 2026 18:15:14 -0400 Subject: [PATCH] =?UTF-8?q?Grade=20collapse:=20mechanism=20traced=20?= =?UTF-8?q?=E2=80=94=20A=20is=20mathematically=20unreachable?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Completes the diagnosis. Report only; no grade logic or thresholds changed. The grade is an integer index (GRADE_SCALE, NEUTRAL_INDEX 3) moved by a flat sum of +/-1.0 and +/-0.5 factor deltas, then clamped and rounded. grade_thresholds.json is NOT an input mapper in the JS path — engine1 reads it BACKWARDS, taking the letter the index already produced and looking up that band's midpoint to manufacture `confidence`. So confidence is a cosmetic re-encoding of the letter: zero information beyond it, and it can never disagree with it. There is no data-sufficiency penalty in the live path (the one CLAUDE.md describes is in mlbGrader.js, which is dead code). Six of thirteen factors are wired to features nothing populates — verified: refreshTeamStats has ZERO production callers (so opp_rank_stat is permanently null, killing a +/-1.0), teamId/season_type/ game_count_in_7d are never passed (gameContext is built as {home_away} and nothing else), and MLB consistency starves on the same dead gameLogService path as Finding 2. Also verified: BOTH l20 branches are delta +1.0 — there is no negative L20 contribution at all. Arithmetic: an A needs sum >= +4.5; the live maximum is +3.0 (+2.0 on a back-to-back, and MLB rest_days is 0 most days). D needs <= -1.51; the live minimum is -1.5 and Math.round(1.5)=2, so it misses by one rounding tick. Reachable band is index 2..6 = {C-,C,C+,B-,B}, which the adapter's FOUR_LETTER_MAP (a 3->1 collapse) renders as exactly {C,B} — the observed output, derived from first principles. Reachable confidences {42,47,52, 57,63} match the live values {47,52,57,63} exactly; C- is truncated by gradeSlateService keeping the higher-confidence side. mlb-grade-degradation.md's "25/25 grade<->confidence agreement" is a TAUTOLOGY, not a validation — confidence is derived from the letter, so it would report 25/25 even if every grade were wrong. Recommends feeding the starving factors (restores A/D on merit) and explicitly REJECTS re-scaling thresholds, which would mint A's without adding information — every "A" would be a relabelled B. Co-Authored-By: Claude Opus 4.8 (1M context) Claude-Session: https://claude.ai/code/session_01SmNjJAwEnqHPtXbvSZR8kA --- specs/audit-data/grade-collapse.md | 103 +++++++++++++++++++++++++++-- 1 file changed, 97 insertions(+), 6 deletions(-) diff --git a/specs/audit-data/grade-collapse.md b/specs/audit-data/grade-collapse.md index 183da03..3b16578 100644 --- a/specs/audit-data/grade-collapse.md +++ b/specs/audit-data/grade-collapse.md @@ -155,10 +155,86 @@ They are **adjacent, not identical**, and both trace to the same missing input: for 100 % of props, so whatever spread the composite was designed to produce is being generated from the surviving features only. -**The 9-discrete-confidence-values pattern is consistent with a small set of additive -rule hits** — a scorer landing on a lattice rather than a continuum. Confirming exactly -where the letter range compresses requires the engine1/threshold trace, which is the one -open thread in this report. +**The 9-discrete-confidence-values pattern is a small set of additive rule hits** — a +scorer landing on a lattice rather than a continuum. Mechanism now traced in full below. + +--- + +## FINDING 3 — THE MECHANISM: **`A` IS MATHEMATICALLY UNREACHABLE** + +Every claim here was verified directly against the source. + +### The grade is an integer index, not a score + +`engine1.js:16-17, 158-163`: +```js +const GRADE_SCALE = ['F','D','C-','C','C+','B-','B','B+','A-','A','A+']; +const NEUTRAL_INDEX = 3; // 'C' +... +let idx = NEUTRAL_INDEX; +for (const f of factors) idx += f.delta; // flat sum of ±1.0 / ±0.5 +idx = clampIndex(Math.round(idx)); +``` + +`grade_thresholds.json` is **not an input mapper in the JS path.** Nothing compares a +probability to those cutoffs. `engine1.js:29-36` reads the table *backwards* — it takes +the letter the index already produced and looks up that band's **midpoint** to +manufacture a confidence number. + +**So `confidence` is a cosmetic re-encoding of the letter.** It carries zero information +beyond the letter and by construction can never disagree with it. There is **no +data-sufficiency penalty in the live path** — the one CLAUDE.md describes lives in +`mlbGrader.js:50-69`, which is DEAD CODE. (`computeFeatures.js:21` still carries a stale +comment claiming the adapter downgrades confidence; it does not.) + +### Six of thirteen factors are wired to features nothing populates + +| Dead factor | Δ | Why it never fires | +|---|---|---| +| `weak/top_opponent_defense` | **±1.0** | needs `opp_rank_stat` ← `team_stats:{sport}:{abbr}` ← **`refreshTeamStats` has ZERO production callers** (verified: only its own export + tests) | +| `consistency_elite/boom_bust` | **±1.0** | MLB consistency logs come from the same dead `gameLogService` path as Finding 2 | +| `opp_starters_out` | +1.0/+0.5 | `featureCache.js:280`: `if (!teamId) return out;` — `computeFeatures` never passes `teamId` | +| playoff factors | ±0.5 | `season_type` never set | +| `heavy_workload_7d` | −0.5 | `game_count_in_7d` never set | +| `ref_*` / `coach_*` | ±0.5 | NBA-flavored caches, absent for MLB | + +`computeFeatures.js:234-236` builds `gameContext` as **`{ home_away }` and nothing else.** + +### The arithmetic + +`idx = clamp(round(3 + Σδ))`. Live-firing factors for MLB reduce to: `l5_*` (±1.0), +`l20_*` (**+1.0 only — verified, BOTH branches are `delta: 1.0`, there is no negative L20 +contribution**), `home_game` (+0.5), rest (±0.5), `trap_composite_high` (−1.0). + +| 4-letter | needs Σδ | live reachable? | +|---|---|---| +| **A** (A−/A/A+) | **≥ +4.5** | **NO — live max is +3.0** (+2.0 on a back-to-back, and MLB `rest_days` is 0 most days) | +| B | +1.5 … +4.49 | yes | +| C | −1.5 … +1.49 | yes | +| **D** | ≤ −1.51 | **NO — live min is −1.5**, and `Math.round(1.5) = 2` → `C−`. Misses by one rounding tick. | +| **F** | ≤ −2.51 | **NO** | + +**An A is short by at least 1.5 index steps — and the ≥1.5 of deltas that would close the +gap (`opp_rank_stat` ±1.0, `consistency` ±1.0, `injury` +1.0) are exactly the permanently- +null features.** The reachable index band is **2…6 = {C−, C, C+, B−, B}**, which +`gradeAdapter.FOUR_LETTER_MAP` (`gradeAdapter.js:31-37`, a 3→1 collapse) renders as +exactly **{C, B}**. That is the observed output, derived from first principles. + +Confidence corroborates exactly: reachable letters carry `{42, 47, 52, 57, 63}`. **Live +today we observe precisely `{47, 52, 57, 63}`** — C− (42) is absent because +`gradeSlateService.js:76` keeps the higher-confidence side of each prop, truncating the +bottom. The older values in the ledger (`55, 45, 35, 25, 20`) are from the pre-`888d103` +hand-rolled table `{10,15,20,25,35,45,55,65,80,90,100}` — the ledger is append-only, so +it contains both eras. + +### Relationship to `mlb-grade-degradation.md` + +**Shared table, different bug — and its "fix" made this collapse invisible.** That audit +redefined confidence as the band midpoint so the letter round-trips through the table. +The resulting "25/25 agreement" is **a tautology, not a validation**: confidence is +derived *from* the letter, so it would report 25/25 even if every grade were wrong. That +audit only examined the output encoding. This collapse is one layer upstream, on the +input side — whether `computeFactors` has enough live features to move the index at all. --- @@ -177,8 +253,23 @@ Suggested order: is offline). Fingerprint that `p_win`/`ev_pct` appear live. 2. **Then C-led** — persist EV that now has values. 3. **Then G-a** — with the flex band still disabled per the standing ruling. -4. **Separately: the grade-range investigation** (engine1 composite + thresholds), which - may be partly a consequence of step 1 and should be re-measured after it. +4. **Then the grade range** — now diagnosed (Finding 3), and it is NOT primarily a + consequence of step 1. It needs its own decision, because there are two very different + fixes and picking wrong bakes in a lie: + - **(a) Feed the starving factors.** Call `refreshTeamStats` (nothing does), pass + `teamId`/`season_type`/`game_count_in_7d` through `gameContext`, give + `safeGetConsistency` the same MLB branch as step 1. This restores ±3.0 of range and + makes A/D reachable **on merit**. + - **(b) Re-scale the index/thresholds** so the current narrow spread spans more + letters. **This is the tempting one and it is the wrong one** — it would mint A's + without adding a single bit of information, and every "A" would be a relabelled B. + It converts a visible limitation into an invisible lie. + **Recommend (a), explicitly reject (b).** If (a) proves infeasible, the honest fallback + is to keep the two-letter output and stop advertising a scale we don't produce — not to + stretch the scale. + +**Copy consequence, either way:** "A-RATED" appears on public surfaces and `AccuracyBadge` +for a grade the engine has never emitted. Until (a) lands, that copy is unsupported. Open question for Kev: NBA/WNBA have no free game-log source on this path with Python down. `espnStatsAdapter.getPlayerGameLog` (Wave 0) already solves exactly this for