diff --git a/specs/audit-data/grade-collapse.md b/specs/audit-data/grade-collapse.md index 183da03..3b16578 100644 --- a/specs/audit-data/grade-collapse.md +++ b/specs/audit-data/grade-collapse.md @@ -155,10 +155,86 @@ They are **adjacent, not identical**, and both trace to the same missing input: for 100 % of props, so whatever spread the composite was designed to produce is being generated from the surviving features only. -**The 9-discrete-confidence-values pattern is consistent with a small set of additive -rule hits** — a scorer landing on a lattice rather than a continuum. Confirming exactly -where the letter range compresses requires the engine1/threshold trace, which is the one -open thread in this report. +**The 9-discrete-confidence-values pattern is a small set of additive rule hits** — a +scorer landing on a lattice rather than a continuum. Mechanism now traced in full below. + +--- + +## FINDING 3 — THE MECHANISM: **`A` IS MATHEMATICALLY UNREACHABLE** + +Every claim here was verified directly against the source. + +### The grade is an integer index, not a score + +`engine1.js:16-17, 158-163`: +```js +const GRADE_SCALE = ['F','D','C-','C','C+','B-','B','B+','A-','A','A+']; +const NEUTRAL_INDEX = 3; // 'C' +... +let idx = NEUTRAL_INDEX; +for (const f of factors) idx += f.delta; // flat sum of ±1.0 / ±0.5 +idx = clampIndex(Math.round(idx)); +``` + +`grade_thresholds.json` is **not an input mapper in the JS path.** Nothing compares a +probability to those cutoffs. `engine1.js:29-36` reads the table *backwards* — it takes +the letter the index already produced and looks up that band's **midpoint** to +manufacture a confidence number. + +**So `confidence` is a cosmetic re-encoding of the letter.** It carries zero information +beyond the letter and by construction can never disagree with it. There is **no +data-sufficiency penalty in the live path** — the one CLAUDE.md describes lives in +`mlbGrader.js:50-69`, which is DEAD CODE. (`computeFeatures.js:21` still carries a stale +comment claiming the adapter downgrades confidence; it does not.) + +### Six of thirteen factors are wired to features nothing populates + +| Dead factor | Δ | Why it never fires | +|---|---|---| +| `weak/top_opponent_defense` | **±1.0** | needs `opp_rank_stat` ← `team_stats:{sport}:{abbr}` ← **`refreshTeamStats` has ZERO production callers** (verified: only its own export + tests) | +| `consistency_elite/boom_bust` | **±1.0** | MLB consistency logs come from the same dead `gameLogService` path as Finding 2 | +| `opp_starters_out` | +1.0/+0.5 | `featureCache.js:280`: `if (!teamId) return out;` — `computeFeatures` never passes `teamId` | +| playoff factors | ±0.5 | `season_type` never set | +| `heavy_workload_7d` | −0.5 | `game_count_in_7d` never set | +| `ref_*` / `coach_*` | ±0.5 | NBA-flavored caches, absent for MLB | + +`computeFeatures.js:234-236` builds `gameContext` as **`{ home_away }` and nothing else.** + +### The arithmetic + +`idx = clamp(round(3 + Σδ))`. Live-firing factors for MLB reduce to: `l5_*` (±1.0), +`l20_*` (**+1.0 only — verified, BOTH branches are `delta: 1.0`, there is no negative L20 +contribution**), `home_game` (+0.5), rest (±0.5), `trap_composite_high` (−1.0). + +| 4-letter | needs Σδ | live reachable? | +|---|---|---| +| **A** (A−/A/A+) | **≥ +4.5** | **NO — live max is +3.0** (+2.0 on a back-to-back, and MLB `rest_days` is 0 most days) | +| B | +1.5 … +4.49 | yes | +| C | −1.5 … +1.49 | yes | +| **D** | ≤ −1.51 | **NO — live min is −1.5**, and `Math.round(1.5) = 2` → `C−`. Misses by one rounding tick. | +| **F** | ≤ −2.51 | **NO** | + +**An A is short by at least 1.5 index steps — and the ≥1.5 of deltas that would close the +gap (`opp_rank_stat` ±1.0, `consistency` ±1.0, `injury` +1.0) are exactly the permanently- +null features.** The reachable index band is **2…6 = {C−, C, C+, B−, B}**, which +`gradeAdapter.FOUR_LETTER_MAP` (`gradeAdapter.js:31-37`, a 3→1 collapse) renders as +exactly **{C, B}**. That is the observed output, derived from first principles. + +Confidence corroborates exactly: reachable letters carry `{42, 47, 52, 57, 63}`. **Live +today we observe precisely `{47, 52, 57, 63}`** — C− (42) is absent because +`gradeSlateService.js:76` keeps the higher-confidence side of each prop, truncating the +bottom. The older values in the ledger (`55, 45, 35, 25, 20`) are from the pre-`888d103` +hand-rolled table `{10,15,20,25,35,45,55,65,80,90,100}` — the ledger is append-only, so +it contains both eras. + +### Relationship to `mlb-grade-degradation.md` + +**Shared table, different bug — and its "fix" made this collapse invisible.** That audit +redefined confidence as the band midpoint so the letter round-trips through the table. +The resulting "25/25 agreement" is **a tautology, not a validation**: confidence is +derived *from* the letter, so it would report 25/25 even if every grade were wrong. That +audit only examined the output encoding. This collapse is one layer upstream, on the +input side — whether `computeFactors` has enough live features to move the index at all. --- @@ -177,8 +253,23 @@ Suggested order: is offline). Fingerprint that `p_win`/`ev_pct` appear live. 2. **Then C-led** — persist EV that now has values. 3. **Then G-a** — with the flex band still disabled per the standing ruling. -4. **Separately: the grade-range investigation** (engine1 composite + thresholds), which - may be partly a consequence of step 1 and should be re-measured after it. +4. **Then the grade range** — now diagnosed (Finding 3), and it is NOT primarily a + consequence of step 1. It needs its own decision, because there are two very different + fixes and picking wrong bakes in a lie: + - **(a) Feed the starving factors.** Call `refreshTeamStats` (nothing does), pass + `teamId`/`season_type`/`game_count_in_7d` through `gameContext`, give + `safeGetConsistency` the same MLB branch as step 1. This restores ±3.0 of range and + makes A/D reachable **on merit**. + - **(b) Re-scale the index/thresholds** so the current narrow spread spans more + letters. **This is the tempting one and it is the wrong one** — it would mint A's + without adding a single bit of information, and every "A" would be a relabelled B. + It converts a visible limitation into an invisible lie. + **Recommend (a), explicitly reject (b).** If (a) proves infeasible, the honest fallback + is to keep the two-letter output and stop advertising a scale we don't produce — not to + stretch the scale. + +**Copy consequence, either way:** "A-RATED" appears on public surfaces and `AccuracyBadge` +for a grade the engine has never emitted. Until (a) lands, that copy is unsupported. Open question for Kev: NBA/WNBA have no free game-log source on this path with Python down. `espnStatsAdapter.getPlayerGameLog` (Wave 0) already solves exactly this for