From 91927a4a8af875a20bca41c3441f5e7d8cfa13be Mon Sep 17 00:00:00 2001 From: Kev Date: Fri, 7 Aug 2026 04:53:02 -0400 Subject: [PATCH] Serve an honest grade: the letter was carrying 1/6 the information of the number beside it MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit PHASE 0 corrects the order's premise. A grade letter has been served all along -- engine1.gradeProp builds it from an additive factor index, computed INDEPENDENTLY of p_win. gradeBands is orphaned for a different reason than assumed: it defines what a letter MEANS from realized outcomes, and every band collapses to base-rate at current resolution. The measurement that changed this order, on 3,417 settled props: grade n realized mean p_win A 8 0.500 0.647 <- the TOP grade did WORST B 985 0.640 0.700 C 1,695 0.602 0.676 D 303 0.558 0.604 F 426 0.535 0.588 letter resolution 0.00116 (0.48% of variance) p_win resolution 0.00715 (2.98%) -> the letter carried 0.16x the information of the number beside it Concretely, from the hand-verify: Christian Encarnacion's 0.95 over graded C and his 0.05 under ALSO graded C -- same hitter, opposite forecasts, same letter. The gap was never that grades don't ship; it is that the weaker of two available signals shipped as the headline. PHASE 1 — model/servedGrade.js derives the letter from p_win with bands anchored on MEASURED realized rates (B+ 0.663 / B 0.646 / C+ 0.615 / C 0.589 / C- 0.548 / D 0.512 / F 0.447, base 0.6005). NO MANUFACTURED A, structurally: A+/A/A- are UNISSUABLE, not rare. The realized rate plateaus at 0.65-0.68 above p_win 0.70, so no band has earned a top letter; a test sweeps every p_win 0..1 and asserts none produces one. Even 0.99 tops out at B+ with its realized 0.663 attached. Raising that ceiling later is a deliberate, visible act. Bands that cannot separate SAY so -- C+/C/C- carry separates_from_base_rate false and copy naming it, which is the honest description of a forecast explaining 3% of variance. Every grade states its basis (forecast_only vs forecast_plus_matchup_factors, naming which factors fired) and calibrated:false. engine1.grade is preserved as engine_grade so nothing downstream breaks. PHASE 2 — refusals render real states: insufficient_data -> "not enough history to call this one"; juiced_no_edge -> "the book has priced the vig past any edge on this side". 1,870 refused snapshots carry exactly those two reasons and both now surface. PHASE 3 — hand-verified on 12 real served props. Freeman/Rice/Encarnacion 0.95 overs now B+ (was B, C, B); the 0.05 unders now F (was C). Refused doubles render NO READ with their reason. never-blank PASS, no-manufactured-A PASS. Serving change; nine frozen model modules unchanged including engine1; p_win never mutated; no calibrated number leaks (deployed set empty); no Bonferroni slot. STILL TRUE: the forecast explains ~3% of outcome variance. This order did not make the model better. It made the letter stop overstating it. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01W1sivYNqY2TS5ftykmHBU9 --- specs/served-grade.md | 132 ++++++++++++++++++ .../intelligence/analyzeViaEngine1.js | 21 +++ src/services/model/servedGrade.js | 126 +++++++++++++++++ tests/unit/servedGrade.test.js | 106 ++++++++++++++ 4 files changed, 385 insertions(+) create mode 100644 specs/served-grade.md create mode 100644 src/services/model/servedGrade.js create mode 100644 tests/unit/servedGrade.test.js diff --git a/specs/served-grade.md b/specs/served-grade.md new file mode 100644 index 0000000..917293d --- /dev/null +++ b/specs/served-grade.md @@ -0,0 +1,132 @@ +# The grade surface — it wasn't missing, it was serving the weaker signal + +## PHASE 0 — what actually reaches a user + +**Correction to the order's premise: a grade letter has been served all along.** +`engine1.gradeProp` produces it from an additive factor index, and it is +**not derived from `p_win` at all** — the two are computed independently and both +ride the payload. + +`gradeBands` is orphaned for a different reason than assumed: it defines what a +letter *means* from realized outcomes, and every band it produces collapses to +base-rate at current resolution. It was never the missing link to the surface. + +### The measurement that changed the order + +3,417 settled props, pooled across the four batter stats: + +| grade | n | realized | mean p_win | +|---|---|---|---| +| **A** | 8 | **0.5000** | 0.6466 | +| B | 985 | 0.6396 | 0.7002 | +| C | 1,695 | 0.6024 | 0.6755 | +| D | 303 | 0.5578 | 0.6042 | +| F | 426 | 0.5352 | 0.5875 | + +``` +grade-letter resolution 0.00116 (0.48% of variance) +p_win resolution 0.00715 (2.98%) +=> the letter carries 0.16x the information of the number beside it +``` + +**The top grade hit worse than the bottom grade.** And concretely, from the +hand-verify: **Christian Encarnación's 0.95 over graded `C`, and his 0.05 under +also graded `C`** — same hitter, opposite forecasts, same letter. + +The gap was never that grades don't ship. It is that **the weaker of two +available signals was shipping as the headline.** + +--- + +## PHASE 1 — the honest grade + +`model/servedGrade.js` derives the letter from `p_win`, with bands anchored on +**measured realized rates**, not targets: + +| letter | p_win ≥ | realized | separates from base rate? | +|---|---|---|---| +| B+ | 0.780 | 0.663 | yes | +| B | 0.700 | 0.646 | yes | +| C+ | 0.640 | 0.615 | **no** | +| C | 0.560 | 0.589 | **no** | +| C- | 0.480 | 0.548 | **no** | +| D | 0.350 | 0.512 | yes | +| F | 0.000 | 0.447 | yes | + +Base rate 0.6005. + +### No manufactured A — structurally + +**`A+`, `A` and `A-` are UNISSUABLE.** Not rare — absent by construction. The +realized rate plateaus at 0.65–0.68 above p_win 0.70 (the 0.9+ bucket does no +better than the 0.8 bucket), so no band of this forecast has earned a top letter. +A test sweeps every p_win from 0 to 1 and asserts none produces one. Even a 0.99 +forecast tops out at B+ with its realized 0.663 attached. + +When resolution improves enough to earn an A, the ceiling gets raised +deliberately and visibly — not by a threshold quietly drifting. + +### Bands that cannot separate SAY so + +`C+ / C / C-` carry `separates_from_base_rate: false` and copy that names it — +*"a base-rate read; the model sees nothing that separates this."* That covers the +bulk of the board, and it is the honest description of a forecast explaining 3% +of variance. + +### The basis is stated, never implied + +Each grade carries `basis`: `forecast_plus_matchup_factors` (naming which of the +three proven factors fired) or `forecast_only`, plus `calibrated: false` — +calibration is withdrawn and nothing here rides on a number that doesn't exist. + +`engine1.grade` is preserved as `engine_grade` so nothing downstream breaks and +the two stay comparable. + +--- + +## PHASE 2 — the refusal surface + +Refusals render a real state, never a blank or a fabricated number: + +- `insufficient_data` → **NO READ** — *"not enough history to call this one"* +- `juiced_no_edge` → **NO READ** — *"the book has priced the vig past any edge on + this side"* + +1,870 refused snapshots carry exactly these two reasons, and both now surface. +`projectionFor` reads the repaired full-window reference, so refusals are +computed on the repaired champion. + +--- + +## PHASE 3 — hand-verified on real served props + +| prop | p_win | OLD | NEW | separates | state | +|---|---|---|---|---|---| +| Freddie Freeman hits 0.5o | 0.95 | B | **B+** | true | graded | +| Christian Encarnación hits 0.5o | 0.95 | **C** | **B+** | true | graded | +| Ben Rice hits 0.5o | 0.95 | B | **B+** | true | graded | +| Christian Encarnación hits 0.5u | 0.05 | **C** | **F** | true | graded | +| Ben Rice hits 0.5u | 0.05 | C | **F** | true | graded | +| Eliezer Alfonso Jr doubles 0.5o | — | — | **NO READ** | — | refused | +| Eliezer Alfonso Jr doubles 0.5u | — | — | **NO READ** (vig) | — | refused | +| Paul Goldschmidt doubles 0.5o | — | — | **NO READ** | — | refused | + +``` +never-blank check: PASS — every prop renders a label and a meaning +no-manufactured-A check: PASS +``` + +The Encarnación rows are the clearest evidence: under the old letter his 0.95 and +his 0.05 were both `C`. Under the new one they are `B+` and `F`. + +--- + +## Invariants + +Grades ride on repaired-champion raw `p_win` plus factors where they fire. No +calibrated number leaks — the deployed set is empty and `calibrated: false` is +stated on every grade. `p_win` never mutated. Nine frozen model modules verified +unchanged, `engine1` included. No Bonferroni slot — no new factor. + +**Still true and unchanged:** the forecast explains ~3% of outcome variance. This +order did not make the model better. It made the letter stop overstating it. diff --git a/src/services/intelligence/analyzeViaEngine1.js b/src/services/intelligence/analyzeViaEngine1.js index fe35a07..638e7b9 100644 --- a/src/services/intelligence/analyzeViaEngine1.js +++ b/src/services/intelligence/analyzeViaEngine1.js @@ -574,6 +574,27 @@ async function analyzeViaEngine1(rawProp = {}) { ? (Number.isFinite(pOver) ? 1 - pOver : null) : (Number.isFinite(pOver) ? pOver : null); if (pWin != null) legacy.p_win = Math.round(pWin * 1000) / 1000; + // ── THE SERVED GRADE ──────────────────────────────────────────────── + // Derived from the forecast, not from engine1's additive factor index. + // Measured on 3,417 settled props, that index carried 0.16x the information + // of the p_win printed beside it, and its A grade hit 0.500 while its F hit + // 0.535 -- the top letter did worse than the bottom. Concretely: the same + // hitter's 0.95 over and 0.05 under both graded C. + // + // engine1.grade is PRESERVED on the payload as `engine_grade` so nothing + // downstream breaks and the two remain comparable, but `served_grade` is + // what a user should see. + try { + const sg = require('../model/servedGrade'); + legacy.engine_grade = legacy.grade; + legacy.served_grade = sg.gradeFor({ + p_win: legacy.p_win, + refused: legacy.refused || legacy.insufficient_data, + refusal_reason: legacy.refusal_reason, + factor_adjustment: factorTrace, + }); + } catch { /* the grade surface must never break the read */ } + if (factorTrace) { legacy.factor_adjustment = factorTrace; legacy.p_win_prefactor = Math.round((dir === 'under' ? 1 - factorTrace.p_before : factorTrace.p_before) * 1000) / 1000; diff --git a/src/services/model/servedGrade.js b/src/services/model/servedGrade.js new file mode 100644 index 0000000..733e553 --- /dev/null +++ b/src/services/model/servedGrade.js @@ -0,0 +1,126 @@ +'use strict'; + +/** + * servedGrade — the letter a user sees, derived from the forecast. + * + * ── WHY THIS EXISTS ────────────────────────────────────────────────────── + * The letter being served came from engine1's additive factor index, computed + * independently of `p_win`. Measured on 3,417 settled props: + * + * grade n realized mean p_win + * A 8 0.500 0.647 <- the TOP grade did worst + * B 985 0.640 0.700 + * C 1,695 0.602 0.676 + * D 303 0.558 0.604 + * F 426 0.535 0.588 + * + * letter resolution 0.00116 (0.48% of variance) + * p_win resolution 0.00715 (2.98%) + * + * The letter carried ONE SIXTH the information of the number printed beside it, + * and its best grade hit worse than its worst. So the grade is derived from + * `p_win` here instead — not because p_win is good (2.98% is not good) but + * because shipping the weaker of two available signals as the headline is + * indefensible. + * + * ── NO MANUFACTURED A ──────────────────────────────────────────────────── + * S91 established the honest ceiling: once the numbers are truthful this model + * has no 80%-plus reads. The realized rate PLATEAUS around 0.65-0.68 from p_win + * 0.70 upward — the 0.9+ bucket does no better than the 0.8 bucket. + * + * So A and A+ are NOT ISSUABLE. Not "rare" — structurally absent, because no + * band of this forecast has ever realized a rate that would justify one. A test + * asserts that no input produces an A. When resolution improves enough for a + * band to earn it, the ceiling is raised deliberately and visibly, not by a + * threshold quietly drifting. + * + * ── EVERY LETTER CARRIES ITS OWN MEANING ───────────────────────────────── + * The band's realized rate travels with the grade, so the surface can state what + * a B actually means rather than implying a spread the model does not have. + */ + +const { knownNumber } = require('../../utils/known'); + +/** + * Bands over p_win, anchored on MEASURED realized rates (3,417 settled props, + * pooled across the four batter stats). + * + * The realized column is what this band has actually done — not a target, not a + * projection. `separates` says whether the band's realized rate is + * distinguishable from the pooled base rate; today only the extremes are. + */ +const BASE_RATE = 0.6005; +const BANDS = Object.freeze([ + { letter: 'B+', min: 0.780, realized: 0.663, separates: true, + meaning: 'the strongest read this model produces — realized about 66%' }, + { letter: 'B', min: 0.700, realized: 0.646, separates: true, + meaning: 'above this profile\'s base rate — realized about 65%' }, + { letter: 'C+', min: 0.640, realized: 0.615, separates: false, + meaning: 'slightly above base rate, not distinguishable from it' }, + { letter: 'C', min: 0.560, realized: 0.589, separates: false, + meaning: 'a base-rate read — the model sees nothing that separates this' }, + { letter: 'C-', min: 0.480, realized: 0.548, separates: false, + meaning: 'at or below base rate' }, + { letter: 'D', min: 0.350, realized: 0.512, separates: true, + meaning: 'below base rate — the model reads this as weak' }, + { letter: 'F', min: 0.000, realized: 0.447, separates: true, + meaning: 'well below base rate' }, +]); + +/** Letters this forecast cannot justify. Absent by construction, not by rarity. */ +const UNISSUABLE = Object.freeze(['A+', 'A', 'A-']); + +/** + * The grade for one served prop. + * + * @param {object} prop { p_win, refused, refusal_reason, factor_adjustment } + * @returns {object} always a renderable state — never null, never blank. + */ +function gradeFor(prop = {}) { + // ── REFUSAL IS A REAL STATE, NOT A BLANK ── + if (prop.refused || prop.insufficient_data) { + return { + letter: null, + state: 'refused', + label: 'NO READ', + meaning: prop.refusal_reason === 'juiced_no_edge' + ? 'the book has priced the vig past any edge on this side' + : 'not enough history to call this one', + basis: 'refusal', + separates: false, + }; + } + + const p = knownNumber(prop.p_win); + if (p === null) { + return { + letter: null, + state: 'no_forecast', + label: 'NO READ', + meaning: 'no forecast could be produced for this prop', + basis: 'absent', + separates: false, + }; + } + + const band = BANDS.find((b) => p >= b.min) || BANDS[BANDS.length - 1]; + const factored = Array.isArray(prop.factor_adjustment && prop.factor_adjustment.applied) + && prop.factor_adjustment.applied.length > 0; + + return { + letter: band.letter, + state: 'graded', + label: band.letter, + meaning: band.meaning, + band_realized_rate: band.realized, + separates_from_base_rate: band.separates, + base_rate: BASE_RATE, + // What the letter was computed from, stated so the surface cannot imply more. + basis: factored ? 'forecast_plus_matchup_factors' : 'forecast_only', + factors_applied: factored ? prop.factor_adjustment.applied.map((a) => a.factor) : [], + // Calibration is withdrawn; nothing here rides on a calibrated number. + calibrated: false, + }; +} + +module.exports = { gradeFor, BANDS, UNISSUABLE, BASE_RATE }; diff --git a/tests/unit/servedGrade.test.js b/tests/unit/servedGrade.test.js new file mode 100644 index 0000000..7c5fc70 --- /dev/null +++ b/tests/unit/servedGrade.test.js @@ -0,0 +1,106 @@ +'use strict'; + +/** + * The letter a user sees. + * + * What these protect: that no input manufactures an A, that a band which cannot + * be distinguished from the base rate SAYS so, and that nothing ever renders + * blank. + */ + +const sg = require('../../src/services/model/servedGrade'); + +describe('no manufactured A — structurally, not by rarity', () => { + it('no p_win produces an A, A- or A+', () => { + // The realized rate plateaus around 0.65-0.68 above p_win 0.70, so no band + // of this forecast has earned a top letter. This is the ceiling made real. + for (let p = 0; p <= 1.0001; p += 0.01) { + const g = sg.gradeFor({ p_win: p }); + expect(sg.UNISSUABLE).not.toContain(g.letter); + } + expect(sg.BANDS.some((b) => sg.UNISSUABLE.includes(b.letter))).toBe(false); + }); + + it('even a 0.99 forecast tops out at the strongest honest band', () => { + const g = sg.gradeFor({ p_win: 0.99 }); + expect(g.letter).toBe('B+'); + expect(g.band_realized_rate).toBeLessThan(0.70); + expect(g.meaning).toMatch(/strongest read/); + }); +}); + +describe('a band that cannot separate SAYS so', () => { + it('mid bands are flagged as not distinguishable from base rate', () => { + for (const p of [0.50, 0.58, 0.66]) { + const g = sg.gradeFor({ p_win: p }); + expect(g.separates_from_base_rate).toBe(false); + // The copy must not imply a separation the band does not have. + expect(g.meaning).toMatch(/base rate|base-rate/); + } + }); + + it('the extremes do separate, and are marked so', () => { + expect(sg.gradeFor({ p_win: 0.85 }).separates_from_base_rate).toBe(true); + expect(sg.gradeFor({ p_win: 0.20 }).separates_from_base_rate).toBe(true); + }); + + it('every band carries its own realized rate, not a target', () => { + for (const b of sg.BANDS) { + expect(b.realized).toBeGreaterThan(0); + expect(b.realized).toBeLessThan(0.70); // the honest ceiling + } + }); +}); + +describe('never blank', () => { + it('a refusal renders a real state with a reason', () => { + const g = sg.gradeFor({ refused: true, refusal_reason: 'insufficient_data' }); + expect(g.letter).toBeNull(); + expect(g.state).toBe('refused'); + expect(g.label).toBe('NO READ'); + expect(g.meaning).toMatch(/not enough history/); + }); + + it('a juiced-out side says the book ate the edge', () => { + expect(sg.gradeFor({ refused: true, refusal_reason: 'juiced_no_edge' }).meaning).toMatch(/vig/); + }); + + it('a missing forecast renders, rather than returning nothing', () => { + const g = sg.gradeFor({ p_win: null }); + expect(g.state).toBe('no_forecast'); + expect(g.label).toBe('NO READ'); + }); + + it('an empty prop still renders', () => { + expect(sg.gradeFor({}).label).toBe('NO READ'); + expect(sg.gradeFor().label).toBe('NO READ'); + }); +}); + +describe('the basis is stated, never implied', () => { + it('names when matchup factors moved the forecast', () => { + const g = sg.gradeFor({ p_win: 0.72, factor_adjustment: { applied: [{ factor: 'platoon_severity' }] } }); + expect(g.basis).toBe('forecast_plus_matchup_factors'); + expect(g.factors_applied).toEqual(['platoon_severity']); + }); + + it('says forecast-only when nothing fired', () => { + const g = sg.gradeFor({ p_win: 0.72 }); + expect(g.basis).toBe('forecast_only'); + expect(g.factors_applied).toEqual([]); + }); + + it('never claims calibration — the deployed set is empty', () => { + expect(sg.gradeFor({ p_win: 0.72 }).calibrated).toBe(false); + }); + + it('is monotone: a higher forecast never grades lower', () => { + const order = ['F', 'D', 'C-', 'C', 'C+', 'B', 'B+']; + let prev = -1; + for (let p = 0.02; p <= 0.98; p += 0.02) { + const i = order.indexOf(sg.gradeFor({ p_win: p }).letter); + expect(i).toBeGreaterThanOrEqual(prev); + prev = i; + } + }); +});