91927a4a8a
number beside it PHASE 0 corrects the order's premise. A grade letter has been served all along -- engine1.gradeProp builds it from an additive factor index, computed INDEPENDENTLY of p_win. gradeBands is orphaned for a different reason than assumed: it defines what a letter MEANS from realized outcomes, and every band collapses to base-rate at current resolution. The measurement that changed this order, on 3,417 settled props: grade n realized mean p_win A 8 0.500 0.647 <- the TOP grade did WORST B 985 0.640 0.700 C 1,695 0.602 0.676 D 303 0.558 0.604 F 426 0.535 0.588 letter resolution 0.00116 (0.48% of variance) p_win resolution 0.00715 (2.98%) -> the letter carried 0.16x the information of the number beside it Concretely, from the hand-verify: Christian Encarnacion's 0.95 over graded C and his 0.05 under ALSO graded C -- same hitter, opposite forecasts, same letter. The gap was never that grades don't ship; it is that the weaker of two available signals shipped as the headline. PHASE 1 — model/servedGrade.js derives the letter from p_win with bands anchored on MEASURED realized rates (B+ 0.663 / B 0.646 / C+ 0.615 / C 0.589 / C- 0.548 / D 0.512 / F 0.447, base 0.6005). NO MANUFACTURED A, structurally: A+/A/A- are UNISSUABLE, not rare. The realized rate plateaus at 0.65-0.68 above p_win 0.70, so no band has earned a top letter; a test sweeps every p_win 0..1 and asserts none produces one. Even 0.99 tops out at B+ with its realized 0.663 attached. Raising that ceiling later is a deliberate, visible act. Bands that cannot separate SAY so -- C+/C/C- carry separates_from_base_rate false and copy naming it, which is the honest description of a forecast explaining 3% of variance. Every grade states its basis (forecast_only vs forecast_plus_matchup_factors, naming which factors fired) and calibrated:false. engine1.grade is preserved as engine_grade so nothing downstream breaks. PHASE 2 — refusals render real states: insufficient_data -> "not enough history to call this one"; juiced_no_edge -> "the book has priced the vig past any edge on this side". 1,870 refused snapshots carry exactly those two reasons and both now surface. PHASE 3 — hand-verified on 12 real served props. Freeman/Rice/Encarnacion 0.95 overs now B+ (was B, C, B); the 0.05 unders now F (was C). Refused doubles render NO READ with their reason. never-blank PASS, no-manufactured-A PASS. Serving change; nine frozen model modules unchanged including engine1; p_win never mutated; no calibrated number leaks (deployed set empty); no Bonferroni slot. STILL TRUE: the forecast explains ~3% of outcome variance. This order did not make the model better. It made the letter stop overstating it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W1sivYNqY2TS5ftykmHBU9
107 lines
3.7 KiB
JavaScript
107 lines
3.7 KiB
JavaScript
'use strict';
|
|
|
|
/**
|
|
* The letter a user sees.
|
|
*
|
|
* What these protect: that no input manufactures an A, that a band which cannot
|
|
* be distinguished from the base rate SAYS so, and that nothing ever renders
|
|
* blank.
|
|
*/
|
|
|
|
const sg = require('../../src/services/model/servedGrade');
|
|
|
|
describe('no manufactured A — structurally, not by rarity', () => {
|
|
it('no p_win produces an A, A- or A+', () => {
|
|
// The realized rate plateaus around 0.65-0.68 above p_win 0.70, so no band
|
|
// of this forecast has earned a top letter. This is the ceiling made real.
|
|
for (let p = 0; p <= 1.0001; p += 0.01) {
|
|
const g = sg.gradeFor({ p_win: p });
|
|
expect(sg.UNISSUABLE).not.toContain(g.letter);
|
|
}
|
|
expect(sg.BANDS.some((b) => sg.UNISSUABLE.includes(b.letter))).toBe(false);
|
|
});
|
|
|
|
it('even a 0.99 forecast tops out at the strongest honest band', () => {
|
|
const g = sg.gradeFor({ p_win: 0.99 });
|
|
expect(g.letter).toBe('B+');
|
|
expect(g.band_realized_rate).toBeLessThan(0.70);
|
|
expect(g.meaning).toMatch(/strongest read/);
|
|
});
|
|
});
|
|
|
|
describe('a band that cannot separate SAYS so', () => {
|
|
it('mid bands are flagged as not distinguishable from base rate', () => {
|
|
for (const p of [0.50, 0.58, 0.66]) {
|
|
const g = sg.gradeFor({ p_win: p });
|
|
expect(g.separates_from_base_rate).toBe(false);
|
|
// The copy must not imply a separation the band does not have.
|
|
expect(g.meaning).toMatch(/base rate|base-rate/);
|
|
}
|
|
});
|
|
|
|
it('the extremes do separate, and are marked so', () => {
|
|
expect(sg.gradeFor({ p_win: 0.85 }).separates_from_base_rate).toBe(true);
|
|
expect(sg.gradeFor({ p_win: 0.20 }).separates_from_base_rate).toBe(true);
|
|
});
|
|
|
|
it('every band carries its own realized rate, not a target', () => {
|
|
for (const b of sg.BANDS) {
|
|
expect(b.realized).toBeGreaterThan(0);
|
|
expect(b.realized).toBeLessThan(0.70); // the honest ceiling
|
|
}
|
|
});
|
|
});
|
|
|
|
describe('never blank', () => {
|
|
it('a refusal renders a real state with a reason', () => {
|
|
const g = sg.gradeFor({ refused: true, refusal_reason: 'insufficient_data' });
|
|
expect(g.letter).toBeNull();
|
|
expect(g.state).toBe('refused');
|
|
expect(g.label).toBe('NO READ');
|
|
expect(g.meaning).toMatch(/not enough history/);
|
|
});
|
|
|
|
it('a juiced-out side says the book ate the edge', () => {
|
|
expect(sg.gradeFor({ refused: true, refusal_reason: 'juiced_no_edge' }).meaning).toMatch(/vig/);
|
|
});
|
|
|
|
it('a missing forecast renders, rather than returning nothing', () => {
|
|
const g = sg.gradeFor({ p_win: null });
|
|
expect(g.state).toBe('no_forecast');
|
|
expect(g.label).toBe('NO READ');
|
|
});
|
|
|
|
it('an empty prop still renders', () => {
|
|
expect(sg.gradeFor({}).label).toBe('NO READ');
|
|
expect(sg.gradeFor().label).toBe('NO READ');
|
|
});
|
|
});
|
|
|
|
describe('the basis is stated, never implied', () => {
|
|
it('names when matchup factors moved the forecast', () => {
|
|
const g = sg.gradeFor({ p_win: 0.72, factor_adjustment: { applied: [{ factor: 'platoon_severity' }] } });
|
|
expect(g.basis).toBe('forecast_plus_matchup_factors');
|
|
expect(g.factors_applied).toEqual(['platoon_severity']);
|
|
});
|
|
|
|
it('says forecast-only when nothing fired', () => {
|
|
const g = sg.gradeFor({ p_win: 0.72 });
|
|
expect(g.basis).toBe('forecast_only');
|
|
expect(g.factors_applied).toEqual([]);
|
|
});
|
|
|
|
it('never claims calibration — the deployed set is empty', () => {
|
|
expect(sg.gradeFor({ p_win: 0.72 }).calibrated).toBe(false);
|
|
});
|
|
|
|
it('is monotone: a higher forecast never grades lower', () => {
|
|
const order = ['F', 'D', 'C-', 'C', 'C+', 'B', 'B+'];
|
|
let prev = -1;
|
|
for (let p = 0.02; p <= 0.98; p += 0.02) {
|
|
const i = order.indexOf(sg.gradeFor({ p_win: p }).letter);
|
|
expect(i).toBeGreaterThanOrEqual(prev);
|
|
prev = i;
|
|
}
|
|
});
|
|
});
|