Files
vyndr/tests/unit/rareEventSuppression.test.js
T
builtbykev 3591c7626e Total grade cutover + the ceiling stated as a position
PHASE 0 caught my own repeat of the failure I diagnosed one order ago.
91927a4 attached `served_grade` BESIDE the old letter and left `grade`
alone -- so the honest grade reached nobody, exactly as gradeBands had
been built-correct-and-unread. grep showed served_grade appearing in one
file (where I set it) and all 14+ consumers -- scan route, dashboard,
parlay, newsletter, desk, content templates, retention -- still reading
`.grade`, i.e. still the dishonest letter.

CUTOVER IS NOW TOTAL: legacy.grade IS the honest letter. Overwriting the
one field every consumer already reads cuts every surface over at once
instead of editing fourteen call sites and missing one. engine1's index is
preserved as `engine_grade` and verified read by ZERO serving code.

Confidence follows the letter: it came from a grade-band midpoint of the
OLD letter, so leaving it would have paired a served B+ with a C's
confidence. Both now derive from p_win, kept on the existing 0-100 scale.

MEASURED BLAST RADIUS before shipping: 303 of 47,991 non-refused
snapshots (0.6%) have a grade but no p_win, and now render NO READ instead
of a letter. That is correct -- their old letter came from the retired
index carrying 0.48% resolution, i.e. noise -- and NO READ is a rendered
state with a reason, so never-blank holds.

PHASE 1 — the ceiling is now a STATED POSITION, not a confusing absence.
servedGrade.SCALE_LEGEND plus web GradeScaleLegend.tsx say it plainly: we
do not issue A grades, no band has hit at a rate that would justify one,
our honest ceiling is a strong B+ (~66% realized vs ~60% baseline), and if
the model earns an A the legend changes and we say why. The
separates_from_base_rate flag renders per band -- C+/C/C- are labelled
"we cannot separate this from the baseline", which is most of any slate.

PHASE 3 hand-verified across every state: B+ with 3 factors (basis
forecast_plus_matchup_factors), B+ with none (forecast_only), C flagged
not-separable, F, and three refusal states rendering NO READ with reasons.
never-blank PASS, no-manufactured-A PASS.

Test fallout was real and is documented rather than papered over: engine
BEHAVIOUR assertions moved to engine_grade, suppression assertions stayed
on grade (a suppressed prop has no letter either way), and the confidence
78 -> 95 change is the grade-band midpoint being replaced by p_win.

No A-threshold loosening. No calibrated number leaks (deployed set empty).
p_win never mutated. Ten frozen modules verified unchanged including
engine1 and probabilityEstimator. No Bonferroni slot.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W1sivYNqY2TS5ftykmHBU9
2026-08-07 14:07:02 -04:00

175 lines
8.9 KiB
JavaScript

'use strict';
/**
* NOTE ON `engine_grade` (2026-08-07 grade cutover).
*
* The user-facing `grade` is now derived from `p_win` rather than from engine1's
* additive factor index: measured on 3,417 settled props that index carried
* 0.16x the information of p_win, and its A hit worse than its F. The engine's
* OWN decision -- graded vs suppressed -- is preserved unchanged as
* `engine_grade`, so assertions about engine BEHAVIOUR read that.
*
* Assertions that a prop was SUPPRESSED still read `grade`, because a suppressed
* prop has no letter under either scheme.
*
* Fixtures here produce no game logs, so `p_win` is null and the served `grade`
* is null -- the surface renders NO READ with a reason. That is deliberate and
* measured: 0.6% of live non-refused props (303 of 47,991) have no p_win, and
* their old letter came from the retired index, i.e. noise.
*/
// Betting-logic audit (2026-07-19) — rare-event 0.5 markets. The UNDER is
// always suppressed (juiced); the OVER grades only when the model genuinely
// projects the event above the line; and the CONSENSUS-vs-MODEL board drops
// no-model rows so a suppressed market can't take the under's rank.
// ── config helpers (pure) ───────────────────────────────────────────────────
const cfg = require('../../src/config/rareEventMarkets');
describe('rareEventMarkets config', () => {
test('the rare stats + threshold are config-driven (tunable, cross-sport)', () => {
expect(cfg.RARE_EVENT_STATS).toEqual(
expect.arrayContaining(['doubles', 'triples', 'home_runs', 'stolen_bases', 'steals', 'blocks']),
);
expect(cfg.RARE_EVENT_LINE_MAX).toBe(0.5);
expect(cfg.JUICE_ODDS_FLOOR).toBe(-400); // the general guard's default
});
test('UNDER on a rare 0.5 market is suppressed', () => {
for (const s of cfg.RARE_EVENT_STATS) {
expect(cfg.isSuppressedRareUnder(s, 0.5, 'under')).toBe(true);
}
expect(cfg.isSuppressedRareUnder('doubles', 0.5, 'over')).toBe(false); // over handled separately
expect(cfg.isSuppressedRareUnder('hits', 0.5, 'under')).toBe(false); // hits is NOT rare
expect(cfg.isSuppressedRareUnder('doubles', 1.5, 'under')).toBe(false); // 1.5 line is a diff market
});
test('OVER on a rare 0.5 market is suppressed UNLESS projection > line', () => {
expect(cfg.isSuppressedRareOver('doubles', 0.5, 'over', 0.2)).toBe(true); // 0.2 <= 0.5 → junk
expect(cfg.isSuppressedRareOver('doubles', 0.5, 'over', 0.5)).toBe(true); // == line → not genuine
expect(cfg.isSuppressedRareOver('doubles', 0.5, 'over', 0.7)).toBe(false); // 0.7 > 0.5 → genuine read
expect(cfg.isSuppressedRareOver('doubles', 0.5, 'under', 0.2)).toBe(false); // under path, not this fn
expect(cfg.isSuppressedRareOver('hits', 0.5, 'over', 0.2)).toBe(false); // not rare
});
});
describe('juice floor — the GENERAL, stat-agnostic guard', () => {
test('graded-side odds are read from the graded direction', () => {
expect(cfg.gradedSideOdds({ direction: 'under', under_odds: -800, over_odds: 120 })).toBe(-800);
expect(cfg.gradedSideOdds({ direction: 'over', under_odds: -800, over_odds: 120 })).toBe(120);
expect(cfg.parseAmerican('+600')).toBe(600);
expect(cfg.parseAmerican(null)).toBeNull();
});
test('a side priced past the floor is too juiced (any stat)', () => {
expect(cfg.isTooJuiced({ direction: 'under', under_odds: -800 })).toBe(true); // the doubles case
expect(cfg.isTooJuiced({ direction: 'under', under_odds: -1100 })).toBe(true);
expect(cfg.isTooJuiced({ direction: 'under', under_odds: -400 })).toBe(true); // == floor
});
test('normal + longshot prices are NOT juiced (legit markets preserved)', () => {
expect(cfg.isTooJuiced({ direction: 'under', under_odds: -110 })).toBe(false);
expect(cfg.isTooJuiced({ direction: 'under', under_odds: -250 })).toBe(false);
expect(cfg.isTooJuiced({ direction: 'over', over_odds: 600 })).toBe(false); // a real longshot over
});
test('no odds → cannot judge from price (falls back to the structural rule)', () => {
expect(cfg.isTooJuiced({ direction: 'under' })).toBe(false);
expect(cfg.isTooJuiced({ direction: 'under', under_odds: null })).toBe(false);
});
});
// ── analyzeViaEngine1 integration ───────────────────────────────────────────
const mockComputeReturn = { current: null };
jest.mock('../../src/services/intelligence/computeFeatures', () => ({
computeFeaturesForProp: async () => mockComputeReturn.current,
}));
const mockEngine1Return = { current: null };
jest.mock('../../src/services/intelligence/engine1', () => ({
gradeProp: () => mockEngine1Return.current,
}));
const { analyzeViaEngine1 } = require('../../src/services/intelligence/analyzeViaEngine1');
const feat = (l5, line, dir) => ({
features: { l5_avg: l5, l20_avg: l5 },
trap: {}, consistency: { consistency: 'reliable', score: 0.7 },
prop: { line, direction: dir }, meta: { sport: 'mlb', errors: [] },
});
beforeEach(() => {
mockComputeReturn.current = null;
mockEngine1Return.current = { grade: 'B', confidence: 0.55, top_factors: [], all_factors: [] };
});
describe('analyzeViaEngine1 — rare-event suppression', () => {
test('doubles UNDER 0.5 is REFUSED (grade null, suppressed) — no compute', async () => {
const out = await analyzeViaEngine1({ player: 'X', stat_type: 'doubles', line: 0.5, direction: 'under' });
expect(out.grade).toBeNull();
expect(out.suppressed).toBe(true);
expect(out.suppressed_reason).toBe('rare_event_under');
expect(out.insufficient_data).toBe(true); // filtered by gradeBestSide
});
test('doubles OVER 0.5 with projection 0.2 (<= line) is REFUSED', async () => {
mockComputeReturn.current = feat(0.2, 0.5, 'over');
const out = await analyzeViaEngine1({ player: 'X', stat_type: 'doubles', line: 0.5, direction: 'over' });
expect(out.grade).toBeNull();
expect(out.suppressed_reason).toBe('rare_event_over_below_line');
});
test('doubles OVER 0.5 with projection 0.7 (> line) GRADES — genuine event read', async () => {
mockComputeReturn.current = feat(0.7, 0.5, 'over');
const out = await analyzeViaEngine1({ player: 'X', stat_type: 'doubles', line: 0.5, direction: 'over' });
expect(out.engine_grade).toBe('B'); // real grade, not suppressed
expect(out.suppressed).toBeUndefined();
});
test('home_runs / stolen_bases / triples UNDER 0.5 all suppressed', async () => {
for (const s of ['home_runs', 'stolen_bases', 'triples']) {
const out = await analyzeViaEngine1({ player: 'X', stat_type: s, line: 0.5, direction: 'under' });
expect(out.grade).toBeNull();
expect(out.suppressed).toBe(true);
}
});
test('a NON-rare under (hits) is unaffected — still grades', async () => {
mockComputeReturn.current = feat(1.2, 0.5, 'under');
const out = await analyzeViaEngine1({ player: 'X', stat_type: 'hits', line: 0.5, direction: 'under' });
expect(out.engine_grade).toBe('B');
expect(out.suppressed).toBeUndefined();
});
test('a rare stat at a 1.5 line (not 0.5) is unaffected', async () => {
mockComputeReturn.current = feat(1.2, 1.5, 'under');
const out = await analyzeViaEngine1({ player: 'X', stat_type: 'home_runs', line: 1.5, direction: 'under' });
expect(out.engine_grade).toBe('B');
});
test('JUICE GUARD — a heavily-juiced side is refused for ANY stat, via price', async () => {
// hits is NOT a rare-event stat, but the book pricing it -800 means no edge.
const out = await analyzeViaEngine1({ player: 'X', stat_type: 'hits', line: 1.5, direction: 'under', under_odds: -800 });
expect(out.grade).toBeNull();
expect(out.suppressed_reason).toBe('juiced_no_edge');
});
test('JUICE GUARD — a normally-priced play still grades', async () => {
mockComputeReturn.current = feat(1.4, 0.5, 'over');
const out = await analyzeViaEngine1({ player: 'X', stat_type: 'hits', line: 0.5, direction: 'over', over_odds: -130 });
expect(out.engine_grade).toBe('B'); // -130 is fine → real read
});
});
// ── board layer: collectBreadth drops no-model rows ─────────────────────────
const { collectBreadth } = require('../../web/src/lib/marketBreadth');
describe('collectBreadth drops null-model rows (suppressed markets off the board)', () => {
const books = [{ book: 'dk', line: 0.5 }, { book: 'fd', line: 0.5 }];
test('a prop with no model (refused/ungraded) is not ranked onto the board', () => {
const rows = collectBreadth([
{ player: 'Real', stat: 'hits', side: 'over', line: 1.5, books: [{ book: 'dk', line: 1.5 }, { book: 'fd', line: 1.5 }], modelValue: 2.1 },
{ player: 'Suppressed', stat: 'doubles', side: 'over', line: 0.5, books, modelValue: null },
], 6);
expect(rows.map((r) => r.player)).toEqual(['Real']); // the null-model doubles is gone
});
});