WNBA truth correction + THE p_win FLIP (live, rollback armed)

PART A -- WNBA TRUTH CORRECTION (no behaviour change).
WNBA does not "abstain" and is not "anti-predictive". The -0.12 that
produced those words was NBA-template machinery run on WNBA data -- WNBA
has never had its own archetypes, variables, conditions or calibration,
which is precisely the "sport stubbed in on another sport's template"
CLAUDE.md forbids. That is an UNBUILT MODEL'S EXPECTED FAILURE, not a
verdict on the sport; reading it as a verdict would quietly retire a sport
we never actually attempted. Its own build is QUEUED, after MLB.

The guard CODE is unchanged -- FORECAST_RANKED_SPORTS = {'mlb'} and the
inheritance test are correct live safety either way. Only the meaning is
corrected, and generalised into the doctrine-as-a-gate: a sport ranks on
p_win ONLY once its OWN model is built and shown to predict (calibration
AND resolution on its own holdout). Others are held out as NOT-BUILT,
never as failed. Re-labelled across gradeRanking, snapshot route, tests,
MASTER-PLAN and the challenger report.

PART B -- THE FLIP, gated on a full-slate re-run.

The re-run found something better than a bigger sample. An induced
snapshot graded 7 props: gradeAndCacheSlate runs with DEFAULT_LIMIT = 25
and ~72% of those refuse for insufficient_data, while 546 props are
gradeable. So 8 props IS the board, structurally -- not a small sample of
it. Logged as its own finding; the cap is a separate order.

For a statistically meaningful delta I used 11 real historical boards
(n=328, board sizes 14-57): 79.9% of rows move, mean 5.16 places per
board, TOP READ CHANGES ON 9 OF 11 BOARDS. The re-ordering holds at real
board size. Query committed.

FLIPPED:
- rankGrades drops its edge key (safe for every sport: removes a
  non-predictive tiebreak without putting p_win in front).
- selectTopGrades leads on forecast_rank, edge key removed.
- flattenToEdgeBoard sorts on forecastRank, not edge -- this board had
  edge as its PRIMARY key, so the whole mobile board was ordered by a
  quantity measured not to predict.
- forecast_rank threaded onto strip props.

Sports whose model is not built supply no forecast_rank, so their boards
fall through to the unchanged grade chain -- the fallback is the guard.

ROLLBACK ARMED: boards sort by forecast_rank WHEN PRESENT, so
FORECAST_RANK=0 reverts every surface on the next response -- no deploy,
no client release.

Edge is still computed, stored, carried and displayed as a labelled
diagnostic. Retired from ranking, not deleted.

Eight superseded tests updated to strictly stronger INVERSE properties --
they now fail if edge is ever re-introduced as a ranking key, which the
originals could not detect.

Gates: 4,045 tests / 323 suites green; next build exit 0.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QJs13VsyiSKYQP6rj3NNmc
This commit is contained in:
Kev
2026-08-01 01:55:43 -04:00
parent ef4ac60b81
commit 6c97f59546
9 changed files with 222 additions and 90 deletions
+33 -14
View File
@@ -25,22 +25,40 @@ describe('flattenToEdgeBoard', () => {
] }),
];
test('flattens every graded prop across games into ranked rows, edge desc', () => {
// SUPERSEDED 2026-08-01: the board no longer ranks on edge (corr -0.010/-0.022
// vs corr(p_win) = +0.26). With no forecast_rank supplied it falls through to
// GRADE rank, which is what this now asserts.
test('flattens every graded prop across games into ranked rows, grade order', () => {
const rows = flattenToEdgeBoard(cards);
expect(rows.map((r) => r.player)).toEqual(['Jokic', 'Edwards', 'Gordon', 'LaVine']); // 8.4, 5.2, 3.4, -1.2
expect(rows.map((r) => r.player)).toEqual(['Jokic', 'Edwards', 'Gordon', 'LaVine']);
expect(rows[0].rank).toBe(1);
expect(rows[3].rank).toBe(4);
expect(rows[1].live).toBe(true); // Edwards' game is live → row carries it
expect(rows[0].away).toBe('DEN'); expect(rows[0].home).toBe('MIN');
});
test('a null edge sorts LAST, never 0-coerced to the top', () => {
const rows = flattenToEdgeBoard([card({ playerStrips: [
strip('NoEdge', [{ stat: 'PTS', line: 20, side: 'O', grade: 'A', edge: null }]),
strip('HasEdge', [{ stat: 'PTS', line: 20, side: 'O', grade: 'C', edge: 2.0 }]),
// SUPERSEDED 2026-08-01 — replaced by the strictly stronger inverse: edge
// cannot move the board AT ALL. The original could not detect edge being
// re-introduced as a key; this fails immediately if it is.
test('EDGE CANNOT RANK — flipping every edge leaves the order identical', () => {
const build = (e1, e2) => flattenToEdgeBoard([card({ playerStrips: [
strip('First', [{ stat: 'PTS', line: 20, side: 'O', grade: 'A', edge: e1 }]),
strip('Second', [{ stat: 'PTS', line: 20, side: 'O', grade: 'A', edge: e2 }]),
] })]);
expect(rows[0].player).toBe('HasEdge'); // +2.0 beats a null (not 0)
expect(rows[1].player).toBe('NoEdge');
expect(build(null, 2.0).map((r) => r.player)).toEqual(['First', 'Second']);
expect(build(2.0, null).map((r) => r.player)).toEqual(['First', 'Second']);
expect(build(-30, 30).map((r) => r.player)).toEqual(['First', 'Second']);
});
test('forecastRank ORDERS the board and a missing rank sorts LAST', () => {
const rows = flattenToEdgeBoard([card({ playerStrips: [
strip('NoRank', [{ stat: 'PTS', line: 20, side: 'O', grade: 'A', edge: 99 }]),
strip('Ranked2', [{ stat: 'PTS', line: 21, side: 'O', grade: 'C', forecastRank: 2 }]),
strip('Ranked1', [{ stat: 'PTS', line: 22, side: 'O', grade: 'D', forecastRank: 1 }]),
] })]);
// rank 1 → rank 2 → unranked. Grade and edge are both overridden.
expect(rows.map((r) => r.player)).toEqual(['Ranked1', 'Ranked2', 'NoRank']);
expect(rows[0].rank).toBe(1);
});
test('awaiting / dead / ungraded rows are excluded from the board', () => {
@@ -72,9 +90,10 @@ describe('flattenToEdgeBoard', () => {
});
// P1-7 — an impossible edge (the pipeline's 60/100/140 placeholder) is not a
// market signal. It must be treated as absent so the board neither DISPLAYS
// nor RANKS on it — a fake +140% must never outrank a real +8.4%.
test('insane edge (>40%) is nulled so it neither displays nor ranks', () => {
// market signal and must not be DISPLAYED as a real percentage. The ranking
// half of this test is retired: edge ranks nothing now, so nulling it can no
// longer change an order. The display guard still matters and still holds.
test('insane edge (>40%) is nulled so it is never DISPLAYED as a real %', () => {
const rows = flattenToEdgeBoard([card({ playerStrips: [
strip('RealEdge', [{ stat: 'PTS', line: 20, side: 'O', grade: 'B', edge: 8.4 }]),
strip('BrokenEdge', [{ stat: 'PTS', line: 20, side: 'O', grade: 'A', edge: 140 }]),
@@ -82,8 +101,8 @@ describe('flattenToEdgeBoard', () => {
const broken = rows.find((r) => r.player === 'BrokenEdge');
const real = rows.find((r) => r.player === 'RealEdge');
expect(broken.edge).toBeNull(); // not displayed as a fake %
expect(real.edge).toBe(8.4); // real edge preserved
// real edge outranks the nulled placeholder despite the placeholder's higher grade
expect(rows[0].player).toBe('RealEdge');
expect(real.edge).toBe(8.4); // real edge preserved as a diagnostic
// Ordering is now grade-driven (A before B) and edge has no say in it.
expect(rows[0].player).toBe('BrokenEdge');
});
});
+35 -16
View File
@@ -46,25 +46,44 @@ describe('COMPUTATION and SORT survive — removing them would re-break the boar
expect(read('web/src/lib/gradeAdapter.js')).toMatch(/function computeEdge/);
});
test('the signed-edge sort fallback is INTACT (nulls last, no abs)', () => {
const src = read('web/src/lib/slateAdapter.js');
expect(src).toMatch(/descNullsLast\(a\.edge, b\.edge\)/);
// Strip comments before asserting the abs() bug is gone — the file's own
// doc-comment QUOTES the old broken key to explain the fix, and matching
// that would be a false positive.
const code = src.replace(/\/\*[\s\S]*?\*\//g, '').replace(/^\s*\/\/.*$/gm, '');
expect(code).not.toMatch(/Math\.abs\(numOr\(g\.edge/);
expect(code).toMatch(/edge: strictNum\(g\.edge\)/);
// SUPERSEDED 2026-08-01 by the p_win flip. These two asserted that the SIGNED
// EDGE SORT was intact. Edge no longer sorts anything — measured on n=200
// settled MLB rows, corr(edge, outcome) = -0.010 / -0.022 against
// corr(p_win, outcome) = +0.26. The retirement these tests guard is now
// deeper: edge is retired from RANKING as well as from hero DISPLAY.
//
// The replacements assert the INVERSE, which is strictly stronger: the sort
// keys must not mention edge at all, in either ranking function.
test('NO ranking function sorts on edge any more', () => {
const strip = (src) => src.replace(/\/\*[\s\S]*?\*\//g, '').replace(/^\s*\/\/.*$/gm, '');
for (const f of ['web/src/lib/slateAdapter.js', 'src/utils/gradeRanking.js']) {
const code = strip(read(f));
expect(code).not.toMatch(/descNullsLast\(a\.edge, b\.edge\)/);
expect(code).not.toMatch(/Math\.abs\(numOr\(g\.edge/);
expect(code).not.toMatch(/edge: strictNum\(g\.edge/);
}
});
test('the sort still orders a real signed edge correctly', () => {
test('edge is still CARRIED on the row — retired from ranking, not deleted', () => {
// Losing the record would be worse than mis-using it: edge stays computed,
// stored and displayed as a labelled diagnostic.
const code = read('web/src/lib/slateAdapter.js');
expect(code).toMatch(/edge: \(rec\.edge_pct != null/);
});
test('the sort is driven by forecast_rank, and edge cannot move it', () => {
const adapter = require('../../web/src/lib/slateAdapter');
const out = adapter.selectTopGrades([
{ player: 'disagrees', grade: 'B', confidence: 50, edge: -40 },
{ player: 'agrees', grade: 'B', confidence: 50, edge: 12 },
{ player: 'absent', grade: 'B', confidence: 50 },
], 10).map((g) => g.player);
expect(out).toEqual(['agrees', 'disagrees', 'absent']);
const names = (rows) => rows.map((g) => g.player);
// identical rows except edge → identical order
expect(names(adapter.selectTopGrades([
{ player: 'a', grade: 'B', confidence: 50, edge: -40 },
{ player: 'b', grade: 'B', confidence: 50, edge: 12 },
], 10))).toEqual(['a', 'b']);
// forecast_rank decides
expect(names(adapter.selectTopGrades([
{ player: 'worseGradeBetterForecast', grade: 'C', confidence: 10, forecast_rank: 1, edge: -40 },
{ player: 'betterGradeWorseForecast', grade: 'A', confidence: 99, forecast_rank: 2, edge: 40 },
], 10))).toEqual(['worseGradeBetterForecast', 'betterGradeWorseForecast']);
});
test('DeskShowcase keeps its own already-honest rendering', () => {
+48 -18
View File
@@ -14,37 +14,67 @@ const { isTakeable: backendIsTakeable } = require('../../src/config/valueEngine'
const names = (rows) => rows.map((r) => r.player);
describe('selectTopGrades — signed signal, missing sorts LAST', () => {
test('a DISAGREEMENT does not outrank an AGREEMENT at equal grade+confidence', () => {
// edge is signed by direction: positive = model agrees with the graded side.
// Old |edge| ranked -80 (strong disagreement) above +10 (mild agreement).
const grades = [
{ player: 'disagrees', grade: 'B', confidence: 50, edge: -80 },
{ player: 'agrees', grade: 'B', confidence: 50, edge: 10 },
// SUPERSEDED 2026-08-01 by the p_win flip. These three asserted that EDGE
// ordered the board (signed, nulls last). Edge no longer ranks anything:
// measured on n=200 settled MLB rows, corr(edge, outcome) = -0.010 under the
// incumbent ruler and -0.022 under the consensus ruler, against
// corr(p_win, outcome) = +0.26.
//
// The replacements are strictly STRONGER — they fail if edge is ever
// re-introduced as a ranking key, which the originals could not detect.
describe('selectTopGrades — EDGE CANNOT RANK (retired 2026-08-01)', () => {
test('flipping edge from -80 to +80 does NOT change the order', () => {
const a = [
{ player: 'first', grade: 'B', confidence: 50, edge: -80 },
{ player: 'second', grade: 'B', confidence: 50, edge: 10 },
];
expect(names(adapter.selectTopGrades(grades, 10))).toEqual(['agrees', 'disagrees']);
const b = [
{ player: 'first', grade: 'B', confidence: 50, edge: 80 },
{ player: 'second', grade: 'B', confidence: 50, edge: -10 },
];
// Equal grade + confidence + no p_win -> stable input order, edge irrelevant.
expect(names(adapter.selectTopGrades(a, 10))).toEqual(['first', 'second']);
expect(names(adapter.selectTopGrades(b, 10))).toEqual(['first', 'second']);
});
test('a MISSING signal sorts LAST and is still PRESENT (never dropped)', () => {
test('a MISSING edge is no longer penalised — rows are still all PRESENT', () => {
const grades = [
{ player: 'no-signal', grade: 'B', confidence: 50 },
{ player: 'weak-but-real', grade: 'B', confidence: 50, edge: 0.1 },
{ player: 'negative-but-real', grade: 'B', confidence: 50, edge: -5 },
];
const out = names(adapter.selectTopGrades(grades, 10));
expect(out).toEqual(['weak-but-real', 'negative-but-real', 'no-signal']);
expect(out).toHaveLength(3); // present, not dropped
expect(out[out.length - 1]).toBe('no-signal');
expect(out).toHaveLength(3); // never dropped
expect(out).toEqual(['no-signal', 'weak-but-real', 'negative-but-real']); // input order
});
test('null/empty-string edge is absent, NOT zero (Number(null) === 0 guard)', () => {
test('forecast_rank LEADS the chain when the server supplied it', () => {
const grades = [
{ player: 'nullish', grade: 'B', confidence: 50, edge: null },
{ player: 'empty', grade: 'B', confidence: 50, edge: '' },
{ player: 'real-negative', grade: 'B', confidence: 50, edge: -1 },
{ player: 'topGrade', grade: 'A', confidence: 90, forecast_rank: 3 },
{ player: 'topForecast', grade: 'C', confidence: 40, forecast_rank: 1 },
];
// A real negative beats two absents; absents keep input order at the bottom.
expect(names(adapter.selectTopGrades(grades, 10))).toEqual(['real-negative', 'nullish', 'empty']);
// p_win-first order wins over the letter: the letter measured r ~ 0.005 and
// is inverted, p_win measured +0.26.
expect(names(adapter.selectTopGrades(grades, 10))).toEqual(['topForecast', 'topGrade']);
});
test('a MISSING forecast_rank sorts LAST and falls back to the grade chain', () => {
const grades = [
{ player: 'noRank', grade: 'A', confidence: 90 },
{ player: 'ranked', grade: 'C', confidence: 40, forecast_rank: 2 },
];
expect(names(adapter.selectTopGrades(grades, 10))).toEqual(['ranked', 'noRank']);
});
test('forecast_rank null/empty is ABSENT, not 0 (Number(null) === 0 guard)', () => {
const grades = [
{ player: 'nullish', grade: 'B', confidence: 50, forecast_rank: null },
{ player: 'empty', grade: 'B', confidence: 50, forecast_rank: '' },
{ player: 'real', grade: 'C', confidence: 10, forecast_rank: 9 },
];
// rank 9 is a REAL rank and must beat two absents — a 0-coercion would
// have put the nulls first.
expect(names(adapter.selectTopGrades(grades, 10))[0]).toBe('real');
});
});
+17 -9
View File
@@ -55,18 +55,22 @@ describe('rankByForecast — the challenger', () => {
});
});
describe('rankGrades — the incumbent is UNTOUCHED (live ordering byte-identical)', () => {
it('still leads with the grade letter and still consults edge', () => {
describe('rankGrades — still grade-first (p_win leading is rankByForecast only)', () => {
it('leads with the grade letter, NOT p_win', () => {
const out = rankGrades([g('lowPwinA', 'A', 0.51), g('highPwinC', 'C', 0.74)]);
expect(out[0].player).toBe('lowPwinA'); // grade-first, unchanged
});
it('edge still breaks a true tie in the incumbent', () => {
// SUPERSEDED 2026-08-01: the edge KEY was removed from rankGrades entirely.
// Removing it is safe for every sport — it takes a non-predictive signal out
// without putting p_win in front (that is `rankByForecast`, gated to sports
// whose own model has passed).
it('edge can no longer break a tie — the key is gone for EVERY sport', () => {
const out = rankGrades([
g('lowEdge', 'B', 0.6, -110, { edge: 1 }),
g('highEdge', 'B', 0.6, -110, { edge: 9 }),
g('first', 'B', 0.6, -110, { edge: 1 }),
g('second', 'B', 0.6, -110, { edge: 9 }),
]);
expect(out[0].player).toBe('highEdge');
expect(out[0].player).toBe('first'); // stable input order, edge ignored
});
});
@@ -97,14 +101,18 @@ describe('rankingDelta — the challenger-first measurement', () => {
});
});
describe('per-sport doctrine — who may rank on the forecast', () => {
it('MLB may; WNBA may NOT (its p_win is anti-predictive, it abstains)', () => {
describe('per-sport doctrine — a sport ranks only once ITS OWN model is built', () => {
// WNBA is held out as NOT-BUILT, not as failed: its -0.12 came from
// NBA-template machinery run on WNBA data, which is an unbuilt model's
// expected failure rather than a verdict on the sport. Same live behaviour,
// corrected meaning.
it('MLB may (its own model passed); WNBA may not (its model is not built yet)', () => {
expect(ranksOnForecast('mlb')).toBe(true);
expect(ranksOnForecast('MLB')).toBe(true);
expect(ranksOnForecast('wnba')).toBe(false);
});
it('no sport inherits MLBs result — unknown sports are excluded', () => {
it('no sport inherits MLBs result — every unbuilt sport is held out', () => {
for (const s of ['nba', 'nfl', 'soccer', 'nhl', 'ncaab', '', null, undefined]) {
expect(ranksOnForecast(s)).toBe(false);
}