builtbykev/vyndr
1
0
Fork 0
Code Issues Pull Requests Actions 6 Packages Projects Releases Wiki Activity

WNBA truth correction + THE p_win FLIP (live, rollback armed)

Browse Source 
PART A -- WNBA TRUTH CORRECTION (no behaviour change).
WNBA does not "abstain" and is not "anti-predictive". The -0.12 that
produced those words was NBA-template machinery run on WNBA data -- WNBA
has never had its own archetypes, variables, conditions or calibration,
which is precisely the "sport stubbed in on another sport's template"
CLAUDE.md forbids. That is an UNBUILT MODEL'S EXPECTED FAILURE, not a
verdict on the sport; reading it as a verdict would quietly retire a sport
we never actually attempted. Its own build is QUEUED, after MLB.

The guard CODE is unchanged -- FORECAST_RANKED_SPORTS = {'mlb'} and the
inheritance test are correct live safety either way. Only the meaning is
corrected, and generalised into the doctrine-as-a-gate: a sport ranks on
p_win ONLY once its OWN model is built and shown to predict (calibration
AND resolution on its own holdout). Others are held out as NOT-BUILT,
never as failed. Re-labelled across gradeRanking, snapshot route, tests,
MASTER-PLAN and the challenger report.

PART B -- THE FLIP, gated on a full-slate re-run.

The re-run found something better than a bigger sample. An induced
snapshot graded 7 props: gradeAndCacheSlate runs with DEFAULT_LIMIT = 25
and ~72% of those refuse for insufficient_data, while 546 props are
gradeable. So 8 props IS the board, structurally -- not a small sample of
it. Logged as its own finding; the cap is a separate order.

For a statistically meaningful delta I used 11 real historical boards
(n=328, board sizes 14-57): 79.9% of rows move, mean 5.16 places per
board, TOP READ CHANGES ON 9 OF 11 BOARDS. The re-ordering holds at real
board size. Query committed.

FLIPPED:
- rankGrades drops its edge key (safe for every sport: removes a
  non-predictive tiebreak without putting p_win in front).
- selectTopGrades leads on forecast_rank, edge key removed.
- flattenToEdgeBoard sorts on forecastRank, not edge -- this board had
  edge as its PRIMARY key, so the whole mobile board was ordered by a
  quantity measured not to predict.
- forecast_rank threaded onto strip props.

Sports whose model is not built supply no forecast_rank, so their boards
fall through to the unchanged grade chain -- the fallback is the guard.

ROLLBACK ARMED: boards sort by forecast_rank WHEN PRESENT, so
FORECAST_RANK=0 reverts every surface on the next response -- no deploy,
no client release.

Edge is still computed, stored, carried and displayed as a labelled
diagnostic. Retired from ranking, not deleted.

Eight superseded tests updated to strictly stronger INVERSE properties --
they now fail if edge is ever re-introduced as a ranking key, which the
originals could not detect.

Gates: 4,045 tests / 323 suites green; next build exit 0.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QJs13VsyiSKYQP6rj3NNmc
This commit is contained in:
Kev
2026-08-01 01:55:43 -04:00
parent ef4ac60b81
commit 6c97f59546
9 changed files with 222 additions and 90 deletions
Download Patch File Download Diff File Expand all files Collapse all files
tests/unit/gradeBoardSort.test.js
+48 -18
View File
@@ -14,37 +14,67 @@ const { isTakeable: backendIsTakeable } = require('../../src/config/valueEngine'
const names = (rows) => rows.map((r) => r.player);
describe('selectTopGrades — signed signal, missing sorts LAST', () => {
test('a DISAGREEMENT does not outrank an AGREEMENT at equal grade+confidence', () => {
// edge is signed by direction: positive = model agrees with the graded side.
// Old |edge| ranked -80 (strong disagreement) above +10 (mild agreement).
const grades = [
{ player: 'disagrees', grade: 'B', confidence: 50, edge: -80 },
{ player: 'agrees', grade: 'B', confidence: 50, edge: 10 },
// SUPERSEDED 2026-08-01 by the p_win flip. These three asserted that EDGE
// ordered the board (signed, nulls last). Edge no longer ranks anything:
// measured on n=200 settled MLB rows, corr(edge, outcome) = -0.010 under the
// incumbent ruler and -0.022 under the consensus ruler, against
// corr(p_win, outcome) = +0.26.
//
// The replacements are strictly STRONGER — they fail if edge is ever
// re-introduced as a ranking key, which the originals could not detect.
describe('selectTopGrades — EDGE CANNOT RANK (retired 2026-08-01)', () => {
test('flipping edge from -80 to +80 does NOT change the order', () => {
const a = [
{ player: 'first', grade: 'B', confidence: 50, edge: -80 },
{ player: 'second', grade: 'B', confidence: 50, edge: 10 },
];
expect(names(adapter.selectTopGrades(grades, 10))).toEqual(['agrees', 'disagrees']);
const b = [
{ player: 'first', grade: 'B', confidence: 50, edge: 80 },
{ player: 'second', grade: 'B', confidence: 50, edge: -10 },
];
// Equal grade + confidence + no p_win -> stable input order, edge irrelevant.
expect(names(adapter.selectTopGrades(a, 10))).toEqual(['first', 'second']);
expect(names(adapter.selectTopGrades(b, 10))).toEqual(['first', 'second']);
});
test('a MISSING signal sorts LAST and is still PRESENT (never dropped)', () => {
test('a MISSING edge is no longer penalised — rows are still all PRESENT', () => {
const grades = [
{ player: 'no-signal', grade: 'B', confidence: 50 },
{ player: 'weak-but-real', grade: 'B', confidence: 50, edge: 0.1 },
{ player: 'negative-but-real', grade: 'B', confidence: 50, edge: -5 },
];
const out = names(adapter.selectTopGrades(grades, 10));
expect(out).toEqual(['weak-but-real', 'negative-but-real', 'no-signal']);
expect(out).toHaveLength(3); // present, not dropped
expect(out[out.length - 1]).toBe('no-signal');
expect(out).toHaveLength(3); // never dropped
expect(out).toEqual(['no-signal', 'weak-but-real', 'negative-but-real']); // input order
});
test('null/empty-string edge is absent, NOT zero (Number(null) === 0 guard)', () => {
test('forecast_rank LEADS the chain when the server supplied it', () => {
const grades = [
{ player: 'nullish', grade: 'B', confidence: 50, edge: null },
{ player: 'empty', grade: 'B', confidence: 50, edge: '' },
{ player: 'real-negative', grade: 'B', confidence: 50, edge: -1 },
{ player: 'topGrade', grade: 'A', confidence: 90, forecast_rank: 3 },
{ player: 'topForecast', grade: 'C', confidence: 40, forecast_rank: 1 },
];
// A real negative beats two absents; absents keep input order at the bottom.
expect(names(adapter.selectTopGrades(grades, 10))).toEqual(['real-negative', 'nullish', 'empty']);
// p_win-first order wins over the letter: the letter measured r ~ 0.005 and
// is inverted, p_win measured +0.26.
expect(names(adapter.selectTopGrades(grades, 10))).toEqual(['topForecast', 'topGrade']);
});
test('a MISSING forecast_rank sorts LAST and falls back to the grade chain', () => {
const grades = [
{ player: 'noRank', grade: 'A', confidence: 90 },
{ player: 'ranked', grade: 'C', confidence: 40, forecast_rank: 2 },
];
expect(names(adapter.selectTopGrades(grades, 10))).toEqual(['ranked', 'noRank']);
});
test('forecast_rank null/empty is ABSENT, not 0 (Number(null) === 0 guard)', () => {
const grades = [
{ player: 'nullish', grade: 'B', confidence: 50, forecast_rank: null },
{ player: 'empty', grade: 'B', confidence: 50, forecast_rank: '' },
{ player: 'real', grade: 'C', confidence: 10, forecast_rank: 9 },
];
// rank 9 is a REAL rank and must beat two absents — a 0-coercion would
// have put the nulls first.
expect(names(adapter.selectTopGrades(grades, 10))[0]).toBe('real');
});
});