Files
vyndr/tests/unit/gradeBands.test.js
T
builtbykev 3081c92e00 Per-archetype grade bands: built, gated, and the rescale blocked twice
The premise does not hold. proven-status.js run fresh: PROVEN_SET is
EMPTY, no archetype x stat reaches the gate. pitcher_contact_profile has
a CI upper bound of exactly 0.0000 and platoon_severity is held on
4.5%-contaminated splits, so the proven set is one factor, pooled, not
three archetype-conditioned ones. The specific pattern the order names --
defense strong for GHOST/BRUSH, null for BOMBER -- is the one I measured
running the OTHER WAY yesterday, both noise-dominated.

But the second blocker is new and matters more, because it would stop the
rescale even if the factors had proved: the grade does not separate
within any archetype. Every archetype collapses to ONE band at the
corrected bar, because bands merge when their intervals overlap and
publishing two letters we cannot tell apart is a distinction we have not
measured.

Uncorrected, so the ranking is visible rather than hidden by the bar,
this INVERTS the order's design. The order gives contact types the
factor-rich treatment and power types honest base-rate, reasoning that
single-game hits are variance for a power profile. Measured:

  BOMBER n=466  corr(p_win,outcome) +0.207  quintiles 0.75 0.62 0.60 0.48 0.48
  GHOST  n=192  corr(p_win,outcome) -0.007  quintiles 0.47 0.63 0.74 0.58 0.45

BOMBER is the one archetype the model ranks, and it splits into a real
A 0.660 / B 0.481 at 95%. GHOST is flat, and non-monotone -- its most
confident reads hit 47% while its middle reads hit 74%. Shipping as
specified would have given the factor-rich treatment to the archetype the
model reads worst and left base-rate on the one it reads best. That is
mechanically sensible in hindsight: a power hitter's hit tracks whether
he can damage the arm, a contact hitter's depends on balls finding holes.

BOMBER's split does not survive the cumulative correction at 106 tests.
Exposing it by loosening the correction is the curve-to-make-A's the
order forbids, so it stays one band.

BUILT: gradeBands.js -- lift against the archetype's OWN base rate (the
same 62% is lift for a 45% profile and a deficit for a 68% one),
indistinguishable neighbours merged, thin bands PROVISIONAL not dropped,
Wilson intervals widened by the cumulative correction. The two-bar rule
is structural: proven-alone, calibrated-alone and neither all return
base_rate with the reason stated, so with nothing proven no
factor-informed band can be produced at all.

reasoning() is built and tested but NOT wired to the card -- there is no
per-archetype band being served, so attaching the copy now would ship
product language for a rescale that does not exist.

NOT BUILT: the specified power-type reason "the matchup edge is in
total_bases". total_bases is recorded INCONCLUSIVE (+0.0038, CI
[-0.068,+0.075]). Wiring it would assert an edge measured as
indistinguishable from zero -- the exact fabricated-reason failure this
module exists to prevent.

BOMBER x hits is 29 rows short of the gate and is the archetype the model
actually reads. That is the first slot to test, not GHOST.

Counter and frozen clusters byte-identical. No letter was moved.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W1sivYNqY2TS5ftykmHBU9
2026-08-06 01:03:10 -04:00

154 lines
6.9 KiB
JavaScript

'use strict';
/**
* What a letter is allowed to mean.
*
* The failure these guard against is the one the whole programme keeps circling:
* a grade that LOOKS like it read tonight's matchup while reading nothing. Here
* that would be a band labelled factor-informed on an archetype where no factor
* ever proved — which is the state of every archetype today.
*/
const gb = require('../../src/services/model/gradeBands');
/** n rows whose outcome rate genuinely tracks p, interleaved (never front-loaded). */
function rows(specs) {
const out = [];
for (const [p, n, rate] of specs) {
for (let i = 0; i < n; i += 1) {
const won = Math.floor((i + 1) * rate) > Math.floor(i * rate) ? 1 : 0;
out.push({ p, won });
}
}
return out;
}
describe('the two-bar rule is structural', () => {
const data = rows([[0.75, 120, 0.75], [0.6, 120, 0.6], [0.45, 120, 0.45]]);
it('with NOTHING proven — the state today — no band can be factor-informed', () => {
const out = gb.buildBands(data, { archetype: 'GHOST' });
expect(out.basis).toBe('base_rate');
expect(out.two_bar.factor_informed_allowed).toBe(false);
expect(out.two_bar.reason).toMatch(/no factor has passed the gate/);
expect(out.bands.every((b) => b.basis === 'base_rate')).toBe(true);
});
it('PROVEN but not calibrated is still a base-rate read, and names why', () => {
// Both bars, or neither claim. A proven factor whose numbers are not
// certified honest cannot carry a letter that asserts a rate.
const out = gb.buildBands(data, { archetype: 'GHOST', proven: true });
expect(out.basis).toBe('base_rate');
expect(out.two_bar.reason).toMatch(/not certified calibrated/);
});
it('CALIBRATED but not proven is still a base-rate read', () => {
const out = gb.buildBands(data, { archetype: 'GHOST', calibrated: true });
expect(out.basis).toBe('base_rate');
expect(out.two_bar.reason).toMatch(/no factor has passed the gate/);
});
it('only BOTH unlocks factor-informed', () => {
const out = gb.buildBands(data, { archetype: 'GHOST', proven: true, calibrated: true });
expect(out.basis).toBe('factor_informed');
expect(out.bands.every((b) => b.basis === 'factor_informed')).toBe(true);
});
});
describe('lift is measured against the archetype\'s OWN base rate', () => {
it('a band is only credited when its interval clears the base rate', () => {
const out = gb.buildBands(rows([[0.8, 200, 0.80], [0.5, 200, 0.50], [0.3, 200, 0.30]]), {
archetype: 'GHOST', targetBands: 3,
});
const top = out.bands[0];
expect(top.shows_lift).toBe(true);
expect(top.separation).toBe('above_base_rate');
expect(top.lift_vs_archetype_base).toBeGreaterThan(0);
const bottom = out.bands[out.bands.length - 1];
expect(bottom.shows_deficit).toBe(true);
});
it('the same realized rate is lift for one archetype and not for another', () => {
// 62% is a real read for a profile that hits 45%, and slightly under water
// for one that hits 68%. A raw-rate band would call both the same letter.
const lowBase = gb.buildBands(rows([[0.62, 300, 0.62], [0.4, 300, 0.40]]), { archetype: 'LOW', targetBands: 2 });
const highBase = gb.buildBands(rows([[0.62, 300, 0.62], [0.75, 300, 0.75]]), { archetype: 'HIGH', targetBands: 2 });
expect(lowBase.bands[0].shows_lift).toBe(true);
expect(highBase.bands.find((b) => b.p_range[0] === 0.62).shows_deficit).toBe(true);
});
it('a band indistinguishable from the base rate claims NO lift', () => {
const flat = gb.buildBands(rows([[0.7, 150, 0.55], [0.5, 150, 0.55], [0.3, 150, 0.55]]), { archetype: 'FLAT' });
for (const b of flat.bands) {
expect(b.shows_lift).toBe(false);
expect(b.separation).toBe('indistinguishable_from_base_rate');
}
expect(flat.bands_showing_lift).toBe(0);
});
});
describe('bands the ledger cannot stand behind', () => {
it('indistinguishable neighbours are MERGED, not published as different letters', () => {
// If we cannot tell two bands apart, calling one A and one B is a
// distinction we have not measured.
const out = gb.buildBands(rows([[0.7, 200, 0.60], [0.65, 200, 0.60], [0.6, 200, 0.60]]), {
archetype: 'SAME', targetBands: 5,
});
expect(out.bands.length).toBe(1);
});
it('thin bands are PROVISIONAL, not silently dropped', () => {
// "Still counting" and "nothing here" are different claims.
const out = gb.buildBands(rows([[0.8, 20, 0.8], [0.3, 20, 0.3]]), { archetype: 'THIN', targetBands: 2 });
expect(out.bands.some((b) => b.provisional)).toBe(true);
});
it('too few outcomes → no bands at all, with the refusal stated', () => {
const out = gb.buildBands(rows([[0.6, 5, 0.6]]), { archetype: 'TINY' });
expect(out.bands).toEqual([]);
expect(out.refused).toMatch(/insufficient settled outcomes/);
});
it('the cumulative correction WIDENS every interval', () => {
const data = rows([[0.75, 150, 0.75], [0.45, 150, 0.45]]);
const one = gb.buildBands(data, { archetype: 'X', cumulativeTests: 1, targetBands: 2 });
const many = gb.buildBands(data, { archetype: 'X', cumulativeTests: 99, targetBands: 2 });
const width = (o) => o.bands[0].ci[1] - o.bands[0].ci[0];
expect(width(many)).toBeGreaterThan(width(one));
});
it('the interval never runs past 0 or 1', () => {
const out = gb.buildBands(rows([[0.99, 60, 1.0], [0.01, 60, 0.0]]), { archetype: 'EDGE', targetBands: 2 });
for (const b of out.bands) {
expect(b.ci[0]).toBeGreaterThanOrEqual(0);
expect(b.ci[1]).toBeLessThanOrEqual(1);
}
});
});
describe('the reason attached to a grade is true or absent', () => {
it('a base-rate band SAYS it is a base-rate read', () => {
const out = gb.buildBands(rows([[0.75, 120, 0.75], [0.45, 120, 0.45]]), { archetype: 'BOMBER', targetBands: 2 });
const text = gb.reasoning(out.bands[0], { archetype: 'BOMBER' });
expect(text).toMatch(/base-rate read/);
expect(text).toMatch(/no matchup factor is proven/);
// The thing it must never do is imply a matchup was read.
expect(text).not.toMatch(/matchup edge|favourable matchup/i);
});
it('a factor-informed band with NO named proven factors returns nothing', () => {
// Better to say nothing than to invent the why.
const out = gb.buildBands(rows([[0.75, 120, 0.75], [0.45, 120, 0.45]]),
{ archetype: 'GHOST', proven: true, calibrated: true, targetBands: 2 });
expect(gb.reasoning(out.bands[0], { archetype: 'GHOST', provenFactors: [] })).toBeNull();
});
it('a factor-informed band names the factors that actually proved', () => {
const out = gb.buildBands(rows([[0.75, 120, 0.75], [0.45, 120, 0.45]]),
{ archetype: 'GHOST', proven: true, calibrated: true, targetBands: 2 });
const text = gb.reasoning(out.bands[0], { archetype: 'GHOST', provenFactors: ['defense_by_direction'] });
expect(text).toMatch(/GHOST: defense_by_direction/);
expect(text).toMatch(/proved for this profile/);
});
});