ff037e40c2
There is nothing to re-adjudicate. The proven set is empty and always has
been -- verified three ways: proven-status reports EMPTY, validatedSkills()
returns {} for every archetype, and zero conditioning entries have ever
reached PROVEN. The one PROVEN feature is recent_frequency_prior, which is the
incumbent counter itself, proven by the S78 ablation as ~100% of the
champion's resolution. It is the baseline every challenger is measured
against, not a conditioning interaction, and demoting it would leave the model
with nothing to grade from.
A correction to the premise: the cumulative gate did NOT catch a false
positive last session. It caught nothing, because there was nothing in the
proven set to catch. What it did was tighten alpha from 0.0026 to 0.0013
within one session, which demonstrated the mechanism working rather than a
demotion. So steps 3 and 4 -- demote, recalibrate -- are vacuous here, and
readjudicateAll says so plainly rather than glossing a no-op.
But the worry behind the order was well founded, and the audit found the real
exposure: promote() did not require the cumulative denominator. It checked n,
lift and CI, and nothing stopped a future session from testing eight
hypotheses, correcting by eight, and promoting on a p-value that would not
survive the programme's real denominator. That is precisely the hole that
makes a retroactive re-adjudication pass necessary later, so it is closed at
promotion time instead. isSufficient now refuses evidence carrying no
correction, evidence corrected against fewer tests than the cumulative count,
and any p-value that does not clear 0.05 over its own test count. The same
rule guards a PROVEN conditioning entry.
The second audit found two of four analysis scripts still correcting
per-session; pitcher-prove-k and tb-solo-and-interactions now use the
cumulative ledger, so the correction is native on every path.
reAblation.js is the standing second line: pure and injectable, so the
decision rule cannot drift from the gate's, and every verdict records both
p-values and both test counts so a demotion is re-derivable by anyone. A
feature promoted at alpha 0.05/20 can demote on the same p-value once the bar
is 0.05/60 -- correct, because the bar rose only after the programme had more
chances to get lucky. No fresh measurement is PENDING_RETEST and never a
demotion: absence of a re-test is not evidence, and demoting on it would
punish whichever stat happens to be off-season.
Net effect on the proven set is zero. No demotions, no recalibrations, and no
public ledger event -- announcing "recalibrated after re-adjudication" when
nothing changed would itself be a false signal of rigour.
4,238 tests green (337 suites); web build exit 0; counter byte-identical.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W1sivYNqY2TS5ftykmHBU9
252 lines
14 KiB
JavaScript
252 lines
14 KiB
JavaScript
'use strict';
|
|
|
|
/**
|
|
* THE FIVE DISCIPLINES, asserted as behaviour.
|
|
*
|
|
* These are not coverage tests. Each block locks one discipline so it cannot be
|
|
* quietly abandoned later — which is exactly how the previous engine drifted
|
|
* into a frequency counter with six wired-but-dead features hanging off it.
|
|
*/
|
|
|
|
const reg = require('../../src/services/model/featureRegistry');
|
|
const sk = require('../../src/services/model/skillProjection');
|
|
|
|
const BOMBER = { k_pct: 0.28, bb_pct: 0.11, barrel_pct: 0.16, hard_hit_pct: 0.52, avg_exit_velo: 93.5, avg_launch_angle: 19 };
|
|
const GHOST = { k_pct: 0.14, bb_pct: 0.07, barrel_pct: 0.02, hard_hit_pct: 0.30, avg_exit_velo: 85.0, avg_launch_angle: 6 };
|
|
const ACE = { k_pct: 0.32, bb_pct: 0.05, hard_hit_pct: 0.30, whiff_pct: 0.33 };
|
|
const BATTING_PRACTICE = { k_pct: 0.15, bb_pct: 0.10, hard_hit_pct: 0.46, whiff_pct: 0.18 };
|
|
|
|
afterEach(() => reg.__reset());
|
|
|
|
// ── DISCIPLINE 3 ──────────────────────────────────────────────────────────
|
|
describe('D3 — earn its place or it is out', () => {
|
|
it('starts with an essentially EMPTY proven set — nothing is assumed to work', () => {
|
|
const live = reg.liveFeatures('mlb');
|
|
// Only the incumbent counter is PROVEN, and only because it was measured.
|
|
expect([...live]).toEqual(['recent_frequency_prior']);
|
|
});
|
|
|
|
it('refuses promotion without a real sample, positive lift, and a CI above zero', () => {
|
|
expect(reg.promote('mlb', 'batter_barrel_pct', null).ok).toBe(false);
|
|
expect(reg.promote('mlb', 'batter_barrel_pct', { n: 10, lift: 0.05, ci95: [0.01, 0.09] }).ok).toBe(false); // n too small
|
|
expect(reg.promote('mlb', 'batter_barrel_pct', { n: 500, lift: -0.02, ci95: [-0.05, -0.01] }).ok).toBe(false); // negative lift
|
|
expect(reg.promote('mlb', 'batter_barrel_pct', { n: 500, lift: 0.03, ci95: [-0.01, 0.07] }).ok).toBe(false); // CI straddles zero
|
|
expect(reg.statusOf('mlb', 'batter_barrel_pct')).toBe(reg.STATUS.CANDIDATE);
|
|
});
|
|
|
|
it('promotes on sufficient evidence, and records WHY', () => {
|
|
// The correction is now part of "sufficient": evidence must carry the
|
|
// CUMULATIVE bonferroni_tests it was corrected against, so a promotion can
|
|
// never use a laxer bar than the programme has earned.
|
|
const ev = {
|
|
n: 600, lift: 0.04, ci95: [0.012, 0.068], measured_at: '2026-09-01',
|
|
bonferroni_tests: 38, p_value: 0.0005,
|
|
};
|
|
expect(reg.promote('mlb', 'batter_barrel_pct', ev, null, { cumulativeTests: 38 }).ok).toBe(true);
|
|
expect(reg.isLive('mlb', 'batter_barrel_pct')).toBe(true);
|
|
expect(reg.liveFeatures('mlb').has('batter_barrel_pct')).toBe(true);
|
|
});
|
|
|
|
it('keeps DEAD features BY NAME so they are not silently rebuilt', () => {
|
|
const s = reg.summary('mlb');
|
|
const deadKeys = s.dead.map((d) => d.key);
|
|
expect(deadKeys).toEqual(expect.arrayContaining([
|
|
'champion_opp_rank_adj', 'champion_home_away_adj', 'champion_consistency_pull',
|
|
]));
|
|
// and each carries the reason it died
|
|
expect(s.dead.find((d) => d.key === 'champion_home_away_adj').why).toMatch(/harmful|IMPROVED/i);
|
|
});
|
|
|
|
it('per-sport: a feature proven for MLB says nothing about another sport', () => {
|
|
reg.promote('mlb', 'batter_barrel_pct', { n: 600, lift: 0.04, ci95: [0.01, 0.07], bonferroni_tests: 38, p_value: 0.0005 }, null, { cumulativeTests: 38 });
|
|
expect(reg.isLive('nba', 'batter_barrel_pct')).toBe(false);
|
|
expect(reg.statusOf('nba', 'batter_barrel_pct')).toBeNull();
|
|
});
|
|
|
|
it('THE GATE: with only PROVEN features allowed, the skill model REFUSES', () => {
|
|
// The proven set carries no contact-quality feature yet, so there is no
|
|
// forward read to make. Refusing is the correct output — this is what stops
|
|
// an unproven model reaching a user by accident.
|
|
const out = sk.projectSkill({
|
|
batter: BOMBER, pitcher: ACE, archetype: 'BOMBER', statType: 'hits', line: 0.5,
|
|
expectedPa: 4.2, allowed: reg.liveFeatures('mlb'),
|
|
});
|
|
expect(out).toBeNull();
|
|
});
|
|
|
|
it('with CANDIDATES allowed (the challenger), it produces a read', () => {
|
|
const out = sk.projectSkill({
|
|
batter: BOMBER, pitcher: ACE, archetype: 'BOMBER', statType: 'hits', line: 0.5,
|
|
expectedPa: 4.2, allowed: reg.candidateFeatures('mlb'),
|
|
});
|
|
expect(out).not.toBeNull();
|
|
expect(out.p_over_line).toBeGreaterThan(0);
|
|
});
|
|
});
|
|
|
|
describe('D3 — verdicts are PER STAT, because the physics differ', () => {
|
|
it('a feature can be DEAD for one stat and still CANDIDATE for another', () => {
|
|
// The first real gate run: every skill feature failed for `hits` (n=570,
|
|
// max |r| 0.062) while hard-hit rate showed the strongest marginal
|
|
// correlation yet seen for `total_bases` (r=0.153). A sport-wide DEAD mark
|
|
// would have killed, on hits evidence, the features most alive on TB.
|
|
reg.recordStatVerdict('mlb', 'hits', 'batter_barrel_pct', reg.STATUS.DEAD,
|
|
{ n: 570, r: -0.037, note: 'failed the gate for hits' });
|
|
expect(reg.statusForStat('mlb', 'hits', 'batter_barrel_pct')).toBe(reg.STATUS.DEAD);
|
|
expect(reg.statusForStat('mlb', 'total_bases', 'batter_barrel_pct')).toBe(reg.STATUS.CANDIDATE);
|
|
expect(reg.candidateFeaturesForStat('mlb', 'hits').has('batter_barrel_pct')).toBe(false);
|
|
expect(reg.candidateFeaturesForStat('mlb', 'total_bases').has('batter_barrel_pct')).toBe(true);
|
|
});
|
|
|
|
it('a stat with no verdict inherits the sport-level status', () => {
|
|
expect(reg.statusForStat('mlb', 'runs', 'batter_exit_velo')).toBe(reg.STATUS.CANDIDATE);
|
|
expect(reg.statusForStat('mlb', 'runs', 'champion_home_away_adj')).toBe(reg.STATUS.DEAD);
|
|
});
|
|
|
|
it('the live gate is per stat too, and still PROVEN-only', () => {
|
|
expect(reg.liveFeaturesForStat('mlb', 'hits').has('batter_barrel_pct')).toBe(false);
|
|
expect([...reg.liveFeaturesForStat('mlb', 'hits')]).toEqual(['recent_frequency_prior']);
|
|
});
|
|
});
|
|
|
|
// ── DISCIPLINE 1 ──────────────────────────────────────────────────────────
|
|
describe('D1 — skill, not results: the pitcher actually moves the read', () => {
|
|
const run = (pitcher) => sk.projectSkill({
|
|
batter: BOMBER, pitcher, archetype: 'BOMBER', statType: 'hits', line: 0.5, expectedPa: 4.2,
|
|
});
|
|
|
|
it('a better pitcher LOWERS the projection for the same hitter', () => {
|
|
const vsAce = run(ACE);
|
|
const vsBP = run(BATTING_PRACTICE);
|
|
expect(vsAce.p_over_line).toBeLessThan(vsBP.p_over_line);
|
|
expect(vsAce.projected_value).toBeLessThan(vsBP.projected_value);
|
|
});
|
|
|
|
it('this is the thing the counter cannot do — no pitcher at all is a different read', () => {
|
|
const noPitcher = run(null);
|
|
expect(noPitcher).not.toBeNull();
|
|
expect(noPitcher.pitcher_applied).toBe(false);
|
|
expect(run(ACE).pitcher_applied).toBe(true);
|
|
});
|
|
|
|
it('the odds ratio returns league when both sides are league (the identity)', () => {
|
|
expect(sk.oddsRatio(0.222, 0.222, 0.222)).toBeCloseTo(0.222, 6);
|
|
});
|
|
|
|
it('an average pitcher leaves the batter rate untouched', () => {
|
|
expect(sk.oddsRatio(0.30, 0.222, 0.222)).toBeCloseTo(0.30, 6);
|
|
});
|
|
|
|
it('shrinks a thin sample toward its anchor, and leaves a full one alone', () => {
|
|
expect(sk.shrink(0.60, 10, 200, 0.39)).toBeLessThan(0.45); // 10-PA callup
|
|
expect(sk.shrink(0.60, 5000, 200, 0.39)).toBeGreaterThan(0.59); // established
|
|
});
|
|
});
|
|
|
|
// ── DISCIPLINE 2 ──────────────────────────────────────────────────────────
|
|
describe('D2 — archetype SELECTS features, it does not nudge', () => {
|
|
it('the SAME hitter read through two archetypes gets materially different reads', () => {
|
|
const asBomber = sk.projectSkill({ batter: BOMBER, pitcher: ACE, archetype: 'BOMBER', statType: 'hits', line: 0.5, expectedPa: 4.2 });
|
|
const asGhost = sk.projectSkill({ batter: BOMBER, pitcher: ACE, archetype: 'GHOST', statType: 'hits', line: 0.5, expectedPa: 4.2 });
|
|
// Not a rounding difference — a different feature map entirely.
|
|
expect(Math.abs(asBomber.p_over_line - asGhost.p_over_line)).toBeGreaterThan(0.15);
|
|
});
|
|
|
|
it("a BOMBER's read is driven by barrels; a GHOST's is nearly blind to them", () => {
|
|
expect(sk.featureMapFor('BOMBER').hitWeights.barrel).toBeGreaterThan(0.4);
|
|
expect(sk.featureMapFor('GHOST').hitWeights.barrel).toBeLessThan(0.1);
|
|
expect(sk.featureMapFor('GHOST').hitWeights.gb_speed).toBeGreaterThan(0.5);
|
|
expect(sk.featureMapFor('BOMBER').hitWeights.gb_speed).toBe(0);
|
|
});
|
|
|
|
it('barrel rate moves a BOMBER and barely moves a GHOST — features silent where they do not apply', () => {
|
|
const move = (arch) => {
|
|
const lo = sk.hitOnContact({ batter: { ...BOMBER, barrel_pct: 0.04 }, pitcher: ACE, park: 1, archetype: arch });
|
|
const hi = sk.hitOnContact({ batter: { ...BOMBER, barrel_pct: 0.20 }, pitcher: ACE, park: 1, archetype: arch });
|
|
return hi - lo;
|
|
};
|
|
expect(move('BOMBER')).toBeGreaterThan(move('GHOST') * 3);
|
|
});
|
|
|
|
it('an unknown archetype falls back to a balanced map, not a refusal', () => {
|
|
const out = sk.projectSkill({ batter: BOMBER, pitcher: ACE, archetype: 'NOT_A_REAL_ONE', statType: 'hits', line: 0.5, expectedPa: 4.2 });
|
|
expect(out).not.toBeNull();
|
|
expect(out.archetype).toBe('NOT_A_REAL_ONE');
|
|
});
|
|
});
|
|
|
|
// ── HONESTY ───────────────────────────────────────────────────────────────
|
|
describe('honesty — unknown is not zero, and absent beats invented', () => {
|
|
it('no hitter profile → NO forward read (null), never a league-average guess', () => {
|
|
expect(sk.projectSkill({ batter: null, pitcher: ACE, statType: 'hits', line: 0.5 })).toBeNull();
|
|
expect(sk.projectSkill({ batter: {}, pitcher: ACE, statType: 'hits', line: 0.5 })).toBeNull();
|
|
});
|
|
|
|
it('a missing skill input is SILENT — it does not act as a measured zero', () => {
|
|
const full = sk.hitOnContact({ batter: BOMBER, pitcher: ACE, park: 1, archetype: 'BOMBER' });
|
|
const noBarrel = sk.hitOnContact({ batter: { ...BOMBER, barrel_pct: null }, pitcher: ACE, park: 1, archetype: 'BOMBER' });
|
|
const zeroBarrel = sk.hitOnContact({ batter: { ...BOMBER, barrel_pct: 0 }, pitcher: ACE, park: 1, archetype: 'BOMBER' });
|
|
expect(noBarrel).not.toBeNull();
|
|
// A REAL zero is a fact and must read far lower than an ABSENT one.
|
|
expect(zeroBarrel).toBeLessThan(noBarrel);
|
|
expect(Math.abs(noBarrel - full)).toBeLessThan(Math.abs(zeroBarrel - full));
|
|
});
|
|
|
|
it('total_bases is COMPOUND, not a relabelled hits curve', () => {
|
|
// The first cut multiplied hits by a constant bases-per-hit, which made
|
|
// P(TB>=2) EXACTLY equal to P(hits>=1) — a relabel carrying no information a
|
|
// hits model did not already have. It was refused rather than shipped. It is
|
|
// now a real convolution over per-PA base outcomes, and this test locks the
|
|
// property that distinguishes the two.
|
|
const tb = sk.projectSkill({ batter: BOMBER, pitcher: ACE, archetype: 'BOMBER', statType: 'total_bases', line: 1.5, expectedPa: 4.2 });
|
|
const hits = sk.projectSkill({ batter: BOMBER, pitcher: ACE, archetype: 'BOMBER', statType: 'hits', line: 0.5, expectedPa: 4.2 });
|
|
expect(tb).not.toBeNull();
|
|
expect(tb.family).toBe('pa_compound_bases_convolution');
|
|
expect(tb.p_over_line).not.toBeCloseTo(hits.p_over_line, 3); // NOT degenerate
|
|
expect(tb.distribution.reduce((a, b) => a + b, 0)).toBeCloseTo(1, 2);
|
|
});
|
|
|
|
it('hit-type shares respond to power skill — a slugger homers more per hit', () => {
|
|
const slugger = sk.hitTypeShares({ batter: BOMBER, archetype: 'BOMBER' });
|
|
const slap = sk.hitTypeShares({ batter: GHOST, archetype: 'GHOST' });
|
|
expect(slugger.homer).toBeGreaterThan(slap.homer * 3);
|
|
expect(slap.single).toBeGreaterThan(slugger.single);
|
|
// Shares are a distribution over hit types.
|
|
for (const s of [slugger, slap]) {
|
|
expect(s.single + s.double + s.triple + s.homer).toBeCloseTo(1, 6);
|
|
}
|
|
});
|
|
|
|
it('an absent power read leaves the shares at league — never a guessed lean', () => {
|
|
const noSkill = sk.hitTypeShares({ batter: { k_pct: 0.2 }, archetype: 'DEFAULT' });
|
|
expect(noSkill).toEqual(sk.LEAGUE_HIT_SHARES);
|
|
});
|
|
|
|
it('the distribution is a real distribution and P(>=k) is monotone', () => {
|
|
const out = sk.projectSkill({ batter: BOMBER, pitcher: ACE, archetype: 'BOMBER', statType: 'hits', line: 0.5, expectedPa: 4.2 });
|
|
expect(out.distribution.reduce((a, b) => a + b, 0)).toBeCloseTo(1, 2);
|
|
let prev = 1;
|
|
for (let k = 1; k <= 5; k += 1) {
|
|
const p = sk.atLeast(out.distribution, k);
|
|
expect(p).toBeLessThanOrEqual(prev + 1e-9);
|
|
prev = p;
|
|
}
|
|
});
|
|
|
|
it('a hitter exactly at league on every axis lands on league BABIP — no invented lean', () => {
|
|
const leagueBat = {
|
|
k_pct: sk.LEAGUE.k_pct, bb_pct: sk.LEAGUE.bb_pct,
|
|
barrel_pct: sk.LEAGUE.barrel_pct, hard_hit_pct: sk.LEAGUE.hard_hit_pct,
|
|
avg_exit_velo: sk.LEAGUE.avg_exit_velo, avg_launch_angle: 12,
|
|
};
|
|
const boc = sk.hitOnContact({ batter: leagueBat, pitcher: { hard_hit_pct: sk.LEAGUE.hard_hit_pct }, park: 1, archetype: 'DEFAULT' });
|
|
expect(boc).toBeCloseTo(sk.LEAGUE.babip, 2);
|
|
});
|
|
|
|
it('opportunity matters: more projected PA means a higher chance of one hit', () => {
|
|
const few = sk.projectSkill({ batter: BOMBER, pitcher: ACE, archetype: 'BOMBER', statType: 'hits', line: 0.5, expectedPa: 2 });
|
|
const many = sk.projectSkill({ batter: BOMBER, pitcher: ACE, archetype: 'BOMBER', statType: 'hits', line: 0.5, expectedPa: 5 });
|
|
expect(many.p_over_line).toBeGreaterThan(few.p_over_line);
|
|
});
|
|
});
|