Per-archetype grade bands: built, gated, and the rescale blocked twice
The premise does not hold. proven-status.js run fresh: PROVEN_SET is EMPTY, no archetype x stat reaches the gate. pitcher_contact_profile has a CI upper bound of exactly 0.0000 and platoon_severity is held on 4.5%-contaminated splits, so the proven set is one factor, pooled, not three archetype-conditioned ones. The specific pattern the order names -- defense strong for GHOST/BRUSH, null for BOMBER -- is the one I measured running the OTHER WAY yesterday, both noise-dominated. But the second blocker is new and matters more, because it would stop the rescale even if the factors had proved: the grade does not separate within any archetype. Every archetype collapses to ONE band at the corrected bar, because bands merge when their intervals overlap and publishing two letters we cannot tell apart is a distinction we have not measured. Uncorrected, so the ranking is visible rather than hidden by the bar, this INVERTS the order's design. The order gives contact types the factor-rich treatment and power types honest base-rate, reasoning that single-game hits are variance for a power profile. Measured: BOMBER n=466 corr(p_win,outcome) +0.207 quintiles 0.75 0.62 0.60 0.48 0.48 GHOST n=192 corr(p_win,outcome) -0.007 quintiles 0.47 0.63 0.74 0.58 0.45 BOMBER is the one archetype the model ranks, and it splits into a real A 0.660 / B 0.481 at 95%. GHOST is flat, and non-monotone -- its most confident reads hit 47% while its middle reads hit 74%. Shipping as specified would have given the factor-rich treatment to the archetype the model reads worst and left base-rate on the one it reads best. That is mechanically sensible in hindsight: a power hitter's hit tracks whether he can damage the arm, a contact hitter's depends on balls finding holes. BOMBER's split does not survive the cumulative correction at 106 tests. Exposing it by loosening the correction is the curve-to-make-A's the order forbids, so it stays one band. BUILT: gradeBands.js -- lift against the archetype's OWN base rate (the same 62% is lift for a 45% profile and a deficit for a 68% one), indistinguishable neighbours merged, thin bands PROVISIONAL not dropped, Wilson intervals widened by the cumulative correction. The two-bar rule is structural: proven-alone, calibrated-alone and neither all return base_rate with the reason stated, so with nothing proven no factor-informed band can be produced at all. reasoning() is built and tested but NOT wired to the card -- there is no per-archetype band being served, so attaching the copy now would ship product language for a rescale that does not exist. NOT BUILT: the specified power-type reason "the matchup edge is in total_bases". total_bases is recorded INCONCLUSIVE (+0.0038, CI [-0.068,+0.075]). Wiring it would assert an edge measured as indistinguishable from zero -- the exact fabricated-reason failure this module exists to prevent. BOMBER x hits is 29 rows short of the gate and is the archetype the model actually reads. That is the first slot to test, not GHOST. Counter and frozen clusters byte-identical. No letter was moved. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W1sivYNqY2TS5ftykmHBU9
This commit is contained in:
@@ -0,0 +1,100 @@
|
||||
#!/usr/bin/env node
|
||||
'use strict';
|
||||
|
||||
/**
|
||||
* build-grade-bands — publish what each letter actually means, per archetype.
|
||||
*
|
||||
* Runs the real settled ledger through gradeBands. Because no factor has passed
|
||||
* the gate for any archetype, every band comes back a BASE-RATE read — which is
|
||||
* the honest answer today, and the output states it rather than leaving a reader
|
||||
* to infer it.
|
||||
*
|
||||
* SUPABASE_URL=... node scripts/build-grade-bands.js
|
||||
*/
|
||||
|
||||
require('dotenv').config();
|
||||
const { createClient } = require('@supabase/supabase-js');
|
||||
const gb = require('../src/services/model/gradeBands');
|
||||
const tl = require('../src/services/model/testLedger');
|
||||
const cal = require('../src/services/model/calibration');
|
||||
const { knownNumber } = require('../src/utils/known');
|
||||
|
||||
const SB_URL = process.env.SUPABASE_URL;
|
||||
const SB_KEY = process.env.SUPABASE_SERVICE_ROLE_KEY || process.env.SUPABASE_SERVICE_KEY;
|
||||
const STAT = process.env.BAND_STAT || 'hits';
|
||||
const PAGE = 1000;
|
||||
|
||||
/**
|
||||
* PROVEN, PER ARCHETYPE. Empty, and that is the measured state — see
|
||||
* specs/per-archetype-re-audit.md. Nothing may be added here that has not
|
||||
* cleared the gate FOR THAT ARCHETYPE; pooled proof does not qualify a slot.
|
||||
*/
|
||||
const PROVEN_BY_ARCHETYPE = Object.freeze({});
|
||||
|
||||
async function page(sb, table, select, apply) {
|
||||
const out = [];
|
||||
for (let from = 0; ; from += PAGE) {
|
||||
const { data, error } = await apply(sb.from(table).select(select)).range(from, from + PAGE - 1);
|
||||
if (error) throw error;
|
||||
if (!data || data.length === 0) break;
|
||||
out.push(...data);
|
||||
if (data.length < PAGE) break;
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
async function main() {
|
||||
const sb = createClient(SB_URL, SB_KEY, { auth: { persistSession: false } });
|
||||
|
||||
const snaps = await page(sb, 'model_snapshots', 'player_key, game_date, archetype, stat',
|
||||
(q) => q.eq('sport', 'mlb').eq('stat', STAT).not('archetype', 'is', null));
|
||||
const archOf = new Map();
|
||||
for (const s of snaps) archOf.set(`${s.player_key}|${s.game_date}`, s.archetype);
|
||||
|
||||
const led = await page(sb, 'ledger_entries', 'player_key, game_date, outcome, p_win, quarantine_reason',
|
||||
(q) => q.eq('sport', 'mlb').is('user_id', null).eq('stat', STAT)
|
||||
.in('outcome', ['hit', 'miss']).not('p_win', 'is', null));
|
||||
const clean = led.filter((r) => !(r.quarantine_reason || '').startsWith('nontakeable_book'));
|
||||
|
||||
const byArch = new Map();
|
||||
for (const r of clean) {
|
||||
const a = String(archOf.get(`${r.player_key}|${r.game_date}`) || 'UNLABELLED').toUpperCase();
|
||||
if (!byArch.has(a)) byArch.set(a, []);
|
||||
byArch.get(a).push({ p: knownNumber(r.p_win), won: r.outcome === 'hit' ? 1 : 0 });
|
||||
}
|
||||
|
||||
const mc = await tl.recordAndCount(tl.supabaseStore(sb),
|
||||
[...byArch.keys()].map((a) => ({
|
||||
sport: 'mlb', stat: STAT, archetype: a === 'UNLABELLED' ? null : a,
|
||||
interaction: 'grade_band_lift', target: 'outcome',
|
||||
})));
|
||||
|
||||
// Calibration is measured, not assumed. Today it is certified for hits only in
|
||||
// a middle band (specs — held-out error 0.477->0.506, 0.587->0.580), which is
|
||||
// NOT the same as an archetype's probabilities being calibrated.
|
||||
const certified = typeof cal.certifyBands === 'function';
|
||||
|
||||
const out = [];
|
||||
for (const [arch, rows] of [...byArch.entries()].sort((a, b) => b[1].length - a[1].length)) {
|
||||
out.push(gb.buildBands(rows, {
|
||||
archetype: arch,
|
||||
cumulativeTests: mc.cumulative_tests,
|
||||
proven: Boolean(PROVEN_BY_ARCHETYPE[arch]),
|
||||
calibrated: false, // no archetype's distribution is certified calibrated
|
||||
}));
|
||||
}
|
||||
|
||||
console.log(JSON.stringify({
|
||||
stat: STAT,
|
||||
settled_rows: clean.length,
|
||||
archetypes: byArch.size,
|
||||
cumulative_tests: mc.cumulative_tests,
|
||||
calibration_helper_present: certified,
|
||||
proven_by_archetype: PROVEN_BY_ARCHETYPE,
|
||||
note: 'every band is a BASE-RATE read — no factor has passed the gate for any archetype',
|
||||
bands: out,
|
||||
}, null, 2));
|
||||
process.exit(0);
|
||||
}
|
||||
|
||||
main().catch((e) => { console.error(e); process.exit(1); });
|
||||
Reference in New Issue
Block a user