total_bases: every power factor is THEATER, and a units bug nearly hid it
PREMISE CORRECTION: the per-archetype rescale is not "proven and live on hits". gradeBands was built, gated and explicitly NOT wired two orders ago -- no hits archetype slot reached sample, every band came back base-rate, and only defense_by_direction proved pooled. This applies an unvalidated-at-archetype-level method to a second stat. FULL-HISTORY AUDIT: 988 clean settled TB rows (101 quarantined, 948 with p_win), 341 players, and only 9 DISTINCT GAME DATES. No archetype slot reaches 500 -- BOMBER 340, GHOST 147, BRUSH 55. Confirmed short on full history, not a windowed artifact. The 9-date figure matters more than the row count: ~49 games means any game- or venue-borne factor has almost no replication here. THE BASELINE HAD TO CHANGE, to a harder null. TB lines vary (1.5 on 559 rows, 0.5 on 345), so a per-line personal base rate would rest on ~2 rows per player-line and would have to be invented. The null is the counter's own p_win, which already prices the line -- beating the champion, not beating "he's due". THE UNITS BUG, caught, and it had produced the best result in the programme. The first run reported barrel_rate at Brier -0.0095, the largest improvement ever measured here. fromStatcastRow returns barrel_pct as a FRACTION (0.06) while the raw table stores 0-100, so (0.06 - 7.8) * 0.018 clamped EVERY row to the maximum negative shift. That uniform downward push "improved" Brier purely by leaning on the counter's over-prediction and contained no barrel information at all. Same family as the S80 trap, inverted. exit_velo was a second bug -- the column is avg_exit_velo, so it read null on every row and reported n=0. A zero is a wiring bug until proven an honest absence. GATE with units fixed, 138 cumulative tests: barrel_rate n=707 shift 0.0364 brier +0.0036 THEATER exit_velo n=707 shift 0.0229 brier +0.0022 THEATER hard_contact_allowed n=707 shift 0.0260 brier +0.0033 THEATER park_weather_hit_type n=651 36 entities PENDING (k<40) platoon_severity n=481 PENDING (n<500) THE PREDICTED INVERSION WENT THE OTHER WAY. BOMBER x barrel_rate is +0.0114, the single most harmful cell in the table, exactly where the strongest proof was predicted. GHOST +0.0012. All sample-blocked so not a verdict, but recorded so it is not claimed later. AND IT IS NOT DOUBLE-COUNTING -- tested and refuted: corr(barrel, p_win) = -0.061, the counter is not pricing barrel at all. The duller answer is corr(barrel, counter RESIDUAL) = -0.012. Barrel is a real skill that carries no information about what the counter gets wrong at this line. That also closes the S81 lead: hard_hit r=0.153 at n=295 drifted to 0.135 at n=383 and is THEATER at n=707. THE REAL FINDING: TB is miscalibrated, not under-factored. mean p_win 0.5698 vs actual 0.5074, bias +0.0624. Held out on a strict time split (fit < 2026-08-02, eval 651 unseen rows): raw 0.25007, constant de-bias 0.24740 (-0.00267), isotonic 0.24621 (-0.00386). Worth more than any factor tested and the only intervention pointing the right way -- and still refused at the corrected bar on 32 clusters. A CANDIDATE, not a result. It also explains the units bug's fake success exactly: a blanket downward shift is a crude de-bias. NO RESCALE. Nothing proved, nothing certified calibrated, no slot at sample -- every band would be the honest base-rate band gradeBands already returns by construction. Counter and frozen clusters byte-identical. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W1sivYNqY2TS5ftykmHBU9
This commit is contained in:
@@ -0,0 +1,345 @@
|
||||
#!/usr/bin/env node
|
||||
'use strict';
|
||||
|
||||
/**
|
||||
* prove-tb-factors — TOTAL BASES IS A DIFFERENT EVENT FROM HITS.
|
||||
*
|
||||
* A hit asks whether the ball found a hole. Total bases asks how hard and how
|
||||
* far it was struck. So the causally-correct factors differ, and the
|
||||
* archetype differential is expected to INVERT: contact defence proved for hits,
|
||||
* where a slap single is worth exactly one base regardless of who fielded it;
|
||||
* for total bases the value should live with the power profiles.
|
||||
*
|
||||
* ── THE BASELINE IS THE CHAMPION, NOT "HE'S DUE" ─────────────────────────
|
||||
* The hits gate used the player's own leave-one-out base rate as the null. That
|
||||
* cannot be reproduced here: total-bases lines VARY (1.5 on 559 rows, 0.5 on
|
||||
* 345, 2.5 on 45), and a player's rate of clearing 1.5 bases is a different
|
||||
* quantity from his rate of clearing 0.5. With 988 rows over 341 players there
|
||||
* are roughly two rows per player-line — far too thin to estimate a per-line
|
||||
* personal base rate without inventing one.
|
||||
*
|
||||
* So the null here is the COUNTER'S OWN FORECAST (p_win), which already prices
|
||||
* the line. That is a strictly HARDER null than a base rate, not an easier one:
|
||||
* a factor must improve on the champion, not merely on "he's due". Stated
|
||||
* plainly because it differs from the hits run and the difference matters when
|
||||
* comparing the two.
|
||||
*
|
||||
* SUPABASE_URL=... node scripts/prove-tb-factors.js
|
||||
*/
|
||||
|
||||
require('dotenv').config();
|
||||
const { createClient } = require('@supabase/supabase-js');
|
||||
const fg = require('../src/services/model/factorGate');
|
||||
const sk = require('../src/services/model/skillProjection');
|
||||
const tl = require('../src/services/model/testLedger');
|
||||
const mlb = require('../src/services/adapters/mlbStatsAdapter');
|
||||
const { knownNumber, knownRate } = require('../src/utils/known');
|
||||
const { nameKey } = require('../src/utils/playerName');
|
||||
const sd = require('../src/services/model/sprayDefense');
|
||||
const pss = require('../src/services/model/platoonSeverity');
|
||||
const pw = require('../src/services/model/parkWeather');
|
||||
|
||||
/** League-typical hit-type shares; the atom reshapes these and TB follows. */
|
||||
const BASE_SHARES = { single: 0.655, double: 0.195, triple: 0.017, home_run: 0.133 };
|
||||
const tbFrom = (s) => s.single + 2 * s.double + 3 * s.triple + 4 * s.home_run;
|
||||
|
||||
const SB_URL = process.env.SUPABASE_URL;
|
||||
const SB_KEY = process.env.SUPABASE_SERVICE_ROLE_KEY || process.env.SUPABASE_SERVICE_KEY;
|
||||
const PAGE = 1000;
|
||||
const ARCHS = (process.env.TB_ARCHETYPES || 'ALL,BOMBER,GHOST,BRUSH,DRIVER').split(',');
|
||||
|
||||
async function page(sb, table, select, apply) {
|
||||
const out = [];
|
||||
for (let from = 0; ; from += PAGE) {
|
||||
const { data, error } = await apply(sb.from(table).select(select)).range(from, from + PAGE - 1);
|
||||
if (error) throw error;
|
||||
if (!data || data.length === 0) break;
|
||||
out.push(...data);
|
||||
if (data.length < PAGE) break;
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
/**
|
||||
* THE FACTORS. Each returns a MULTIPLIER on the base rate, or null when the
|
||||
* input is absent — an absent factor must leave the baseline untouched rather
|
||||
* than nudge it toward some default.
|
||||
*/
|
||||
const FACTORS = [
|
||||
{
|
||||
key: 'barrel_rate',
|
||||
needs: ['barrel_pct'],
|
||||
entity: (r) => r.player_key,
|
||||
mechanism: 'THE EXTRA-BASE SKILL ITSELF. A barrel is the exit-velocity and launch-angle combination that produces extra bases; it is the most direct expression of what total bases measures, where for hits it is largely irrelevant to whether a grounder finds a hole.',
|
||||
// UNITS: fromStatcastRow returns barrel_pct as a FRACTION (0.06), not the
|
||||
// 0-100 the raw table stores. Writing this against the percentage scale
|
||||
// clamped every row to the maximum negative shift, which then "improved"
|
||||
// Brier only by leaning on the counter's known global over-prediction.
|
||||
apply: (r) => 1 + Math.max(-0.20, Math.min(0.20, (r.barrel_pct - 0.078) * 1.8)),
|
||||
},
|
||||
{
|
||||
key: 'exit_velo',
|
||||
needs: ['avg_exit_velo'],
|
||||
entity: (r) => r.player_key,
|
||||
mechanism: 'How hard the ball leaves the bat. Separates a double in the gap from a fly out, which is exactly the margin total bases lives on.',
|
||||
apply: (r) => 1 + Math.max(-0.15, Math.min(0.15, (r.avg_exit_velo - 88.9) * 0.020)),
|
||||
},
|
||||
{
|
||||
key: 'hard_contact_allowed',
|
||||
needs: ['pitcher_hard_hit_allowed'],
|
||||
entity: (r) => r.starter_id,
|
||||
mechanism: 'A pitcher who concedes hard contact concedes EXTRA BASES, not just hits. For total bases this should read stronger than it did for hits.',
|
||||
apply: (r) => 1 + Math.max(-0.15, Math.min(0.15, (r.pitcher_hard_hit_allowed - 0.389) * 1.2)),
|
||||
},
|
||||
{
|
||||
key: 'park_weather_hit_type',
|
||||
needs: ['park_weather_ratio'],
|
||||
entity: (r) => r.park_weather_ratio,
|
||||
mechanism: 'Whether a struck ball becomes a double, clears the fence, or dies at the track. The atom reshapes HIT TYPE rather than P(hit), which is the only form that can express a total-bases effect.',
|
||||
apply: (r) => r.park_weather_ratio,
|
||||
caveat: 'venue-borne: replication caps at the number of distinct park readings, not the row count',
|
||||
},
|
||||
{
|
||||
key: 'platoon_severity',
|
||||
needs: ['platoon_severity_mult'],
|
||||
entity: (r) => r.player_key,
|
||||
mechanism: "The hitter's OWN measured split, shrunk by the smaller side's plate appearances and refused below a floor.",
|
||||
apply: (r) => r.platoon_severity_mult,
|
||||
},
|
||||
];
|
||||
|
||||
async function main() {
|
||||
if (!SB_URL || !SB_KEY) throw new Error('SUPABASE_URL / service key required');
|
||||
const sb = createClient(SB_URL, SB_KEY, { auth: { persistSession: false } });
|
||||
|
||||
const statcast = await page(sb, 'statcast_aggregates', '*', (q) => q.eq('sport', 'mlb'));
|
||||
const batters = new Map(); const pitchersById = new Map();
|
||||
for (const r of statcast) {
|
||||
const prof = sk.fromStatcastRow(r);
|
||||
if (r.role === 'pitcher' && r.source_id != null) pitchersById.set(Number(r.source_id), prof);
|
||||
if (r.role === 'batter' && r.player_key) batters.set(r.player_key, prof);
|
||||
}
|
||||
const sprayRows = await page(sb, 'batter_spray', '*', (q) => q.eq('sport', 'mlb'));
|
||||
const sprayByKey = new Map();
|
||||
for (const r of sprayRows) {
|
||||
if (!r.player_key) continue;
|
||||
const prev = sprayByKey.get(r.player_key);
|
||||
if (!prev || String(r.as_of_date) > String(prev.as_of_date)) sprayByKey.set(r.player_key, r);
|
||||
}
|
||||
|
||||
const platRows = await page(sb, 'platoon_splits', '*', (q) => q.eq('sport', 'mlb'));
|
||||
const platByKey = new Map();
|
||||
for (const r of platRows) {
|
||||
if (!r.player_key) continue;
|
||||
const prev = platByKey.get(r.player_key);
|
||||
if (!prev || String(r.as_of_date) > String(prev.as_of_date)) platByKey.set(r.player_key, r);
|
||||
}
|
||||
|
||||
const parkRows = await page(sb, 'park_dimensions', '*', (q) => q.eq('sport', 'mlb'));
|
||||
const parkByVenue = new Map();
|
||||
for (const p of parkRows) if (!parkByVenue.has(p.venue_id)) parkByVenue.set(p.venue_id, p);
|
||||
const parkLeague = pw.leagueGeometry([...parkByVenue.values()]);
|
||||
const ctxRows = await page(sb, 'game_context', 'game_id, venue_id, wx_temp_f, wx_wind_speed_mph, wx_wind_direction_deg', (q) => q);
|
||||
const ctxBy = new Map(ctxRows.map((c) => [c.game_id, c]));
|
||||
|
||||
const defRows = await page(sb, 'team_defense', '*', (q) => q.eq('sport', 'mlb'));
|
||||
const defByTeam = new Map();
|
||||
for (const d of defRows) defByTeam.set(d.team, d);
|
||||
|
||||
const snaps = await page(sb, 'model_snapshots', 'player_key, game_date, archetype, stat',
|
||||
(q) => q.eq('sport', 'mlb').eq('stat', 'total_bases').not('archetype', 'is', null));
|
||||
const archOf = new Map();
|
||||
for (const s of snaps) archOf.set(`${s.player_key}|${s.game_date}`, s.archetype);
|
||||
|
||||
const led = await page(sb, 'ledger_entries',
|
||||
'id, game_id, player_key, player_name, line, side, outcome, game_date, p_win, quarantine_reason, env_park_base',
|
||||
(q) => q.eq('sport', 'mlb').is('user_id', null).eq('stat', 'total_bases')
|
||||
.in('outcome', ['hit', 'miss']).not('p_win', 'is', null));
|
||||
const clean = led.filter((r) => !(r.quarantine_reason || '').startsWith('nontakeable_book'));
|
||||
|
||||
// Opponent faced, from each hitter's own game log.
|
||||
const names = new Map();
|
||||
for (const r of clean) if (!names.has(r.player_key)) names.set(r.player_key, r.player_name);
|
||||
const oppBy = new Map(); const startersBy = new Map();
|
||||
const dates = [...new Set(clean.map((r) => r.game_date))].sort();
|
||||
for (const d of dates) {
|
||||
try {
|
||||
const games = await mlb.getScheduleWithPitchers(d);
|
||||
for (const g of games) {
|
||||
if (!g.home || !g.away) continue;
|
||||
if (g.home.probablePitcher) startersBy.set(`${d}|OPP:${g.home.team}`, g.home.probablePitcher.id);
|
||||
if (g.away.probablePitcher) startersBy.set(`${d}|OPP:${g.away.team}`, g.away.probablePitcher.id);
|
||||
}
|
||||
} catch { /* absent slate */ }
|
||||
}
|
||||
for (const [key, name] of names) {
|
||||
try {
|
||||
const found = await mlb.searchPlayer(name);
|
||||
if (!found || !found.id) continue;
|
||||
const log = await mlb.getPlayerGameLog(found.id);
|
||||
for (const g of log || []) if (g && g.date && g.opponent) oppBy.set(`${key}|${String(g.date).slice(0, 10)}`, g.opponent);
|
||||
} catch { /* no log */ }
|
||||
}
|
||||
|
||||
// Per-player base rate — the honest null: "he's due", no reading of tonight.
|
||||
const byPlayer = new Map();
|
||||
for (const r of clean) {
|
||||
const cur = byPlayer.get(r.player_key) || { n: 0, w: 0 };
|
||||
cur.n += 1; cur.w += r.outcome === 'hit' ? 1 : 0;
|
||||
byPlayer.set(r.player_key, cur);
|
||||
}
|
||||
|
||||
const loss = { no_batter_profile: 0, thin_base_rate: 0, no_opponent: 0, no_pitcher: 0, kept: 0 };
|
||||
const rows = [];
|
||||
for (const r of clean) {
|
||||
const bat = batters.get(r.player_key);
|
||||
const bp = byPlayer.get(r.player_key);
|
||||
if (!bat) loss.no_batter_profile += 1;
|
||||
if (!bp || bp.n < 3) { loss.thin_base_rate += 1; continue; }
|
||||
// THE NULL IS THE CHAMPION. Total-bases lines vary, so a per-line personal
|
||||
// base rate cannot be estimated from ~2 rows per player-line without
|
||||
// inventing one. p_win already prices the line, and beating it is a harder
|
||||
// bar than beating "he's due".
|
||||
const baseline = knownNumber(r.p_win);
|
||||
if (baseline === null) { loss.thin_base_rate += 1; continue; }
|
||||
const faced = oppBy.get(`${r.player_key}|${r.game_date}`) || null;
|
||||
const nick = faced ? String(faced).split(' ').pop() : null;
|
||||
const def = faced ? (defByTeam.get(faced) || defByTeam.get(nick)) : null;
|
||||
if (!faced) loss.no_opponent += 1;
|
||||
const starterId = faced ? startersBy.get(`${r.game_date}|OPP:${faced}`) : null;
|
||||
const pit = starterId != null ? pitchersById.get(Number(starterId)) : null;
|
||||
if (faced && !pit) loss.no_pitcher += 1;
|
||||
loss.kept += 1;
|
||||
rows.push({
|
||||
id: r.id,
|
||||
// Errors are correlated WITHIN a game — shared starter, park, weather and
|
||||
// the game's own randomness — so the interval must be clustered on it.
|
||||
// Three of these factors (pitcher profile, team defence, park) are also
|
||||
// CONSTANT across every hitter facing that starter, which makes row
|
||||
// resampling straightforwardly wrong for them.
|
||||
cluster: r.game_id,
|
||||
opp: faced,
|
||||
starter_id: starterId != null ? Number(starterId) : null,
|
||||
player_key: r.player_key,
|
||||
archetype: archOf.get(`${r.player_key}|${r.game_date}`) || null,
|
||||
won: r.outcome === 'hit' ? 1 : 0,
|
||||
baseline,
|
||||
team_defense: def ? knownNumber(def.oaa_sum) : null,
|
||||
pitcher_hard_hit_allowed: pit ? knownRate(pit.hard_hit_pct) : null,
|
||||
line: knownNumber(r.line),
|
||||
barrel_pct: bat ? knownRate(bat.barrel_pct) : null,
|
||||
avg_exit_velo: bat ? knownNumber(bat.avg_exit_velo) : null,
|
||||
park_weather_ratio: (() => {
|
||||
const c = ctxBy.get(r.game_id);
|
||||
if (!c || c.venue_id == null) return null;
|
||||
const dims = parkByVenue.get(c.venue_id);
|
||||
if (!dims) return null;
|
||||
const read = pw.parkWeatherRead({ dims, wx: c, league: parkLeague });
|
||||
if (!read) return null;
|
||||
const shaped = pw.applyToShares(BASE_SHARES, read);
|
||||
return tbFrom(shaped) / tbFrom(BASE_SHARES);
|
||||
})(),
|
||||
platoon_severity_mult: (() => {
|
||||
const sp = platByKey.get(r.player_key);
|
||||
if (!sp || !bat || !bat.bats || !pit || !pit.throws) return null;
|
||||
const out = pss.platoonRead({
|
||||
splits: {
|
||||
vl: { pa: sp.vl_pa, atBats: sp.vl_ab, hits: sp.vl_hits },
|
||||
vr: { pa: sp.vr_pa, atBats: sp.vr_ab, hits: sp.vr_hits },
|
||||
},
|
||||
bats: bat.bats, throws: pit.throws,
|
||||
});
|
||||
return out && out.readable ? out.multiplier : null;
|
||||
})(),
|
||||
spray_multiplier: (() => {
|
||||
const sp = sprayByKey.get(r.player_key);
|
||||
const posOaa = def && def.position_oaa ? def.position_oaa : null;
|
||||
if (!sp || !posOaa || !bat || !bat.bats) return null;
|
||||
const out = sd.sprayDefenseMultiplier({ spray: sp, bats: bat.bats, positionOaa: posOaa });
|
||||
return out ? out.multiplier : null;
|
||||
})(),
|
||||
platoon_edge: (bat && pit && bat.bats && pit.throws)
|
||||
? (String(bat.bats)[0] !== String(pit.throws)[0] ? 1 : -1) : null,
|
||||
});
|
||||
}
|
||||
|
||||
// Cumulative Bonferroni across the programme lifetime.
|
||||
const store = tl.supabaseStore(sb);
|
||||
const mc = await tl.recordAndCount(store, FACTORS.flatMap((f) =>
|
||||
ARCHS.map((a) => ({ sport: 'mlb', stat: 'total_bases', archetype: a === 'ALL' ? null : a, interaction: `factor:${f.key}`, target: 'outcome' }))));
|
||||
|
||||
// STEP 1 — FULL-HISTORY SAMPLE AUDIT PER SLOT, before any gating.
|
||||
const audit = [];
|
||||
for (const f of FACTORS) {
|
||||
for (const arch of ARCHS) {
|
||||
const slot = arch === 'ALL' ? rows : rows.filter((r) => String(r.archetype || '').toUpperCase() === arch);
|
||||
const usable = slot.filter((r) => f.needs.every((k) => knownNumber(r[k]) !== null));
|
||||
audit.push({
|
||||
factor: f.key,
|
||||
archetype: arch,
|
||||
rows: usable.length,
|
||||
games: new Set(usable.map((r) => r.cluster).filter(Boolean)).size,
|
||||
players: new Set(usable.map((r) => r.player_key)).size,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
const results = [];
|
||||
for (const arch of ARCHS) {
|
||||
const slot = arch === 'ALL' ? rows : rows.filter((r) => String(r.archetype || '').toUpperCase() === arch);
|
||||
for (const f of FACTORS) {
|
||||
const usable = slot.filter((r) => f.needs.every((k) => knownNumber(r[k]) !== null));
|
||||
// A park effect is replicated across PARKS, not across games: 619 rows in
|
||||
// 45 games still only ever saw ~23 ballparks, and unmodelled park
|
||||
// heterogeneity is confounded with the very thing being estimated. So the
|
||||
// cluster is the COARSER of the game and the entity the treatment rides on.
|
||||
const ents = f.entity ? new Set(usable.map((r) => String(f.entity(r)))) : null;
|
||||
const games = new Set(usable.map((r) => String(r.cluster)));
|
||||
const useEntity = ents && ents.size < games.size;
|
||||
const paired = usable.map((r) => {
|
||||
const mult = f.apply(r);
|
||||
const cond = mult === null ? null : Math.min(0.99, Math.max(0.01, r.baseline * mult));
|
||||
return {
|
||||
baseline: r.baseline,
|
||||
conditioned: cond,
|
||||
won: r.won,
|
||||
cluster: useEntity ? `e:${f.entity(r)}` : r.cluster,
|
||||
};
|
||||
});
|
||||
const v = fg.adjudicate(paired, {
|
||||
factor: f.key, archetype: arch, stat: 'total_bases',
|
||||
cumulativeTests: mc.cumulative_tests, // native cumulative correction
|
||||
});
|
||||
results.push({
|
||||
archetype: arch, factor: f.key, n: v.movement.n,
|
||||
clusters: v.improvement ? v.improvement.effective_n : null,
|
||||
cluster_unit: useEntity ? 'treatment_entity' : 'game',
|
||||
distinct_games: games.size,
|
||||
distinct_entities: ents ? ents.size : null,
|
||||
mean_abs_shift: v.movement.mean_abs_shift,
|
||||
brier_delta: v.improvement ? v.improvement.brier_delta : null,
|
||||
ci: v.improvement ? v.improvement.ci : null,
|
||||
ci_level: v.improvement ? v.improvement.ci_level : null,
|
||||
verdict: v.verdict,
|
||||
reason: v.reason,
|
||||
...(f.caveat ? { input_caveat: f.caveat } : {}),
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
console.log(JSON.stringify({
|
||||
baseline: "each row scored against the player's OWN leave-one-out base rate — the honest 'he's due' null",
|
||||
total_rows: rows.length,
|
||||
slot_audit: audit,
|
||||
clean_settled_rows_available: clean.length,
|
||||
row_loss: loss,
|
||||
cumulative_bonferroni: mc,
|
||||
gate: 'a factor must MOVE the prediction AND improve out-of-sample Brier; movement alone is THEATER',
|
||||
results,
|
||||
proven: results.filter((r) => r.verdict === 'PROVES'),
|
||||
theater: results.filter((r) => r.verdict === 'THEATER'),
|
||||
}, null, 2));
|
||||
process.exit(0);
|
||||
}
|
||||
|
||||
main().catch((e) => { console.error(e); process.exit(1); });
|
||||
@@ -0,0 +1,146 @@
|
||||
# total_bases — every power factor is THEATER, and the units bug nearly hid it
|
||||
|
||||
**Nothing proved. The predicted inversion did not appear. And the one real
|
||||
finding is that the counter is badly miscalibrated on this stat, not that it is
|
||||
missing a factor.**
|
||||
|
||||
---
|
||||
|
||||
## Premise correction
|
||||
|
||||
The order describes the per-archetype rescale method as "PROVEN and LIVE on
|
||||
hits". It is neither. `gradeBands` was built and gated two orders ago and
|
||||
explicitly **not wired**: no hits archetype slot reached sample, every band came
|
||||
back base-rate, and only `defense_by_direction` proved pooled. This is therefore
|
||||
applying an **unvalidated-at-archetype-level** method to a second stat, not
|
||||
rolling out a proven one.
|
||||
|
||||
## STEP 1 — Full-history audit
|
||||
|
||||
988 clean settled total_bases rows (101 quarantined, 948 carrying `p_win`), 341
|
||||
players, **9 distinct game dates**.
|
||||
|
||||
| archetype | n | vs gate |
|
||||
|---|---|---|
|
||||
| UNLABELLED | 373 | short 127 |
|
||||
| **BOMBER** | **340** | short 160 |
|
||||
| GHOST | 147 | short 353 |
|
||||
| BRUSH | 55 | short 445 |
|
||||
| DRIVER | 40 | short 460 |
|
||||
|
||||
**No slot reaches 500 — confirmed on full history, not a windowed artifact.**
|
||||
|
||||
The 9-date figure matters more than the row count: with only ~49 games, any
|
||||
game-borne or venue-borne factor has almost no replication here.
|
||||
|
||||
## The baseline had to change, and it is a HARDER null
|
||||
|
||||
The hits gate used the player's leave-one-out base rate. That cannot be
|
||||
reproduced: TB lines vary (1.5 on 559 rows, 0.5 on 345, 2.5 on 45), and a
|
||||
player's rate of clearing 1.5 bases is a different quantity from 0.5. At ~2 rows
|
||||
per player-line, a per-line personal base rate would have to be invented.
|
||||
|
||||
So the null here is the **counter's own forecast**, which already prices the
|
||||
line. A factor must beat the champion, not "he's due" — strictly harder.
|
||||
|
||||
---
|
||||
|
||||
## THE UNITS BUG (caught, and it had produced the best result in the programme)
|
||||
|
||||
The first run reported `barrel_rate` at Brier **−0.0095**, the largest
|
||||
improvement ever measured here. It was an artifact.
|
||||
|
||||
`fromStatcastRow` returns `barrel_pct` as a **FRACTION** (0.06); the raw table
|
||||
stores 0–100. The factor was written against the percentage scale, so
|
||||
`(0.06 − 7.8) × 0.018` clamped **every row** to the maximum negative shift. That
|
||||
uniform downward push "improved" Brier only by leaning on the counter's
|
||||
over-prediction bias — it contained no barrel information whatsoever.
|
||||
|
||||
Same family as the S80 units trap, inverted: there the raw percentages were fed
|
||||
in unconverted; here the converted fractions were read as percentages.
|
||||
`exit_velo` was a second, simpler wiring bug — the column is `avg_exit_velo`, so
|
||||
the factor read null on every row and reported n=0. **A zero is a wiring bug
|
||||
until proven an honest absence.**
|
||||
|
||||
---
|
||||
|
||||
## STEP 2/3 — The gate, units fixed (138 cumulative tests)
|
||||
|
||||
| factor | n | entities | clustered on | k | shift | Brier Δ | CI | verdict |
|
||||
|---|---|---|---|---|---|---|---|---|
|
||||
| `barrel_rate` | 707 | 179 | game | 49 | 0.0364 | **+0.0036** | [−0.0025, +0.0101] | **THEATER** |
|
||||
| `exit_velo` | 707 | 179 | game | 49 | 0.0229 | **+0.0022** | [−0.0030, +0.0072] | **THEATER** |
|
||||
| `hard_contact_allowed` | 707 | 93 | game | 49 | 0.0260 | **+0.0033** | [−0.0019, +0.0082] | **THEATER** |
|
||||
| `park_weather_hit_type` | 651 | **36** | treatment entity | 36 | 0.0070 | +0.0016 | [+0.0001, +0.0038] | PENDING — k<40 |
|
||||
| `platoon_severity` | 481 | 120 | game | 49 | 0.0243 | +0.0006 | [−0.0045, +0.0055] | PENDING — n<500 |
|
||||
|
||||
All three contact-quality factors move the number and make it **worse**.
|
||||
|
||||
### The predicted inversion did not appear — it went the other way
|
||||
|
||||
The order expects barrel and park→hit-type to prove for POWER archetypes, since
|
||||
extra bases are where their value lives. Measured:
|
||||
|
||||
| | BOMBER | GHOST |
|
||||
|---|---|---|
|
||||
| `barrel_rate` | **+0.0114** (most harmful cell in the table) | +0.0012 |
|
||||
| `exit_velo` | +0.0067 | +0.0011 |
|
||||
|
||||
**BOMBER × barrel_rate is the single worst result**, exactly where the strongest
|
||||
proof was predicted. All slots are sample-blocked so none of this is a verdict —
|
||||
but the direction is recorded so it is not claimed later.
|
||||
|
||||
### Why — and it is NOT double-counting
|
||||
|
||||
The obvious explanation is that the counter already prices power, so adding
|
||||
barrel double-counts it. **Tested and refuted:** corr(barrel_pct, p_win) =
|
||||
**−0.061**. The counter is not pricing barrel at all.
|
||||
|
||||
The actual answer is duller and more useful: corr(barrel_pct, counter
|
||||
**residual**) = **−0.012**. Barrel carries essentially no information about what
|
||||
the counter gets wrong. It is a real skill that does not help at this line.
|
||||
|
||||
This also closes the S81 lead: `hard_hit_pct` marginal r = 0.153 at n=295, which
|
||||
drifted to 0.135 at n=383 (S82) and is now THEATER at n=707. An estimate
|
||||
regressing as noise averages out, followed to its conclusion.
|
||||
|
||||
---
|
||||
|
||||
## The real finding: TB is miscalibrated, not under-factored
|
||||
|
||||
```
|
||||
mean p_win 0.5698 actual hit rate 0.5074 counter bias +0.0624
|
||||
```
|
||||
|
||||
Held out on a strict time split (fit on dates < 2026-08-02, evaluated on 651 rows
|
||||
the fit never saw):
|
||||
|
||||
| | Brier | Δ |
|
||||
|---|---|---|
|
||||
| raw counter | 0.25007 | — |
|
||||
| constant de-bias | 0.24740 | −0.00267 |
|
||||
| **isotonic** | **0.24621** | **−0.00386** |
|
||||
|
||||
**The calibration fix is worth more than any factor tested, and it is the only
|
||||
intervention pointing the right way.** It is nonetheless refused at the corrected
|
||||
bar — 32 game clusters in the eval window against a 40 floor — so it is a
|
||||
CANDIDATE, not a result.
|
||||
|
||||
That also explains the units bug's fake success precisely: a blanket downward
|
||||
shift is a crude de-bias, and it "worked" for that reason alone.
|
||||
|
||||
## STEP 4 — No rescale
|
||||
|
||||
Two-bar rule: nothing proved, nothing calibrated-and-certified for TB, no
|
||||
archetype slot at sample. Every band would be an honest base-rate band, which is
|
||||
what `gradeBands` already returns by construction. Running it would add nothing.
|
||||
|
||||
## Next, in order of value
|
||||
|
||||
1. **Calibrate total_bases** — largest measured effect, needs game-date accrual
|
||||
to clear the cluster floor, not new inputs.
|
||||
2. **Stop adding contact-quality factors to TB.** Three tested, three THEATER,
|
||||
and the residual correlation says there is nothing there to find.
|
||||
3. Archetype slots need ~160 more BOMBER rows before any per-archetype claim.
|
||||
|
||||
Counter and frozen clusters byte-identical.
|
||||
+1
-1
File diff suppressed because one or more lines are too long
Reference in New Issue
Block a user