Build the two-part factor gate: one factor proves, and zero are theatre

The question was whether the hit grade reads tonight's game or just says he is
due. Answering it needed a gate that correlation cannot provide, because
correlation cannot separate the two ways a factor looks alive: it reads the
game, or it moves the number and reads nothing. The second is what a product
ships by accident -- arch-v1 moved 76% of rows by 2.5 points, changed
resolution by 0.0000, and was live for months, and no user could have told.

So a factor must now clear both conditions: move the prediction off the
player's own leave-one-out base rate, AND improve out-of-sample Brier. Brier
rather than correlation, because correlation asks whether the ordering improved
and this asks whether the NUMBER got closer to what happened -- and for a graded
probability the number is the product.

The correction applies to the interval itself, which turned out to matter more
than expected. A plain 95% CI is the right bar for one test; at fifty
cumulative tests roughly two or three intervals exclude zero by chance alone.
Widening to 1 - 0.05/tests, currently 99.9%, flipped both defence and platoon
out of "proves". A 95% interval would have shipped two unproven factors into
the grade, with reasoning text explaining them to users.

That forced a distinction I had initially collapsed. Defence and platoon have
FAVOURABLE point estimates whose corrected intervals merely span zero, and
calling that THEATER would repeat the error this codebase keeps correcting:
insufficient evidence is not evidence of absence. THEATER is now reserved for
its one real meaning -- moves the number, reads nothing -- and
NOT_PROVEN_AT_CORRECTED_BAR names a real candidate held to a bar that rises with
every hypothesis the programme tests.

Result on 741 settled hits rows: pitcher_contact_profile PROVES, improving
Brier by 0.0066 with a 99.9% interval of [-0.0114, -0.0016]. Defence (-0.0043)
and platoon (-0.0039) are not proven at the corrected bar. Park is
sample-blocked at n=405. Zero factors are theatre, which is the genuinely good
news: nothing decorative is being wired. Per-archetype every slot is
sample-blocked (BOMBER 252-294, GHOST 67-125).

Two spec gaps worth recording. The approach identities the order names -- SPRAY,
DAMAGE-DEALER, COUNT-WORKER -- do not exist in the registry; the MLB batter
archetypes are BOMBER, GHOST, TORCH, BRUSH, DRIVER, FLEX, ALPHA, HYBRID and
CATALYST. And parkFactors maps hits to run_base, so there is no hits-specific
park factor at all: a park that turns outs into hits without producing runs is
invisible to the input we have.

The grade rescale is NOT run. It was explicitly gated on the factor proving,
and one pooled factor worth 0.0066 of Brier is not a factor-informed
distribution -- rescaling on it would dress a base-rate model as a matchup
model, which is the exact thing this gate was built to prevent.

4,286 tests green (340 suites); web build exit 0.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W1sivYNqY2TS5ftykmHBU9

scripts/prove-hit-factors.js

@@ -0,0 +1,207 @@
+#!/usr/bin/env node
+'use strict';
+
+/**
+ * prove-hit-factors — does the hit grade read tonight's game, or say "he's due"?
+ *
+ * Each factor is conditioned against the player's OWN base rate and put through
+ * the two-part gate: it must MOVE the prediction and the moved prediction must
+ * be MORE ACCURATE out-of-sample. Movement alone is THEATER — a grade that
+ * swings on park and platoon looks like it read the matchup, and a user cannot
+ * tell the difference from outside.
+ *
+ * The baseline is deliberately the honest null this order describes: the
+ * player's base rate, i.e. "he's due" with no reading of tonight at all. A
+ * factor earns its place only by beating that.
+ *
+ *   SUPABASE_URL=... node scripts/prove-hit-factors.js
+ */
+
+require('dotenv').config();
+const { createClient } = require('@supabase/supabase-js');
+const fg = require('../src/services/model/factorGate');
+const sk = require('../src/services/model/skillProjection');
+const tl = require('../src/services/model/testLedger');
+const mlb = require('../src/services/adapters/mlbStatsAdapter');
+const { knownNumber, knownRate } = require('../src/utils/known');
+const { nameKey } = require('../src/utils/playerName');
+
+const SB_URL = process.env.SUPABASE_URL;
+const SB_KEY = process.env.SUPABASE_SERVICE_ROLE_KEY || process.env.SUPABASE_SERVICE_KEY;
+const PAGE = 1000;
+const ARCHS = (process.env.HF_ARCHETYPES || 'BOMBER,GHOST,ALL').split(',');
+
+async function page(sb, table, select, apply) {
+  const out = [];
+  for (let from = 0; ; from += PAGE) {
+    const { data, error } = await apply(sb.from(table).select(select)).range(from, from + PAGE - 1);
+    if (error) throw error;
+    if (!data || data.length === 0) break;
+    out.push(...data);
+    if (data.length < PAGE) break;
+  }
+  return out;
+}
+
+/**
+ * THE FACTORS. Each returns a MULTIPLIER on the base rate, or null when the
+ * input is absent — an absent factor must leave the baseline untouched rather
+ * than nudge it toward some default.
+ */
+const FACTORS = [
+  {
+    key: 'defense',
+    needs: ['team_defense'],
+    mechanism: 'A ball in play becomes a hit or an out partly by who is standing behind the pitcher. Should matter most where contact stays in the park.',
+    // More outs converted above average -> fewer hits.
+    apply: (r) => 1 - Math.max(-0.12, Math.min(0.12, r.team_defense / 250)),
+  },
+  {
+    key: 'pitcher_contact_profile',
+    needs: ['pitcher_hard_hit_allowed'],
+    mechanism: 'A contact-allowing arm concedes better contact than a bat-misser; hit probability should follow the quality of contact he permits.',
+    apply: (r) => 1 + Math.max(-0.15, Math.min(0.15, (r.pitcher_hard_hit_allowed - 0.389) * 1.2)),
+  },
+  {
+    key: 'park_hits',
+    needs: ['park_factor'],
+    mechanism: 'Some parks turn outs into hits without producing runs — big outfields, high walls, deep gaps.',
+    apply: (r) => r.park_factor,
+    caveat: 'STAT_BASE maps hits -> run_base, so this is a RUN factor standing in for a HITS factor. A park that converts outs to hits without scoring is invisible to it.',
+  },
+  {
+    key: 'platoon',
+    needs: ['platoon_edge'],
+    mechanism: 'Handedness advantage — a hitter facing the opposite hand sees the ball better and hits it harder.',
+    apply: (r) => (r.platoon_edge > 0 ? 1.06 : 0.96),
+  },
+];
+
+async function main() {
+  if (!SB_URL || !SB_KEY) throw new Error('SUPABASE_URL / service key required');
+  const sb = createClient(SB_URL, SB_KEY, { auth: { persistSession: false } });
+
+  const statcast = await page(sb, 'statcast_aggregates', '*', (q) => q.eq('sport', 'mlb'));
+  const batters = new Map(); const pitchersById = new Map();
+  for (const r of statcast) {
+    const prof = sk.fromStatcastRow(r);
+    if (r.role === 'pitcher' && r.source_id != null) pitchersById.set(Number(r.source_id), prof);
+    if (r.role === 'batter' && r.player_key) batters.set(r.player_key, prof);
+  }
+  const defRows = await page(sb, 'team_defense', '*', (q) => q.eq('sport', 'mlb'));
+  const defByTeam = new Map();
+  for (const d of defRows) defByTeam.set(d.team, d);
+
+  const snaps = await page(sb, 'model_snapshots', 'player_key, game_date, archetype, stat',
+    (q) => q.eq('sport', 'mlb').eq('stat', 'hits').not('archetype', 'is', null));
+  const archOf = new Map();
+  for (const s of snaps) archOf.set(`${s.player_key}|${s.game_date}`, s.archetype);
+
+  const led = await page(sb, 'ledger_entries',
+    'id, player_key, player_name, line, side, outcome, game_date, p_win, quarantine_reason, env_park_base',
+    (q) => q.eq('sport', 'mlb').is('user_id', null).eq('stat', 'hits')
+      .in('outcome', ['hit', 'miss']).not('p_win', 'is', null));
+  const clean = led.filter((r) => !(r.quarantine_reason || '').startsWith('nontakeable_book'));
+
+  // Opponent faced, from each hitter's own game log.
+  const names = new Map();
+  for (const r of clean) if (!names.has(r.player_key)) names.set(r.player_key, r.player_name);
+  const oppBy = new Map(); const startersBy = new Map();
+  const dates = [...new Set(clean.map((r) => r.game_date))].sort();
+  for (const d of dates) {
+    try {
+      const games = await mlb.getScheduleWithPitchers(d);
+      for (const g of games) {
+        if (!g.home || !g.away) continue;
+        if (g.home.probablePitcher) startersBy.set(`${d}|OPP:${g.home.team}`, g.home.probablePitcher.id);
+        if (g.away.probablePitcher) startersBy.set(`${d}|OPP:${g.away.team}`, g.away.probablePitcher.id);
+      }
+    } catch { /* absent slate */ }
+  }
+  for (const [key, name] of names) {
+    try {
+      const found = await mlb.searchPlayer(name);
+      if (!found || !found.id) continue;
+      const log = await mlb.getPlayerGameLog(found.id);
+      for (const g of log || []) if (g && g.date && g.opponent) oppBy.set(`${key}|${String(g.date).slice(0, 10)}`, g.opponent);
+    } catch { /* no log */ }
+  }
+
+  // Per-player base rate — the honest null: "he's due", no reading of tonight.
+  const byPlayer = new Map();
+  for (const r of clean) {
+    const cur = byPlayer.get(r.player_key) || { n: 0, w: 0 };
+    cur.n += 1; cur.w += r.outcome === 'hit' ? 1 : 0;
+    byPlayer.set(r.player_key, cur);
+  }
+
+  const rows = [];
+  for (const r of clean) {
+    const bat = batters.get(r.player_key);
+    const bp = byPlayer.get(r.player_key);
+    if (!bp || bp.n < 3) continue;
+    // Leave-one-out so a row never contributes to its own baseline.
+    const baseline = (bp.w - (r.outcome === 'hit' ? 1 : 0)) / (bp.n - 1);
+    const faced = oppBy.get(`${r.player_key}|${r.game_date}`) || null;
+    const nick = faced ? String(faced).split(' ').pop() : null;
+    const def = faced ? (defByTeam.get(faced) || defByTeam.get(nick)) : null;
+    const starterId = faced ? startersBy.get(`${r.game_date}|OPP:${faced}`) : null;
+    const pit = starterId != null ? pitchersById.get(Number(starterId)) : null;
+    rows.push({
+      id: r.id,
+      archetype: archOf.get(`${r.player_key}|${r.game_date}`) || null,
+      won: r.outcome === 'hit' ? 1 : 0,
+      baseline,
+      team_defense: def ? knownNumber(def.oaa_sum) : null,
+      pitcher_hard_hit_allowed: pit ? knownRate(pit.hard_hit_pct) : null,
+      park_factor: knownNumber(r.env_park_base),
+      platoon_edge: (bat && pit && bat.bats && pit.throws)
+        ? (String(bat.bats)[0] !== String(pit.throws)[0] ? 1 : -1) : null,
+    });
+  }
+
+  // Cumulative Bonferroni across the programme lifetime.
+  const store = tl.supabaseStore(sb);
+  const mc = await tl.recordAndCount(store, FACTORS.flatMap((f) =>
+    ARCHS.map((a) => ({ sport: 'mlb', stat: 'hits', archetype: a === 'ALL' ? null : a, interaction: `factor:${f.key}`, target: 'outcome' }))));
+
+  const results = [];
+  for (const arch of ARCHS) {
+    const slot = arch === 'ALL' ? rows : rows.filter((r) => String(r.archetype || '').toUpperCase() === arch);
+    for (const f of FACTORS) {
+      const usable = slot.filter((r) => f.needs.every((k) => knownNumber(r[k]) !== null));
+      const paired = usable.map((r) => {
+        const mult = f.apply(r);
+        const cond = mult === null ? null : Math.min(0.99, Math.max(0.01, r.baseline * mult));
+        return { baseline: r.baseline, conditioned: cond, won: r.won };
+      });
+      const v = fg.adjudicate(paired, {
+        factor: f.key, archetype: arch, stat: 'hits',
+        cumulativeTests: mc.cumulative_tests,      // native cumulative correction
+      });
+      results.push({
+        archetype: arch, factor: f.key, n: v.movement.n,
+        mean_abs_shift: v.movement.mean_abs_shift,
+        brier_delta: v.improvement ? v.improvement.brier_delta : null,
+        ci: v.improvement ? v.improvement.ci : null,
+        ci_level: v.improvement ? v.improvement.ci_level : null,
+        verdict: v.verdict,
+        reason: v.reason,
+        ...(f.caveat ? { input_caveat: f.caveat } : {}),
+      });
+    }
+  }
+
+  console.log(JSON.stringify({
+    baseline: "each row scored against the player's OWN leave-one-out base rate — the honest 'he's due' null",
+    total_rows: rows.length,
+    cumulative_bonferroni: mc,
+    gate: 'a factor must MOVE the prediction AND improve out-of-sample Brier; movement alone is THEATER',
+    results,
+    proven: results.filter((r) => r.verdict === 'PROVES'),
+    theater: results.filter((r) => r.verdict === 'THEATER'),
+  }, null, 2));
+  process.exit(0);
+}
+
+main().catch((e) => { console.error(e); process.exit(1); });