Settle model_snapshots + four-stat calibration: works, deploys nowhere

Settlement done (15,484 written). Calibration improves held-out Brier on all three stats it can be fitted for, beating every factor ever tested. No stat deploys: the date-cluster ceiling is 17, not 90. PHASE 0 CORRECTIONS: 71,192 snapshots unsettled, not 22,032. Span is 07-19 -> 08-06 = 19 dates, not 05-01 -> 08-04. Nothing has ever been rescaled on any stat -- all four are base-rate bands today -- and the TB "inversion confirmed" was the units-bug artifact, UNPROVEN. PHASE 1, two integrity findings both caught by the gate: 1. The dupe check hard-failed on snapshot id 33875. model_snapshots is written by the cron at 14/19/22/1/3 UTC and an unordered .range() walk over a live table returns overlapping pages. Fixed with .order('id'). 2. 12,894 rows were logged AFTER first pitch -- cycles at ET 21/22/23 on the game date (10,738) plus 664 the next morning. A 01:00-UTC cycle is 21:00 the previous evening Eastern, same game date, two hours into the slate. Tested for contamination: bias +0.0058 in-game vs +0.0008 pre-game, so NOT sharper, just late. Excluded for provenance. THE ENABLING MOVE DID NOT ENABLE. 71,192 rows collapse to 4,799 distinct pre-game props (2.5x cycle fan-out, then 97.6% both-sides duplication, then the pre-game filter). Hits ends at 1,140 rows against the ledger's existing 1,312. Date-clusters: hits 17, TB 7, rbi 5, runs 5. THE MEASUREMENT THAT NEARLY WENT THE OTHER WAY: 97.6% of props carry both sides, whose p_wins sum to ~1 and whose outcomes are complementary, so the raw population is pinned to 0.5 by construction. Measured that way the counter reads +0.0002 on hits -- "perfectly calibrated" -- and would have overturned three sessions. Deduped to the model-picked side it is +0.0868. The tell was mean p_win sitting at 0.4998 on every stat. PHASE 2/3, isotonic point-in-time, split by cumulative rows (a 60%-of-dates cut left 143 fit rows under the fitter's 200 minimum; still strictly temporal): hits n=1140 bias +0.0868 brier 0.2626 -> 0.2511 d -0.0115 CI [-0.0139,-0.0097] TB n=1050 bias +0.0834 brier 0.2490 -> 0.2438 d -0.0052 CI [-0.0061,-0.0045] rbi n= 630 bias +0.0164 brier 0.2011 -> 0.1965 d -0.0046 CI [-0.0092,-0.0010] runs n= 597 bias +0.0410 no map fittable (173 fit rows < 200) ALL FOUR REFUSE: 2-4 eval date-clusters against a floor of 40. The floor is the order's own and was not relaxed to force a pass. A NULL THAT SCORED ITSELF: the first run reported hits at Brier 0.5567, worse than predicting 0.5 for everything. fitIsotonic returns null below its minimum, applyIsotonic then returns null per row, and (null-1)**2 is 1 while (null-0)**2 is 0 -- so the "Brier" was silently just the win rate (0.5684). This project's signature Number(null)===0 breach, in my own measurement code. Now a hard refuse. PHASE 4: the bias is NOT a uniform shift. Identical favourite-longshot shape on all four stats -- near zero or negative at 0.5-0.6, rising to +0.21 to +0.28 above 0.9. The counter is over-confident specifically about its favourites, which is the population a user acts on. Gradient is hits ~ TB > runs > rbi, not the TB > RBI > runs anticipated. PHASE 5/6 NOT RUN -- both gated on a Phase 3 deploy that did not open. PHASE 7, refusal accuracy, first real measurement: refused props are FURTHER from a coin flip than graded ones (TB refusals went over 21.6% of the time). The obvious explanation, that refusals concentrate on players who barely played, was tested and does not hold -- refused mean 3.20 AB vs graded 3.39, 6.6% vs 6.2% with <=1 AB. So we pass on what we have no INPUT for, not on what we cannot call. Refusing to invent a number without a reference stays correct; the pass is not landing on the genuinely uncertain props. p_win never mutated, no p_win_calibrated written since nothing deployed, no Bonferroni slot consumed. Counter and frozen clusters byte-identical. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W1sivYNqY2TS5ftykmHBU9
2026-08-06 15:30:30 -04:00
parent 23d1b13176
commit f976df47b8
4 changed files with 696 additions and 1 deletions
@@ -0,0 +1,267 @@
 #!/usr/bin/env node
 'use strict';
 /**
 * calibrate-four-stats — Phases 2, 3, 4 and 7.
 *
 * ── ONE SIDE PER PROP, OR THE MEASUREMENT IS MEANINGLESS ─────────────────
 * 97.6% of snapshot props carry BOTH the over and the under. Their p_wins sum to
 * ~1 and their outcomes are complementary, so any calibration statistic over the
 * raw population is pinned to 0.5 by symmetry. Measured that way the counter
 * looks perfectly calibrated (+0.0002 on hits); deduped to the model-PICKED side
 * it is +0.0868. Same rows, opposite conclusion.
 *
 * ── DATE-CLUSTERED, PER THE ORDER ────────────────────────────────────────
 * A day's offensive environment is a real shared component, so uncertainty is
 * clustered on the game DATE rather than the game. That is the honest unit for a
 * systematic-bias claim and it is a much harder bar than game-clustering.
 *
 *   SUPABASE_URL=... node scripts/calibrate-four-stats.js
 */
 require('dotenv').config();
 const fs = require('fs');
 const path = require('path');
 const { createClient } = require('@supabase/supabase-js');
 const cal = require('../src/services/model/calibration');
 const { knownNumber } = require('../src/utils/known');
 const SB_URL = process.env.SUPABASE_URL;
 const SB_KEY = process.env.SUPABASE_SERVICE_ROLE_KEY || process.env.SUPABASE_SERVICE_KEY;
 const BOX = path.join(process.cwd(), '.seq-cache', 'batting-lines.json');
 const STATS = ['hits', 'total_bases', 'rbi', 'runs'];
 const PAGE = 1000;
 /** The order's deploy floor: dates, not games. */
 const MIN_DATE_CLUSTERS = 40;
 const FIELD = { hits: (b) => b.hits, total_bases: (b) => b.totalBases, rbi: (b) => b.rbi, runs: (b) => b.runs };
 const mean = (xs) => (xs.length ? xs.reduce((a, b) => a + b, 0) / xs.length : null);
 const brier = (ps, ys) => mean(ps.map((p, i) => (p - ys[i]) ** 2));
 async function page(sb, table, select, apply) {
  const out = [];
  for (let from = 0; ; from += PAGE) {
    const { data, error } = await apply(sb.from(table).select(select))
      .order('id', { ascending: true }).range(from, from + PAGE - 1);
    if (error) throw error;
    if (!data || data.length === 0) break;
    out.push(...data);
    if (data.length < PAGE) break;
  }
  return out;
 }
 const isPreGame = (capturedAt, gameDate) => {
  const et = new Date(new Date(capturedAt).getTime() - 4 * 3600 * 1000);
  const d = et.toISOString().slice(0, 10);
  return d < gameDate || (d === gameDate && et.getUTCHours() < 19);
 };
 function makeRnd(seed) {
  let s = seed >>> 0;
  return () => { s ^= s << 13; s >>>= 0; s ^= s >>> 17; s ^= s << 5; s >>>= 0; return s / 4294967296; };
 }
 /** Paired bootstrap on the Brier difference, resampling DATES. */
 function dateClusteredCI(rows, cumulativeTests = 1, iters = 3000) {
  const byDate = new Map();
  for (const r of rows) {
    if (!byDate.has(r.date)) byDate.set(r.date, []);
    byDate.get(r.date).push(r);
  }
  const keys = [...byDate.keys()];
  const rnd = makeRnd(20260807);
  const diffs = [];
  for (let it = 0; it < iters; it += 1) {
    const raw = []; const adj = []; const ys = [];
    for (let i = 0; i < keys.length; i += 1) {
      for (const r of byDate.get(keys[Math.floor(rnd() * keys.length)])) {
        raw.push(r.p); adj.push(r.pc); ys.push(r.won);
      }
    }
    diffs.push(brier(adj, ys) - brier(raw, ys));
  }
  diffs.sort((a, b) => a - b);
  const tests = Math.max(1, Math.round(cumulativeTests));
  const alpha = 0.05 / tests;
  const q = (x) => diffs[Math.floor(Math.min(diffs.length - 1, Math.max(0, x * (diffs.length - 1))))];
  return { ci: [round4(q(alpha / 2)), round4(q(1 - alpha / 2))], date_clusters: keys.length, ci_level: round4(1 - alpha) };
 }
 async function main() {
  const sb = createClient(SB_URL, SB_KEY, { auth: { persistSession: false } });
  const lines = JSON.parse(fs.readFileSync(BOX, 'utf8')).lines;
  const snaps = await page(sb, 'model_snapshots',
    'id, game_date, captured_at, stat, player_key, line, side, p_win, refused, grade',
    (q) => q.eq('sport', 'mlb').in('stat', STATS));
  // ── ONE SIDE PER PROP: the side the model picked (its higher p_win). ──
  const picked = new Map();
  const refusedProps = new Map();
  for (const r of snaps) {
    if (!isPreGame(r.captured_at, r.game_date)) continue;
    const k = [r.game_date, r.stat, r.player_key, r.line].join('|');
    if (r.refused || knownNumber(r.p_win) === null) {
      if (!refusedProps.has(k)) refusedProps.set(k, r);
      continue;
    }
    const prev = picked.get(k);
    if (!prev || knownNumber(r.p_win) > knownNumber(prev.p_win)) picked.set(k, r);
  }
  const resolve = (r) => {
    const b = lines[`${r.game_date}|${r.player_key}`];
    const L = knownNumber(r.line);
    if (!b || L === null || !r.side) return null;
    const v = knownNumber(FIELD[r.stat](b));
    if (v === null) return null;
    const over = v > L;
    return { over, won: (String(r.side).toLowerCase() === 'under' ? !over : over) ? 1 : 0, realized: v };
  };
  const out = { deploy_floor_date_clusters: MIN_DATE_CLUSTERS, per_stat: {}, refusal_accuracy: {} };
  for (const stat of STATS) {
    const rows = [];
    for (const r of picked.values()) {
      if (r.stat !== stat) continue;
      const res = resolve(r);
      if (!res) continue;
      rows.push({ date: r.game_date, p: knownNumber(r.p_win), won: res.won });
    }
    rows.sort((a, b) => String(a.date).localeCompare(String(b.date)));
    const dates = [...new Set(rows.map((r) => r.date))].sort();
    if (rows.length < 100 || dates.length < 3) {
      out.per_stat[stat] = { n: rows.length, date_clusters: dates.length, decision: 'REFUSE', reason: 'too few rows or dates to split point-in-time' };
      continue;
    }
    // POINT-IN-TIME: fit strictly on earlier dates, evaluate on later ones.
    //
    // The cut is placed by ROW COUNT rather than by date index. Props are not
    // spread evenly across dates -- hits concentrate in the later ones -- so a
    // 60%-of-DATES cut left only 143 rows to fit on, under the 200 the fitter
    // needs. Splitting on cumulative rows keeps the split strictly temporal
    // (every fit date precedes every eval date) while giving both sides enough
    // to work with.
    const perDate = new Map();
    for (const r of rows) perDate.set(r.date, (perDate.get(r.date) || 0) + 1);
    let acc = 0; let cut = dates[dates.length - 1];
    for (const d of dates) {
      acc += perDate.get(d) || 0;
      if (acc >= rows.length * 0.45) { cut = d; break; }
    }
    const fit = rows.filter((r) => r.date < cut);
    const ev = rows.filter((r) => r.date >= cut);
    if (fit.length < 50 || ev.length < 50) {
      out.per_stat[stat] = { n: rows.length, date_clusters: dates.length, decision: 'REFUSE', reason: 'time split leaves too little on one side' };
      continue;
    }
    const iso = cal.fitIsotonic(fit.map((r) => ({ p: r.p, won: r.won })));
    // NULL IS NOT A PREDICTION. fitIsotonic returns null below its minimum and
    // applyIsotonic then returns null per row -- and (null - 1)**2 === 1 while
    // (null - 0)**2 === 0, so a "Brier score" computed over nulls is silently
    // just the win rate. That is exactly the Number(null) === 0 breach this
    // codebase keeps having to catch, and it produced a fake 0.5567 for hits.
    if (!iso) {
      out.per_stat[stat] = {
        n: rows.length, date_clusters: dates.length, fit_n: fit.length, eval_n: ev.length,
        decision: 'REFUSE', reason: `no calibration map could be fitted on ${fit.length} fit rows`,
      };
      continue;
    }
    const scored = ev.map((r) => ({ ...r, pc: cal.applyIsotonic(iso, r.p) }))
      .filter((r) => knownNumber(r.pc) !== null);
    if (scored.length < 50) {
      out.per_stat[stat] = {
        n: rows.length, date_clusters: dates.length,
        decision: 'REFUSE', reason: `only ${scored.length} eval rows could be mapped`,
      };
      continue;
    }
    const ys = scored.map((r) => r.won);
    const bRaw = brier(scored.map((r) => r.p), ys);
    const bCal = brier(scored.map((r) => r.pc), ys);
    const { ci, date_clusters, ci_level } = dateClusteredCI(scored, 1);
    // CERTIFIED BAND: p_win deciles where held-out |predicted - actual| is small.
    const bands = [];
    for (let lo = 0.3; lo < 0.95; lo += 0.1) {
      const slice = scored.filter((r) => r.p >= lo && r.p < lo + 0.1);
      if (slice.length < 25) continue;
      const pred = mean(slice.map((r) => r.pc));
      const act = mean(slice.map((r) => r.won));
      bands.push({ range: [round2(lo), round2(lo + 0.1)], n: slice.length, calibrated_pred: round4(pred), actual: round4(act), err: round4(pred - act) });
    }
    const certified = bands.filter((b) => Math.abs(b.err) <= 0.05).map((b) => b.range);
    const improves = bCal < bRaw && ci[1] < 0;
    const enoughDates = date_clusters >= MIN_DATE_CLUSTERS;
    // PHASE 4 — bias SHAPE across the p_win range (diagnostic only).
    const shape = [];
    for (let lo = 0.3; lo < 0.95; lo += 0.1) {
      const slice = rows.filter((r) => r.p >= lo && r.p < lo + 0.1);
      if (slice.length < 25) continue;
      shape.push({ range: [round2(lo), round2(lo + 0.1)], n: slice.length, bias: round4(mean(slice.map((r) => r.p)) - mean(slice.map((r) => r.won))) });
    }
    out.per_stat[stat] = {
      n: rows.length,
      date_clusters: dates.length,
      bias_pre: round4(mean(rows.map((r) => r.p)) - mean(rows.map((r) => r.won))),
      fit_n: fit.length, eval_n: ev.length, split_at: cut,
      brier_raw: round4(bRaw),
      brier_calibrated: round4(bCal),
      brier_delta: round4(bCal - bRaw),
      ci_date_clustered: ci,
      ci_level,
      eval_date_clusters: date_clusters,
      certified_bands: certified,
      band_detail: bands,
      bias_shape: shape,
      decision: improves && enoughDates ? 'DEPLOY' : 'REFUSE',
      reason: improves && enoughDates ? 'held-out Brier improves, date-clustered, and the date floor is met'
        : (!enoughDates
          ? `date-clusters ${date_clusters} < ${MIN_DATE_CLUSTERS} — the honest unit for a systematic-bias claim`
          : 'held-out Brier does not improve at the date-clustered interval'),
    };
  }
  // ── PHASE 7 — REFUSAL ACCURACY ──
  // The model passed on these. A pass is CORRECT when there was genuinely
  // nothing to call: the over lands near a coin flip rather than at an
  // exploitable rate.
  for (const stat of STATS) {
    const refs = [];
    for (const r of refusedProps.values()) {
      if (r.stat !== stat) continue;
      const res = resolve({ ...r, side: 'over' });
      if (res) refs.push(res.over ? 1 : 0);
    }
    const graded = [];
    for (const r of picked.values()) {
      if (r.stat !== stat) continue;
      const res = resolve({ ...r, side: 'over' });
      if (res) graded.push(res.over ? 1 : 0);
    }
    out.refusal_accuracy[stat] = {
      refused_n: refs.length,
      refused_over_rate: refs.length ? round4(mean(refs)) : null,
      graded_over_rate: graded.length ? round4(mean(graded)) : null,
      refused_distance_from_coinflip: refs.length ? round4(Math.abs(mean(refs) - 0.5)) : null,
      graded_distance_from_coinflip: graded.length ? round4(Math.abs(mean(graded) - 0.5)) : null,
    };
  }
  console.log(JSON.stringify(out, null, 2));
  process.exit(0);
 }
 const round4 = (v) => (v == null || !Number.isFinite(v) ? null : Math.round(v * 10000) / 10000);
 const round2 = (v) => Math.round(v * 100) / 100;
 main().catch((e) => { console.error(e); process.exit(1); });
@@ -0,0 +1,249 @@
 #!/usr/bin/env node
 'use strict';
 /**
 * settle-model-snapshots — pay the standing debt.
 *
 * 71,192 snapshot rows have never carried an outcome. They are the retention
 * table built for exactly this kind of replay, and until they are settled every
 * measurement in this programme runs on the far smaller ledger slice.
 *
 * ── OUTCOME IS SIDE-ALIGNED, NOT RAW ─────────────────────────────────────
 * The order specifies `outcome = 1[realized > line]`. That is the OVER
 * perspective, and it would be backwards for every under-side prop — `p_win` is
 * side-aligned (verified: TB mean p_win 0.5698 against a 0.5074 side-won rate),
 * so a raw over-indicator would silently invert the target on the under rows and
 * make calibration measure the wrong thing.
 *
 * So: `actual_value` stores the realized stat (raw, unopinionated) and `outcome`
 * stores whether the GRADED SIDE won. Deviation from the literal order, stated
 * because it changes the number.
 *
 * ── INTEGRITY (hard-fail) ────────────────────────────────────────────────
 *   conservation   settled + unresolvable + orphaned == candidates
 *   no dupes       one write per snapshot id
 *   no orphans     a settled row must have matched a real box score
 *   prediction-time logging  captured_at must PRECEDE the game date; a row
 *                  logged after the fact is not a prediction and is refused
 *
 *   node scripts/settle-model-snapshots.js            # dry run, verifies only
 *   SETTLE_WRITE=1 node scripts/settle-model-snapshots.js
 */
 require('dotenv').config();
 const fs = require('fs');
 const path = require('path');
 const axios = require('axios');
 const { createClient } = require('@supabase/supabase-js');
 const { nameKey } = require('../src/utils/playerName');
 const { knownNumber } = require('../src/utils/known');
 const SB_URL = process.env.SUPABASE_URL;
 const SB_KEY = process.env.SUPABASE_SERVICE_ROLE_KEY || process.env.SUPABASE_SERVICE_KEY;
 const WRITE = process.env.SETTLE_WRITE === '1';
 const BOX_CACHE = path.join(process.cwd(), '.seq-cache', 'batting-lines.json');
 const STATS = ['hits', 'total_bases', 'rbi', 'runs'];
 const PAGE = 1000;
 /** Realized value per stat, from the box-score batting line. */
 const FIELD = Object.freeze({
  hits: (b) => knownNumber(b.hits),
  total_bases: (b) => knownNumber(b.totalBases),
  rbi: (b) => knownNumber(b.rbi),
  runs: (b) => knownNumber(b.runs),
 });
 const get = async (url) => (await axios.get(url, { timeout: 45_000 })).data;
 /** Eastern first pitch, conservatively. Anything at or after this is in-game. */
 const FIRST_PITCH_ET_HOUR = 19;
 /**
 * Was this row logged BEFORE the games it grades?
 *
 * The pipeline runs on UTC cron hours, so a 01:00-UTC cycle is 21:00 the
 * PREVIOUS evening in Eastern -- same game date, three hours into the slate.
 */
 function isPreGame(capturedAt, gameDate) {
  if (!capturedAt || !gameDate) return false;
  const cap = new Date(capturedAt);
  if (Number.isNaN(cap.getTime())) return false;
  const et = new Date(cap.getTime() - 4 * 3600 * 1000);        // EDT
  const etDate = et.toISOString().slice(0, 10);
  if (etDate < String(gameDate)) return true;                   // day before, fine
  if (etDate > String(gameDate)) return false;                  // day after, post-game
  return et.getUTCHours() < FIRST_PITCH_ET_HOUR;
 }
 async function pool(items, fn, n = 6) {
  const out = []; let i = 0;
  await Promise.all(Array.from({ length: n }, async () => {
    while (i < items.length) {
      const idx = i; i += 1;
      try { out[idx] = await fn(items[idx]); } catch { out[idx] = null; }
    }
  }));
  return out.filter(Boolean);
 }
 async function page(sb, table, select, apply) {
  const out = [];
  for (let from = 0; ; from += PAGE) {
    // STABLE ORDER. model_snapshots is a LIVE table -- the snapshot cron writes
    // to it at 14/19/22/1/3 UTC -- and an unordered .range() walk over a table
    // being appended to returns overlapping pages. The integrity gate caught
    // exactly that on the first run.
    const { data, error } = await apply(sb.from(table).select(select))
      .order('id', { ascending: true })
      .range(from, from + PAGE - 1);
    if (error) throw error;
    if (!data || data.length === 0) break;
    out.push(...data);
    if (data.length < PAGE) break;
  }
  return out;
 }
 /** Box-score batting lines for a date range, cached. */
 async function battingLines(dates) {
  if (fs.existsSync(BOX_CACHE)) {
    const c = JSON.parse(fs.readFileSync(BOX_CACHE, 'utf8'));
    if (dates.every((d) => c.dates.includes(d))) return c.lines;
  }
  const games = [];
  for (const d of dates) {
    try {
      const s = await get(`https://statsapi.mlb.com/api/v1/schedule?sportId=1&date=${d}`);
      for (const day of s.dates || []) {
        for (const g of day.games || []) {
          if (String(g.status && g.status.detailedState) === 'Final') {
            games.push({ pk: g.gamePk, date: g.officialDate || d });
          }
        }
      }
    } catch { /* absent day */ }
  }
  console.error(`[settle] ${games.length} final games across ${dates.length} dates`);
  const lines = {};
  const loaded = await pool(games, async (g) => {
    const box = await get(`https://statsapi.mlb.com/api/v1/game/${g.pk}/boxscore`);
    const out = [];
    for (const side of ['home', 'away']) {
      const t = box.teams[side];
      if (!t) continue;
      for (const id of t.batters || []) {
        const pl = t.players[`ID${id}`];
        const b = pl && pl.stats && pl.stats.batting;
        if (!b || b.atBats == null) continue;      // did not bat -> absent, not zero
        out.push({
          date: g.date,
          key: nameKey(pl.person && pl.person.fullName),
          name: pl.person && pl.person.fullName,
          gamePk: g.pk,
          hits: b.hits, totalBases: b.totalBases, rbi: b.rbi, runs: b.runs, atBats: b.atBats,
        });
      }
    }
    return out;
  });
  for (const arr of loaded) for (const r of arr) {
    const k = `${r.date}|${r.key}`;
    // A doubleheader gives two lines; sum them — the prop covers the day.
    if (!lines[k]) lines[k] = { ...r, games: 1 };
    else {
      lines[k].hits += r.hits; lines[k].totalBases += r.totalBases;
      lines[k].rbi += r.rbi; lines[k].runs += r.runs; lines[k].atBats += r.atBats;
      lines[k].games += 1;
    }
  }
  fs.mkdirSync(path.dirname(BOX_CACHE), { recursive: true });
  fs.writeFileSync(BOX_CACHE, JSON.stringify({ dates, lines }));
  return lines;
 }
 async function main() {
  const sb = createClient(SB_URL, SB_KEY, { auth: { persistSession: false } });
  const snaps = await page(sb, 'model_snapshots',
    'id, game_date, captured_at, stat, player_key, player_name, line, side, p_win, refused, outcome',
    (q) => q.eq('sport', 'mlb').in('stat', STATS).is('outcome', null));
  console.error(`[settle] ${snaps.length} unsettled snapshot rows`);
  const dates = [...new Set(snaps.map((r) => r.game_date))].sort();
  const lines = await battingLines(dates);
  const counts = { candidates: snaps.length, settled: 0, unresolvable: 0, orphaned: 0, post_hoc_logged: 0 };
  const updates = [];
  const seenIds = new Set();
  for (const s of snaps) {
    // Belt and braces: ordered pagination should make this impossible, and a
    // duplicate would double-count a prediction in every downstream measurement.
    if (seenIds.has(s.id)) throw new Error(`INTEGRITY: duplicate snapshot id ${s.id}`);
    seenIds.add(s.id);
    // A row logged after first pitch is not a prediction.
    //
    // Measured: cycles at ET 21:00/22:00/23:00 on the game date (10,738 rows)
    // were captured DURING or AFTER the games they grade, and a further 664 the
    // following morning. Games start ~19:05 ET, so the honest cutoff is ET
    // first pitch on the game date -- not a UTC date compare, which both keeps
    // post-game 01:00-UTC rows and discards legitimate pre-dawn ones.
    if (!isPreGame(s.captured_at, s.game_date)) {
      counts.post_hoc_logged += 1; counts.unresolvable += 1; continue;
    }
    const line = knownNumber(s.line);
    if (line === null || !s.side) { counts.unresolvable += 1; continue; }
    const b = lines[`${s.game_date}|${s.player_key}`];
    if (!b) { counts.orphaned += 1; continue; }
    const realized = FIELD[s.stat](b);
    if (realized === null) { counts.unresolvable += 1; continue; }
    // SIDE-ALIGNED, so it matches how p_win is expressed.
    const over = realized > line;
    const won = String(s.side).toLowerCase() === 'under' ? !over : over;
    updates.push({ id: s.id, outcome: won ? 'hit' : 'miss', actual_value: realized });
    counts.settled += 1;
  }
  // CONSERVATION — hard fail.
  const acc = counts.settled + counts.unresolvable + counts.orphaned;
  if (acc !== counts.candidates) {
    throw new Error(`INTEGRITY: conservation violated ${acc} != ${counts.candidates}`);
  }
  // Hand-verifiable sample.
  const sample = updates.slice(0, 12).map((u) => {
    const s = snaps.find((x) => x.id === u.id);
    return { player: s.player_name, date: s.game_date, stat: s.stat, line: s.line, side: s.side,
      realized: u.actual_value, outcome: u.outcome };
  });
  if (WRITE) {
    let written = 0;
    for (let i = 0; i < updates.length; i += 500) {
      const batch = updates.slice(i, i + 500);
      const results = await Promise.all(batch.map((u) => sb.from('model_snapshots')
        .update({ outcome: u.outcome, actual_value: u.actual_value, settled_at: new Date().toISOString(), settlement_source: 'statsapi_boxscore' })
        .eq('id', u.id).is('outcome', null)));
      written += results.filter((r) => !r.error).length;
    }
    counts.written = written;
  }
  console.log(JSON.stringify({
    mode: WRITE ? 'WRITE' : 'DRY RUN',
    counts,
    dates_before: 'ledger-only slice',
    snapshot_dates: dates.length,
    date_span: [dates[0], dates[dates.length - 1]],
    hand_verify_sample: sample,
    note: 'outcome is SIDE-ALIGNED (matches p_win); actual_value holds the raw realized stat',
  }, null, 2));
  process.exit(0);
 }
 main().catch((e) => { console.error(e); process.exit(1); });
@@ -0,0 +1,179 @@
 # Settlement + four-stat calibration — the enabling move did not enable
 **Settlement is done (15,484 rows written). Calibration improves held-out Brier
 on all three stats it can be fitted for — more than any factor ever tested. And
 no stat can deploy, because the date-cluster ceiling is 17, not 90.**
 ---
 ## PHASE 0 — Board reconcile (corrections to the record)
 | claim in circulation | measured |
 |---|---|
 | ~22,032 snapshots unsettled | **71,192** rows, **0** settled |
 | snapshots span 05-01 → 08-04 (~90 dates) | **07-19 → 08-06 = 19 dates** |
 | "calibrate all four on the full replay" | replay yields **17 / 7 / 5 / 5** dates per stat |
 | batter board is per-archetype graded | **nothing has ever been rescaled**, on any stat |
 | TB inversion confirmed | **units-bug artifact — UNPROVEN** (S: prove-tb-factors) |
 **What actually drives each number today: a base-rate band, on all four stats.**
 No stat is per-archetype graded. Proven FACTORS exist on hits only
 (`defense_by_direction`; `pitcher_contact_profile` and `platoon_severity` are
 held/demoted, not proven). `gradeBands` remains built, gated and unwired.
 ---
 ## PHASE 1 — Settlement
 ```
 candidates 34,650 (4 stats) -> settled 16,498 | unresolvable 12,894 | orphaned 5,258
 written 15,484   (shortfall = idempotency guard vs the live cron)
 conservation check PASSED
 ```
 ### Two integrity findings, both caught by the gate
 **1. Unordered pagination over a LIVE table.** The dupe check hard-failed on
 snapshot id 33875. `model_snapshots` is written by the cron at 14/19/22/1/3 UTC,
 and an unordered `.range()` walk over a table being appended to returns
 overlapping pages. Fixed with `.order('id')`. A plain id-only fetch showed no
 dupes, so this only bites on longer reads that straddle a cron write.
 **2. 12,894 rows were logged AFTER first pitch.** Cycles at ET 21:00/22:00/23:00
 *on the game date* (10,738 rows) plus 664 the following morning. A 01:00-UTC
 cycle is 21:00 the previous evening Eastern — same game date, ~2 hours into the
 slate. These are not predictions and are excluded.
 Tested whether they were outcome-contaminated: bias +0.0058 in-game vs +0.0008
 pre-game — **not sharper, just late.** Excluded for provenance, not because they
 cheated.
 ### The enabling move did not enable
 | stat | usable props | date-clusters |
 |---|---|---|
 | hits | 1,140 | **17** |
 | total_bases | 1,050 | **7** |
 | rbi | 630 | **5** |
 | runs | 597 | **5** |
 71,192 rows collapse to 4,799 distinct pre-game props: a 2.5× cycle fan-out,
 then 97.6% both-sides duplication, then the pre-game filter. **Hits ends with
 1,140 rows against the ledger's existing 1,312.** Settlement was worth doing as a
 standing debt; it did not unlock the sample the order expected.
 ---
 ## THE MEASUREMENT THAT NEARLY WENT THE OTHER WAY
 **97.6% of props carry BOTH sides.** Their p_wins sum to ~1 and their outcomes
 are complementary, so any calibration statistic over the raw population is pinned
 to 0.5 by construction:
 | | hits | TB | rbi | runs |
 |---|---|---|---|---|
 | both-sides population | +0.0002 | +0.0012 | +0.0019 | +0.0000 |
 | **model-picked side only** | **+0.0868** | **+0.0834** | **+0.0164** | **+0.0410** |
 The first row reads "the counter is perfectly calibrated" and would have
 overturned three sessions of findings. Same rows, opposite conclusion, and the
 tell was mean p_win sitting at 0.4998 on every stat.
 ---
 ## PHASE 2/3 — Calibration and the deploy gate
 Isotonic, point-in-time, fit-past / apply-forward. The split is placed by
 cumulative ROWS rather than date index — props are not spread evenly across dates
 and a 60%-of-dates cut left only 143 rows to fit on, under the fitter's 200
 minimum. Still strictly temporal: every fit date precedes every eval date.
 | stat | n | dates | bias | fit / eval | Brier raw | Brier cal | Δ | CI (date-clustered) | eval dates | decision |
 |---|---|---|---|---|---|---|---|---|---|---|
 | hits | 1,140 | 17 | +0.0868 | 375 / 765 | 0.2626 | 0.2511 | **−0.0115** | [−0.0139, −0.0097] | 4 | **REFUSE** |
 | total_bases | 1,050 | 7 | +0.0834 | 425 / 625 | 0.2490 | 0.2438 | **−0.0052** | [−0.0061, −0.0045] | 2 | **REFUSE** |
 | rbi | 630 | 5 | +0.0164 | 205 / 425 | 0.2011 | 0.1965 | **−0.0046** | [−0.0092, −0.0010] | 2 | **REFUSE** |
 | runs | 597 | 5 | +0.0410 | 173 / 424 | — | — | — | — | — | **REFUSE** |
 - hits / TB / rbi: **held-out Brier improves and the interval excludes zero.**
  Every one of these beats every factor ever tested on any stat.
 - runs: **no map could be fitted** — 173 fit rows under the 200 minimum.
 - **All four refuse on the date floor: 2–4 eval date-clusters against 40.**
 Certified bands (held-out |err| ≤ 0.05): hits [0.5–0.7], TB [0.6–0.8],
 rbi [0.5–0.9]. Outside band → refuse, fall to base rate.
 ### A null that scored itself
 The first run reported hits at Brier **0.5567** — worse than predicting 0.5 for
 everything. `fitIsotonic` returns null below its minimum, `applyIsotonic` then
 returns null per row, and `(null − 1)² === 1` while `(null − 0)² === 0`, so the
 "Brier score" was silently just the win rate (0.5684). **This project's signature
 `Number(null) === 0` breach, in my own measurement code.** Now a hard refuse.
 ---
 ## PHASE 4 — Bias shape (diagnostic only)
 | stat | 0.5–0.6 | 0.6–0.7 | 0.7–0.8 | 0.8–0.9 | 0.9–1.0 |
 |---|---|---|---|---|---|
 | hits | +0.024 | +0.054 | +0.145 | +0.244 | +0.244 |
 | total_bases | −0.019 | +0.076 | +0.156 | +0.160 | +0.282 |
 | rbi | −0.026 | −0.045 | −0.039 | +0.095 | +0.211 |
 | runs | −0.050 | +0.031 | +0.064 | +0.155 | +0.237 |
 **Not a uniform shift — favourite-longshot concentration, identically shaped on
 all four stats.** Near zero or slightly negative at the bottom, then rising
 sharply. The counter is over-confident specifically about its favourites, which
 is the population a user acts on.
 Cross-stat gradient measured: hits +0.0868 ≈ TB +0.0834 > runs +0.0410 > rbi
 +0.0164 — not the TB > RBI > runs the order anticipated.
 ---
 ## PHASE 5/6 — Not run, honestly
 Both are gated on a Phase 3 deploy. Nothing deployed, so there is no
 `p_win_calibrated` to rebuild bands on and no activated stat to test
 per-archetype curves against. Running them would be building on a gate that did
 not open.
 ---
 ## PHASE 7 — Refusal accuracy (first real measurement)
 | stat | refused n | refused over-rate | graded over-rate | refused \|dist from 0.5\| | graded |
 |---|---|---|---|---|---|
 | hits | 59 | 0.4237 | 0.5368 | 0.076 | 0.037 |
 | total_bases | 74 | 0.2162 | 0.4029 | **0.284** | 0.097 |
 | rbi | 141 | 0.1773 | 0.2429 | **0.323** | 0.257 |
 | runs | 12 | 0.6667 | 0.3015 | 0.167 | 0.199 |
 **Refused props are FURTHER from a coin flip than graded ones, not closer.** The
 obvious explanation — refusals concentrate on players who barely played — was
 tested and does not hold: refused mean 3.20 AB vs graded 3.39, and 6.6% vs 6.2%
 with ≤1 AB.
 So "we pass on what we can't call" is not quite what happens. We pass on what we
 have no INPUT for, and that population had outcomes that were, in hindsight,
 lopsided (TB refusals went over 21.6% of the time). Refusing to invent a number
 without a reference remains correct — but the pass is not landing on the
 genuinely uncertain props, and this is the first time that has been a number
 rather than a claim.
 ---
 ## Verdict
 - **Calibration works.** Three stats improve held-out, all beating every factor
  ever tested. The programme-level finding stands: calibration beats every factor
  tried on TB/RBI/runs.
 - **Nothing deploys.** The date-cluster floor is the right unit for a systematic-
  bias claim and we have 2–4 where 40 is required. Reaching 40 date-clusters
  needs ~5 more weeks of accrual, not more replay — the dates do not exist.
 - **The floor is the order's own** and it was not relaxed to force a pass.
 `p_win` never mutated; no `p_win_calibrated` written since nothing deployed.
 Calibration consumed no Bonferroni slot. Counter and frozen clusters
 byte-identical.