Reliever chain: Link 1 proves, Link 2 does not, and the premise inverts

The causal insight is right -- the game is a sequence and the matchup does shift mid-game. The direction is backwards, measured on 93,663 plate appearances from 1,238 games pulled free from statsapi. LINK 1 PROVES. Starter batters-faced, point-in-time from his own prior starts only, clustered on the pitcher: MAE 3.2226 -> 2.7990, delta -0.4236, CI [-0.6006,-0.2731] at 0.9995 corrected for 107 tests, 1,706 starts across 204 pitchers. It finds the tail the chain needed -- early exits are a 23.2% base rate, model-flagged starts are 34.0% early, lift +10.8pp. Scope correction inside Link 1: the order specifies fatigue x GAME SCRIPT, but game script is not available at grade time -- whether he gets hit tonight is the thing being projected, not an input to it. Only the workload half is measured; the in-game half is recorded as a live feature, out of scope, rather than quietly folded in. LINK 2 DOES NOT PROVE, twice over. Model accuracy 17.2% vs an 8.6% baseline -- doubling it sounds good and is not, since naming a specific arm is wrong five times in six. And structurally the entity is the BULLPEN: 39,629 post-starter plate appearances across 30 clubs is 30 readings, below the 40-cluster floor, the same permanent ceiling as park geometry and team defence. LINK 3 NOT RUN, per the order's own rule. THE PREMISE IS REFUTED, and this chains on nothing so it was safe to measure: vs STARTER n=48,492 hit rate 0.2444 +/-0.0038 vs BULLPEN n=35,760 hit rate 0.2373 +/-0.0044 The pen is 0.7pp HARDER. The specific effect the chain exists to exploit -- early exit making later at-bats softer -- is +0.0010 on 35,760 PAs. A well-powered null, not a sample problem. What IS real is times through the order: TTO1 0.2351 -> TTO2 0.2515 -> TTO3 0.2518. A starter does decay as the lineup sees him again, but that advantage is SURRENDERED when he leaves, not extended -- the pen is harder than his second and third time through. A modern bullpen is a queue of fresh specialists throwing one inning each; there is no tiring arm to punish. So the insight survives inverted, and Link 1 stays valuable for the opposite reason it was built: a likely early hook predicts the hitter LOSES his third-time-through look (0.2518 -> 0.2373 on that PA). The mispricing is on hitters who get an EXTRA look at a starter going deep. BUILT: predictionGate.js + tests -- the two-part gate for a continuous prediction. factorGate binarises outcomes for Brier, which would destroy a target like batters faced. Same discipline, same THEATER verdict, real scale. PRE-REGISTERED NOT RUN: Link 2' using a PA-weighted bullpen AGGREGATE rather than a named arm. Recorded rather than substituted in -- running Link 3 on a swapped-in Link 2 is the assumed-link failure the order forbids. Given the premise result its expected value is now low. PARALLEL TRACK logged: total_bases n=948 pooled, BOMBER x TB 340, short by 160. Sample-readiness only, not a verdict. Counter and frozen clusters byte-identical. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W1sivYNqY2TS5ftykmHBU9
2026-08-06 01:58:38 -04:00
parent 3081c92e00
commit e4dae0e6b0
8 changed files with 819 additions and 1 deletions
@@ -0,0 +1,160 @@
+'use strict';
+
+/**
+ * predictionGate — the two-part gate for a CONTINUOUS prediction.
+ *
+ * factorGate answers this for probabilities, where the loss is Brier. A link in
+ * the reliever chain predicts a quantity — how many batters the starter faces,
+ * which arm throws the fourth plate appearance — so the loss is squared error or
+ * a hit rate, and the binarisation factorGate applies to outcomes would silently
+ * destroy the target.
+ *
+ * The discipline is identical and deliberately so:
+ *
+ *   (a) MOVEMENT     the prediction differs from the naive baseline at all
+ *   (b) IMPROVEMENT  paired bootstrap on the loss difference, interval excluding
+ *                    zero at the cumulative-corrected level
+ *
+ * A link that predicts the league average very precisely has learned nothing, and
+ * without (a) it would pass (b) by tying. That is the same THEATER failure the
+ * probability gate exists to name, wearing different units.
+ *
+ * ── WHY CLUSTERING MATTERS HERE TOO ──────────────────────────────────────
+ * The same starter appears many times in a season, so his starts are not
+ * independent readings: a pitcher the model happens to fit well contributes a
+ * run of correlated wins. Clustering on the pitcher makes the interval reflect
+ * how many ARMS we have read, not how many starts we have counted.
+ */
+
+const { knownNumber } = require('../../utils/known');
+
+const MIN_N = 500;
+const MIN_CLUSTERS = 40;
+
+const mean = (xs) => (xs.length ? xs.reduce((a, b) => a + b, 0) / xs.length : null);
+
+/** Absolute error, the honest default for a quantity a user would read. */
+const absLoss = (pred, actual) => Math.abs(pred - actual);
+/** Squared error, when large misses should dominate. */
+const sqLoss = (pred, actual) => (pred - actual) ** 2;
+
+function makeRnd(seed) {
+  let s = seed >>> 0;
+  return () => { s ^= s << 13; s >>>= 0; s ^= s >>> 17; s ^= s << 5; s >>>= 0; return s / 4294967296; };
+}
+
+/**
+ * @param {Array} rows [{ baseline, prediction, actual, cluster? }]
+ * @param {object} opts { loss, iters, seed, cumulativeTests, minN, minClusters }
+ */
+function adjudicate(rows, opts = {}) {
+  const loss = opts.loss === 'squared' ? sqLoss : absLoss;
+  const usable = (rows || []).filter((r) =>
+    knownNumber(r && r.baseline) !== null
+    && knownNumber(r && r.prediction) !== null
+    && knownNumber(r && r.actual) !== null);
+
+  const n = usable.length;
+  const shifts = usable.map((r) => Math.abs(r.prediction - r.baseline));
+  const movement = {
+    n,
+    mean_abs_shift: n ? round4(mean(shifts)) : null,
+    max_abs_shift: n ? round4(Math.max(...shifts)) : null,
+  };
+
+  const minN = opts.minN ?? MIN_N;
+  const minClusters = opts.minClusters ?? MIN_CLUSTERS;
+  const base = { link: opts.link || null, movement };
+
+  if (n < minN) {
+    return { ...base, verdict: 'PENDING_SAMPLE', reason: `n ${n} < ${minN}`, rows_needed: minN - n };
+  }
+
+  const clustered = usable.some((r) => r.cluster != null);
+  const groups = new Map();
+  if (clustered) {
+    for (const r of usable) {
+      const k = String(r.cluster);
+      if (!groups.has(k)) groups.set(k, []);
+      groups.get(k).push(r);
+    }
+  }
+  const keys = clustered ? [...groups.keys()] : null;
+  if (clustered && keys.length < minClusters) {
+    return {
+      ...base,
+      verdict: 'PENDING_SAMPLE',
+      reason: `${n} rows but only ${keys.length} independent clusters < ${minClusters}`,
+      clusters_needed: minClusters - keys.length,
+    };
+  }
+
+  const lossBase = mean(usable.map((r) => loss(r.baseline, r.actual)));
+  const lossPred = mean(usable.map((r) => loss(r.prediction, r.actual)));
+  const delta = lossPred - lossBase;          // negative = the link is better
+
+  const rnd = makeRnd(opts.seed ?? 20260806);
+  const iters = opts.iters ?? 3000;
+  const diffs = [];
+  for (let it = 0; it < iters; it += 1) {
+    const b = []; const p = [];
+    if (clustered) {
+      for (let i = 0; i < keys.length; i += 1) {
+        for (const r of groups.get(keys[Math.floor(rnd() * keys.length)])) {
+          b.push(loss(r.baseline, r.actual)); p.push(loss(r.prediction, r.actual));
+        }
+      }
+    } else {
+      for (let i = 0; i < n; i += 1) {
+        const r = usable[Math.floor(rnd() * n)];
+        b.push(loss(r.baseline, r.actual)); p.push(loss(r.prediction, r.actual));
+      }
+    }
+    diffs.push(mean(p) - mean(b));
+  }
+  diffs.sort((x, y) => x - y);
+  const tests = Math.max(1, Math.round(knownNumber(opts.cumulativeTests) ?? 1));
+  const alpha = 0.05 / tests;
+  const q = (x) => round4(diffs[Math.floor(Math.min(diffs.length - 1, Math.max(0, x * (diffs.length - 1))))]);
+  const ci = [q(alpha / 2), q(1 - alpha / 2)];
+
+  const improvement = {
+    n,
+    effective_n: clustered ? keys.length : n,
+    cluster_unit: clustered ? 'cluster' : 'row',
+    loss_baseline: round4(lossBase),
+    loss_prediction: round4(lossPred),
+    loss_delta: round4(delta),
+    ci,
+    ci_level: round4(1 - alpha),
+    bonferroni_tests: tests,
+    improves: ci[1] < 0,
+    degrades: ci[0] > 0,
+  };
+  const out = { ...base, improvement };
+
+  if (movement.mean_abs_shift === null || movement.mean_abs_shift < (opts.minMovement ?? 0)) {
+    return { ...out, verdict: 'INERT', reason: 'the link never departs from the naive baseline' };
+  }
+  if (improvement.improves) {
+    return { ...out, verdict: 'PROVES', reason: `beats the naive baseline by ${-improvement.loss_delta} (CI ${JSON.stringify(ci)} at ${improvement.ci_level}, corrected for ${tests} tests)` };
+  }
+  if (delta < 0) {
+    return {
+      ...out,
+      verdict: 'NOT_PROVEN_AT_CORRECTED_BAR',
+      reason: `point estimate improves by ${-improvement.loss_delta} but the corrected interval spans zero (${JSON.stringify(ci)})`,
+      note: 'a real candidate held to a rising bar — not theatre',
+    };
+  }
+  return {
+    ...out,
+    verdict: 'THEATER',
+    reason: `moves ${movement.mean_abs_shift} off the baseline while accuracy does NOT improve (delta ${improvement.loss_delta})`,
+    consequence: 'wiring this would make the projection LOOK like it read the game script while reading nothing',
+  };
+}
+
+const round4 = (v) => (v == null || !Number.isFinite(v) ? null : Math.round(v * 10000) / 10000);
+
+module.exports = { adjudicate, absLoss, sqLoss, MIN_N, MIN_CLUSTERS };