Reliever chain: Link 1 proves, Link 2 does not, and the premise inverts

The causal insight is right -- the game is a sequence and the matchup does shift mid-game. The direction is backwards, measured on 93,663 plate appearances from 1,238 games pulled free from statsapi. LINK 1 PROVES. Starter batters-faced, point-in-time from his own prior starts only, clustered on the pitcher: MAE 3.2226 -> 2.7990, delta -0.4236, CI [-0.6006,-0.2731] at 0.9995 corrected for 107 tests, 1,706 starts across 204 pitchers. It finds the tail the chain needed -- early exits are a 23.2% base rate, model-flagged starts are 34.0% early, lift +10.8pp. Scope correction inside Link 1: the order specifies fatigue x GAME SCRIPT, but game script is not available at grade time -- whether he gets hit tonight is the thing being projected, not an input to it. Only the workload half is measured; the in-game half is recorded as a live feature, out of scope, rather than quietly folded in. LINK 2 DOES NOT PROVE, twice over. Model accuracy 17.2% vs an 8.6% baseline -- doubling it sounds good and is not, since naming a specific arm is wrong five times in six. And structurally the entity is the BULLPEN: 39,629 post-starter plate appearances across 30 clubs is 30 readings, below the 40-cluster floor, the same permanent ceiling as park geometry and team defence. LINK 3 NOT RUN, per the order's own rule. THE PREMISE IS REFUTED, and this chains on nothing so it was safe to measure: vs STARTER n=48,492 hit rate 0.2444 +/-0.0038 vs BULLPEN n=35,760 hit rate 0.2373 +/-0.0044 The pen is 0.7pp HARDER. The specific effect the chain exists to exploit -- early exit making later at-bats softer -- is +0.0010 on 35,760 PAs. A well-powered null, not a sample problem. What IS real is times through the order: TTO1 0.2351 -> TTO2 0.2515 -> TTO3 0.2518. A starter does decay as the lineup sees him again, but that advantage is SURRENDERED when he leaves, not extended -- the pen is harder than his second and third time through. A modern bullpen is a queue of fresh specialists throwing one inning each; there is no tiring arm to punish. So the insight survives inverted, and Link 1 stays valuable for the opposite reason it was built: a likely early hook predicts the hitter LOSES his third-time-through look (0.2518 -> 0.2373 on that PA). The mispricing is on hitters who get an EXTRA look at a starter going deep. BUILT: predictionGate.js + tests -- the two-part gate for a continuous prediction. factorGate binarises outcomes for Brier, which would destroy a target like batters faced. Same discipline, same THEATER verdict, real scale. PRE-REGISTERED NOT RUN: Link 2' using a PA-weighted bullpen AGGREGATE rather than a named arm. Recorded rather than substituted in -- running Link 3 on a swapped-in Link 2 is the assumed-link failure the order forbids. Given the premise result its expected value is now low. PARALLEL TRACK logged: total_bases n=948 pooled, BOMBER x TB 340, short by 160. Sample-readiness only, not a verdict. Counter and frozen clusters byte-identical. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W1sivYNqY2TS5ftykmHBU9
2026-08-06 01:58:38 -04:00
parent 3081c92e00
commit e4dae0e6b0
8 changed files with 819 additions and 1 deletions
@@ -0,0 +1,131 @@
+#!/usr/bin/env node
+'use strict';
+
+/**
+ * LINK 2 — can we say WHICH arm faces the later plate appearances?
+ *
+ * Link 1 proved, so this link is allowed to be attempted at all. It is gated the
+ * same way: predict the reliever who actually threw a given post-starter plate
+ * appearance, against a naive baseline, point-in-time.
+ *
+ *   BASELINE     the team's most-used reliever to date — "guess the busiest arm"
+ *   PREDICTION   the reliever that team has most often used IN THIS INNING to
+ *                date, which is the cheapest expression of bullpen ROLE
+ *
+ * Loss is misclassification: 0 when the named arm actually threw it, 1 otherwise.
+ *
+ * ── THE REPLICATION UNIT IS THE BULLPEN, AND THERE ARE THIRTY ────────────
+ * Bullpen usage is a team-level process — the same manager, the same arms, the
+ * same roles all season — so errors are correlated within team and the entity
+ * this prediction rides on is the club. That caps replication at 30 whatever the
+ * row count, exactly like park geometry and team defence. Reported explicitly
+ * rather than dissolved into a row count of tens of thousands.
+ *
+ *   node scripts/link2-reliever-identity.js
+ */
+
+require('dotenv').config();
+const fs = require('fs');
+const path = require('path');
+const pg = require('../src/services/model/predictionGate');
+const tl = require('../src/services/model/testLedger');
+const { createClient } = require('@supabase/supabase-js');
+
+const CACHE = process.env.SEQ_OUT || path.join(process.cwd(), '.seq-cache', 'sequences.json');
+
+/** Pick the key with the highest count; null when there is nothing to pick from. */
+function argmax(counter) {
+  let best = null; let bestN = -1;
+  for (const [k, v] of counter) if (v > bestN) { best = k; bestN = v; }
+  return bestN > 0 ? best : null;
+}
+
+function build() {
+  const { games } = JSON.parse(fs.readFileSync(CACHE, 'utf8'));
+  games.sort((a, b) => String(a.date).localeCompare(String(b.date)) || a.gamePk - b.gamePk);
+
+  // Point-in-time bullpen histories, accumulated as we walk forward in time.
+  const overall = new Map();          // team -> Map(pitcherId -> appearances)
+  const byInning = new Map();         // `team|inning` -> Map(pitcherId -> appearances)
+  const rows = [];
+
+  for (const g of games) {
+    for (const side of ['home', 'away']) {
+      const team = g[side].abbr || g[side].team;
+      if (!team) continue;
+      const starter = (g[side].arms || []).find((a) => a.started);
+      if (!starter) continue;
+      const relievers = new Set((g[side].arms || []).filter((a) => !a.started).map((a) => a.id));
+      if (!relievers.size) continue;
+
+      // This side PITCHES in the opposite half-inning.
+      const half = side === 'home' ? 'top' : 'bottom';
+      const post = g.pas.filter((p) => p.half === half && p.pitcher !== starter.id);
+
+      for (const pa of post) {
+        const ov = overall.get(team);
+        const inn = byInning.get(`${team}|${pa.inning}`);
+        const basePick = ov ? argmax(ov) : null;
+        const modelPick = inn ? argmax(inn) : basePick;
+        // No history yet is honestly unreadable, not a wrong guess.
+        if (basePick === null || modelPick === null) continue;
+        rows.push({
+          cluster: team,
+          baseline: Number(basePick) === pa.pitcher ? 1 : 0,
+          prediction: Number(modelPick) === pa.pitcher ? 1 : 0,
+          actual: 1,
+          inning: pa.inning,
+        });
+      }
+
+      // Now fold this game into history — never before predicting from it.
+      if (!overall.has(team)) overall.set(team, new Map());
+      const ovm = overall.get(team);
+      for (const r of relievers) ovm.set(String(r), (ovm.get(String(r)) || 0) + 1);
+      for (const pa of post) {
+        const k = `${team}|${pa.inning}`;
+        if (!byInning.has(k)) byInning.set(k, new Map());
+        const m = byInning.get(k);
+        m.set(String(pa.pitcher), (m.get(String(pa.pitcher)) || 0) + 1);
+      }
+    }
+  }
+  return rows;
+}
+
+(async () => {
+  const rows = build();
+
+  let cumulative = 1;
+  try {
+    const sb = createClient(process.env.SUPABASE_URL,
+      process.env.SUPABASE_SERVICE_ROLE_KEY || process.env.SUPABASE_SERVICE_KEY,
+      { auth: { persistSession: false } });
+    const mc = await tl.recordAndCount(tl.supabaseStore(sb), [
+      { sport: 'mlb', stat: 'reliever_identity', archetype: null, interaction: 'link2:reliever_identity', target: 'actual_arm' },
+    ]);
+    cumulative = mc.cumulative_tests;
+  } catch { /* offline */ }
+
+  const verdict = pg.adjudicate(rows, {
+    link: 'link2_reliever_identity',
+    loss: 'absolute',
+    cumulativeTests: cumulative,
+  });
+
+  const acc = (k) => (rows.length ? rows.filter((r) => r[k] === 1).length / rows.length : null);
+
+  console.log(JSON.stringify({
+    link: 'LINK 2 — reliever identity',
+    post_starter_plate_appearances: rows.length,
+    distinct_bullpens: new Set(rows.map((r) => r.cluster)).size,
+    cumulative_tests: cumulative,
+    baseline_accuracy: round4(acc('baseline')),
+    model_accuracy: round4(acc('prediction')),
+    verdict,
+    structural_note: 'the entity this prediction rides on is the BULLPEN, and there are 30 — row count cannot create replication that does not exist',
+  }, null, 2));
+  process.exit(0);
+})();
+
+const round4 = (v) => (v == null ? null : Math.round(v * 10000) / 10000);