Audit the LODO instrument: it cannot evaluate any stat, and both prior

FAILs were false

PHASE 0 — the gate at 1f40014 was mine and was an incoherent pair. A 1-SE
informativeness bar with a ZERO-reversal rule: at exactly 1 SE a stable
stat's drop reverses with prob Phi(-1)=0.1587, so on four informative
drops P(>=1 reversal | perfectly stable) = 1 - 0.8413^4 = 0.50. It failed
stable stats half the time by construction. And the pooled n*=70
mis-credited EVERY stat -- too low for hits (own 77) and runs (81), too
high for total_bases (60) and rbi (54).

PHASE 1, blind. Per-stat (g, sigma_row): hits -0.01288/0.11251, TB
-0.01380/0.10680, rbi -0.00884/0.06459, runs -0.00902/0.08080. All four
clear z=1.96 at full n, so none is NO-EFFECT. Committed k=1 with per-stat
n* and a binomial cutoff holding FP at 0.004-0.031.

THE FINDING THAT DOMINATES: the test has no power. Against a strong
instability (date-to-date SD equal to the effect) it detects a failure
1.4%-9.3% of the time, and across every k from 1.0 to 2.0 the best any
stat reaches is 0.337. A gate that cannot fail cannot pass, so
LODO_POWER_FLOOR=0.50 makes UNTESTABLE structural -- "could not test" can
never read as "passed".

PHASE 2/3 cold, at each stat's OWN n*:

  hits  5 informative, 0 reversals, cutoff 2, power 0.093  UNTESTABLE
  TB    5 informative, 0 reversals, cutoff 2, power 0.093  UNTESTABLE
  rbi   4 informative, 1 reversal,  cutoff 2, power 0.045  UNTESTABLE
  runs  3 informative, 2 reversals, cutoff 2, power 0.014  UNTESTABLE

Setting the power floor aside entirely, NOT ONE STAT EXCEEDS ITS CUTOFF.

PHASE 4 — rbi's FAIL was false, as the order suspected. So was RUNS' --
which the order did not anticipate, having classified it DATE-DRIVEN on a
244-row reversal; two reversals in three drops does not clear a cutoff of
2. TB's PASS was vacuous: the test could not have failed it. hits' own n*
is LARGER than the pooled one (77 vs 70), and it remains untestable.

PHASE 5 — deploy basis is now the date-clustered CI alone:

  hits  CI [-0.0139,-0.0097], 4 date clusters   relabelled ci_only
  TB    CI [-0.0061,-0.0045], 2 date clusters   RELABELLED, kept
  rbi   CI [-0.0092,-0.0010], 2 date clusters   NEWLY DEPLOYED
  runs  no fittable map at its split            REFUSE, no CI either

Every deployed stat carries calibration_basis ci_only_lodo_untestable and
auto-demotion is the SOLE stability guard, not a backstop to a passed
test. Stated plainly: those intervals rest on 2-4 date clusters, which is
thin, and it is now the only support. rbi gains chainAcross stackability;
its bands rebuilt on p_win_calibrated (425 rows) are every-archetype
base_rate. runs is queued for the low-param calibrator for the ordinary
reason -- no fittable map -- not on the date-driven finding, which was an
artefact.

PHASE 6 — the deploy set was set by a coin-flip-power ruler; it is now set
by a per-stat power-coherent pre-committed test whose first act was to
report that it cannot evaluate anything. The audit was permitted to wound
the live deploy and did: total_bases lost its LODO claim. Standing
question unchanged -- 18 archetype slots across three deployed stats, every
one a single band indistinguishable from base rate.

Blind ordering held. p_win never mutated. No Bonferroni slot. Counter and
frozen clusters verified file-by-file.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W1sivYNqY2TS5ftykmHBU9

scripts/derive-lodo-test.js

@@ -0,0 +1,198 @@
+#!/usr/bin/env node
+'use strict';
+
+/**
+ * PHASE 1 — derive a COHERENT LODO test, blind to reversals.
+ *
+ * ── THE DEFECT BEING FIXED ───────────────────────────────────────────────
+ * The gate at 1f40014 paired a 1-SE per-drop informativeness bar with a
+ * zero-reversal decision rule. Those two are incoherent. At exactly 1 SE, a
+ * genuinely STABLE stat's drop reverses with probability Phi(-1) = 0.159, so on
+ * four informative drops the chance of at least one reversal is
+ * 1 - 0.841^4 = 0.50. The rule failed stable stats half the time by construction.
+ *
+ * And n* was pooled across four stats whose signed effects differ several-fold,
+ * so "informative" meant different things for different stats while being
+ * treated as one number.
+ *
+ * ── THE FIX ──────────────────────────────────────────────────────────────
+ * The two halves have to be chosen together:
+ *
+ *   informative bar   n*_k = k^2 * (sigma_row / |g|)^2      PER STAT
+ *   decision rule     FAIL iff reversals > c, where under stability
+ *                     R ~ Binomial(D, Phi(-k)) and c is the smallest cutoff
+ *                     with P(R > c) <= 0.05
+ *
+ * `g` is the mean SIGNED per-row improvement — the quantity whose sign a
+ * reversal flips. Not a mean-absolute, and not pooled: a reversal is a claim
+ * about THIS stat's effect changing sign.
+ *
+ * THIS SCRIPT PRINTS NO REVERSAL AND NO VERDICT. It is blind by construction and
+ * must run, and its constants be committed, before any stat is re-read.
+ *
+ *   SUPABASE_URL=... node scripts/derive-lodo-test.js
+ */
+
+require('dotenv').config();
+const fs = require('fs');
+const path = require('path');
+const { createClient } = require('@supabase/supabase-js');
+const cal = require('../src/services/model/calibration');
+const guards = require('../src/services/model/calibrationGuards');
+const { knownNumber } = require('../src/utils/known');
+
+const BOX = path.join(process.cwd(), '.seq-cache', 'batting-lines.json');
+const STATS = ['hits', 'total_bases', 'rbi', 'runs'];
+const PAGE = 1000;
+/** |g| must clear this many SE at the stat's full n or there is no effect to test. */
+const EFFECT_Z = 1.96;
+/** Target false-positive rate for the whole per-stat test. */
+const TARGET_FP = 0.05;
+
+const FIELD = { hits: (b) => b.hits, total_bases: (b) => b.totalBases, rbi: (b) => b.rbi, runs: (b) => b.runs };
+const mean = (xs) => (xs.length ? xs.reduce((a, b) => a + b, 0) / xs.length : null);
+
+/** Standard normal CDF (Abramowitz–Stegun 7.1.26 via erf). */
+function normCdf(z) {
+  const t = 1 / (1 + 0.2316419 * Math.abs(z));
+  const d = 0.3989422804014327 * Math.exp(-z * z / 2);
+  const p = d * t * (0.319381530 + t * (-0.356563782 + t * (1.781477937 + t * (-1.821255978 + t * 1.330274429))));
+  return z >= 0 ? 1 - p : p;
+}
+
+const binomPmf = (n, k, p) => {
+  let logC = 0;
+  for (let i = 0; i < k; i += 1) logC += Math.log(n - i) - Math.log(i + 1);
+  return Math.exp(logC + k * Math.log(p) + (n - k) * Math.log(1 - p));
+};
+/** P(R > c) for R ~ Binomial(n, p). */
+const binomTail = (n, c, p) => {
+  let s = 0;
+  for (let k = c + 1; k <= n; k += 1) s += binomPmf(n, k, p);
+  return s;
+};
+/** Smallest cutoff c with P(R > c) <= target. */
+function cutoffFor(D, p, target) {
+  for (let c = 0; c <= D; c += 1) if (binomTail(D, c, p) <= target) return { cutoff: c, fp: binomTail(D, c, p) };
+  return { cutoff: D, fp: 0 };
+}
+
+async function page(sb, t, s, f) {
+  const o = [];
+  for (let i = 0; ; i += PAGE) {
+    const { data, error } = await f(sb.from(t).select(s)).order('id', { ascending: true }).range(i, i + PAGE - 1);
+    if (error) throw error;
+    if (!data.length) break;
+    o.push(...data);
+    if (data.length < PAGE) break;
+  }
+  return o;
+}
+const isPreGame = (c, g) => {
+  const et = new Date(new Date(c).getTime() - 4 * 3600 * 1000);
+  const d = et.toISOString().slice(0, 10);
+  return d < g || (d === g && et.getUTCHours() < 19);
+};
+
+(async () => {
+  const sb = createClient(process.env.SUPABASE_URL,
+    process.env.SUPABASE_SERVICE_ROLE_KEY || process.env.SUPABASE_SERVICE_KEY, { auth: { persistSession: false } });
+  const lines = JSON.parse(fs.readFileSync(BOX, 'utf8')).lines;
+
+  const snaps = await page(sb, 'model_snapshots',
+    'id, game_date, captured_at, stat, player_key, line, side, p_win, refused',
+    (q) => q.eq('sport', 'mlb').in('stat', STATS));
+
+  const picked = new Map();
+  for (const r of snaps) {
+    if (!isPreGame(r.captured_at, r.game_date) || r.refused || knownNumber(r.p_win) === null) continue;
+    const k = [r.game_date, r.stat, r.player_key, r.line].join('|');
+    const prev = picked.get(k);
+    if (!prev || knownNumber(r.p_win) > knownNumber(prev.p_win)) picked.set(k, r);
+  }
+  guards.assertPickedSideDedup([...picked.values()].map((r) => ({
+    propKey: [r.game_date, r.stat, r.player_key, r.line].join('|'), side: r.side, p: knownNumber(r.p_win),
+  })));
+
+  const perStat = {};
+  const dateSizes = {};
+
+  for (const stat of STATS) {
+    const rows = [];
+    for (const r of picked.values()) {
+      if (r.stat !== stat) continue;
+      const b = lines[`${r.game_date}|${r.player_key}`];
+      const L = knownNumber(r.line);
+      if (!b || L === null || !r.side) continue;
+      const v = knownNumber(FIELD[stat](b));
+      if (v === null) continue;
+      const over = v > L;
+      rows.push({ date: r.game_date, p: knownNumber(r.p_win), won: (String(r.side).toLowerCase() === 'under' ? !over : over) ? 1 : 0 });
+    }
+    const map = cal.fitIsotonic(rows.map((r) => ({ p: r.p, won: r.won })));
+    if (!map) { perStat[stat] = { n: rows.length, fittable: false }; continue; }
+
+    const d = [];
+    for (const r of rows) {
+      const pc = cal.applyIsotonic(map, r.p);
+      if (knownNumber(pc) === null) continue;
+      d.push((pc - r.won) ** 2 - (r.p - r.won) ** 2);
+    }
+    const g = mean(d);
+    const sigma = Math.sqrt(d.reduce((s, x) => s + (x - g) ** 2, 0) / (d.length - 1));
+    const seFull = sigma / Math.sqrt(d.length);
+
+    // Date sizes are sample STRUCTURE, not outcomes — safe to read here.
+    const sizes = new Map();
+    for (const r of rows) sizes.set(r.date, (sizes.get(r.date) || 0) + 1);
+    dateSizes[stat] = [...sizes.values()].sort((a, b) => b - a);
+
+    perStat[stat] = {
+      n: d.length,
+      g_signed: round5(g),
+      sigma_row: round5(sigma),
+      se_full: round5(seFull),
+      effect_z_at_full_n: round3(Math.abs(g) / seFull),
+      improves: g < 0,
+      no_effect: Math.abs(g) / seFull < EFFECT_Z,
+      fittable: true,
+    };
+  }
+
+  // ── Choose k jointly. Blind: uses only (g, sigma) and date SIZES. ──
+  const kTable = [];
+  for (const k of [1.0, 1.25, 1.5, 1.75, 2.0]) {
+    const pNoise = normCdf(-k);
+    const row = { k, per_drop_noise_prob: round4(pNoise), stats: {} };
+    for (const stat of STATS) {
+      const ps = perStat[stat];
+      if (!ps || !ps.fittable) continue;
+      const nStar = Math.ceil(k * k * (ps.sigma_row / Math.abs(ps.g_signed)) ** 2);
+      const D = (dateSizes[stat] || []).filter((n) => n >= nStar).length;
+      const { cutoff, fp } = D > 0 ? cutoffFor(D, pNoise, TARGET_FP) : { cutoff: null, fp: null };
+      // FN at a stated alternative: date-to-date SD of the effect equals |g|.
+      const pAlt = D > 0 ? normCdf(-Math.abs(ps.g_signed) / Math.sqrt(ps.g_signed ** 2 + (ps.sigma_row ** 2) / nStar)) : null;
+      const fn = D > 0 && cutoff !== null ? 1 - binomTail(D, cutoff, pAlt) : null;
+      row.stats[stat] = {
+        n_star: nStar, informative_drops: D, cutoff, fp: fp === null ? null : round4(fp),
+        fn_at_tau_equals_g: fn === null ? null : round4(fn),
+      };
+    }
+    kTable.push(row);
+  }
+
+  console.log(JSON.stringify({
+    phase: 'PHASE 1 — coherent LODO test derivation, BLIND',
+    defect_being_fixed: 'a 1-SE informative bar with a zero-reversal rule: P(>=1 reversal | stable, 4 drops) = 0.50',
+    per_stat_effect: perStat,
+    date_sizes: dateSizes,
+    k_selection_table: kTable,
+    target_fp: TARGET_FP,
+    blind: 'no reversal, no verdict, no reversing date referenced anywhere in this output',
+  }, null, 2));
+  process.exit(0);
+})().catch((e) => { console.error(e); process.exit(1); });
+
+const round5 = (v) => (v == null || !Number.isFinite(v) ? null : Math.round(v * 100000) / 100000);
+const round4 = (v) => (v == null || !Number.isFinite(v) ? null : Math.round(v * 10000) / 10000);
+const round3 = (v) => (v == null || !Number.isFinite(v) ? null : Math.round(v * 1000) / 1000);

scripts/lodo-calibration.js

@@ -45,7 +45,7 @@ const FAVOURITE_FLOOR = 0.9;
  * from here -- it is imported so the value that decides the verdicts cannot be
  * edited alongside them.
  */
-const { LODO_MIN_HELD_ROWS: MIN_HELD_ROWS } = require('../src/services/model/calibrationRegistry');
+const { LODO_TEST, LODO_POWER_FLOOR } = require('../src/services/model/calibrationRegistry');
 
 const FIELD = { hits: (b) => b.hits, total_bases: (b) => b.totalBases, rbi: (b) => b.rbi, runs: (b) => b.runs };
 const mean = (xs) => (xs.length ? xs.reduce((a, b) => a + b, 0) / xs.length : null);
@@ -131,7 +131,9 @@ async function main() {
       continue;
     }
 
-    // ── LEAVE ONE DATE OUT ──
+    // ── LEAVE ONE DATE OUT, at THIS stat's own informative bar ──
+    const spec = LODO_TEST[stat];
+    const MIN_HELD_ROWS = spec ? spec.n_star : Infinity;
     const table = [];
     for (const d of dates) {
       const fit = rows.filter((r) => r.date !== d);
@@ -168,10 +170,19 @@ async function main() {
     }
 
     const informative = table.filter((t) => t.verdict !== 'UNINFORMATIVE');
-    const anyReversal = informative.some((t) => t.verdict === 'REVERSES');
-    const signTested = informative.filter((t) => t.favourite_sign_holds !== null);
-    const anySignFlip = signTested.some((t) => t.favourite_sign_holds === false);
-    const passes = informative.length > 0 && !anyReversal && !anySignFlip;
+    const reversals = informative.filter((t) => t.verdict === 'REVERSES');
+
+    // THE DECISION RULE IS BINOMIAL, not zero-tolerance. Under stability each
+    // informative drop reverses with prob Phi(-k), so demanding zero reversals
+    // failed stable stats roughly half the time.
+    const cutoff = spec ? spec.cutoff : 0;
+    const exceedsCutoff = reversals.length > cutoff;
+
+    // AND THE TEST MUST BE ABLE TO FAIL. Below the power floor it cannot, so it
+    // cannot pass either -- "could not test" must never read as "passed".
+    const underpowered = !spec || spec.power < LODO_POWER_FLOOR;
+    const verdict = underpowered ? 'UNTESTABLE_BY_LODO' : (exceedsCutoff ? 'FAIL' : 'PASS');
+    const passes = verdict === 'PASS';
 
     out.per_stat[stat] = {
       n: rows.length,
@@ -179,14 +190,19 @@ async function main() {
       lodo_table: table,
       informative_drops: informative.length,
       brier_reversals: informative.filter((t) => t.verdict === 'REVERSES').length,
-      favourite_sign_flips: signTested.filter((t) => t.favourite_sign_holds === false).length,
-      favourite_sign_untested: informative.length - signTested.length,
-      lodo: passes ? 'PASS' : 'FAIL',
-      reason: passes
-        ? 'improvement never reverses and the favourite over-prediction never flips sign across any single-date drop'
-        : (anyReversal
-          ? `improvement reverses when ${informative.filter((t) => t.verdict === 'REVERSES').map((t) => t.dropped).join(', ')} is dropped — the effect is date-driven`
-          : `the favourite over-prediction flips sign when ${signTested.filter((t) => t.favourite_sign_holds === false).map((t) => t.dropped).join(', ')} is dropped`),
+      favourite_sign_flips: informative.filter((t) => t.favourite_sign_holds === false).length,
+      favourite_sign_untested: informative.filter((t) => t.favourite_sign_holds === null).length,
+      n_star: spec ? spec.n_star : null,
+      cutoff,
+      reversal_count: reversals.length,
+      reversing_dates: reversals.map((t) => ({ date: t.dropped, held_n: t.held_n, delta: t.brier_delta })),
+      test_power: spec ? spec.power : null,
+      lodo: verdict,
+      reason: underpowered
+        ? `power ${spec ? spec.power : 0} < floor ${LODO_POWER_FLOOR} — this test would miss a real date-driven failure more than nine times in ten, so it can neither pass nor fail the stat`
+        : (exceedsCutoff
+          ? `${reversals.length} reversals among ${informative.length} informative drops exceeds the cutoff of ${cutoff}`
+          : `${reversals.length} reversals among ${informative.length} informative drops is within the cutoff of ${cutoff}`),
     };
   }
 

specs/lodo-instrument-audit.md

@@ -0,0 +1,170 @@
+# Auditing the LODO instrument — it cannot evaluate any stat
+
+**All four stats are UNTESTABLE-BY-LODO. Not one exceeds its reversal cutoff, so
+rbi's and runs' prior FAILs were both false. And no stat may claim LODO
+stability, because at this date count the test cannot fail.**
+
+---
+
+## PHASE 0 — the defect, on record
+
+The gate at `1f40014` was mine, and it was an incoherent pair:
+
+**1. A 1-SE informativeness bar coupled to a ZERO-reversal rule.** At exactly 1
+SE, a genuinely STABLE stat's drop reverses with probability Φ(−1) = 0.1587. On
+four informative drops:
+
+```
+P(>= 1 reversal | perfectly stable) = 1 - 0.8413^4 = 0.50
+```
+
+**The rule failed stable stats half the time by construction.** A test cannot
+have a 1-SE noise floor and a zero-tolerance decision rule; the two have to be
+chosen together.
+
+**2. A pooled n\*.** The four stats' signed effects differ several-fold, so one
+threshold meant four different things. Measured, the pooled 70 was:
+
+| stat | own n\* | pooled 70 was |
+|---|---|---|
+| hits | 77 | **too low** |
+| total_bases | 60 | too high |
+| rbi | 54 | too high |
+| runs | 81 | **too low** |
+
+It mis-credited **every** stat, in both directions.
+
+Neither defect touched the counter or `p_win`. Both touched only which
+calibrations were judged stable.
+
+---
+
+## PHASE 1 — the coherent test, derived blind
+
+### (a)(b)(c) per-stat effect
+
+| stat | n | g (signed) | σ_row | SE_full | effect z | NO-EFFECT? |
+|---|---|---|---|---|---|---|
+| hits | 1,140 | −0.01288 | 0.11251 | 0.00333 | 3.87 | no |
+| total_bases | 1,050 | −0.01380 | 0.10680 | 0.00330 | 4.19 | no |
+| rbi | 630 | −0.00884 | 0.06459 | 0.00257 | 3.44 | no |
+| runs | 597 | −0.00902 | 0.08080 | 0.00331 | 2.73 | no |
+
+All four have a real effect at full n. None is NO-EFFECT — there is something for
+stability to be tested *of* in every case.
+
+### The committed test pair
+
+`k = 1`, chosen because informative drops (D) are the binding scarcity here and
+k=1 maximises them while the binomial cutoff holds the false-positive rate.
+
+| stat | n\*=k²(σ/\|g\|)² | informative drops D | cutoff | FP | **power at τ=\|g\|** |
+|---|---|---|---|---|---|
+| hits | 77 | 5 | 2 | 0.031 | **0.093** |
+| total_bases | 60 | 5 | 2 | 0.031 | **0.093** |
+| rbi | 54 | 4 | 2 | 0.014 | **0.045** |
+| runs | 81 | 3 | 2 | 0.004 | **0.014** |
+
+Per-drop noise probability under stability Φ(−1) = 0.1587. FAIL iff reversals > cutoff.
+
+### The finding that dominates everything else: the test has no power
+
+Against a **strong** instability — date-to-date SD of the effect equal to the
+effect itself — this test detects a failure between **1.4% and 9.3%** of the
+time. Across every k examined (1.0 → 2.0), the best any stat reaches is 0.337,
+and reaching even that costs all but two informative drops.
+
+**A gate that cannot fail cannot pass.** `LODO_POWER_FLOOR = 0.50` makes that
+structural: below it a stat is UNTESTABLE-BY-LODO regardless of its reversal
+count, so "could not test" can never be read as "passed".
+
+Committed as `LODO_K`, `LODO_TEST`, `LODO_POWER_FLOOR`; a test recomputes each
+n\* from (σ, g) and each cutoff from the binomial tail, and asserts the old
+zero-reversal rule's ~0.50 false-fail rate. The stale pooled constant is nulled
+so nothing can read it.
+
+---
+
+## PHASE 2/3 — cold re-read at each stat's own n\*
+
+| stat | own n\* | informative | reversals | cutoff | power | verdict |
+|---|---|---|---|---|---|---|
+| hits | 77 | 5 | 0 | 2 | 0.093 | **UNTESTABLE** |
+| total_bases | 60 | 5 | 0 | 2 | 0.093 | **UNTESTABLE** |
+| rbi | 54 | 4 | 1 | 2 | 0.045 | **UNTESTABLE** |
+| runs | 81 | 3 | 2 | 2 | 0.014 | **UNTESTABLE** |
+
+**Setting the power floor aside entirely, not one stat exceeds its cutoff.**
+
+---
+
+## PHASE 4 — reconcile against 1f40014
+
+| stat | 1f40014 | now | why it changed |
+|---|---|---|---|
+| hits | PASS | UNTESTABLE | the PASS was from a test that cannot fail; 0 reversals is uninformative at 9% power |
+| total_bases | PASS | UNTESTABLE | same — and it "passed" at a pooled n\*=70 above its own 60, so its drop count was under-credited too |
+| **rbi** | **FAIL** | UNTESTABLE (1 reversal, cutoff 2) | **FALSE FAIL.** One reversal on four drops is a ~16%-per-drop coin flip, not evidence |
+| **runs** | **FAIL** | UNTESTABLE (2 reversals, cutoff 2) | **ALSO A FALSE FAIL** under the coherent rule — this was not anticipated |
+
+Answering the order's three questions directly:
+
+- **Is rbi's FAIL a false fail?** Yes. And so is runs' — which the order did not
+  anticipate, having classified runs as DATE-DRIVEN on the strength of a 244-row
+  reversal. Under a rule with a stated error rate, two reversals in three drops
+  does not clear the cutoff.
+- **Was TB's PASS real?** No. It was vacuous: the test could not have failed it.
+- **Does hits still pass at its own smaller n\*?** Its own n\* is *larger* (77 vs
+  the pooled 70), it still shows zero reversals, and it is still untestable.
+
+---
+
+## PHASE 5 — deploy, withdraw, route
+
+| stat | basis | decision |
+|---|---|---|
+| hits | date-clustered CI [−0.0139, −0.0097], **4 date clusters** | DEPLOY-PROVISIONAL, relabelled `ci_only_lodo_untestable` |
+| total_bases | CI [−0.0061, −0.0045], **2 date clusters** | **RELABELLED** — kept, no longer claims LODO stability |
+| **rbi** | CI [−0.0092, −0.0010], **2 date clusters** | **NEWLY DEPLOYED** — its FAIL was false |
+| runs | **no fittable map** at its point-in-time split | REFUSE — no CI to stand on either |
+
+Every deployed stat now carries `calibration_basis: 'ci_only_lodo_untestable'`.
+**Auto-demotion is the sole stability guard**, not a backstop to a passed test.
+
+The honest weakness, stated rather than buried: those intervals rest on **2 to 4
+date clusters**. That is thin support, and it is now the *only* support.
+
+**rbi chainAcross stackability is newly granted**; hits' remains from `1f40014`.
+rbi bands rebuilt on `p_win_calibrated` (425 eval rows): every archetype
+indistinguishable from its base rate, two-bar rule keeping them `base_rate`.
+
+runs is **not** routed to the low-parameter calibrator on a date-driven finding —
+that finding was an artefact. It is queued for the ordinary reason: no isotonic
+map is fittable at its sample.
+
+---
+
+## PHASE 6 — logged
+
+**The deploy set at `1f40014` was set by a coin-flip-power ruler.** It is now set
+by a per-stat, power-coherent, pre-committed test with a stated false-positive
+rate — and that test's first act was to report that it cannot evaluate anything,
+which is a more useful answer than either verdict it replaced.
+
+**The audit was permitted to wound the live deploy, and did**: total_bases lost
+its LODO claim and now stands on a two-date-cluster interval. That it could is
+the integrity property.
+
+**Standing question unchanged.** Calibrated `p_win` still separates within
+archetype no better than raw — 18 archetype slots across three deployed stats,
+every one a single band indistinguishable from its base rate. Per-archetype
+separation comes from proven factors or it does not exist.
+
+---
+
+## Invariants
+
+Blind ordering held: (g, σ_row) and the committed (n\*, cutoff, power floor) were
+derived and locked with no reversal or verdict in view, before any stat was
+re-read. `p_win` never mutated. No Bonferroni slot consumed. Counter and frozen
+clusters verified file-by-file.

src/services/model/calibrationRegistry.js

@@ -29,41 +29,57 @@ const { knownNumber } = require('../../utils/known');
 const STATUS = Object.freeze({ NONE: 'none', PROVISIONAL: 'provisional', PROMOTED: 'promoted' });
 
 /**
- * LODO_MIN_HELD_ROWS — POWER-DERIVED, PRE-COMMITTED, NOT OPERATOR-CHOSEN.
+ * THE LODO TEST — a COHERENT pair, replacing the incoherent one at 1f40014.
  *
- * A leave-one-date-out reversal is only informative if that date's held-out
- * Brier delta is distinguishable from zero at its row count. Below that, a
- * "reversal" is a coin flip wearing a decimal point — which is what made the
- * previous verdict depend on a number someone picked.
+ * ── WHAT WAS WRONG ───────────────────────────────────────────────────────
+ * The previous gate paired a 1-SE per-drop informativeness bar with a
+ * ZERO-reversal decision rule. At exactly 1 SE a genuinely STABLE stat's drop
+ * reverses with probability Phi(-1) = 0.159, so on four informative drops the
+ * chance of at least one reversal is 1 - 0.841^4 = 0.50. The rule failed stable
+ * stats half the time BY CONSTRUCTION. And n* was pooled across four stats whose
+ * signed effects differ several-fold, so one number meant four different things:
+ * measured, the pooled 70 was too LOW for hits (77) and runs (81) and too HIGH
+ * for total_bases (60) and rbi (54).
  *
- * The per-row Brier difference is d_i = (pc_i - y_i)^2 - (p_i - y_i)^2, so a
- * date's delta is mean(d) and SE(n) = SD(d)/sqrt(n). The smallest n at which a
- * typical effect clears one standard error is n* = (SD(d)/|effect|)^2.
+ * ── THE COHERENT PAIR ────────────────────────────────────────────────────
+ * The bar and the rule are chosen TOGETHER, per stat, for a stated error rate:
  *
- * Measured 2026-08-07, pooled across all four stats so that no single stat's
- * verdict could shape the threshold that decides it:
+ *   informative bar   n*_k = k^2 * (sigma_row / |g|)^2      per stat
+ *   decision rule     FAIL iff reversals > cutoff, where under stability
+ *                     R ~ Binomial(D, Phi(-k)) and cutoff is the smallest c
+ *                     with P(R > c) <= 0.05
  *
- *     pooled rows          3,417
- *     SD(d)                0.09816
- *     |effect|             0.01175
- *     n* = (0.09816/0.01175)^2 = 69.8  ->  70
+ * `g` is the mean SIGNED per-row improvement — the quantity whose sign a
+ * reversal flips. k = 1 is chosen because it maximises informative drops (D),
+ * which is the binding scarcity here, while the binomial cutoff holds the
+ * false-positive rate at 0.004-0.031 across the four stats.
  *
- * SE-vs-n: n=20 gives effect/SE 0.54, n=50 gives 0.85, n=75 gives 1.04. So
- * anything under ~70 held rows cannot tell a real reversal from noise.
+ * ── AND THE TEST STILL HAS ALMOST NO POWER ───────────────────────────────
+ * At the stated alternative (date-to-date SD of the effect equal to |g| — a
+ * strong instability), power is 0.093 / 0.093 / 0.045 / 0.014. The test would
+ * MISS a real date-driven failure more than nine times in ten. Across every k
+ * examined, the best any stat reaches is 0.337.
  *
- * DERIVED BLIND — the derivation script prints no stat verdict, no date and no
- * reversal. It ran, and this constant was committed, before any stat was
- * re-read. That ordering is the integrity property; a test locks the value so it
- * cannot be silently tuned afterwards.
+ * So a PASS here means "no instability was detected", NOT "it is stable", and a
+ * gate that cannot fail is not a gate. LODO_POWER_FLOOR makes that structural: a
+ * stat whose test power falls below it is UNTESTABLE-BY-LODO and may not claim
+ * LODO stability at all, whatever its reversal count.
+ *
+ * Derived BLIND — the derivation script prints no reversal, no verdict and no
+ * reversing date. It ran, and these were committed, before any stat was re-read.
  */
-const LODO_MIN_HELD_ROWS = 70;
-const LODO_THRESHOLD_BASIS = Object.freeze({
-  pooled_rows: 3417,
-  per_row_brier_diff_sd: 0.09816,
-  pooled_effect_abs_mean: 0.01175,
-  rule: 'n* = (SD(d) / |effect|)^2',
-  derived_blind: true,
+const LODO_K = 1.0;
+/** Below this power the test cannot fail, so it cannot pass either. */
+const LODO_POWER_FLOOR = 0.50;
+/** Per-stat, from (g, sigma_row) measured blind. */
+const LODO_TEST = Object.freeze({
+  hits:        { g: -0.01288, sigma_row: 0.11251, n_star: 77, informative_drops: 5, cutoff: 2, fp: 0.0310, power: 0.093 },
+  total_bases: { g: -0.01380, sigma_row: 0.10680, n_star: 60, informative_drops: 5, cutoff: 2, fp: 0.0310, power: 0.093 },
+  rbi:         { g: -0.00884, sigma_row: 0.06459, n_star: 54, informative_drops: 4, cutoff: 2, fp: 0.0141, power: 0.045 },
+  runs:        { g: -0.00902, sigma_row: 0.08080, n_star: 81, informative_drops: 3, cutoff: 2, fp: 0.0040, power: 0.014 },
 });
+/** Legacy name kept so nothing silently reads a stale pooled value. */
+const LODO_MIN_HELD_ROWS = null;
 /** The ORIGINAL floor, correctly scoped: promotion, not deploy. */
 const PROMOTION_DATE_CLUSTERS = 40;
 
@@ -144,4 +160,4 @@ function createRegistry(initial = {}) {
   return { deploy, reverify, serves, get, all, log: () => log.slice() };
 }
 
-module.exports = { createRegistry, STATUS, PROMOTION_DATE_CLUSTERS, LODO_MIN_HELD_ROWS, LODO_THRESHOLD_BASIS };
+module.exports = { createRegistry, STATUS, PROMOTION_DATE_CLUSTERS, LODO_K, LODO_TEST, LODO_POWER_FLOOR, LODO_MIN_HELD_ROWS };

src/services/snapshotService.js

@@ -276,27 +276,37 @@ async function loadPitcherArsenals(sport) {
 }
 
 /**
- * Stats whose calibration passed leave-one-date-out and may serve a calibrated
- * number. PROVISIONAL: auto-demoted the first time the held-out interval stops
- * excluding zero or the favourite over-prediction flips sign.
+ * Stats that may serve a calibrated number, and ON WHAT BASIS.
  *
- * Gated at LODO_MIN_HELD_ROWS = 70, which is POWER-DERIVED and pre-committed:
- * below ~70 held rows a date's Brier delta cannot be told from a coin flip, so a
- * "reversal" there carries no information.
+ * ── LODO CANNOT EVALUATE ANY OF THEM ─────────────────────────────────────
+ * The LODO gate was audited and rebuilt as a coherent pair (per-stat
+ * informativeness bar + binomial reversal cutoff at FP <= 0.05). At this date
+ * count its POWER against a strong date-driven instability is 0.093 / 0.093 /
+ * 0.045 / 0.014 — it would miss a real failure more than nine times in ten. A
+ * gate that cannot fail cannot pass, so every stat is UNTESTABLE-BY-LODO and
+ * none of these deploys claim LODO stability.
  *
- * hits was WITHDRAWN at 6ae11f1 and is RESTORED here. That is not a reversal of
- * the earlier call — it was correct on the instrument available then, which
- * admitted 20- and 25-row dates as evidence. With the threshold derived from
- * power rather than chosen, hits reverses on nothing. The restoration came
- * through the gate, not around it.
+ * The previous zero-reversal rule was incoherent: at a 1-SE bar a stable stat
+ * reverses on ~16% of drops, so demanding zero failed stable stats ~50% of the
+ * time. Re-read under the binomial cutoff, NEITHER rbi (1 reversal) NOR runs
+ * (2) exceeds its cutoff of 2 — both prior FAILs were false.
  *
- * rbi and runs remain ABSENT, and their failures are NOT underpowered: each
- * reverses on a date comfortably above the threshold (rbi 2026-08-01 n=99; runs
- * 2026-08-01 n=86 and 2026-08-05 n=244). Those are DATE-DRIVEN failures — no
- * threshold and no further accrual rescues them, and isotonic is fitting
- * day-structure. Routed to the low-parameter calibrator queue.
+ * ── SO THE DEPLOY BASIS IS THE DATE-CLUSTERED CI ALONE ───────────────────
+ * hits, total_bases and rbi each have a point-in-time held-out interval
+ * excluding zero. That is the ONLY support they have, and it is thin — the
+ * interval rests on 4, 2 and 2 date clusters respectively. Auto-demotion is
+ * therefore the sole stability guard, not a backstop to a passed test.
+ *
+ * runs is absent: no isotonic map was fittable at its point-in-time split, so it
+ * has no CI support to stand on either.
  */
-const CALIBRATION_DEPLOYED = Object.freeze(['hits', 'total_bases']);
+const CALIBRATION_DEPLOYED = Object.freeze(['hits', 'total_bases', 'rbi']);
+/** Why each deployed stat is allowed to serve. Not one of them is LODO-stable. */
+const CALIBRATION_BASIS = Object.freeze({
+  hits: 'ci_only_lodo_untestable',
+  total_bases: 'ci_only_lodo_untestable',
+  rbi: 'ci_only_lodo_untestable',
+});
 
 async function runSnapshot(sport, opts = {}) {
   const sp = String(sport || '').toLowerCase();
@@ -759,6 +769,7 @@ async function runSnapshot(sport, opts = {}) {
             g.calibrated = out.calibrated;
             g.calibration_reason = out.reason;
             g.calibration_status = 'provisional';
+          g.calibration_basis = CALIBRATION_BASIS[stat] || null;
             if (out.calibrated) marked += 1;
           }
           console.log(`[calibration] ${sp} ${stat} (PROVISIONAL) — ${marked} stackable; fit n=${calibrator.fit_n} through ${calibrator.fitted_through}`);
@@ -881,5 +892,6 @@ module.exports = {
   pushTickerItems,
   ACTIVE_SPORTS,
   CALIBRATION_DEPLOYED,
+  CALIBRATION_BASIS,
   __internals: { propKey, gradedAtFor, indexOdds, lastName, isTopGrade, DELTA_NOISE, DELTA_MOVE, TICKER_CAP },
 };

tests/unit/calibrationDeployGate.test.js

@@ -1,43 +1,45 @@
 'use strict';
 
 /**
- * Which stats serve a calibrated number in the live pipeline.
+ * Which stats serve a calibrated number, and on what basis.
  *
- * The rule this locks: a stat that cannot survive dropping a single settled date
- * was never calibrated — it was fitted to that date. hits WAS served calibrated
- * and is not any more, which is the honest consequence of measuring it.
+ * The thing these lock is that "could not test" never reads as "passed". LODO
+ * has 1.4%-9.3% power at this date count, so no deployed stat claims stability
+ * from it — each rides a thin date-clustered interval and auto-demotion alone.
  */
 
 const snapshotService = require('../../src/services/snapshotService');
+const { LODO_TEST, LODO_POWER_FLOOR } = require('../../src/services/model/calibrationRegistry');
 
-describe('the deployed set is LODO-gated', () => {
-  it('serves the stats that passed LODO at the powered threshold', () => {
-    expect(snapshotService.CALIBRATION_DEPLOYED).toContain('total_bases');
-    // hits was withdrawn at 6ae11f1 under a hand-chosen threshold that admitted
-    // 20-row dates as evidence, and is restored here because the powered
-    // instrument finds no reversal. Through the gate, not around it.
-    expect(snapshotService.CALIBRATION_DEPLOYED).toContain('hits');
+describe('the deploy set rests on the CI, not on a passed LODO', () => {
+  it('serves the three stats with a point-in-time interval excluding zero', () => {
+    expect(snapshotService.CALIBRATION_DEPLOYED).toEqual(['hits', 'total_bases', 'rbi']);
   });
 
-  it('does NOT serve rbi or runs — both fail on dates ABOVE the threshold', () => {
-    // These are DATE-DRIVEN failures, not underpowered ones: rbi reverses on a
-    // 99-row date and runs on 86- and 244-row dates. No threshold rescues them.
-    for (const stat of ['rbi', 'runs']) {
-      expect(snapshotService.CALIBRATION_DEPLOYED).not.toContain(stat);
+  it('does NOT serve runs — no fittable map, so no CI to stand on', () => {
+    expect(snapshotService.CALIBRATION_DEPLOYED).not.toContain('runs');
+  });
+
+  it('labels every deployed stat as LODO-untestable rather than LODO-stable', () => {
+    for (const stat of snapshotService.CALIBRATION_DEPLOYED) {
+      expect(snapshotService.CALIBRATION_BASIS[stat]).toBe('ci_only_lodo_untestable');
     }
   });
 
-  it('every deployed stat cleared the power-derived threshold, not a chosen one', () => {
-    const { LODO_MIN_HELD_ROWS, LODO_THRESHOLD_BASIS } = require('../../src/services/model/calibrationRegistry');
-    expect(LODO_MIN_HELD_ROWS).toBe(70);
-    expect(LODO_THRESHOLD_BASIS.derived_blind).toBe(true);
-    // The reversing dates that keep rbi/runs out are all at or above it, so
-    // their exclusion cannot be an artefact of the threshold.
-    for (const n of [99, 86, 244]) expect(n).toBeGreaterThanOrEqual(LODO_MIN_HELD_ROWS);
+  it('no stat may claim LODO stability, because the test cannot fail', () => {
+    for (const t of Object.values(LODO_TEST)) expect(t.power).toBeLessThan(LODO_POWER_FLOOR);
   });
 
-  it('is frozen, so a stat cannot be added at runtime without a code change', () => {
+  it('rbi and runs were FALSE FAILS under the old zero-reversal rule', () => {
+    // Re-read under the binomial cutoff: 1 and 2 reversals, cutoff 2 for both.
+    expect(LODO_TEST.rbi.cutoff).toBe(2);
+    expect(LODO_TEST.runs.cutoff).toBe(2);
+    // rbi returns to the deploy set on CI support; runs still has none.
+    expect(snapshotService.CALIBRATION_DEPLOYED).toContain('rbi');
+  });
+
+  it('is frozen, so a stat cannot be added at runtime', () => {
     expect(Object.isFrozen(snapshotService.CALIBRATION_DEPLOYED)).toBe(true);
-    expect(() => { snapshotService.CALIBRATION_DEPLOYED.push('hits'); }).toThrow();
+    expect(() => { snapshotService.CALIBRATION_DEPLOYED.push('runs'); }).toThrow();
   });
 });

tests/unit/calibrationRegistry.test.js

@@ -119,34 +119,58 @@ describe('serving is band-limited', () => {
   });
 });
 
-describe('the LODO held-row threshold is power-derived, not operator-chosen', () => {
-  const { LODO_MIN_HELD_ROWS, LODO_THRESHOLD_BASIS } = require('../../src/services/model/calibrationRegistry');
+describe('the LODO test is a COHERENT pair, not a bar plus an unrelated rule', () => {
+  const { LODO_K, LODO_TEST, LODO_POWER_FLOOR } = require('../../src/services/model/calibrationRegistry');
 
-  it('is the value its own stated derivation produces', () => {
-    // n* = (SD(d) / |effect|)^2 -- recomputed here so the constant cannot drift
-    // away from the basis that justifies it.
-    const { per_row_brier_diff_sd: sd, pooled_effect_abs_mean: eff } = LODO_THRESHOLD_BASIS;
-    expect(Math.ceil((sd / eff) ** 2)).toBe(LODO_MIN_HELD_ROWS);
+  const normCdf = (z) => {
+    const t = 1 / (1 + 0.2316419 * Math.abs(z));
+    const d = 0.3989422804014327 * Math.exp(-z * z / 2);
+    const p = d * t * (0.319381530 + t * (-0.356563782 + t * (1.781477937 + t * (-1.821255978 + t * 1.330274429))));
+    return z >= 0 ? 1 - p : p;
+  };
+  const binomPmf = (n, k, p) => {
+    let logC = 0;
+    for (let i = 0; i < k; i += 1) logC += Math.log(n - i) - Math.log(i + 1);
+    return Math.exp(logC + k * Math.log(p) + (n - k) * Math.log(1 - p));
+  };
+  const tail = (n, c, p) => { let s = 0; for (let k = c + 1; k <= n; k += 1) s += binomPmf(n, k, p); return s; };
+
+  it('each n* is what its own (sigma_row, g) produce — no pooled value', () => {
+    for (const [stat, t] of Object.entries(LODO_TEST)) {
+      expect(Math.ceil(LODO_K ** 2 * (t.sigma_row / Math.abs(t.g)) ** 2)).toBe(t.n_star);
+    }
+    // And the four differ, which is exactly why one pooled number mis-credited them.
+    const stars = Object.values(LODO_TEST).map((t) => t.n_star);
+    expect(new Set(stars).size).toBeGreaterThan(1);
   });
 
-  it('carries its derivation basis, and was derived blind', () => {
-    expect(LODO_THRESHOLD_BASIS.rule).toBe('n* = (SD(d) / |effect|)^2');
-    expect(LODO_THRESHOLD_BASIS.derived_blind).toBe(true);
-    expect(LODO_THRESHOLD_BASIS.pooled_rows).toBeGreaterThan(1000);
-  });
-
-  it('rejects the thresholds that were previously chosen by hand', () => {
-    // 20 was the operator-chosen value whose verdict moved with it; anything
-    // below n* cannot distinguish a reversal from a coin flip.
-    for (const weak of [10, 20, 25, 30, 50]) {
-      const se = LODO_THRESHOLD_BASIS.per_row_brier_diff_sd / Math.sqrt(weak);
-      expect(LODO_THRESHOLD_BASIS.pooled_effect_abs_mean).toBeLessThan(se);
-      expect(weak).toBeLessThan(LODO_MIN_HELD_ROWS);
+  it('each cutoff is the smallest one holding the false-positive rate at 0.05', () => {
+    const p = normCdf(-LODO_K);
+    for (const [stat, t] of Object.entries(LODO_TEST)) {
+      expect(tail(t.informative_drops, t.cutoff, p)).toBeLessThanOrEqual(0.05);
+      if (t.cutoff > 0) expect(tail(t.informative_drops, t.cutoff - 1, p)).toBeGreaterThan(0.05);
+      expect(t.fp).toBeCloseTo(tail(t.informative_drops, t.cutoff, p), 3);
     }
   });
 
-  it('is informative at the committed value', () => {
-    const se = LODO_THRESHOLD_BASIS.per_row_brier_diff_sd / Math.sqrt(LODO_MIN_HELD_ROWS);
-    expect(LODO_THRESHOLD_BASIS.pooled_effect_abs_mean).toBeGreaterThanOrEqual(se * 0.99);
+  it('the OLD rule is demonstrably incoherent — it failed stable stats ~half the time', () => {
+    // Zero-reversal rule at a 1-SE bar, on four informative drops.
+    const pNoise = normCdf(-1);
+    const falseFail = 1 - (1 - pNoise) ** 4;
+    expect(falseFail).toBeGreaterThan(0.45);
+    expect(falseFail).toBeLessThan(0.55);
+  });
+
+  it('every stat falls below the power floor, so none may claim LODO stability', () => {
+    // A gate that cannot fail is not a gate. This is the honest state at this
+    // date count, and the floor makes it structural rather than a footnote.
+    for (const [stat, t] of Object.entries(LODO_TEST)) {
+      expect(t.power).toBeLessThan(LODO_POWER_FLOOR);
+    }
+  });
+
+  it('the stale pooled threshold is nulled so nothing can read it', () => {
+    const { LODO_MIN_HELD_ROWS } = require('../../src/services/model/calibrationRegistry');
+    expect(LODO_MIN_HELD_ROWS).toBeNull();
   });
 });