Audit the LODO instrument: it cannot evaluate any stat, and both prior

FAILs were false

PHASE 0 — the gate at 1f40014 was mine and was an incoherent pair. A 1-SE
informativeness bar with a ZERO-reversal rule: at exactly 1 SE a stable
stat's drop reverses with prob Phi(-1)=0.1587, so on four informative
drops P(>=1 reversal | perfectly stable) = 1 - 0.8413^4 = 0.50. It failed
stable stats half the time by construction. And the pooled n*=70
mis-credited EVERY stat -- too low for hits (own 77) and runs (81), too
high for total_bases (60) and rbi (54).

PHASE 1, blind. Per-stat (g, sigma_row): hits -0.01288/0.11251, TB
-0.01380/0.10680, rbi -0.00884/0.06459, runs -0.00902/0.08080. All four
clear z=1.96 at full n, so none is NO-EFFECT. Committed k=1 with per-stat
n* and a binomial cutoff holding FP at 0.004-0.031.

THE FINDING THAT DOMINATES: the test has no power. Against a strong
instability (date-to-date SD equal to the effect) it detects a failure
1.4%-9.3% of the time, and across every k from 1.0 to 2.0 the best any
stat reaches is 0.337. A gate that cannot fail cannot pass, so
LODO_POWER_FLOOR=0.50 makes UNTESTABLE structural -- "could not test" can
never read as "passed".

PHASE 2/3 cold, at each stat's OWN n*:

  hits  5 informative, 0 reversals, cutoff 2, power 0.093  UNTESTABLE
  TB    5 informative, 0 reversals, cutoff 2, power 0.093  UNTESTABLE
  rbi   4 informative, 1 reversal,  cutoff 2, power 0.045  UNTESTABLE
  runs  3 informative, 2 reversals, cutoff 2, power 0.014  UNTESTABLE

Setting the power floor aside entirely, NOT ONE STAT EXCEEDS ITS CUTOFF.

PHASE 4 — rbi's FAIL was false, as the order suspected. So was RUNS' --
which the order did not anticipate, having classified it DATE-DRIVEN on a
244-row reversal; two reversals in three drops does not clear a cutoff of
2. TB's PASS was vacuous: the test could not have failed it. hits' own n*
is LARGER than the pooled one (77 vs 70), and it remains untestable.

PHASE 5 — deploy basis is now the date-clustered CI alone:

  hits  CI [-0.0139,-0.0097], 4 date clusters   relabelled ci_only
  TB    CI [-0.0061,-0.0045], 2 date clusters   RELABELLED, kept
  rbi   CI [-0.0092,-0.0010], 2 date clusters   NEWLY DEPLOYED
  runs  no fittable map at its split            REFUSE, no CI either

Every deployed stat carries calibration_basis ci_only_lodo_untestable and
auto-demotion is the SOLE stability guard, not a backstop to a passed
test. Stated plainly: those intervals rest on 2-4 date clusters, which is
thin, and it is now the only support. rbi gains chainAcross stackability;
its bands rebuilt on p_win_calibrated (425 rows) are every-archetype
base_rate. runs is queued for the low-param calibrator for the ordinary
reason -- no fittable map -- not on the date-driven finding, which was an
artefact.

PHASE 6 — the deploy set was set by a coin-flip-power ruler; it is now set
by a per-stat power-coherent pre-committed test whose first act was to
report that it cannot evaluate anything. The audit was permitted to wound
the live deploy and did: total_bases lost its LODO claim. Standing
question unchanged -- 18 archetype slots across three deployed stats, every
one a single band indistinguishable from base rate.

Blind ordering held. p_win never mutated. No Bonferroni slot. Counter and
frozen clusters verified file-by-file.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W1sivYNqY2TS5ftykmHBU9

scripts/lodo-calibration.js

@@ -45,7 +45,7 @@ const FAVOURITE_FLOOR = 0.9;
  * from here -- it is imported so the value that decides the verdicts cannot be
  * edited alongside them.
  */
-const { LODO_MIN_HELD_ROWS: MIN_HELD_ROWS } = require('../src/services/model/calibrationRegistry');
+const { LODO_TEST, LODO_POWER_FLOOR } = require('../src/services/model/calibrationRegistry');
 
 const FIELD = { hits: (b) => b.hits, total_bases: (b) => b.totalBases, rbi: (b) => b.rbi, runs: (b) => b.runs };
 const mean = (xs) => (xs.length ? xs.reduce((a, b) => a + b, 0) / xs.length : null);
@@ -131,7 +131,9 @@ async function main() {
       continue;
     }
 
-    // ── LEAVE ONE DATE OUT ──
+    // ── LEAVE ONE DATE OUT, at THIS stat's own informative bar ──
+    const spec = LODO_TEST[stat];
+    const MIN_HELD_ROWS = spec ? spec.n_star : Infinity;
     const table = [];
     for (const d of dates) {
       const fit = rows.filter((r) => r.date !== d);
@@ -168,10 +170,19 @@ async function main() {
     }
 
     const informative = table.filter((t) => t.verdict !== 'UNINFORMATIVE');
-    const anyReversal = informative.some((t) => t.verdict === 'REVERSES');
-    const signTested = informative.filter((t) => t.favourite_sign_holds !== null);
-    const anySignFlip = signTested.some((t) => t.favourite_sign_holds === false);
-    const passes = informative.length > 0 && !anyReversal && !anySignFlip;
+    const reversals = informative.filter((t) => t.verdict === 'REVERSES');
+
+    // THE DECISION RULE IS BINOMIAL, not zero-tolerance. Under stability each
+    // informative drop reverses with prob Phi(-k), so demanding zero reversals
+    // failed stable stats roughly half the time.
+    const cutoff = spec ? spec.cutoff : 0;
+    const exceedsCutoff = reversals.length > cutoff;
+
+    // AND THE TEST MUST BE ABLE TO FAIL. Below the power floor it cannot, so it
+    // cannot pass either -- "could not test" must never read as "passed".
+    const underpowered = !spec || spec.power < LODO_POWER_FLOOR;
+    const verdict = underpowered ? 'UNTESTABLE_BY_LODO' : (exceedsCutoff ? 'FAIL' : 'PASS');
+    const passes = verdict === 'PASS';
 
     out.per_stat[stat] = {
       n: rows.length,
@@ -179,14 +190,19 @@ async function main() {
       lodo_table: table,
       informative_drops: informative.length,
       brier_reversals: informative.filter((t) => t.verdict === 'REVERSES').length,
-      favourite_sign_flips: signTested.filter((t) => t.favourite_sign_holds === false).length,
-      favourite_sign_untested: informative.length - signTested.length,
-      lodo: passes ? 'PASS' : 'FAIL',
-      reason: passes
-        ? 'improvement never reverses and the favourite over-prediction never flips sign across any single-date drop'
-        : (anyReversal
-          ? `improvement reverses when ${informative.filter((t) => t.verdict === 'REVERSES').map((t) => t.dropped).join(', ')} is dropped — the effect is date-driven`
-          : `the favourite over-prediction flips sign when ${signTested.filter((t) => t.favourite_sign_holds === false).map((t) => t.dropped).join(', ')} is dropped`),
+      favourite_sign_flips: informative.filter((t) => t.favourite_sign_holds === false).length,
+      favourite_sign_untested: informative.filter((t) => t.favourite_sign_holds === null).length,
+      n_star: spec ? spec.n_star : null,
+      cutoff,
+      reversal_count: reversals.length,
+      reversing_dates: reversals.map((t) => ({ date: t.dropped, held_n: t.held_n, delta: t.brier_delta })),
+      test_power: spec ? spec.power : null,
+      lodo: verdict,
+      reason: underpowered
+        ? `power ${spec ? spec.power : 0} < floor ${LODO_POWER_FLOOR} — this test would miss a real date-driven failure more than nine times in ten, so it can neither pass nor fail the stat`
+        : (exceedsCutoff
+          ? `${reversals.length} reversals among ${informative.length} informative drops exceeds the cutoff of ${cutoff}`
+          : `${reversals.length} reversals among ${informative.length} informative drops is within the cutoff of ${cutoff}`),
     };
   }