LODO-gated provisional calibration: total_bases deploys, hits withdrawn

PHASE 0 — I applied factorGate's >=40 date-cluster floor to a calibration
layer without challenging the binding. That floor is a cluster-robust
interval bar for a CAUSAL claim. Calibration makes no causal claim, has a
bounded failure mode (it can only over- or under-shrink) and consumes no
Bonferroni slot. Its real risk is that the correction is DATE-DRIVEN, and
leave-one-date-out tests that directly -- a STRICTER bar, since a cluster
count cannot detect a single day carrying the effect. The >=40 floor is
retained, correctly scoped as the PROMOTION bar.

PHASE 1 — both guards codified, 11 tests, green before Phase 2.
Demonstrated on live data: raw population violated=true, mean_p 0.4962,
both_sides_share 0.9763; after dedup violated=false, mean_p 0.6694. The
null guard's test demonstrates the trap explicitly, since (null-1)**2 is
1 and (null-0)**2 is 0 so a Brier over nulls equals the win rate.

PHASE 2 — LODO:

  hits         n=1140 dates=17  2 reversals (07-22 n=20, 07-26 n=25)  FAIL
  total_bases  n=1050 dates=7   0 reversals, 0 sign flips             PASS
  rbi          n= 630 dates=5   1 reversal  (08-01 n=99)              FAIL
  runs         n= 597 dates=5   2 reversals (08-01 n=86, 08-05 n=244) FAIL

Threshold sensitivity reported because the verdict moves: total_bases
passes at every held-size threshold, runs fails at every one, and hits
fails ONLY when 20/25-row dates are admitted. I fixed MIN_HELD_ROWS=20
before seeing which stats passed and did not move it afterwards to
preserve a deploy. Honest caveat: a per-date Brier delta on 20 rows has a
standard error several times the effect, so the instrument is
underpowered per-drop -- an argument for pre-registering a higher
threshold, which is a Roundtable call, not one to make while holding the
results.

PHASE 3 — total_bases DEPLOY-PROVISIONAL, band [0.6-0.8]. hits, rbi and
runs REFUSE.

HITS WAS BEING SERVED CALIBRATED AND IS NOT ANY MORE. snapshotService
hardcoded it since S91; it fails LODO, so it is out. A stat that cannot
survive dropping one day was never calibrated, it was fitted to that day.
The consequence is real -- hits props become unstackable for
chain.chainAcross -- and it errs toward withdrawing a claim rather than
preserving one on a fragile verdict. Deployment is now driven by a frozen,
tested CALIBRATION_DEPLOYED set, not a hardcoded stat name.

PHASE 4 — calibrationRegistry, 14 tests. Deploy needs BOTH gates, neither
waivable. reverify auto-demotes on the first breach (CI stops excluding
zero, or the favourite bias flips sign) and logs the breaking date.
Promotion needs the original >=40 bar. A provisional deploy that cannot be
taken away is just a deploy.

PHASE 5 — TB bands rebuilt on calibrated values, 625 eval rows. The
two-bar rule still bites: calibrated YES, proven NO, so they stay a
base-rate read, now honestly numbered. Every archetype still collapses to
one band -- calibrated p_win separates within archetype no better than raw.

PHASE 6 logged only: the dead gradient is buried (hits~TB > runs > RBI,
and RBI has the SMALLEST bias, so the skill-driven-gradient mechanism did
not survive); the refused set is a map of missing inputs; a low-parameter
calibrator is queued unbuilt.

p_win never mutated; calibration rides as p_win_calibrated with
calibration_status provisional. No Bonferroni slot consumed. Counter and
frozen clusters byte-identical.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W1sivYNqY2TS5ftykmHBU9

tests/unit/calibrationDeployGate.test.js

@@ -0,0 +1,30 @@
+'use strict';
+
+/**
+ * Which stats serve a calibrated number in the live pipeline.
+ *
+ * The rule this locks: a stat that cannot survive dropping a single settled date
+ * was never calibrated — it was fitted to that date. hits WAS served calibrated
+ * and is not any more, which is the honest consequence of measuring it.
+ */
+
+const snapshotService = require('../../src/services/snapshotService');
+
+describe('the deployed set is LODO-gated', () => {
+  it('serves total_bases — it passed at every held-size threshold', () => {
+    expect(snapshotService.CALIBRATION_DEPLOYED).toContain('total_bases');
+  });
+
+  it('does NOT serve hits, rbi or runs — each fails LODO', () => {
+    // hits reverses when 2026-07-22 or 2026-07-26 is dropped; rbi on 2026-08-01;
+    // runs on 2026-08-01 and 2026-08-05.
+    for (const stat of ['hits', 'rbi', 'runs']) {
+      expect(snapshotService.CALIBRATION_DEPLOYED).not.toContain(stat);
+    }
+  });
+
+  it('is frozen, so a stat cannot be added at runtime without a code change', () => {
+    expect(Object.isFrozen(snapshotService.CALIBRATION_DEPLOYED)).toBe(true);
+    expect(() => { snapshotService.CALIBRATION_DEPLOYED.push('hits'); }).toThrow();
+  });
+});

tests/unit/calibrationGuards.test.js

@@ -0,0 +1,112 @@
+'use strict';
+
+/**
+ * The two ways a calibration measurement lies.
+ *
+ * Both produced a confident, plausible, completely wrong number in the
+ * settlement session, and neither was visible in the output. These lock them out.
+ */
+
+const g = require('../../src/services/model/calibrationGuards');
+const cal = require('../../src/services/model/calibration');
+
+/** A population carrying BOTH sides of each prop, as the snapshot table does. */
+function bothSides(n) {
+  const rows = [];
+  for (let i = 0; i < n; i += 1) {
+    const p = 0.55 + (i % 7) * 0.05;
+    rows.push({ propKey: `prop${i}`, side: 'over', p });
+    rows.push({ propKey: `prop${i}`, side: 'under', p: 1 - p });
+  }
+  return rows;
+}
+
+describe('GUARD 1 — the both-sides tell', () => {
+  it('catches the 0.4998 signature: both sides present AND mean pinned at 0.5', () => {
+    const rows = bothSides(200);
+    const r = g.checkPickedSideDedup(rows);
+    expect(r.violated).toBe(true);
+    expect(r.both_sides_share).toBe(1);
+    expect(Math.abs(r.mean_p - 0.5)).toBeLessThanOrEqual(g.BALANCED_TOLERANCE);
+    expect(r.reason).toMatch(/balanced by construction/);
+  });
+
+  it('assert form REFUSES rather than returning a number', () => {
+    expect(() => g.assertPickedSideDedup(bothSides(100))).toThrow(/CALIBRATION GUARD/);
+  });
+
+  it('passes once deduped to the model-picked side', () => {
+    // The picked side is the one the model favoured, so the mean sits well
+    // above 0.5 — which is what a real forecaster's book looks like.
+    const picked = bothSides(200).filter((r) => r.p > 0.5);
+    const r = g.checkPickedSideDedup(picked);
+    expect(r.violated).toBe(false);
+    expect(r.mean_p).toBeGreaterThan(0.5 + g.BALANCED_TOLERANCE);
+  });
+
+  it('does NOT fire on a genuinely balanced one-sided book', () => {
+    // Either condition alone is unremarkable. A book of one-sided picks that
+    // happens to average 0.5 is honest, and flagging it would be a false alarm.
+    const rows = Array.from({ length: 300 }, (_, i) => ({
+      propKey: `p${i}`, side: 'over', p: i % 2 ? 0.45 : 0.55,
+    }));
+    const r = g.checkPickedSideDedup(rows);
+    expect(r.both_sides_props).toBe(0);
+    expect(r.violated).toBe(false);
+  });
+
+  it('does NOT fire when both sides are present but the mean is skewed', () => {
+    const rows = bothSides(50).concat(
+      Array.from({ length: 400 }, (_, i) => ({ propKey: `x${i}`, side: 'over', p: 0.8 })));
+    const r = g.checkPickedSideDedup(rows);
+    expect(r.both_sides_props).toBeGreaterThan(0);
+    expect(r.violated).toBe(false);          // already deduped elsewhere
+  });
+});
+
+describe('GUARD 2 — a null must never score itself', () => {
+  it('(null-1)**2 can no longer pass as a metric', () => {
+    // This is the exact breach: JS scores null as 1 against a win and 0 against
+    // a loss, so the "Brier" silently equals the win rate.
+    const outcomes = [1, 1, 0, 1, 0];
+    const naive = outcomes.reduce((s, y, i) => s + ((null - y) ** 2), 0) / outcomes.length;
+    const winRate = outcomes.reduce((a, b) => a + b, 0) / outcomes.length;
+    expect(naive).toBeCloseTo(winRate, 10);          // the trap, demonstrated
+
+    expect(g.safeBrier([null, null, null, null, null], outcomes)).toBeNull();
+  });
+
+  it('refuses when ANY single prediction is null', () => {
+    expect(g.safeBrier([0.6, 0.4, null], [1, 0, 1])).toBeNull();
+  });
+
+  it('can be made to hard-fail instead of refusing', () => {
+    expect(() => g.safeBrier([0.6, null], [1, 0], { onNull: 'throw' }))
+      .toThrow(/null prediction reached a Brier term/);
+  });
+
+  it('scores normally when every prediction is real', () => {
+    expect(g.safeBrier([1, 0], [1, 0])).toBe(0);
+    expect(g.safeBrier([0.5, 0.5], [1, 0])).toBeCloseTo(0.25, 10);
+  });
+
+  it('an unfittable map refuses instead of producing null predictions', () => {
+    // fitIsotonic returns null below its minimum; this is what must happen next.
+    const map = cal.fitIsotonic([{ p: 0.6, won: 1 }, { p: 0.4, won: 0 }]);
+    expect(map).toBeNull();
+    const out = g.applyOrRefuse(map, [{ p: 0.6 }], cal.applyIsotonic);
+    expect(out.ok).toBe(false);
+    expect(out.reason).toMatch(/no calibration map/);
+    expect(out.rows).toEqual([]);
+  });
+
+  it('drops unmappable rows rather than passing nulls downstream', () => {
+    const fit = [];
+    for (let i = 0; i < 400; i += 1) fit.push({ p: 0.3 + (i % 60) / 100, won: i % 3 === 0 ? 1 : 0 });
+    const map = cal.fitIsotonic(fit);
+    expect(map).not.toBeNull();
+    const out = g.applyOrRefuse(map, [{ p: 0.5 }, { p: null }], cal.applyIsotonic);
+    expect(out.rows.length).toBe(1);
+    expect(out.dropped).toBe(1);
+  });
+});

tests/unit/calibrationRegistry.test.js

@@ -0,0 +1,120 @@
+'use strict';
+
+/**
+ * Which stats may serve a calibrated number.
+ *
+ * The thing these protect is the meaning of PROVISIONAL: a provisional deploy
+ * that cannot be taken away is just a deploy.
+ */
+
+const { createRegistry, STATUS, PROMOTION_DATE_CLUSTERS } = require('../../src/services/model/calibrationRegistry');
+
+const MAP = [{ lo: 0.5, hi: 0.7, value: 0.55, n: 300 }];
+const GOOD = { lodo_pass: true, ci: [-0.0061, -0.0045], map: MAP, certified_bands: [[0.6, 0.8]], date_clusters: 7, at: '2026-08-06' };
+
+describe('deploy needs BOTH gates', () => {
+  it('deploys when LODO passes and the interval excludes zero', () => {
+    const r = createRegistry();
+    expect(r.deploy('total_bases', GOOD).status).toBe(STATUS.PROVISIONAL);
+  });
+
+  it('refuses on a LODO failure however good the interval', () => {
+    const r = createRegistry();
+    const out = r.deploy('runs', { ...GOOD, lodo_pass: false });
+    expect(out.ok).toBe(false);
+    expect(out.reason).toMatch(/date-driven/);
+  });
+
+  it('refuses when the interval spans zero however clean the LODO', () => {
+    const r = createRegistry();
+    const out = r.deploy('hits', { ...GOOD, ci: [-0.01, 0.002] });
+    expect(out.ok).toBe(false);
+    expect(out.reason).toMatch(/does not exclude zero/);
+  });
+
+  it('refuses without a map — there is nothing to serve', () => {
+    const r = createRegistry();
+    expect(r.deploy('hits', { ...GOOD, map: null }).ok).toBe(false);
+  });
+});
+
+describe('auto-demotion is what makes provisional honest', () => {
+  it('demotes on the first date where the interval stops excluding zero', () => {
+    const r = createRegistry();
+    r.deploy('total_bases', GOOD);
+    const out = r.reverify('total_bases', { ci: [-0.004, 0.001], date: '2026-08-07' });
+    expect(out.status).toBe(STATUS.NONE);
+    expect(out.reason).toBe('ci_no_longer_excludes_zero');
+    expect(out.breaking_date).toBe('2026-08-07');
+    expect(r.serves('total_bases', 0.65).serve).toBe(false);
+  });
+
+  it('demotes when the favourite over-prediction flips sign', () => {
+    // A flip means the correction is now pushing the wrong way.
+    const r = createRegistry();
+    r.deploy('total_bases', GOOD);
+    const out = r.reverify('total_bases', { ci: [-0.006, -0.004], favourite_bias: -0.03, date: '2026-08-08' });
+    expect(out.status).toBe(STATUS.NONE);
+    expect(out.reason).toBe('favourite_bias_flipped');
+  });
+
+  it('logs the demotion with its breaking date', () => {
+    const r = createRegistry();
+    r.deploy('total_bases', GOOD);
+    r.reverify('total_bases', { ci: [0.001, 0.004], date: '2026-08-09' });
+    const ev = r.log().find((e) => e.event === 'auto_demoted');
+    expect(ev).toMatchObject({ stat: 'total_bases', at: '2026-08-09' });
+  });
+
+  it('stays deployed while both conditions hold', () => {
+    const r = createRegistry();
+    r.deploy('total_bases', GOOD);
+    const out = r.reverify('total_bases', { ci: [-0.007, -0.003], favourite_bias: 0.17, date: '2026-08-07' });
+    expect(out.status).toBe(STATUS.PROVISIONAL);
+    expect(out.changed).toBe(false);
+  });
+});
+
+describe('the >=40 date-cluster bar is the PROMOTION bar, not the deploy bar', () => {
+  it('does not block deployment', () => {
+    const r = createRegistry();
+    expect(r.deploy('total_bases', { ...GOOD, date_clusters: 7 }).ok).toBe(true);
+  });
+
+  it('promotes out of provisional once it is met', () => {
+    const r = createRegistry();
+    r.deploy('total_bases', GOOD);
+    const out = r.reverify('total_bases', { ci: [-0.006, -0.004], date_clusters: PROMOTION_DATE_CLUSTERS, date: '2026-09-15' });
+    expect(out.status).toBe(STATUS.PROMOTED);
+  });
+
+  it('does not promote while the interval has stopped holding', () => {
+    const r = createRegistry();
+    r.deploy('total_bases', GOOD);
+    const out = r.reverify('total_bases', { ci: [-0.001, 0.003], date_clusters: 60, date: '2026-09-15' });
+    expect(out.status).toBe(STATUS.NONE);
+  });
+});
+
+describe('serving is band-limited', () => {
+  it('serves inside the certified band and refuses outside it', () => {
+    const r = createRegistry();
+    r.deploy('total_bases', GOOD);
+    expect(r.serves('total_bases', 0.65).serve).toBe(true);
+    expect(r.serves('total_bases', 0.65).provisional).toBe(true);
+    expect(r.serves('total_bases', 0.95).serve).toBe(false);
+    expect(r.serves('total_bases', 0.95).reason).toMatch(/outside the certified band/);
+  });
+
+  it('an undeployed stat never serves', () => {
+    const r = createRegistry();
+    expect(r.serves('hits', 0.6).serve).toBe(false);
+    expect(r.serves('hits', 0.6).reason).toBe('not deployed');
+  });
+
+  it('a missing p_win serves nothing', () => {
+    const r = createRegistry();
+    r.deploy('total_bases', GOOD);
+    expect(r.serves('total_bases', null).serve).toBe(false);
+  });
+});