Audit the LODO instrument: it cannot evaluate any stat, and both prior
FAILs were false
PHASE 0 — the gate at 1f40014 was mine and was an incoherent pair. A 1-SE
informativeness bar with a ZERO-reversal rule: at exactly 1 SE a stable
stat's drop reverses with prob Phi(-1)=0.1587, so on four informative
drops P(>=1 reversal | perfectly stable) = 1 - 0.8413^4 = 0.50. It failed
stable stats half the time by construction. And the pooled n*=70
mis-credited EVERY stat -- too low for hits (own 77) and runs (81), too
high for total_bases (60) and rbi (54).
PHASE 1, blind. Per-stat (g, sigma_row): hits -0.01288/0.11251, TB
-0.01380/0.10680, rbi -0.00884/0.06459, runs -0.00902/0.08080. All four
clear z=1.96 at full n, so none is NO-EFFECT. Committed k=1 with per-stat
n* and a binomial cutoff holding FP at 0.004-0.031.
THE FINDING THAT DOMINATES: the test has no power. Against a strong
instability (date-to-date SD equal to the effect) it detects a failure
1.4%-9.3% of the time, and across every k from 1.0 to 2.0 the best any
stat reaches is 0.337. A gate that cannot fail cannot pass, so
LODO_POWER_FLOOR=0.50 makes UNTESTABLE structural -- "could not test" can
never read as "passed".
PHASE 2/3 cold, at each stat's OWN n*:
hits 5 informative, 0 reversals, cutoff 2, power 0.093 UNTESTABLE
TB 5 informative, 0 reversals, cutoff 2, power 0.093 UNTESTABLE
rbi 4 informative, 1 reversal, cutoff 2, power 0.045 UNTESTABLE
runs 3 informative, 2 reversals, cutoff 2, power 0.014 UNTESTABLE
Setting the power floor aside entirely, NOT ONE STAT EXCEEDS ITS CUTOFF.
PHASE 4 — rbi's FAIL was false, as the order suspected. So was RUNS' --
which the order did not anticipate, having classified it DATE-DRIVEN on a
244-row reversal; two reversals in three drops does not clear a cutoff of
2. TB's PASS was vacuous: the test could not have failed it. hits' own n*
is LARGER than the pooled one (77 vs 70), and it remains untestable.
PHASE 5 — deploy basis is now the date-clustered CI alone:
hits CI [-0.0139,-0.0097], 4 date clusters relabelled ci_only
TB CI [-0.0061,-0.0045], 2 date clusters RELABELLED, kept
rbi CI [-0.0092,-0.0010], 2 date clusters NEWLY DEPLOYED
runs no fittable map at its split REFUSE, no CI either
Every deployed stat carries calibration_basis ci_only_lodo_untestable and
auto-demotion is the SOLE stability guard, not a backstop to a passed
test. Stated plainly: those intervals rest on 2-4 date clusters, which is
thin, and it is now the only support. rbi gains chainAcross stackability;
its bands rebuilt on p_win_calibrated (425 rows) are every-archetype
base_rate. runs is queued for the low-param calibrator for the ordinary
reason -- no fittable map -- not on the date-driven finding, which was an
artefact.
PHASE 6 — the deploy set was set by a coin-flip-power ruler; it is now set
by a per-stat power-coherent pre-committed test whose first act was to
report that it cannot evaluate anything. The audit was permitted to wound
the live deploy and did: total_bases lost its LODO claim. Standing
question unchanged -- 18 archetype slots across three deployed stats, every
one a single band indistinguishable from base rate.
Blind ordering held. p_win never mutated. No Bonferroni slot. Counter and
frozen clusters verified file-by-file.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W1sivYNqY2TS5ftykmHBU9
This commit is contained in:
@@ -276,27 +276,37 @@ async function loadPitcherArsenals(sport) {
|
||||
}
|
||||
|
||||
/**
|
||||
* Stats whose calibration passed leave-one-date-out and may serve a calibrated
|
||||
* number. PROVISIONAL: auto-demoted the first time the held-out interval stops
|
||||
* excluding zero or the favourite over-prediction flips sign.
|
||||
* Stats that may serve a calibrated number, and ON WHAT BASIS.
|
||||
*
|
||||
* Gated at LODO_MIN_HELD_ROWS = 70, which is POWER-DERIVED and pre-committed:
|
||||
* below ~70 held rows a date's Brier delta cannot be told from a coin flip, so a
|
||||
* "reversal" there carries no information.
|
||||
* ── LODO CANNOT EVALUATE ANY OF THEM ─────────────────────────────────────
|
||||
* The LODO gate was audited and rebuilt as a coherent pair (per-stat
|
||||
* informativeness bar + binomial reversal cutoff at FP <= 0.05). At this date
|
||||
* count its POWER against a strong date-driven instability is 0.093 / 0.093 /
|
||||
* 0.045 / 0.014 — it would miss a real failure more than nine times in ten. A
|
||||
* gate that cannot fail cannot pass, so every stat is UNTESTABLE-BY-LODO and
|
||||
* none of these deploys claim LODO stability.
|
||||
*
|
||||
* hits was WITHDRAWN at 6ae11f1 and is RESTORED here. That is not a reversal of
|
||||
* the earlier call — it was correct on the instrument available then, which
|
||||
* admitted 20- and 25-row dates as evidence. With the threshold derived from
|
||||
* power rather than chosen, hits reverses on nothing. The restoration came
|
||||
* through the gate, not around it.
|
||||
* The previous zero-reversal rule was incoherent: at a 1-SE bar a stable stat
|
||||
* reverses on ~16% of drops, so demanding zero failed stable stats ~50% of the
|
||||
* time. Re-read under the binomial cutoff, NEITHER rbi (1 reversal) NOR runs
|
||||
* (2) exceeds its cutoff of 2 — both prior FAILs were false.
|
||||
*
|
||||
* rbi and runs remain ABSENT, and their failures are NOT underpowered: each
|
||||
* reverses on a date comfortably above the threshold (rbi 2026-08-01 n=99; runs
|
||||
* 2026-08-01 n=86 and 2026-08-05 n=244). Those are DATE-DRIVEN failures — no
|
||||
* threshold and no further accrual rescues them, and isotonic is fitting
|
||||
* day-structure. Routed to the low-parameter calibrator queue.
|
||||
* ── SO THE DEPLOY BASIS IS THE DATE-CLUSTERED CI ALONE ───────────────────
|
||||
* hits, total_bases and rbi each have a point-in-time held-out interval
|
||||
* excluding zero. That is the ONLY support they have, and it is thin — the
|
||||
* interval rests on 4, 2 and 2 date clusters respectively. Auto-demotion is
|
||||
* therefore the sole stability guard, not a backstop to a passed test.
|
||||
*
|
||||
* runs is absent: no isotonic map was fittable at its point-in-time split, so it
|
||||
* has no CI support to stand on either.
|
||||
*/
|
||||
const CALIBRATION_DEPLOYED = Object.freeze(['hits', 'total_bases']);
|
||||
const CALIBRATION_DEPLOYED = Object.freeze(['hits', 'total_bases', 'rbi']);
|
||||
/** Why each deployed stat is allowed to serve. Not one of them is LODO-stable. */
|
||||
const CALIBRATION_BASIS = Object.freeze({
|
||||
hits: 'ci_only_lodo_untestable',
|
||||
total_bases: 'ci_only_lodo_untestable',
|
||||
rbi: 'ci_only_lodo_untestable',
|
||||
});
|
||||
|
||||
async function runSnapshot(sport, opts = {}) {
|
||||
const sp = String(sport || '').toLowerCase();
|
||||
@@ -759,6 +769,7 @@ async function runSnapshot(sport, opts = {}) {
|
||||
g.calibrated = out.calibrated;
|
||||
g.calibration_reason = out.reason;
|
||||
g.calibration_status = 'provisional';
|
||||
g.calibration_basis = CALIBRATION_BASIS[stat] || null;
|
||||
if (out.calibrated) marked += 1;
|
||||
}
|
||||
console.log(`[calibration] ${sp} ${stat} (PROVISIONAL) — ${marked} stackable; fit n=${calibrator.fit_n} through ${calibrator.fitted_through}`);
|
||||
@@ -881,5 +892,6 @@ module.exports = {
|
||||
pushTickerItems,
|
||||
ACTIVE_SPORTS,
|
||||
CALIBRATION_DEPLOYED,
|
||||
CALIBRATION_BASIS,
|
||||
__internals: { propKey, gradedAtFor, indexOdds, lastName, isTopGrade, DELTA_NOISE, DELTA_MOVE, TICKER_CAP },
|
||||
};
|
||||
|
||||
Reference in New Issue
Block a user