3 Commits

Author	SHA1	Message	Date
builtbykev	ced40421ed	Audit the LODO instrument: it cannot evaluate any stat, and both prior ... FAILs were false PHASE 0 — the gate at 1f40014 was mine and was an incoherent pair. A 1-SE informativeness bar with a ZERO-reversal rule: at exactly 1 SE a stable stat's drop reverses with prob Phi(-1)=0.1587, so on four informative drops P(>=1 reversal | perfectly stable) = 1 - 0.8413^4 = 0.50. It failed stable stats half the time by construction. And the pooled n*=70 mis-credited EVERY stat -- too low for hits (own 77) and runs (81), too high for total_bases (60) and rbi (54). PHASE 1, blind. Per-stat (g, sigma_row): hits -0.01288/0.11251, TB -0.01380/0.10680, rbi -0.00884/0.06459, runs -0.00902/0.08080. All four clear z=1.96 at full n, so none is NO-EFFECT. Committed k=1 with per-stat n* and a binomial cutoff holding FP at 0.004-0.031. THE FINDING THAT DOMINATES: the test has no power. Against a strong instability (date-to-date SD equal to the effect) it detects a failure 1.4%-9.3% of the time, and across every k from 1.0 to 2.0 the best any stat reaches is 0.337. A gate that cannot fail cannot pass, so LODO_POWER_FLOOR=0.50 makes UNTESTABLE structural -- "could not test" can never read as "passed". PHASE 2/3 cold, at each stat's OWN n*: hits 5 informative, 0 reversals, cutoff 2, power 0.093 UNTESTABLE TB 5 informative, 0 reversals, cutoff 2, power 0.093 UNTESTABLE rbi 4 informative, 1 reversal, cutoff 2, power 0.045 UNTESTABLE runs 3 informative, 2 reversals, cutoff 2, power 0.014 UNTESTABLE Setting the power floor aside entirely, NOT ONE STAT EXCEEDS ITS CUTOFF. PHASE 4 — rbi's FAIL was false, as the order suspected. So was RUNS' -- which the order did not anticipate, having classified it DATE-DRIVEN on a 244-row reversal; two reversals in three drops does not clear a cutoff of 2. TB's PASS was vacuous: the test could not have failed it. hits' own n* is LARGER than the pooled one (77 vs 70), and it remains untestable. PHASE 5 — deploy basis is now the date-clustered CI alone: hits CI [-0.0139,-0.0097], 4 date clusters relabelled ci_only TB CI [-0.0061,-0.0045], 2 date clusters RELABELLED, kept rbi CI [-0.0092,-0.0010], 2 date clusters NEWLY DEPLOYED runs no fittable map at its split REFUSE, no CI either Every deployed stat carries calibration_basis ci_only_lodo_untestable and auto-demotion is the SOLE stability guard, not a backstop to a passed test. Stated plainly: those intervals rest on 2-4 date clusters, which is thin, and it is now the only support. rbi gains chainAcross stackability; its bands rebuilt on p_win_calibrated (425 rows) are every-archetype base_rate. runs is queued for the low-param calibrator for the ordinary reason -- no fittable map -- not on the date-driven finding, which was an artefact. PHASE 6 — the deploy set was set by a coin-flip-power ruler; it is now set by a per-stat power-coherent pre-committed test whose first act was to report that it cannot evaluate anything. The audit was permitted to wound the live deploy and did: total_bases lost its LODO claim. Standing question unchanged -- 18 archetype slots across three deployed stats, every one a single band indistinguishable from base rate. Blind ordering held. p_win never mutated. No Bonferroni slot. Counter and frozen clusters verified file-by-file. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W1sivYNqY2TS5ftykmHBU9	2026-08-06 23:01:57 -04:00
builtbykev	1f40014256	Power-derive the LODO threshold: hits restored through the gate, rbi/runs ... routed as date-driven PHASE 0 — threshold derived BLIND, before any stat was re-read. A reversal is informative only if that date's Brier delta is distinguishable from zero at its row count. Per-row Brier difference d_i = (pc-y)^2 - (p-y)^2, so SE(n) = SD(d)/sqrt(n) and n* = (SD(d)/|effect|)^2. Pooled across all four stats so no single stat's verdict could shape the threshold deciding it: pooled rows 3,417 | SD(d) 0.09816 | |effect| 0.01175 n* = (0.09816/0.01175)^2 = 69.8 -> 70 The hand-chosen 20 sat at 0.54 SE -- a coin flip. That is the defect this removes, and why the previous verdict moved with the number. Committed as calibrationRegistry.LODO_MIN_HELD_ROWS = 70 with LODO_THRESHOLD_BASIS; a test recomputes (SD/effect)^2 and asserts it equals the constant, so it cannot drift from its own justification. The derivation script prints no stat verdict, no date and no reversal. PHASE 1 — LODO at n*, applied cold: hits 5 informative drops, 0 reversals PASS total_bases 4 informative drops, 0 reversals PASS rbi reverses 2026-08-01 (n=99) FAIL runs reverses 08-01 (n=86), 08-05 (244) FAIL hits held-out deltas -0.0041/-0.0080/-0.0192/-0.0140/-0.0139 across 123-272 row dates, favourite sign holding on every testable drop. THIS IS THE INSTRUMENT FINALLY POWERED, NOT VINDICATION OF A PREDICTION -- the withdrawal at 6ae11f1 was correct on the instrument available then, which admitted 20- and 25-row dates as evidence. Nothing about hits changed; the threshold stopped being chosen. PHASE 2 — both failures are DATE-DRIVEN, not underpowered. Every reversal sits above n*=70 (99, 86, 244), so no threshold and no further accrual rescues either: isotonic is fitting day-structure. Routed to the low-parameter calibrator queue (Platt/beta), not built here. PHASE 3 — CALIBRATION_DEPLOYED is now ['hits','total_bases'], frozen and tested, both PROVISIONAL with auto-demotion armed and the >=40 date-cluster promotion bar unchanged. hits stackability for chain.chainAcross is RESTORED, and the record shows it returned through the powered gate rather than by fiat. hits bands rebuilt on p_win_calibrated (765 eval rows): every archetype still one band, still base_rate -- calibrated YES, proven-per-archetype NO. PHASE 4 logged: the deploy set is now set by a power-derived, pre-committed, tested constant rather than an operator-chosen number. At 6ae11f1 that rule moved the live path AGAINST the operator; it has now moved it back on the same evidence because the instrument changed. Both directions are the rule working. And calibrated p_win separates within archetype no better than raw across 13 archetype slots on two deployed stats -- per-archetype separation will come from proven factors or not at all. p_win never mutated; no Bonferroni slot consumed; counter and frozen clusters verified byte-identical file by file. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W1sivYNqY2TS5ftykmHBU9	2026-08-06 20:15:32 -04:00
builtbykev	6ae11f1193	LODO-gated provisional calibration: total_bases deploys, hits withdrawn ... PHASE 0 — I applied factorGate's >=40 date-cluster floor to a calibration layer without challenging the binding. That floor is a cluster-robust interval bar for a CAUSAL claim. Calibration makes no causal claim, has a bounded failure mode (it can only over- or under-shrink) and consumes no Bonferroni slot. Its real risk is that the correction is DATE-DRIVEN, and leave-one-date-out tests that directly -- a STRICTER bar, since a cluster count cannot detect a single day carrying the effect. The >=40 floor is retained, correctly scoped as the PROMOTION bar. PHASE 1 — both guards codified, 11 tests, green before Phase 2. Demonstrated on live data: raw population violated=true, mean_p 0.4962, both_sides_share 0.9763; after dedup violated=false, mean_p 0.6694. The null guard's test demonstrates the trap explicitly, since (null-1)**2 is 1 and (null-0)**2 is 0 so a Brier over nulls equals the win rate. PHASE 2 — LODO: hits n=1140 dates=17 2 reversals (07-22 n=20, 07-26 n=25) FAIL total_bases n=1050 dates=7 0 reversals, 0 sign flips PASS rbi n= 630 dates=5 1 reversal (08-01 n=99) FAIL runs n= 597 dates=5 2 reversals (08-01 n=86, 08-05 n=244) FAIL Threshold sensitivity reported because the verdict moves: total_bases passes at every held-size threshold, runs fails at every one, and hits fails ONLY when 20/25-row dates are admitted. I fixed MIN_HELD_ROWS=20 before seeing which stats passed and did not move it afterwards to preserve a deploy. Honest caveat: a per-date Brier delta on 20 rows has a standard error several times the effect, so the instrument is underpowered per-drop -- an argument for pre-registering a higher threshold, which is a Roundtable call, not one to make while holding the results. PHASE 3 — total_bases DEPLOY-PROVISIONAL, band [0.6-0.8]. hits, rbi and runs REFUSE. HITS WAS BEING SERVED CALIBRATED AND IS NOT ANY MORE. snapshotService hardcoded it since S91; it fails LODO, so it is out. A stat that cannot survive dropping one day was never calibrated, it was fitted to that day. The consequence is real -- hits props become unstackable for chain.chainAcross -- and it errs toward withdrawing a claim rather than preserving one on a fragile verdict. Deployment is now driven by a frozen, tested CALIBRATION_DEPLOYED set, not a hardcoded stat name. PHASE 4 — calibrationRegistry, 14 tests. Deploy needs BOTH gates, neither waivable. reverify auto-demotes on the first breach (CI stops excluding zero, or the favourite bias flips sign) and logs the breaking date. Promotion needs the original >=40 bar. A provisional deploy that cannot be taken away is just a deploy. PHASE 5 — TB bands rebuilt on calibrated values, 625 eval rows. The two-bar rule still bites: calibrated YES, proven NO, so they stay a base-rate read, now honestly numbered. Every archetype still collapses to one band -- calibrated p_win separates within archetype no better than raw. PHASE 6 logged only: the dead gradient is buried (hits~TB > runs > RBI, and RBI has the SMALLEST bias, so the skill-driven-gradient mechanism did not survive); the refused set is a map of missing inputs; a low-parameter calibrator is queued unbuilt. p_win never mutated; calibration rides as p_win_calibrated with calibration_status provisional. No Bonferroni slot consumed. Counter and frozen clusters byte-identical. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W1sivYNqY2TS5ftykmHBU9	2026-08-06 18:31:19 -04:00