— the proven factors were never wired in
PHASE 0 — two truths recorded. The swap is a BET, not an OOS win:
isotonic beat low-param on identical held-out rows (hits +0.0028, rbi
+0.0042, TB tied) and we serve low-param anyway on an untestable prior
about shared daily structure. At 19 dates nothing here can test it. And
the MIN_SLOPE catch is preserved as standing rationale: a near-zero or
negative slope collapses toward base-rate-for-everything, which LOWERS
Brier while destroying all resolution -- a metric win that guts the
product.
PHASE 1 — the duel is now falsifiable. Both corrections computed on every
hits/TB prop; p_win_lowparam served, p_win_isotonic_shadow logged in its
own try so it can never break serving. calibrationDuel.adjudicate encodes
the rule IN CODE before any forward date exists: >=10 forward dates and
isotonic winning with a date-block CI excluding zero => REFUTED, revert;
otherwise UPHELD; under 10 dates PENDING regardless of the numbers. A
date counts as forward only if NEITHER map was fitted on it -- otherwise
we would be scoring which map memorised better. Nothing swaps now.
PHASE 2 — the ceiling, quantified via Murphy decomposition:
stat reliability RESOLUTION uncertainty variance explained
hits 0.01353 0.00252 0.24532 1.03%
TB 0.01419 0.00442 0.24329 1.82%
rbi 0.00654 0.03268 0.22531 14.51%
runs 0.00788 0.00130 0.23182 0.56%
Calibration did exactly what theory says and nothing more: hits
reliability 0.01353 -> 0.00233 (-0.0112, 83% of the error removed) while
resolution moved -0.0002. Unexpected: rbi has 13x the resolution of hits
and is the one stat we do NOT serve corrected -- it needs calibration
least and discriminates most.
PHASE 2 DIAGNOSIS — NOT-TRANSMITTED, and not weak, ABSENT. Traced in code:
sprayDefense.js and platoonSeverity.js are required by NOTHING in src/,
only by analysis scripts and their own tests. The served p_win
(intelligence/probabilityEstimator.js:54) reads exactly four inputs --
game-log frequency, opp_rank_stat +/-0.03, home_away +/-0.015, and a cv
pull -- with zero occurrences of spray, platoon, hard-hit or
contact-profile. And snapshotService grades at line 454 while computing
challenger/context at 640+, so everything proven is computed DOWNSTREAM of
the grade it would inform. The three proven hits factors have never once
moved a served number.
That reframes the recent nulls: "calibrated p_win does not separate within
archetype" was never a statement about factors. The factors were not in
the forecast.
PHASE 3 — bands rebuilt on SERVED values (hits/TB low-param, rbi/runs
raw): 28 archetype slots across four stats, ZERO show lift. No longer an
open shrug -- it is the arithmetic of resolution 0.0013-0.0327 against
uncertainty ~0.23. A forecast explaining 1% of variance cannot produce
separating bands, and no correction to its numbers will change that.
HEADLINE: calibration is complete, delivered honest numbers on two stats
and zero grade separation, because the counter has no resolution -- and
the proven factors are not wired into the forecast at all. The second is
the reason for the first, and it is plumbing rather than a modelling wall.
Per-archetype grades need proven factors that actually reach p_win. Last
calibration order.
Serving unchanged from 74cf1ce. p_win never mutated. No Bonferroni slot.
Counter and frozen clusters verified file-by-file (15 modules).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W1sivYNqY2TS5ftykmHBU9
6.5 KiB
The resolution ceiling — calibration is complete, and it was never the lever
PHASE 0 — two honest truths, on record
1. The swap is a BET, not an OOS win
On identical held-out rows the isotonic map scored BETTER: hits +0.0028 (CI [0.0013, 0.0045]), rbi +0.0042 (CI [0.0003, 0.0092]), total_bases tied.
We serve the low-parameter map anyway, on the untestable prior that isotonic's in-window edge is daily structure shared between the fit and evaluation windows and will not transmit forward. At 19 dates no instrument here can test that prior — LODO has 1.4–9.3% power against it.
Named as a bet, logged, not evidence. Phase 1 makes it falsifiable.
2. The MIN_SLOPE catch, as a standing guard rationale
runs fitted a = −0.032. A near-zero or negative slope collapses the curve
toward base-rate-for-everything, which lowers Brier — shrinking a
miscalibrated forecaster toward its base rate always does — while destroying all
resolution.
A metric win that guts the product. Any calibration layer must refuse a
non-positive slope on principle, not on inspection. That is now MIN_SLOPE.
PHASE 1 — the duel, instrumented forward
Both corrections are computed on every hits/TB prop:
p_win_lowparam (served) and p_win_isotonic_shadow (logged, never read by
serving or by chainAcross). The shadow runs in its own try — it can never
break serving.
calibrationDuel.adjudicate encodes the rule in code, before any forward date
exists, so the bar cannot drift toward whichever answer arrives:
| condition | verdict | action |
|---|---|---|
| ≥10 forward dates AND isotonic wins, date-block CI excluding zero | REFUTED | revert hits/TB to isotonic, log the reversal |
| ≥10 forward dates, isotonic does not win | UPHELD | keep serving low-param |
| <10 forward dates | PENDING | keep serving, no verdict |
A date counts as forward only if neither map was fitted on it — scoring inside a fit window would ask which map memorised better. Rows lacking that provenance are dropped, never assumed forward. Tests lock all of it, including that a decisive shadow win at 5 dates is still PENDING.
Nothing swaps now. The season decides.
PHASE 2 — the resolution ceiling, quantified
Murphy decomposition: Brier = reliability − resolution + uncertainty.
Reliability is what calibration fixes. Resolution is discrimination, and a
monotone map cannot create it — it relabels bins without re-sorting the rows
inside them.
| stat | n | base | reliability | resolution | uncertainty | share of variance explained |
|---|---|---|---|---|---|---|
| hits | 1,140 | 0.5684 | 0.01353 | 0.00252 | 0.24532 | 1.03% |
| total_bases | 1,050 | 0.5819 | 0.01419 | 0.00442 | 0.24329 | 1.82% |
| rbi | 630 | 0.6571 | 0.00654 | 0.03268 | 0.22531 | 14.51% |
| runs | 597 | 0.6348 | 0.00788 | 0.00130 | 0.23182 | 0.56% |
What calibration did, exactly as theory predicts
| stat | reliability | resolution |
|---|---|---|
| hits | 0.01353 → 0.00233 (−0.0112) | 0.00252 → 0.00231 (−0.0002) |
| total_bases | 0.01419 → 0.00527 (−0.0089) | 0.00442 → 0.00414 (−0.0003) |
Calibration removed 83% of hits' reliability error and moved resolution by essentially nothing. It did the whole of its job, and its job was never the one the grade product needs.
Unexpected: rbi has 13× the resolution of hits and is the one stat we
do not serve corrected — it needs calibration least (reliability 0.0065) and
discriminates most. Worth carrying into the factor arc.
PHASE 2 — the HITS factor-transmission diagnosis
Verdict: NOT-TRANSMITTED. Not weak — absent. This is a plumbing defect, and it is the highest-value finding in the order.
Evidence, traced in code rather than recalled:
-
sprayDefense.jsandplatoonSeverity.jsare required by NOTHING insrc/. Only by analysis scripts and their own tests. The two causally-correct atoms that passed the two-part gate have never been on the serving path. -
The served
p_winreads exactly four inputs (intelligence/probabilityEstimator.js:54): game-log frequency over the line,opp_rank_stat(±0.03),home_away(±0.015), and a cv consistency pull. Zero occurrences of spray, platoon, hard-hit or contact-profile. -
Ordering makes it structural.
snapshotServicegrades at line 454 (gradeAndCacheSlate) and only computes challenger/context at line 640+. Everything proven is computed downstream of the grade it would inform.
So the three proven hits factors — defense_by_direction,
pitcher_contact_profile, platoon_severity — were measured on ledger rows by
analysis scripts and have never once moved a served number.
That reframes every null in this programme's recent history. "Calibrated p_win does not separate within archetype" was never a statement about factors. The factors were not in the forecast.
PHASE 3 — bands on served values
| stat | basis | eval rows | slots | slots with lift |
|---|---|---|---|---|
| hits | p_win_lowparam (served) |
765 | 7 | 0 |
| total_bases | p_win_lowparam (served) |
625 | 7 | 0 |
| rbi | raw p_win |
425 | 7 | 0 |
| runs | raw p_win |
424 | 7 | 0 |
28 archetype slots across four stats. Zero show lift. Every slot is one band indistinguishable from its own base rate.
This is no longer an open shrug. It is the arithmetic consequence of resolution of 0.0013–0.0327 against uncertainty of ~0.23: a forecast explaining 1% of the outcome's variance cannot produce bands that separate, and no correction to its numbers will change that.
THE HEADLINE
Calibration is complete. It delivered honest numbers on two stats and ZERO grade separation, because the counter has no resolution — 1.03% of variance on hits, 0.56% on runs.
And the three proven hits factors are NOT WIRED INTO THE FORECAST AT ALL.
Those two facts together are the programme's position. The second is the reason for the first, and it is a plumbing defect rather than a modelling wall — which makes it the cheapest high-value fix available.
Per-archetype grades require proven factors that actually reach p_win. That
is the next and central arc. This is the last calibration order.
Invariants
Serving unchanged from 74cf1ce — this order logs and diagnoses. p_win never
mutated; p_win_lowparam served, p_win_isotonic_shadow logged. No Bonferroni
slot. Counter and frozen clusters verified file-by-file.