builtbykev
e872eff4ce
— the proven factors were never wired in
PHASE 0 — two truths recorded. The swap is a BET, not an OOS win:
isotonic beat low-param on identical held-out rows (hits +0.0028, rbi
+0.0042, TB tied) and we serve low-param anyway on an untestable prior
about shared daily structure. At 19 dates nothing here can test it. And
the MIN_SLOPE catch is preserved as standing rationale: a near-zero or
negative slope collapses toward base-rate-for-everything, which LOWERS
Brier while destroying all resolution -- a metric win that guts the
product.
PHASE 1 — the duel is now falsifiable. Both corrections computed on every
hits/TB prop; p_win_lowparam served, p_win_isotonic_shadow logged in its
own try so it can never break serving. calibrationDuel.adjudicate encodes
the rule IN CODE before any forward date exists: >=10 forward dates and
isotonic winning with a date-block CI excluding zero => REFUTED, revert;
otherwise UPHELD; under 10 dates PENDING regardless of the numbers. A
date counts as forward only if NEITHER map was fitted on it -- otherwise
we would be scoring which map memorised better. Nothing swaps now.
PHASE 2 — the ceiling, quantified via Murphy decomposition:
stat reliability RESOLUTION uncertainty variance explained
hits 0.01353 0.00252 0.24532 1.03%
TB 0.01419 0.00442 0.24329 1.82%
rbi 0.00654 0.03268 0.22531 14.51%
runs 0.00788 0.00130 0.23182 0.56%
Calibration did exactly what theory says and nothing more: hits
reliability 0.01353 -> 0.00233 (-0.0112, 83% of the error removed) while
resolution moved -0.0002. Unexpected: rbi has 13x the resolution of hits
and is the one stat we do NOT serve corrected -- it needs calibration
least and discriminates most.
PHASE 2 DIAGNOSIS — NOT-TRANSMITTED, and not weak, ABSENT. Traced in code:
sprayDefense.js and platoonSeverity.js are required by NOTHING in src/,
only by analysis scripts and their own tests. The served p_win
(intelligence/probabilityEstimator.js:54) reads exactly four inputs --
game-log frequency, opp_rank_stat +/-0.03, home_away +/-0.015, and a cv
pull -- with zero occurrences of spray, platoon, hard-hit or
contact-profile. And snapshotService grades at line 454 while computing
challenger/context at 640+, so everything proven is computed DOWNSTREAM of
the grade it would inform. The three proven hits factors have never once
moved a served number.
That reframes the recent nulls: "calibrated p_win does not separate within
archetype" was never a statement about factors. The factors were not in
the forecast.
PHASE 3 — bands rebuilt on SERVED values (hits/TB low-param, rbi/runs
raw): 28 archetype slots across four stats, ZERO show lift. No longer an
open shrug -- it is the arithmetic of resolution 0.0013-0.0327 against
uncertainty ~0.23. A forecast explaining 1% of variance cannot produce
separating bands, and no correction to its numbers will change that.
HEADLINE: calibration is complete, delivered honest numbers on two stats
and zero grade separation, because the counter has no resolution -- and
the proven factors are not wired into the forecast at all. The second is
the reason for the first, and it is plumbing rather than a modelling wall.
Per-archetype grades need proven factors that actually reach p_win. Last
calibration order.
Serving unchanged from 74cf1ce. p_win never mutated. No Bonferroni slot.
Counter and frozen clusters verified file-by-file (15 modules).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W1sivYNqY2TS5ftykmHBU9
157 lines
6.5 KiB
Markdown
157 lines
6.5 KiB
Markdown
# The resolution ceiling — calibration is complete, and it was never the lever
|
||
|
||
## PHASE 0 — two honest truths, on record
|
||
|
||
### 1. The swap is a BET, not an OOS win
|
||
|
||
On identical held-out rows the **isotonic map scored BETTER**: hits +0.0028
|
||
(CI [0.0013, 0.0045]), rbi +0.0042 (CI [0.0003, 0.0092]), total_bases tied.
|
||
|
||
We serve the low-parameter map anyway, on the untestable prior that isotonic's
|
||
in-window edge is daily structure shared between the fit and evaluation windows
|
||
and will not transmit forward. At 19 dates **no instrument here can test that
|
||
prior** — LODO has 1.4–9.3% power against it.
|
||
|
||
**Named as a bet, logged, not evidence.** Phase 1 makes it falsifiable.
|
||
|
||
### 2. The MIN_SLOPE catch, as a standing guard rationale
|
||
|
||
`runs` fitted `a = −0.032`. A near-zero or negative slope collapses the curve
|
||
toward *base-rate-for-everything*, which **lowers Brier** — shrinking a
|
||
miscalibrated forecaster toward its base rate always does — while destroying all
|
||
resolution.
|
||
|
||
**A metric win that guts the product.** Any calibration layer must refuse a
|
||
non-positive slope on principle, not on inspection. That is now `MIN_SLOPE`.
|
||
|
||
---
|
||
|
||
## PHASE 1 — the duel, instrumented forward
|
||
|
||
Both corrections are computed on every hits/TB prop:
|
||
`p_win_lowparam` (served) and `p_win_isotonic_shadow` (logged, never read by
|
||
serving or by `chainAcross`). The shadow runs in its own try — it can never
|
||
break serving.
|
||
|
||
`calibrationDuel.adjudicate` encodes the rule **in code, before any forward date
|
||
exists**, so the bar cannot drift toward whichever answer arrives:
|
||
|
||
| condition | verdict | action |
|
||
|---|---|---|
|
||
| ≥10 forward dates AND isotonic wins, date-block CI excluding zero | **REFUTED** | revert hits/TB to isotonic, log the reversal |
|
||
| ≥10 forward dates, isotonic does not win | **UPHELD** | keep serving low-param |
|
||
| <10 forward dates | **PENDING** | keep serving, no verdict |
|
||
|
||
A date counts as forward **only if neither map was fitted on it** — scoring
|
||
inside a fit window would ask which map memorised better. Rows lacking that
|
||
provenance are dropped, never assumed forward. Tests lock all of it, including
|
||
that a decisive shadow win at 5 dates is still PENDING.
|
||
|
||
**Nothing swaps now.** The season decides.
|
||
|
||
---
|
||
|
||
## PHASE 2 — the resolution ceiling, quantified
|
||
|
||
Murphy decomposition: `Brier = reliability − resolution + uncertainty`.
|
||
Reliability is what calibration fixes. **Resolution is discrimination, and a
|
||
monotone map cannot create it** — it relabels bins without re-sorting the rows
|
||
inside them.
|
||
|
||
| stat | n | base | reliability | **resolution** | uncertainty | **share of variance explained** |
|
||
|---|---|---|---|---|---|---|
|
||
| hits | 1,140 | 0.5684 | 0.01353 | **0.00252** | 0.24532 | **1.03%** |
|
||
| total_bases | 1,050 | 0.5819 | 0.01419 | **0.00442** | 0.24329 | **1.82%** |
|
||
| rbi | 630 | 0.6571 | 0.00654 | **0.03268** | 0.22531 | **14.51%** |
|
||
| runs | 597 | 0.6348 | 0.00788 | **0.00130** | 0.23182 | **0.56%** |
|
||
|
||
### What calibration did, exactly as theory predicts
|
||
|
||
| stat | reliability | resolution |
|
||
|---|---|---|
|
||
| hits | 0.01353 → 0.00233 (**−0.0112**) | 0.00252 → 0.00231 (−0.0002) |
|
||
| total_bases | 0.01419 → 0.00527 (**−0.0089**) | 0.00442 → 0.00414 (−0.0003) |
|
||
|
||
**Calibration removed 83% of hits' reliability error and moved resolution by
|
||
essentially nothing.** It did the whole of its job, and its job was never the
|
||
one the grade product needs.
|
||
|
||
**Unexpected:** `rbi` has **13× the resolution of hits** and is the one stat we
|
||
do *not* serve corrected — it needs calibration least (reliability 0.0065) and
|
||
discriminates most. Worth carrying into the factor arc.
|
||
|
||
---
|
||
|
||
## PHASE 2 — the HITS factor-transmission diagnosis
|
||
|
||
**Verdict: NOT-TRANSMITTED. Not weak — absent.** This is a plumbing defect, and
|
||
it is the highest-value finding in the order.
|
||
|
||
Evidence, traced in code rather than recalled:
|
||
|
||
1. **`sprayDefense.js` and `platoonSeverity.js` are required by NOTHING in
|
||
`src/`.** Only by analysis scripts and their own tests. The two
|
||
causally-correct atoms that passed the two-part gate have never been on the
|
||
serving path.
|
||
|
||
2. **The served `p_win` reads exactly four inputs**
|
||
(`intelligence/probabilityEstimator.js:54`): game-log frequency over the line,
|
||
`opp_rank_stat` (±0.03), `home_away` (±0.015), and a cv consistency pull.
|
||
Zero occurrences of spray, platoon, hard-hit or contact-profile.
|
||
|
||
3. **Ordering makes it structural.** `snapshotService` grades at line 454
|
||
(`gradeAndCacheSlate`) and only computes challenger/context at line 640+.
|
||
Everything proven is computed **downstream of the grade it would inform**.
|
||
|
||
So the three proven hits factors — `defense_by_direction`,
|
||
`pitcher_contact_profile`, `platoon_severity` — were measured on ledger rows by
|
||
analysis scripts and **have never once moved a served number.**
|
||
|
||
That reframes every null in this programme's recent history. "Calibrated p_win
|
||
does not separate within archetype" was never a statement about factors. The
|
||
factors were not in the forecast.
|
||
|
||
---
|
||
|
||
## PHASE 3 — bands on served values
|
||
|
||
| stat | basis | eval rows | slots | slots with lift |
|
||
|---|---|---|---|---|
|
||
| hits | `p_win_lowparam` (served) | 765 | 7 | **0** |
|
||
| total_bases | `p_win_lowparam` (served) | 625 | 7 | **0** |
|
||
| rbi | raw `p_win` | 425 | 7 | **0** |
|
||
| runs | raw `p_win` | 424 | 7 | **0** |
|
||
|
||
**28 archetype slots across four stats. Zero show lift.** Every slot is one band
|
||
indistinguishable from its own base rate.
|
||
|
||
This is no longer an open shrug. It is the arithmetic consequence of resolution
|
||
of 0.0013–0.0327 against uncertainty of ~0.23: **a forecast explaining 1% of the
|
||
outcome's variance cannot produce bands that separate**, and no correction to its
|
||
numbers will change that.
|
||
|
||
---
|
||
|
||
## THE HEADLINE
|
||
|
||
**Calibration is complete. It delivered honest numbers on two stats and ZERO
|
||
grade separation, because the counter has no resolution — 1.03% of variance on
|
||
hits, 0.56% on runs.**
|
||
|
||
**And the three proven hits factors are NOT WIRED INTO THE FORECAST AT ALL.**
|
||
|
||
Those two facts together are the programme's position. The second is the reason
|
||
for the first, and it is a plumbing defect rather than a modelling wall — which
|
||
makes it the cheapest high-value fix available.
|
||
|
||
**Per-archetype grades require proven factors that actually reach `p_win`. That
|
||
is the next and central arc.** This is the last calibration order.
|
||
|
||
---
|
||
|
||
## Invariants
|
||
|
||
Serving unchanged from `74cf1ce` — this order logs and diagnoses. `p_win` never
|
||
mutated; `p_win_lowparam` served, `p_win_isotonic_shadow` logged. No Bonferroni
|
||
slot. Counter and frozen clusters verified file-by-file.
|