report: p_win recalibration holdout — MLB qualifies on isotonic, WNBA abstains

Measure-only. p_win not flipped live, no grade rebuilt, no calibrator deployed.
Per the doctrine, MLB and WNBA were fitted, selected and judged as SEPARATE
models — and they reach opposite verdicts. No global instrument was fitted.

METHOD: time-forward split per sport (earlier fits, later proves). Both
instruments fitted on TRAIN only — single-parameter Platt and
isotonic-with-pooling. Inputs p_win + outcome only; no market field, no closing
value, no lookahead. Nothing about edge/CLV/beat-the-close enters any pass/fail
line.

MEASUREMENT CORRECTION made mid-run: the first pass reported mean|p - outcome|
(~0.46-0.51), which is NOT calibration — it is noise-dominated individual error
on 0/1 rows and would have made every instrument look identical. Reliability is
only meaningful on BUCKETS (bucket mean predicted vs bucket actual rate,
n-weighted), the metric T0 used. All reported numbers use the corrected metric.

HOLDOUT RELIABILITY (lower better): MLB n=119/4 buckets — raw 0.1038, Platt
0.1120, ISOTONIC 0.0939. WNBA n=93/3 buckets — raw 0.1322, Platt 0.0491,
isotonic 0.0667.
HOLDOUT RESOLUTION: MLB raw 0.1388 -> Platt 0.1284 -> isotonic 0.1225.
WNBA raw -0.1201 -> Platt +0.1269 -> isotonic +0.0322.
Fitted Platt: MLB a=-0.381 b=+0.705; WNBA a=+0.040 b=-0.081.

MLB QUALIFIES, MODESTLY — instrument selected BY HOLDOUT, not assumed: isotonic
beats both raw and Platt, and Platt actually made MLB worse. Reliability improves
0.1038 -> 0.0939 (~10% relative, real but modest) and resolution SURVIVES
(0.1388 -> 0.1225, not crushed). Both Mandate-3 conditions hold.

WNBA ABSTAINS — its Platt result is the best number in the report and is REJECTED
as a fake win. The fitted slope is b = -0.081, negative and near zero, so
sigmoid(0.040 - 0.081*logit p) is nearly constant at ~0.51 for every input: it
"calibrates" by discarding the prediction and emitting the base rate, which is
exactly the failure Mandate 3 pre-registered. Its apparent resolution gain
(-0.120 -> +0.127) is the sign flip, not skill — it would serve the opposite of
its own forecast, fitted on n~96 of anti-signal. Isotonic says the same quietly
(resolution collapses to +0.032).

HONEST CEILING: MLB is a usable-but-unimpressive forecaster (holdout resolution
~0.12, reliability ~0.094, n=119); WNBA has no honest forecast today. Holdout n
and bucket counts (4 and 3) suffice to reject WNBA and prefer isotonic for MLB,
NOT to certify a letter ladder, and the T0 pathology is reduced rather than cured.

CANNOT DETERMINE: per-archetype calibration (Mandate 3d) — bucket n falls below
the reporting floor once split by sport AND archetype on 442 rows.

Queries committed at scripts/pwin-calibration-holdout.sql.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QJs13VsyiSKYQP6rj3NNmc

scripts/pwin-calibration-holdout.sql

@@ -0,0 +1,34 @@
+-- p_win RECALIBRATION — per-sport, time-forward holdout. Measure-only.
+-- specs/pwin-recalibration-holdout.md. NOTE: reliability is measured on BUCKETS
+-- (predicted vs actual rate, n-weighted). mean|p-outcome| on 0/1 rows is NOT
+-- calibration — it is noise-dominated individual error.
+with r as (
+  select sport, game_date, p_win::numeric p, (outcome='hit')::int won,
+         ntile(2) over (partition by sport order by game_date) half
+  from ledger_entries
+  where user_id is null and outcome in ('hit','miss') and p_win is not null
+), tagged as (
+  select *, case when half=1 then 'train' else 'holdout' end split,
+         width_bucket(p,0.1,1.0,5) bkt, ln(p/(1-p)) lg from r
+), tb as (   -- TRAIN buckets = the fit
+  select sport, bkt, count(*) n, avg(lg) mean_lg, avg(won::numeric) rate
+  from tagged where split='train' group by sport, bkt having count(*) >= 8
+), platt as (
+  select sport,
+    regr_slope(ln(greatest(least(rate,.98),.02)/(1-greatest(least(rate,.98),.02))), mean_lg) b,
+    regr_intercept(ln(greatest(least(rate,.98),.02)/(1-greatest(least(rate,.98),.02))), mean_lg) a
+  from tb group by sport
+), hb as (
+  select h.sport, h.bkt, count(*) n, avg(h.p) pred_raw,
+    avg(1/(1+exp(-(pl.a+pl.b*h.lg)))) pred_platt,
+    avg(coalesce(tb.rate,h.p)) pred_iso, avg(h.won::numeric) actual
+  from tagged h join platt pl on pl.sport=h.sport
+  left join tb on tb.sport=h.sport and tb.bkt=h.bkt
+  where h.split='holdout' group by h.sport,h.bkt having count(*) >= 8
+)
+select sport, sum(n) holdout_n, count(*) buckets,
+  sum(n*abs(pred_raw-actual))/sum(n)   raw_reliability_dev,
+  sum(n*abs(pred_platt-actual))/sum(n) platt_reliability_dev,
+  sum(n*abs(pred_iso-actual))/sum(n)   isotonic_reliability_dev
+from hb group by sport order by sport;
+-- Resolution (ordering survives?) is the corr(pred, won) variant of the same CTEs.