From ea1157d709f8035556f09fd89cfc820486452d12 Mon Sep 17 00:00:00 2001 From: Kev Date: Fri, 31 Jul 2026 19:57:21 -0400 Subject: [PATCH] =?UTF-8?q?report:=20grade=20fix=20Part=201=20=E2=80=94=20?= =?UTF-8?q?the=20p=5Fwin-vs-fair=5Fprob=20rebuild=20is=20REFUTED=20by=20th?= =?UTF-8?q?e=20data?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit STOPPED at the Part 1 gate. Nothing rebuilt, no grade changed, no cutover. THE FINDING: grading on p_win vs fair_prob does not work. All three candidate edge formulations correlate NEGATIVELY with outcomes, on both sports, overall, and in both time splits (n=432 decided rows carrying p_win AND fair_prob_lock): ALL n=432 champ -0.0016 p_win ALONE +0.1221 additive -0.0615 ratio -0.1161 logodds -0.0438 MLB n=240 champ +0.0984 p_win ALONE +0.2278 additive -0.0336 ratio -0.1350 logodds -0.0124 WNBA n=192 champ -0.1143 p_win ALONE -0.0842 additive -0.1326 ratio -0.1281 logodds -0.1243 Subtracting the market's lock-time fair probability destroys and inverts the signal. The plain reading: props where the model most disagrees with the market are LESS likely to hit — the market is better than the model, so "edge vs market" is anti-predictive here, while the raw probability retains some skill alone. WHAT DOES CARRY SIGNAL: p_win alone, MLB only, and it is modest. Time-forward split — TRAIN (07-21..07-26, n=120) r=0.2770; HOLDOUT (07-26..07-30, n=120) r=0.1647, with the additive edge negative in BOTH halves. So p_win survives forward validation directionally but the holdout is NOT significant (t~1.81, p~0.07). Suggestive, not proven. WNBA MUST ABSTAIN: every measure negative including p_win itself (-0.084). Forcing one threshold across both sports would make a coin-flip sport look sharp, which the order forbids. LOOKAHEAD GUARD SATISFIED: fair_prob_lock is the lock-time field, populated on 432 decided rows, range 0.145-0.713. closing_prob (415 rows) is the CLOSE and was NOT used in any correlation — using it would have manufactured a correlation. SAMPLE REALITY: 1103 decided rows but only 432 carry both instrument fields, so a per-sport train/holdout split leaves ~120 per half — enough to show direction, not to certify a letter ladder. I did not tune toward a win: three pre-registered candidates were tested and all three failed; picking a fourth because the first three lost is the overfitting the order guards against. Recommended instead: grade MLB on p_win alone with WNBA abstaining and label it modest/accruing (A-RATED hold stays); or wait ~6 weeks for n~500 MLB; or investigate WHY the market-relative edge inverts, which is the more valuable question. Both queries committed at scripts/grade-correlation-proof.sql so no number here has to be taken on trust. Working settlement untouched; dead resolve endpoint not wired. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01QJs13VsyiSKYQP6rj3NNmc --- scripts/grade-correlation-proof.sql | 31 ++++++++ specs/grade-fix-part1-investigation.md | 105 +++++++++++++++++++++++++ 2 files changed, 136 insertions(+) create mode 100644 scripts/grade-correlation-proof.sql create mode 100644 specs/grade-fix-part1-investigation.md diff --git a/scripts/grade-correlation-proof.sql b/scripts/grade-correlation-proof.sql new file mode 100644 index 0000000..10e528a --- /dev/null +++ b/scripts/grade-correlation-proof.sql @@ -0,0 +1,31 @@ +-- GRADE CORRELATION PROOF (2026-07-31). Re-runnable evidence for +-- specs/grade-fix-part1-investigation.md. LOOKAHEAD GUARD: fair_prob_lock is the +-- LOCK-TIME market probability; closing_prob (the close) is deliberately unused. + +-- 1) Candidate edge formulations vs outcome, per sport. +with r as ( + select sport, (outcome='hit')::int as won, p_win::numeric as p, fair_prob_lock::numeric as f, + case grade when 'A+' then 10 when 'A' then 9 when 'A-' then 8 when 'B+' then 7 when 'B' then 6 + when 'B-' then 5 when 'C+' then 4 when 'C' then 3 when 'C-' then 2 when 'D' then 1 else 0 end as champ_idx + from ledger_entries + where user_id is null and outcome in ('hit','miss') + and p_win is not null and fair_prob_lock is not null and grade is not null +) +select sport, count(*) n, + corr(champ_idx, won) champ_letter_r, + corr(p, won) p_win_alone_r, + corr(p - f, won) additive_edge_r, + corr(p / nullif(f,0), won) ratio_edge_r, + corr(ln(p/(1-p)) - ln(f/(1-f)), won) logodds_edge_r +from r group by sport; + +-- 2) Time-forward split (overfitting guard), MLB. +with r as ( + select game_date, (outcome='hit')::int as won, p_win::numeric as p, fair_prob_lock::numeric as f + from ledger_entries + where user_id is null and outcome in ('hit','miss') and sport='mlb' + and p_win is not null and fair_prob_lock is not null +), s as (select *, ntile(2) over (order by game_date) half from r) +select half, count(*) n, min(game_date) from_date, max(game_date) to_date, + corr(p, won) p_win_alone_r, corr(p - f, won) additive_edge_r +from s group by half order by half; diff --git a/specs/grade-fix-part1-investigation.md b/specs/grade-fix-part1-investigation.md new file mode 100644 index 0000000..9393106 --- /dev/null +++ b/specs/grade-fix-part1-investigation.md @@ -0,0 +1,105 @@ +# THE GRADE FIX — PART 1 INVESTIGATION (report; STOPPED at the gate) +2026-07-31. Nothing rebuilt, no grade changed, no surface cut over. + +# 🔴 HEADLINE: THE PROPOSED REBUILD IS REFUTED BY THE DATA + +**Grading on p_win-vs-fair_prob does NOT work. Every edge formulation correlates +NEGATIVELY with outcomes — on both sports, overall, and in both time splits.** +The order said to verify rather than assume, and to STOP on a wrong premise. This +is that case. + +## The evidence (n=432 decided rows carrying BOTH p_win and fair_prob_lock) + +| sport | n | champ letter r | **p_win ALONE r** | additive (p−f) r | ratio r | log-odds r | +|---|---|---|---|---|---|---| +| **ALL** | 432 | −0.0016 | **+0.1221** | **−0.0615** | **−0.1161** | **−0.0438** | +| **MLB** | 240 | +0.0984 | **+0.2278** | **−0.0336** | **−0.1350** | **−0.0124** | +| **WNBA** | 192 | −0.1143 | **−0.0842** | **−0.1326** | **−0.1281** | **−0.1243** | + +**All three candidate edge definitions (2a) are negative everywhere.** Subtracting +the market's lock-time fair probability does not sharpen the signal — it destroys +it and inverts it. + +**The plain reading:** the props where our model most disagrees with the market are +*less* likely to hit. That is the market being better than the model. "Edge vs the +market" is therefore anti-predictive here, while the raw probability estimate +retains some skill on its own. + +## What DOES carry signal: `p_win` alone, MLB only — and it is modest + +Time-forward split (overfitting guard — fit-free, just correlation on each half): + +| MLB split | dates | n | **p_win alone r** | additive edge r | hit% | +|---|---|---|---|---|---| +| TRAIN (older) | 07-21 → 07-26 | 120 | **+0.2770** | −0.0027 | 50.8 | +| **HOLDOUT (newer)** | 07-26 → 07-30 | 120 | **+0.1647** | **−0.0688** | 57.5 | + +`p_win` alone **survives forward validation directionally** (positive in both halves). +**But the holdout is NOT statistically significant: r=0.165 at n=120 → t≈1.81, p≈0.07.** +Suggestive, not proven. The additive edge is negative in both halves. + +**WNBA must ABSTAIN (2c/2d).** Every measure is negative, including p_win itself +(−0.084). There is no honest grade to build there — forcing one threshold across both +sports would make a coin-flip sport look sharp, which 2c explicitly forbids. + +## Sample-size reality (2c per-sport honesty) +1,103 decided rows exist, but only **442 carry p_win and 432 carry `fair_prob_lock`** — +the instrument fields only began populating recently. After a per-sport split that is +**MLB 240 / WNBA 192**, and a train/holdout split leaves **~120 per half**. Threshold +fitting on 120 rows, validated on 120, is thin: it can show direction but cannot +certify a letter ladder. + +## LOOKAHEAD GUARD — satisfied, and the field is named +**`ledger_entries.fair_prob_lock` is the lock-time truth** and is populated on 432 +decided rows, range 0.145–0.713 (a probability, correct scale). `p_win` ranges +0.100–0.950 (the estimator's clamp). `closing_prob` (415 rows) is the CLOSE and was +**not** used in any correlation above — using it would have manufactured a +correlation out of lookahead. + +## Why the served letter is r≈0 (confirmed) +`engine1.gradeProp` sums flat ±1.0/±0.5 factor deltas onto a neutral 'C'; the file +contains **zero references to any probability**. `edge_pct` — the price-free +`(proj−line)/line` — is not an input to the letter either (engine1 has no `edge` +reference); it drives the *board sort* and *was* the display figure until it was +retired 07-31. So the letter is a factor index that never sees price or probability, +which is why it lands at r≈0.005 and inverts (B 52.4% < C 56.9%). + +## Provenance of the r=0.236 figure +It is mine, measured this session on 354 settled rows (MLB 224) — the probability +letter banded through `grade_thresholds.json`. Today's recomputation on the +p_win/fair_prob_lock population gives **MLB p_win r = 0.2278 (n=240)**, consistent. + +--- + +# WHAT I RECOMMEND INSTEAD (not built — needs your call) + +**Do not ship the specified rebuild.** The defensible options: + +1. **Grade MLB on `p_win` alone; WNBA abstains.** Honest, matches the evidence. + But the holdout is p≈0.07 — so it ships as *"modest, directional, accruing"*, not + as a proven edge, and the A-RATED hold stays. +2. **Wait for sample.** The instrument fields are ~10 days old. At ~90 decided + MLB rows/week, n≈500 MLB (a 250/250 split) is ~6 weeks out and would move p≈0.07 + toward significance if the effect is real. +3. **Investigate WHY the market-relative edge inverts.** That negative correlation is + itself information: it says the model's disagreement with the market is noise. That + is the more valuable question than any threshold ladder. + +**What I will not do:** tune a formulation until a positive number appears. Three +pre-registered candidates were tested and all three failed; picking a fourth because +the first three lost is exactly the overfitting the order guards against. + +## Part 4 loose ends (unchanged, tracked) +`dclv` 0 rows/'unknown' — its own follow-up. NBA/soccer never settle — a per-sport +feed decision, and the dead `/api/grading/resolve` must NOT be wired (it would +double-count the working settlement). **Working settlement untouched by this order.** + +## REPRODUCIBLE +Both queries are committed at `scripts/grade-correlation-proof.sql` — re-runnable, +so no number here has to be taken on trust. + +## TAGS +VERIFIED: all correlations above; `fair_prob_lock` is lock-time and populated; the +served letter has no probability input. CANNOT DETERMINE: whether p_win-alone clears +significance — n is too small (holdout p≈0.07). **BLOCKED: Parts 2/3/4 — the +specified rebuild is refuted, so Part 2's ship gate cannot be met as written.**