builtbykev
ea1157d709
STOPPED at the Part 1 gate. Nothing rebuilt, no grade changed, no cutover. THE FINDING: grading on p_win vs fair_prob does not work. All three candidate edge formulations correlate NEGATIVELY with outcomes, on both sports, overall, and in both time splits (n=432 decided rows carrying p_win AND fair_prob_lock): ALL n=432 champ -0.0016 p_win ALONE +0.1221 additive -0.0615 ratio -0.1161 logodds -0.0438 MLB n=240 champ +0.0984 p_win ALONE +0.2278 additive -0.0336 ratio -0.1350 logodds -0.0124 WNBA n=192 champ -0.1143 p_win ALONE -0.0842 additive -0.1326 ratio -0.1281 logodds -0.1243 Subtracting the market's lock-time fair probability destroys and inverts the signal. The plain reading: props where the model most disagrees with the market are LESS likely to hit — the market is better than the model, so "edge vs market" is anti-predictive here, while the raw probability retains some skill alone. WHAT DOES CARRY SIGNAL: p_win alone, MLB only, and it is modest. Time-forward split — TRAIN (07-21..07-26, n=120) r=0.2770; HOLDOUT (07-26..07-30, n=120) r=0.1647, with the additive edge negative in BOTH halves. So p_win survives forward validation directionally but the holdout is NOT significant (t~1.81, p~0.07). Suggestive, not proven. WNBA MUST ABSTAIN: every measure negative including p_win itself (-0.084). Forcing one threshold across both sports would make a coin-flip sport look sharp, which the order forbids. LOOKAHEAD GUARD SATISFIED: fair_prob_lock is the lock-time field, populated on 432 decided rows, range 0.145-0.713. closing_prob (415 rows) is the CLOSE and was NOT used in any correlation — using it would have manufactured a correlation. SAMPLE REALITY: 1103 decided rows but only 432 carry both instrument fields, so a per-sport train/holdout split leaves ~120 per half — enough to show direction, not to certify a letter ladder. I did not tune toward a win: three pre-registered candidates were tested and all three failed; picking a fourth because the first three lost is the overfitting the order guards against. Recommended instead: grade MLB on p_win alone with WNBA abstaining and label it modest/accruing (A-RATED hold stays); or wait ~6 weeks for n~500 MLB; or investigate WHY the market-relative edge inverts, which is the more valuable question. Both queries committed at scripts/grade-correlation-proof.sql so no number here has to be taken on trust. Working settlement untouched; dead resolve endpoint not wired. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QJs13VsyiSKYQP6rj3NNmc
106 lines
5.6 KiB
Markdown
106 lines
5.6 KiB
Markdown
# THE GRADE FIX — PART 1 INVESTIGATION (report; STOPPED at the gate)
|
||
2026-07-31. Nothing rebuilt, no grade changed, no surface cut over.
|
||
|
||
# 🔴 HEADLINE: THE PROPOSED REBUILD IS REFUTED BY THE DATA
|
||
|
||
**Grading on p_win-vs-fair_prob does NOT work. Every edge formulation correlates
|
||
NEGATIVELY with outcomes — on both sports, overall, and in both time splits.**
|
||
The order said to verify rather than assume, and to STOP on a wrong premise. This
|
||
is that case.
|
||
|
||
## The evidence (n=432 decided rows carrying BOTH p_win and fair_prob_lock)
|
||
|
||
| sport | n | champ letter r | **p_win ALONE r** | additive (p−f) r | ratio r | log-odds r |
|
||
|---|---|---|---|---|---|---|
|
||
| **ALL** | 432 | −0.0016 | **+0.1221** | **−0.0615** | **−0.1161** | **−0.0438** |
|
||
| **MLB** | 240 | +0.0984 | **+0.2278** | **−0.0336** | **−0.1350** | **−0.0124** |
|
||
| **WNBA** | 192 | −0.1143 | **−0.0842** | **−0.1326** | **−0.1281** | **−0.1243** |
|
||
|
||
**All three candidate edge definitions (2a) are negative everywhere.** Subtracting
|
||
the market's lock-time fair probability does not sharpen the signal — it destroys
|
||
it and inverts it.
|
||
|
||
**The plain reading:** the props where our model most disagrees with the market are
|
||
*less* likely to hit. That is the market being better than the model. "Edge vs the
|
||
market" is therefore anti-predictive here, while the raw probability estimate
|
||
retains some skill on its own.
|
||
|
||
## What DOES carry signal: `p_win` alone, MLB only — and it is modest
|
||
|
||
Time-forward split (overfitting guard — fit-free, just correlation on each half):
|
||
|
||
| MLB split | dates | n | **p_win alone r** | additive edge r | hit% |
|
||
|---|---|---|---|---|---|
|
||
| TRAIN (older) | 07-21 → 07-26 | 120 | **+0.2770** | −0.0027 | 50.8 |
|
||
| **HOLDOUT (newer)** | 07-26 → 07-30 | 120 | **+0.1647** | **−0.0688** | 57.5 |
|
||
|
||
`p_win` alone **survives forward validation directionally** (positive in both halves).
|
||
**But the holdout is NOT statistically significant: r=0.165 at n=120 → t≈1.81, p≈0.07.**
|
||
Suggestive, not proven. The additive edge is negative in both halves.
|
||
|
||
**WNBA must ABSTAIN (2c/2d).** Every measure is negative, including p_win itself
|
||
(−0.084). There is no honest grade to build there — forcing one threshold across both
|
||
sports would make a coin-flip sport look sharp, which 2c explicitly forbids.
|
||
|
||
## Sample-size reality (2c per-sport honesty)
|
||
1,103 decided rows exist, but only **442 carry p_win and 432 carry `fair_prob_lock`** —
|
||
the instrument fields only began populating recently. After a per-sport split that is
|
||
**MLB 240 / WNBA 192**, and a train/holdout split leaves **~120 per half**. Threshold
|
||
fitting on 120 rows, validated on 120, is thin: it can show direction but cannot
|
||
certify a letter ladder.
|
||
|
||
## LOOKAHEAD GUARD — satisfied, and the field is named
|
||
**`ledger_entries.fair_prob_lock` is the lock-time truth** and is populated on 432
|
||
decided rows, range 0.145–0.713 (a probability, correct scale). `p_win` ranges
|
||
0.100–0.950 (the estimator's clamp). `closing_prob` (415 rows) is the CLOSE and was
|
||
**not** used in any correlation above — using it would have manufactured a
|
||
correlation out of lookahead.
|
||
|
||
## Why the served letter is r≈0 (confirmed)
|
||
`engine1.gradeProp` sums flat ±1.0/±0.5 factor deltas onto a neutral 'C'; the file
|
||
contains **zero references to any probability**. `edge_pct` — the price-free
|
||
`(proj−line)/line` — is not an input to the letter either (engine1 has no `edge`
|
||
reference); it drives the *board sort* and *was* the display figure until it was
|
||
retired 07-31. So the letter is a factor index that never sees price or probability,
|
||
which is why it lands at r≈0.005 and inverts (B 52.4% < C 56.9%).
|
||
|
||
## Provenance of the r=0.236 figure
|
||
It is mine, measured this session on 354 settled rows (MLB 224) — the probability
|
||
letter banded through `grade_thresholds.json`. Today's recomputation on the
|
||
p_win/fair_prob_lock population gives **MLB p_win r = 0.2278 (n=240)**, consistent.
|
||
|
||
---
|
||
|
||
# WHAT I RECOMMEND INSTEAD (not built — needs your call)
|
||
|
||
**Do not ship the specified rebuild.** The defensible options:
|
||
|
||
1. **Grade MLB on `p_win` alone; WNBA abstains.** Honest, matches the evidence.
|
||
But the holdout is p≈0.07 — so it ships as *"modest, directional, accruing"*, not
|
||
as a proven edge, and the A-RATED hold stays.
|
||
2. **Wait for sample.** The instrument fields are ~10 days old. At ~90 decided
|
||
MLB rows/week, n≈500 MLB (a 250/250 split) is ~6 weeks out and would move p≈0.07
|
||
toward significance if the effect is real.
|
||
3. **Investigate WHY the market-relative edge inverts.** That negative correlation is
|
||
itself information: it says the model's disagreement with the market is noise. That
|
||
is the more valuable question than any threshold ladder.
|
||
|
||
**What I will not do:** tune a formulation until a positive number appears. Three
|
||
pre-registered candidates were tested and all three failed; picking a fourth because
|
||
the first three lost is exactly the overfitting the order guards against.
|
||
|
||
## Part 4 loose ends (unchanged, tracked)
|
||
`dclv` 0 rows/'unknown' — its own follow-up. NBA/soccer never settle — a per-sport
|
||
feed decision, and the dead `/api/grading/resolve` must NOT be wired (it would
|
||
double-count the working settlement). **Working settlement untouched by this order.**
|
||
|
||
## REPRODUCIBLE
|
||
Both queries are committed at `scripts/grade-correlation-proof.sql` — re-runnable,
|
||
so no number here has to be taken on trust.
|
||
|
||
## TAGS
|
||
VERIFIED: all correlations above; `fair_prob_lock` is lock-time and populated; the
|
||
served letter has no probability input. CANNOT DETERMINE: whether p_win-alone clears
|
||
significance — n is too small (holdout p≈0.07). **BLOCKED: Parts 2/3/4 — the
|
||
specified rebuild is refuted, so Part 2's ship gate cannot be met as written.**
|