MLB re-run vs consensus ruler: premise dissolved, isotonic DECIDED
MEASURE-ONLY. No promotion, no flip, no tier spend. Live path
byte-identical: CURRENT_RULER_VERSION still v1_first_book, model still
consumes MODEL_BOOKS only.
MANDATE 1'S PREMISE DOES NOT HOLD. The p_win calibration is
RULER-INDEPENDENT, confirmed two ways: estimateProbability takes
{gameLogs, line, statType, features} and never sees a market price, and
the calibration fits p_win against OUTCOMES. Reliability and resolution
are both p_win-vs-outcome measures, so fair_prob cannot enter either.
There is nothing to re-fit -- the ruler changes edge, CLV and takeable,
not calibration.
I RETRACT MY OWN LABEL. I declared the MLB isotonic result PROVISIONAL
"because it was measured against the bent ruler". That over-applied the
ruler caveat to a measurement the ruler never touched. The result was
never contaminated; it moves PROVISIONAL -> DECIDED, not by re-running but
because the gate I attached does not apply.
RAN THE GENUINELY RULER-DEPENDENT QUESTION INSTEAD -- does a median
consensus rescue EDGE? Timing held constant (both rulers at close; a
lock-time reconstruction joins only 43 rows, and mixing lock-incumbent
with close-consensus would confound WHEN with WHAT).
n=200 MLB settled rows: mean |ruler gap| 0.0085. corr(edge_v1, outcome)
-0.0101; corr(edge_v2, outcome) -0.0220; corr(p_win, outcome) +0.2598.
THE HEADLINE: p_win predicts outcomes at +0.26 while p_win minus the
market predicts nothing under EITHER ruler. Subtracting the market price
destroys the signal -- a direct empirical vindication of the identity now
at the top of CLAUDE.md. Market edge is not merely a poor criterion here;
it is a strictly worse instrument than the raw forecast.
CALIBRATION REFRESH (ruler-independent, but n grew 119 -> 250):
time-forward holdout n=125, reliability 0.0846 (was 0.0939), resolution
0.190 (was 0.123). Both hold and both improved on a fresh later window
the earlier fit never saw. Independent replication.
THE LIMITATION THAT BLOCKS A FULL VERDICT: closing_captures holds only
MODEL books -- exchange quotes were never stored, because normalizeProps
discarded them until yesterday. Mean 1.97 books in the historical join. So
this tested a US-books-median ruler, not the exchange-inclusive consensus
whose live delta showed p90 +10 points. That ruler is UNTESTABLE on
existing data at any n. Per Mandate 4's third outcome: inconclusive, not
forced.
SEPARATE FINDING -- LIVE FEED REGRESSION: pinnacle MLB captures went 4,022
-> 0 on 2026-07-31 and have not returned, while every other book continued
(103,940 captures in the prior 10 days). This also corrects an Order Zero
claim of mine: "no sharp anchor exists in our feed" was accurate for the
day measured but wrong generally -- pinnacle was there until 07-30 with
17,090 two-sided captures. line_type='sharp' is a label in closingCapture
via SHARP_BOOKS, not a separate provider. We had a sharp anchor and lost
it two days ago; not caused by anything in this session.
Both queries committed: scripts/ruler-comparison.sql,
scripts/pwin-timeforward.sql.
Gates: 4,028 tests / 322 suites green; next build exit 0.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QJs13VsyiSKYQP6rj3NNmc
This commit is contained in:
+21
-9
@@ -29,16 +29,25 @@ stay provisional until re-run** · documented ≠ verified.
|
||||
|
||||
## ▶ NEXT EXECUTABLE ORDER
|
||||
|
||||
**The MLB calibration RE-RUN against the consensus ruler.** Everything
|
||||
model-shaped is downstream of it:
|
||||
**DONE 2026-08-01** — the MLB re-run landed, and it dissolved rather than answered
|
||||
its own question: **calibration is ruler-independent**, so MLB isotonic was never
|
||||
gated on the ruler. See `specs/mlb-recalibration-vs-consensus-ruler.md`.
|
||||
|
||||
- MLB isotonic `p_win` cannot promote until re-run (it was calibrated against
|
||||
`v1_first_book`).
|
||||
- The newly-visible props cannot feed the model until it promotes.
|
||||
- Every edge/CLV number resets to `ruler_version = v2_consensus` at that boundary
|
||||
and **must not be pooled** with what came before.
|
||||
**The next order follows from what that measurement found, not from the plan:**
|
||||
|
||||
*Blocked on nothing. This is the next build.*
|
||||
> **`p_win` predicts outcomes (r = +0.26). `p_win − fair_prob` does not
|
||||
> (r = −0.01 to −0.02, either ruler).** Subtracting the market destroys the
|
||||
> signal. So the build is: **make the served product rank on `p_win`, not on
|
||||
> edge** — and retire market-relative edge from every ranking, gate and display
|
||||
> where it still sits.
|
||||
|
||||
Two things gate on accrual instead of on code, and cannot be rushed:
|
||||
- The **exchange-inclusive ruler** is untestable until v2-era captures accrue.
|
||||
- The **edge verdict** may change under that ruler — or may not; today it is
|
||||
unproven either way, and 'unproven' is the honest label.
|
||||
|
||||
**Open, cheap, and unrelated to the model:** the 🔴 pinnacle feed regression, and
|
||||
the display layer still shows nothing of the widened multi-book data.
|
||||
|
||||
---
|
||||
|
||||
@@ -51,8 +60,11 @@ model-shaped is downstream of it:
|
||||
| **books/prop, WNBA** | **4.21** feed → 1.20 | **WNBA is BETTER covered than MLB** |
|
||||
| **consensus ruler** | **MARKET, not SHARP** | `pinnacle`/`matchbook`/`polymarket` = **0%** on both sports. No sharp anchor exists in our feed. Permanent limitation, not a milestone |
|
||||
| **ruler delta** (consensus − incumbent) | MLB mean +1.50 pts, median 0, **17% of props move ≥5 pts** | rulers genuinely differ; "better" is unproven |
|
||||
| **MLB isotonic `p_win`** | **PROVISIONAL** | calibrated on the bent ruler; does not promote until re-run |
|
||||
| **MLB isotonic `p_win`** | **DECIDED** — reliability **0.0846**, resolution **0.190**, holdout **n=125** | **PROVISIONAL label RETRACTED 2026-08-01.** Calibration is **ruler-independent** (`estimateProbability` never sees a price; the fit is p_win-vs-outcome). Replicated on a fresh later window, both metrics improved |
|
||||
| **edge vs the ruler** | corr(edge, outcome) **−0.010** (v1) → **−0.022** (v2), n=200 · corr(**p_win**, outcome) **+0.26** | **Subtracting the market DESTROYS the signal.** The consensus ruler does not rescue edge: *differs ≠ better* |
|
||||
| **exchange-inclusive ruler** | **UNTESTABLE on existing data** | exchange quotes were never stored (discarded until 2026-08-01). Becomes testable only as v2-era captures accrue |
|
||||
| **WNBA** | **still abstains** | a MODEL problem, not a coverage problem — coverage was never its constraint |
|
||||
| 🔴 **pinnacle feed** | **0 captures since 2026-07-31** (103,940 in the prior 10 days) | a live regression; **we had a sharp anchor and lost it.** Not caused by our changes |
|
||||
| **soccer** | **settles** — ~15 competitions, 30d | "grades into a void" is a **$19/mo Pro-tier** problem, not a data problem |
|
||||
| **CLV + results feeds** | `/odds/closing` + `/movement` **redacted**; `/results` **403 `required_tier: hobby`**; `/exports/resolved-props` **403 `required_tier: pro`** | **verified on our keys** — plain tier exclusion, not a key or plan fault. **$9/mo** buys CLV + steam + results; **$19/mo** adds the 90-day settlement export |
|
||||
| **books SERVED** | 5 → **13**; props rendered **546 → 2,780** (5.1×); mean **4.22** books/prop | **AGGREGATOR widening is LIVE.** Of 2,234 newly-visible props, **31.2% carry a real non-DFS price**; **68.8% are DFS-only** — shown, tagged, never a market |
|
||||
|
||||
Reference in New Issue
Block a user