MLB re-run vs consensus ruler: premise dissolved, isotonic DECIDED
MEASURE-ONLY. No promotion, no flip, no tier spend. Live path
byte-identical: CURRENT_RULER_VERSION still v1_first_book, model still
consumes MODEL_BOOKS only.
MANDATE 1'S PREMISE DOES NOT HOLD. The p_win calibration is
RULER-INDEPENDENT, confirmed two ways: estimateProbability takes
{gameLogs, line, statType, features} and never sees a market price, and
the calibration fits p_win against OUTCOMES. Reliability and resolution
are both p_win-vs-outcome measures, so fair_prob cannot enter either.
There is nothing to re-fit -- the ruler changes edge, CLV and takeable,
not calibration.
I RETRACT MY OWN LABEL. I declared the MLB isotonic result PROVISIONAL
"because it was measured against the bent ruler". That over-applied the
ruler caveat to a measurement the ruler never touched. The result was
never contaminated; it moves PROVISIONAL -> DECIDED, not by re-running but
because the gate I attached does not apply.
RAN THE GENUINELY RULER-DEPENDENT QUESTION INSTEAD -- does a median
consensus rescue EDGE? Timing held constant (both rulers at close; a
lock-time reconstruction joins only 43 rows, and mixing lock-incumbent
with close-consensus would confound WHEN with WHAT).
n=200 MLB settled rows: mean |ruler gap| 0.0085. corr(edge_v1, outcome)
-0.0101; corr(edge_v2, outcome) -0.0220; corr(p_win, outcome) +0.2598.
THE HEADLINE: p_win predicts outcomes at +0.26 while p_win minus the
market predicts nothing under EITHER ruler. Subtracting the market price
destroys the signal -- a direct empirical vindication of the identity now
at the top of CLAUDE.md. Market edge is not merely a poor criterion here;
it is a strictly worse instrument than the raw forecast.
CALIBRATION REFRESH (ruler-independent, but n grew 119 -> 250):
time-forward holdout n=125, reliability 0.0846 (was 0.0939), resolution
0.190 (was 0.123). Both hold and both improved on a fresh later window
the earlier fit never saw. Independent replication.
THE LIMITATION THAT BLOCKS A FULL VERDICT: closing_captures holds only
MODEL books -- exchange quotes were never stored, because normalizeProps
discarded them until yesterday. Mean 1.97 books in the historical join. So
this tested a US-books-median ruler, not the exchange-inclusive consensus
whose live delta showed p90 +10 points. That ruler is UNTESTABLE on
existing data at any n. Per Mandate 4's third outcome: inconclusive, not
forced.
SEPARATE FINDING -- LIVE FEED REGRESSION: pinnacle MLB captures went 4,022
-> 0 on 2026-07-31 and have not returned, while every other book continued
(103,940 captures in the prior 10 days). This also corrects an Order Zero
claim of mine: "no sharp anchor exists in our feed" was accurate for the
day measured but wrong generally -- pinnacle was there until 07-30 with
17,090 two-sided captures. line_type='sharp' is a label in closingCapture
via SHARP_BOOKS, not a separate provider. We had a sharp anchor and lost
it two days ago; not caused by anything in this session.
Both queries committed: scripts/ruler-comparison.sql,
scripts/pwin-timeforward.sql.
Gates: 4,028 tests / 322 suites green; next build exit 0.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QJs13VsyiSKYQP6rj3NNmc
This commit is contained in:
@@ -0,0 +1,75 @@
|
||||
-- ruler-comparison.sql — Order: MLB CALIBRATION RE-RUN vs CONSENSUS RULER (2026-08-01)
|
||||
-- MEASURE-ONLY. Run against prod (Supabase MCP execute_sql).
|
||||
--
|
||||
-- WHAT THIS DOES AND DOES NOT ANSWER
|
||||
--
|
||||
-- It does NOT re-fit the p_win calibration. It cannot: `estimateProbability`
|
||||
-- takes {gameLogs, line, statType, features} and never sees a market price, and
|
||||
-- the calibration query fits p_win against OUTCOMES. Reliability and resolution
|
||||
-- are both p_win-vs-outcome measures, so the ruler cannot enter either. See
|
||||
-- pwin-timeforward.sql for the (ruler-independent) calibration refresh.
|
||||
--
|
||||
-- What IS ruler-dependent is EDGE (p_win - fair_prob). This measures whether
|
||||
-- swapping the incumbent single-book ruler for a median consensus rescues it.
|
||||
--
|
||||
-- TIMING IS HELD CONSTANT: both rulers are read at CLOSE. The lock-time
|
||||
-- reconstruction is impossible at usable n (only 43 settled rows join
|
||||
-- lock_lines with >=2 two-sided books), and mixing a lock-time incumbent with
|
||||
-- a close-time consensus would confound WHEN with WHAT.
|
||||
--
|
||||
-- LIMITATION, load-bearing: closing_captures contains ONLY MODEL books
|
||||
-- (draftkings/betmgm/betrivers/fanduel/pinnacle). Exchange quotes were never
|
||||
-- stored, because normalizeProps discarded them until 2026-08-01. So this can
|
||||
-- only test a US-books-median ruler, NOT the exchange-inclusive consensus. The
|
||||
-- exchange ruler is untestable on existing data at any n.
|
||||
|
||||
with imp as (
|
||||
select id, player_key, stat, game_date, line, side, p_win, book, (outcome='hit')::int won
|
||||
from public.ledger_entries
|
||||
where sport='mlb' and user_id is null and outcome in ('hit','miss') and p_win is not null),
|
||||
|
||||
-- latest CLOSE capture per (prop, book); two-sided only -- a one-sided quote
|
||||
-- cannot be de-vigged, so it cannot price a ruler.
|
||||
cap as (
|
||||
select distinct on (player_key,stat,game_date,line,book)
|
||||
player_key,stat,game_date,line,book,over_odds,under_odds
|
||||
from public.closing_captures
|
||||
where sport='mlb' and over_odds is not null and under_odds is not null
|
||||
order by player_key,stat,game_date,line,book,captured_at desc),
|
||||
|
||||
d as (select *,
|
||||
case when over_odds>0 then 100.0/(over_odds+100) else (-over_odds)/((-over_odds)+100.0) end po,
|
||||
case when under_odds>0 then 100.0/(under_odds+100) else (-under_odds)/((-under_odds)+100.0) end pu
|
||||
from cap),
|
||||
|
||||
-- multiplicative two-way de-vig, per book (matches src/utils/devig.js)
|
||||
f as (select player_key,stat,game_date,line,book, po/(po+pu) fo, pu/(po+pu) fu
|
||||
from d where po+pu > 0),
|
||||
|
||||
j as (select i.*, f.book ref_book,
|
||||
case when lower(i.side)='under' then f.fu else f.fo end fair_side
|
||||
from imp i join f
|
||||
on f.player_key=i.player_key and f.stat=i.stat
|
||||
and f.game_date=i.game_date and f.line=i.line),
|
||||
|
||||
a as (select id, p_win, won, game_date,
|
||||
count(*) n_books,
|
||||
percentile_cont(0.5) within group (order by fair_side) cons, -- v2: MEDIAN
|
||||
max(case when ref_book=book then fair_side end) own -- v1: the locked book
|
||||
from j group by 1,2,3,4),
|
||||
|
||||
e as (select *, p_win-own edge_v1, p_win-cons edge_v2
|
||||
from a where n_books>=2 and own is not null)
|
||||
|
||||
select count(*) n,
|
||||
round(avg(won)::numeric,4) base_rate,
|
||||
round(avg(abs(cons-own))::numeric,4) mean_abs_ruler_gap,
|
||||
round(avg(cons-own)::numeric,4) mean_signed_ruler_gap,
|
||||
round(corr(edge_v1, won::numeric)::numeric,4) corr_edge_v1_won,
|
||||
round(corr(edge_v2, won::numeric)::numeric,4) corr_edge_v2_won,
|
||||
round(corr(p_win, won::numeric)::numeric,4) corr_pwin_won,
|
||||
round(avg(edge_v1) filter (where won=1)::numeric,4) edge_v1_winners,
|
||||
round(avg(edge_v1) filter (where won=0)::numeric,4) edge_v1_losers,
|
||||
round(avg(edge_v2) filter (where won=1)::numeric,4) edge_v2_winners,
|
||||
round(avg(edge_v2) filter (where won=0)::numeric,4) edge_v2_losers
|
||||
from e;
|
||||
Reference in New Issue
Block a user