builtbykev
7140e62b65
MEASURE-ONLY. No promotion, no flip, no tier spend. Live path
byte-identical: CURRENT_RULER_VERSION still v1_first_book, model still
consumes MODEL_BOOKS only.
MANDATE 1'S PREMISE DOES NOT HOLD. The p_win calibration is
RULER-INDEPENDENT, confirmed two ways: estimateProbability takes
{gameLogs, line, statType, features} and never sees a market price, and
the calibration fits p_win against OUTCOMES. Reliability and resolution
are both p_win-vs-outcome measures, so fair_prob cannot enter either.
There is nothing to re-fit -- the ruler changes edge, CLV and takeable,
not calibration.
I RETRACT MY OWN LABEL. I declared the MLB isotonic result PROVISIONAL
"because it was measured against the bent ruler". That over-applied the
ruler caveat to a measurement the ruler never touched. The result was
never contaminated; it moves PROVISIONAL -> DECIDED, not by re-running but
because the gate I attached does not apply.
RAN THE GENUINELY RULER-DEPENDENT QUESTION INSTEAD -- does a median
consensus rescue EDGE? Timing held constant (both rulers at close; a
lock-time reconstruction joins only 43 rows, and mixing lock-incumbent
with close-consensus would confound WHEN with WHAT).
n=200 MLB settled rows: mean |ruler gap| 0.0085. corr(edge_v1, outcome)
-0.0101; corr(edge_v2, outcome) -0.0220; corr(p_win, outcome) +0.2598.
THE HEADLINE: p_win predicts outcomes at +0.26 while p_win minus the
market predicts nothing under EITHER ruler. Subtracting the market price
destroys the signal -- a direct empirical vindication of the identity now
at the top of CLAUDE.md. Market edge is not merely a poor criterion here;
it is a strictly worse instrument than the raw forecast.
CALIBRATION REFRESH (ruler-independent, but n grew 119 -> 250):
time-forward holdout n=125, reliability 0.0846 (was 0.0939), resolution
0.190 (was 0.123). Both hold and both improved on a fresh later window
the earlier fit never saw. Independent replication.
THE LIMITATION THAT BLOCKS A FULL VERDICT: closing_captures holds only
MODEL books -- exchange quotes were never stored, because normalizeProps
discarded them until yesterday. Mean 1.97 books in the historical join. So
this tested a US-books-median ruler, not the exchange-inclusive consensus
whose live delta showed p90 +10 points. That ruler is UNTESTABLE on
existing data at any n. Per Mandate 4's third outcome: inconclusive, not
forced.
SEPARATE FINDING -- LIVE FEED REGRESSION: pinnacle MLB captures went 4,022
-> 0 on 2026-07-31 and have not returned, while every other book continued
(103,940 captures in the prior 10 days). This also corrects an Order Zero
claim of mine: "no sharp anchor exists in our feed" was accurate for the
day measured but wrong generally -- pinnacle was there until 07-30 with
17,090 two-sided captures. line_type='sharp' is a label in closingCapture
via SHARP_BOOKS, not a separate provider. We had a sharp anchor and lost
it two days ago; not caused by anything in this session.
Both queries committed: scripts/ruler-comparison.sql,
scripts/pwin-timeforward.sql.
Gates: 4,028 tests / 322 suites green; next build exit 0.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QJs13VsyiSKYQP6rj3NNmc
34 lines
1.7 KiB
SQL
34 lines
1.7 KiB
SQL
-- pwin-timeforward.sql — calibration refresh on the CURRENT MLB sample (2026-08-01)
|
|
-- MEASURE-ONLY. Time-forward: earlier games fit/observe, later games prove.
|
|
--
|
|
-- Deliberately RULER-INDEPENDENT, and that is the finding: reliability
|
|
-- (predicted vs actual hit rate) and resolution (does higher p_win hit more)
|
|
-- are both p_win-vs-outcome measures. No fair_prob appears anywhere below,
|
|
-- because none can. This is why the consensus ruler cannot change the
|
|
-- calibration verdict.
|
|
--
|
|
-- reliability = n-weighted mean |predicted - actual| across deciles. NOTE:
|
|
-- mean|p - outcome| on 0/1 rows is NOT calibration -- it is noise-dominated
|
|
-- individual error. Bucket first.
|
|
--
|
|
-- MLB only. WNBA abstains on its own data and is not re-litigated here.
|
|
|
|
with base as (
|
|
select sport, game_date, p_win::numeric p, (outcome='hit')::int won,
|
|
ntile(2) over (order by game_date, id) half
|
|
from public.ledger_entries
|
|
where sport='mlb' and user_id is null and outcome in ('hit','miss') and p_win is not null),
|
|
s as (select *, case when half=1 then 'train' else 'holdout' end split from base),
|
|
b as (select split, width_bucket(p, 0.0, 1.0, 10) bkt,
|
|
count(*) n, avg(p) pred, avg(won::numeric) actual
|
|
from s group by 1,2)
|
|
select split,
|
|
sum(n) total_n,
|
|
count(*) buckets,
|
|
round(sum(n*abs(pred-actual))/sum(n),4) reliability_mean_abs_dev,
|
|
round((select corr(p, won::numeric) from s s2 where s2.split=b.split)::numeric,4) resolution_corr,
|
|
round((select avg(won::numeric) from s s3 where s3.split=b.split)::numeric,4) base_rate,
|
|
(select min(game_date) from s s4 where s4.split=b.split) first_game,
|
|
(select max(game_date) from s s5 where s5.split=b.split) last_game
|
|
from b group by split order by split desc;
|