builtbykev
e29ab6fd6a
PART 1 verified by inducing the REAL rowsFromSnapshot over REAL lock_lines
rows from prod. Three cases, 0 non-takeable anchors:
Narvaez (dabble/kalshi/prizepicks/smarkets, NO takeable book)
-> book=null, price=null, takeable=null [honest absent]
Schwarber(bovada/dabble/novig/PINNACLE before draftkings)
-> draftkings +102 [pinnacle SKIPPED, proving TAKEABLE not MODEL]
Ohtani (dabble/onexbet before draftkings) -> draftkings -266
Narvaez is the case that matters: pre-fix he was stamped dabble +104
takeable=true; he is now honestly absent.
A HARNESS BUG RECORDED: my first verification pulled live /api/odds/mlb,
which returned {"error":"Odds data temporarily unavailable"}. The script
read that as 0 props and printed "all from takeable books? true" -- a
VACUOUSLY TRUE pass. I caught it only because I also printed the book list
and it was empty. Same family as the silent-false traps: a probe that finds
nothing looks identical to a probe that finds nothing wrong.
PART 2: 1,006 rows tagged via the purpose-built quarantine_reason at ROW
level with three sub-cases (recoverable_same_line 936, no_takeable_quote
49, takeable_line_differs 21). getModelAggregate ALREADY excluded
quarantined rows, so the public record and the n>=20 gate were clean
automatically; all five committed holdout scripts now carry the exclusion
explicitly.
PART 3 -- the re-stamp call is now fact-based. The takeable LOCK-TIME price
is recoverable for 936/1,006 (93.0%) from lock_lines, the correct
instrument. Only 431 appear in closing_captures, which is the wrong timing
for a lock price anyway.
LINE CONTAMINATION ANSWERED (previously unverified): the stored line
MATCHES a takeable book's line on 936 (93.0%), DIFFERS on 21 (2.1%), and is
unverifiable on 49 (4.9%) where no takeable book quoted the prop at all.
That makes it cleanly row-level: re-stamp the 936 as an honest JOIN and
recover 886 pending rows for the holdouts, or leave all 1,006 excluded.
Either way the 21 + 49 stay out -- re-stamping those would invent a lock
price, or a line, we never captured. Nothing re-stamped; Kev's call.
Gates: 4,111 tests / 330 suites green; next build exit 0.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QJs13VsyiSKYQP6rj3NNmc
62 lines
3.1 KiB
SQL
62 lines
3.1 KiB
SQL
-- matchup-axis-holdout.sql — per-axis proof, RUN WHEN n IS ADEQUATE.
|
|
--
|
|
-- Filtered to MATCHUP-CARRYING rows only. Including untouched rows would
|
|
-- dilute the comparison with rows where challenger === champion BY
|
|
-- CONSTRUCTION, biasing toward a false positive (the trap opportunity_drift
|
|
-- established).
|
|
--
|
|
-- MATCHUP'S OWN CONTRIBUTION IS KEPT VISIBLE, not just the combined challenger:
|
|
-- arch-v1 composes archetype + environment + opportunity + matchup into ONE
|
|
-- p_win_challenger, so a combined-only view cannot tell which axis earned the
|
|
-- movement. `matchup_nudge` is pulled out of the adjustments array so the axis
|
|
-- can be judged on its own terms and, if it is the one dragging, shelved alone.
|
|
--
|
|
-- BOTH reliability AND resolution must improve for the axis to promote.
|
|
|
|
--
|
|
-- CONTAMINATION EXCLUSION (2026-08-02, MANDATORY). Rows whose price/book/takeable
|
|
-- were stamped from a NON-TAKEABLE book (DFS / offshore / exchange) between
|
|
-- 2026-08-01 and the write-path fix are tagged `quarantine_reason LIKE
|
|
-- 'nontakeable_book%'`. They are EXCLUDED here and must never be pooled with
|
|
-- clean rows: their locked price -- and therefore the `takeable` flag computed
|
|
-- from it -- describes a market you could not have bet.
|
|
|
|
with rows_ as (
|
|
select
|
|
l.game_date, l.id,
|
|
l.p_win::numeric champ,
|
|
l.p_win_challenger::numeric chal,
|
|
(l.outcome='hit')::int won,
|
|
(select (a->>'nudge')::numeric
|
|
from jsonb_array_elements(l.challenger_adjustments) a
|
|
where a->>'axis' = 'matchup' limit 1) matchup_nudge,
|
|
(select a->>'tier'
|
|
from jsonb_array_elements(l.challenger_adjustments) a
|
|
where a->>'axis' = 'matchup' limit 1) matchup_tier
|
|
from public.ledger_entries l
|
|
where l.sport='mlb' and l.user_id is null
|
|
and (l.quarantine_reason is null or l.quarantine_reason not like 'nontakeable_book%')
|
|
and l.outcome in ('hit','miss')
|
|
and l.p_win is not null and l.p_win_challenger is not null
|
|
and l.challenger_adjustments::text like '%matchup%'
|
|
),
|
|
split as (
|
|
select *, case when ntile(2) over (order by game_date, id) = 1 then 'train' else 'holdout' end split
|
|
from rows_
|
|
),
|
|
b_champ as (select split, width_bucket(champ,0,1,10) bkt, count(*) n, avg(champ) pred, avg(won::numeric) actual from split group by 1,2),
|
|
b_chal as (select split, width_bucket(chal ,0,1,10) bkt, count(*) n, avg(chal ) pred, avg(won::numeric) actual from split group by 1,2)
|
|
select
|
|
s.split,
|
|
count(*) n,
|
|
count(distinct s.matchup_tier) tiers,
|
|
round(avg(abs(s.matchup_nudge))::numeric,4) mean_abs_matchup_nudge,
|
|
round((select sum(n*abs(pred-actual))/nullif(sum(n),0) from b_champ c where c.split=s.split),4) reliability_champion,
|
|
round((select sum(n*abs(pred-actual))/nullif(sum(n),0) from b_chal c where c.split=s.split),4) reliability_challenger,
|
|
round(corr(s.champ, s.won::numeric)::numeric,4) resolution_champion,
|
|
round(corr(s.chal , s.won::numeric)::numeric,4) resolution_challenger,
|
|
-- does the matchup nudge ITSELF point the right way?
|
|
round(corr(s.matchup_nudge, s.won::numeric)::numeric,4) matchup_nudge_vs_outcome,
|
|
round(avg(s.won::numeric),4) base_rate
|
|
from split s group by s.split order by s.split desc;
|