From e29ab6fd6aaf9f1b3ad5b5d81e5de10e5efd2752 Mon Sep 17 00:00:00 2001 From: Kev Date: Sun, 2 Aug 2026 14:24:28 -0400 Subject: [PATCH] Takeable enforcement: verified on real rows, 1,006 tagged, re-stamp call ready PART 1 verified by inducing the REAL rowsFromSnapshot over REAL lock_lines rows from prod. Three cases, 0 non-takeable anchors: Narvaez (dabble/kalshi/prizepicks/smarkets, NO takeable book) -> book=null, price=null, takeable=null [honest absent] Schwarber(bovada/dabble/novig/PINNACLE before draftkings) -> draftkings +102 [pinnacle SKIPPED, proving TAKEABLE not MODEL] Ohtani (dabble/onexbet before draftkings) -> draftkings -266 Narvaez is the case that matters: pre-fix he was stamped dabble +104 takeable=true; he is now honestly absent. A HARNESS BUG RECORDED: my first verification pulled live /api/odds/mlb, which returned {"error":"Odds data temporarily unavailable"}. The script read that as 0 props and printed "all from takeable books? true" -- a VACUOUSLY TRUE pass. I caught it only because I also printed the book list and it was empty. Same family as the silent-false traps: a probe that finds nothing looks identical to a probe that finds nothing wrong. PART 2: 1,006 rows tagged via the purpose-built quarantine_reason at ROW level with three sub-cases (recoverable_same_line 936, no_takeable_quote 49, takeable_line_differs 21). getModelAggregate ALREADY excluded quarantined rows, so the public record and the n>=20 gate were clean automatically; all five committed holdout scripts now carry the exclusion explicitly. PART 3 -- the re-stamp call is now fact-based. The takeable LOCK-TIME price is recoverable for 936/1,006 (93.0%) from lock_lines, the correct instrument. Only 431 appear in closing_captures, which is the wrong timing for a lock price anyway. LINE CONTAMINATION ANSWERED (previously unverified): the stored line MATCHES a takeable book's line on 936 (93.0%), DIFFERS on 21 (2.1%), and is unverifiable on 49 (4.9%) where no takeable book quoted the prop at all. That makes it cleanly row-level: re-stamp the 936 as an honest JOIN and recover 886 pending rows for the holdouts, or leave all 1,006 excluded. Either way the 21 + 49 stay out -- re-stamping those would invent a lock price, or a line, we never captured. Nothing re-stamped; Kev's call. Gates: 4,111 tests / 330 suites green; next build exit 0. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01QJs13VsyiSKYQP6rj3NNmc --- scripts/matchup-axis-holdout.sql | 9 ++ scripts/opportunity-axis-holdout.sql | 9 ++ scripts/pwin-timeforward.sql | 11 ++- scripts/ruler-comparison.sql | 11 ++- scripts/tb-compound-holdout.sql | 9 ++ specs/takeable-enforcement.md | 132 +++++++++++++++++++++++++++ 6 files changed, 179 insertions(+), 2 deletions(-) create mode 100644 specs/takeable-enforcement.md diff --git a/scripts/matchup-axis-holdout.sql b/scripts/matchup-axis-holdout.sql index 8c7a80a..035c35b 100644 --- a/scripts/matchup-axis-holdout.sql +++ b/scripts/matchup-axis-holdout.sql @@ -13,6 +13,14 @@ -- -- BOTH reliability AND resolution must improve for the axis to promote. +-- +-- CONTAMINATION EXCLUSION (2026-08-02, MANDATORY). Rows whose price/book/takeable +-- were stamped from a NON-TAKEABLE book (DFS / offshore / exchange) between +-- 2026-08-01 and the write-path fix are tagged `quarantine_reason LIKE +-- 'nontakeable_book%'`. They are EXCLUDED here and must never be pooled with +-- clean rows: their locked price -- and therefore the `takeable` flag computed +-- from it -- describes a market you could not have bet. + with rows_ as ( select l.game_date, l.id, @@ -27,6 +35,7 @@ with rows_ as ( where a->>'axis' = 'matchup' limit 1) matchup_tier from public.ledger_entries l where l.sport='mlb' and l.user_id is null + and (l.quarantine_reason is null or l.quarantine_reason not like 'nontakeable_book%') and l.outcome in ('hit','miss') and l.p_win is not null and l.p_win_challenger is not null and l.challenger_adjustments::text like '%matchup%' diff --git a/scripts/opportunity-axis-holdout.sql b/scripts/opportunity-axis-holdout.sql index 947b4ad..a46684b 100644 --- a/scripts/opportunity-axis-holdout.sql +++ b/scripts/opportunity-axis-holdout.sql @@ -12,6 +12,14 @@ -- FIRST: mean|p - outcome| on 0/1 rows is noise-dominated individual error, not -- calibration. +-- +-- CONTAMINATION EXCLUSION (2026-08-02, MANDATORY). Rows whose price/book/takeable +-- were stamped from a NON-TAKEABLE book (DFS / offshore / exchange) between +-- 2026-08-01 and the write-path fix are tagged `quarantine_reason LIKE +-- 'nontakeable_book%'`. They are EXCLUDED here and must never be pooled with +-- clean rows: their locked price -- and therefore the `takeable` flag computed +-- from it -- describes a market you could not have bet. + with rows_ as ( select game_date, id, p_win::numeric champ, @@ -19,6 +27,7 @@ with rows_ as ( (outcome='hit')::int won from public.ledger_entries where sport='mlb' and user_id is null + and (quarantine_reason is null or quarantine_reason not like 'nontakeable_book%') and outcome in ('hit','miss') and p_win is not null and p_win_challenger is not null -- ONLY rows the opportunity axis actually touched. Including untouched rows diff --git a/scripts/pwin-timeforward.sql b/scripts/pwin-timeforward.sql index 1a4ebde..05d9b00 100644 --- a/scripts/pwin-timeforward.sql +++ b/scripts/pwin-timeforward.sql @@ -13,11 +13,20 @@ -- -- MLB only. WNBA abstains on its own data and is not re-litigated here. +-- +-- CONTAMINATION EXCLUSION (2026-08-02, MANDATORY). Rows whose price/book/takeable +-- were stamped from a NON-TAKEABLE book (DFS / offshore / exchange) between +-- 2026-08-01 and the write-path fix are tagged `quarantine_reason LIKE +-- 'nontakeable_book%'`. They are EXCLUDED here and must never be pooled with +-- clean rows: their locked price -- and therefore the `takeable` flag computed +-- from it -- describes a market you could not have bet. + with base as ( select sport, game_date, p_win::numeric p, (outcome='hit')::int won, ntile(2) over (order by game_date, id) half from public.ledger_entries - where sport='mlb' and user_id is null and outcome in ('hit','miss') and p_win is not null), + where sport='mlb' and user_id is null + and (quarantine_reason is null or quarantine_reason not like 'nontakeable_book%') and outcome in ('hit','miss') and p_win is not null), s as (select *, case when half=1 then 'train' else 'holdout' end split from base), b as (select split, width_bucket(p, 0.0, 1.0, 10) bkt, count(*) n, avg(p) pred, avg(won::numeric) actual diff --git a/scripts/ruler-comparison.sql b/scripts/ruler-comparison.sql index 4835683..0dde528 100644 --- a/scripts/ruler-comparison.sql +++ b/scripts/ruler-comparison.sql @@ -23,10 +23,19 @@ -- only test a US-books-median ruler, NOT the exchange-inclusive consensus. The -- exchange ruler is untestable on existing data at any n. +-- +-- CONTAMINATION EXCLUSION (2026-08-02, MANDATORY). Rows whose price/book/takeable +-- were stamped from a NON-TAKEABLE book (DFS / offshore / exchange) between +-- 2026-08-01 and the write-path fix are tagged `quarantine_reason LIKE +-- 'nontakeable_book%'`. They are EXCLUDED here and must never be pooled with +-- clean rows: their locked price -- and therefore the `takeable` flag computed +-- from it -- describes a market you could not have bet. + with imp as ( select id, player_key, stat, game_date, line, side, p_win, book, (outcome='hit')::int won from public.ledger_entries - where sport='mlb' and user_id is null and outcome in ('hit','miss') and p_win is not null), + where sport='mlb' and user_id is null + and (quarantine_reason is null or quarantine_reason not like 'nontakeable_book%') and outcome in ('hit','miss') and p_win is not null), -- latest CLOSE capture per (prop, book); two-sided only -- a one-sided quote -- cannot be de-vigged, so it cannot price a ruler. diff --git a/scripts/tb-compound-holdout.sql b/scripts/tb-compound-holdout.sql index a5290d3..c5ff917 100644 --- a/scripts/tb-compound-holdout.sql +++ b/scripts/tb-compound-holdout.sql @@ -12,6 +12,14 @@ -- champion. If it does NOT, the family-mismatch hypothesis is WRONG and the -- mean-weakness / similarity branch REOPENS. Record which. +-- +-- CONTAMINATION EXCLUSION (2026-08-02, MANDATORY). Rows whose price/book/takeable +-- were stamped from a NON-TAKEABLE book (DFS / offshore / exchange) between +-- 2026-08-01 and the write-path fix are tagged `quarantine_reason LIKE +-- 'nontakeable_book%'`. They are EXCLUDED here and must never be pooled with +-- clean rows: their locked price -- and therefore the `takeable` flag computed +-- from it -- describes a market you could not have bet. + with tb as ( select game_date, id, lower(side) side, (outcome='hit')::int won, @@ -22,6 +30,7 @@ with tb as ( else proj_tb_p_over::numeric end tbv1_al from public.ledger_entries where sport='mlb' and user_id is null + and (quarantine_reason is null or quarantine_reason not like 'nontakeable_book%') and stat = 'total_bases' and outcome in ('hit','miss') and p_win is not null diff --git a/specs/takeable-enforcement.md b/specs/takeable-enforcement.md new file mode 100644 index 0000000..c50f215 --- /dev/null +++ b/specs/takeable-enforcement.md @@ -0,0 +1,132 @@ +# TAKEABLE ENFORCEMENT — FIXED, TAGGED, AND THE RE-STAMP CALL + +**Date:** 2026-08-02 · shipped before tonight's settle · served path, champion, +ranking and the reference ruler **untouched**. + +**Gates:** 4,111 tests / 330 suites green · `next build` exit 0 · write path +verified on real prod rows · 1,006 rows tagged and excluded. + +--- + +## PART 1 — THE WRITE PATH: TWO LEAKS, NOT ONE + +The audit found `ledgerService.indexProps`. Tracing the lock price found +**`snapshotService.indexOdds` has the same defect** — it also indexed the full +props list, so `gradedAt.odds`, the price a grade is *locked* at, could itself be +a DFS or exchange price. **Fixing only the ledger would have left contamination +flowing in through the lock.** + +Both now gate on **`TAKEABLE_BOOKS`** — deliberately **not** `MODEL_BOOKS`. +`pinnacle` is model-eligible and correctly not takeable; a MODEL gate would +re-break this the moment pinnacle's feed recovers. + +**Two indexes, two roles**, because the row needs different things with different +correctness rules: + +- **PRICE / BOOK / TAKEABLE** → takeable books only. +- **GAME FACTS** (`game_time`, `game_date`, team/opponent) → **book-independent**. + First pitch is first pitch whichever book listed it; gating these too would drop + valid rows for no gain. + +**Collapsing those roles into one index was the bug.** + +### Verified on REAL prod feed rows + +Real `lock_lines` rows, run through the real `rowsFromSnapshot`: + +| player | books available (in feed order) | result | +|---|---|---| +| Carlos Narváez | dabble, kalshi, prizepicks, smarkets — **no takeable book** | **`book=null, price=null, takeable=null`** — honest absent | +| Kyle Schwarber (TB) | bovada, dabble, novig, **pinnacle**, draftkings | **draftkings +102**, `takeable=true` | +| Shohei Ohtani | dabble, onexbet, draftkings | **draftkings −266**, `takeable=false` | + +**0 non-takeable anchored rows.** Narváez is the case that matters most: before the +fix he would have been stamped `dabble +104, takeable=true`. He is now **honestly +absent** — no price beats a price you cannot bet. And **pinnacle was skipped**, +which proves the gate is TAKEABLE and not MODEL. + +**A harness bug worth recording:** my first verification pulled live `/api/odds/mlb`, +which returned `{"error": "Odds data temporarily unavailable"}`. My script read +that as *0 props* and printed **"all from takeable books? true"** — a **vacuously +true pass**. I caught it only because I also printed the book list and it was +empty. Sixth-ish instance of this family: *a probe that finds nothing looks +identical to a probe that finds nothing wrong.* + +--- + +## PART 2 — TAGGED AND EXCLUDED (never pooled) + +**1,006 rows tagged** via the purpose-built `quarantine_reason`, at row level with +three distinct sub-cases so the re-stamp decision can be made per row: + +| tag | rows | settled | pending | +|---|---:|---:|---:| +| `nontakeable_book:recoverable_same_line` | **936** | 47 | 886 | +| `nontakeable_book:no_takeable_quote` | 49 | 0 | 48 | +| `nontakeable_book:takeable_line_differs` | 21 | 0 | 21 | + +**`getModelAggregate` already excludes quarantined rows** (`.is('quarantine_reason', +null)`) — a pre-existing guard doing exactly the right thing, so the public model +record and the n≥20 gate are clean automatically. **All five committed holdout +scripts** now carry the same exclusion explicitly. + +--- + +## PART 3 — THE RE-STAMP CALL, ON FACT + +**The takeable book's LOCK-TIME price is recoverable for 936 of 1,006 (93.0%)** +from `lock_lines` — the correct instrument (lock time, not close time). For +reference, only 431 appear in `closing_captures`, which is the wrong timing for a +lock price anyway. + +**So re-stamping is a legitimate JOIN for 93%, not a fabrication.** It would pull +the captured takeable price onto the row — a value we really recorded, at the +right moment. + +### And the line question — answered + +| | rows | verdict | +|---|---:|---| +| stored line **matches** a takeable book's line | **936 (93.0%)** | line is **CLEAN** | +| a takeable book existed but posted a **different** line | **21 (2.1%)** | line **IS** contaminated | +| **no takeable book quoted the prop at all** | **49 (4.9%)** | unverifiable — and arguably should never have been graded | + +**This makes the decision cleanly row-level:** + +> **OPTION A — re-stamp the 936.** The takeable lock price and the same line both +> exist in the archive. Honest join, sample preserved, 886 pending rows recovered +> for the holdouts. +> +> **OPTION B — leave all 1,006 tagged and excluded.** Simplest, loses 886 pending +> rows from the accruing verdicts, costs nothing in correctness. +> +> **Either way, the 21 + 49 stay excluded.** Re-stamping those would invent a lock +> price, or a line, we never captured. + +**I have not re-stamped anything. This is your call, and it is now a fact-based +one.** + +--- + +## STACK BEHIND A CLEAN LEDGER + +**(b) Structural `Number(null) === 0` guard** — a shared unknown-vs-zero helper. +**`hits` will re-trigger it**: its 0.5 lines make P(0) the whole game, exactly +where a null read as zero is maximally wrong. + +**(c) `hits` fix** — on both invariants, not before them. + +**Carry-forward:** tb-v1 verdict (accruing) · the third pre-registered branch +(improves-ranking-but-low → promote **and** recalibrate) · **the 100s Cloudflare +origin timeout vs a ~115s snapshot** — the work completes server-side, so a **524 +is not a failure**; worth a flag before someone reads it as one. + +## TAGS + +**VERIFIED:** both leaks fixed and gated on TAKEABLE (pinnacle skipped, proving +it) · 0 non-takeable anchors on real prod rows · no-takeable-book → honest absent · +1,006 rows tagged with three sub-cases · `getModelAggregate` already excluded +them · 93.0% recoverable from `lock_lines` · line clean 93.0% / contaminated 2.1% / +unverifiable 4.9%. + +**AWAITING KEV:** re-stamp the 936, or leave all 1,006 excluded.