Scoreboard every challenger; diagnose the 429 as odds-api, not PropLine

PROMOTE-THE-EARNED. Nothing was promoted, because nothing earned it -- not
because the bar was held high. Measured on the same bar that refuted hits-v1:
own rows only, direction-aligned, paired bootstrap, promote only on a CI
excluding zero.

  arch-v1        n=1741  delta 0.0000  CI[-0.0050,+0.0054]  inconclusive
  contact-v1     n=1055  delta +0.0008 CI[-0.0052,+0.0069]  inconclusive
  proj-v1.1      n=1664  delta -0.0301 CI[-0.0543,-0.0060]  reliably WORSE
  matchup/tb-v1/hits-v1  n=0  genuinely pending (rows dated 08-02+)

arch-v1 is the interesting one: it MOVED 76% of rows by 2.5 points on average
and resolution is identical to the champion to four decimals, on the moved
rows too. That is active movement carrying no information -- a finding, not a
pending verdict.

These are true prospective holdouts: arch-v1 and contact-v1 wrote p_win at
grade time into their own columns before the game. Nothing recomputed.

THE 429, read-only. The premise was that we re-pull the full picture every
slot and blow the quota. Measured: PropLine is at 5 calls of 3,000/day --
0.17%. One snapshot is ONE PropLine call per sport, all markets comma-joined.
There is no request-pattern problem, so a change-based pull cannot fix it and
no tier upgrade is needed.

The 429 is odds-api: 478/500 MONTHLY, blocked at 95%. oddsService falls
through silently when PropLine returns empty, and the backup's quota gate
throws the error -- so an empty slate is indistinguishable from an outage and
the message names the wrong provider. Flagged for its own order.

Could NOT verify PropLine movement endpoints: docs are auth-gated and the keys
are production-only. Not asserted either way. The movement-as-data argument
stands on its own merits and should be justified that way, not as a quota fix
it isn't.

Book-breadth invariant written down: we never discard books. All are kept and
shown (DISPLAY_BOOKS = MODEL + REFERENCE + DFS); DFS pick'em is excluded from
PRICING only, because a fixed-payout shaded number is not a market price.
Verified this is already what bookRoles.js does.

Champion byte-identical; every challenger stays wired.
4,159 tests green (332 suites); web build exit 0.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W1sivYNqY2TS5ftykmHBU9
This commit is contained in:
Kev
2026-08-02 22:07:43 -04:00
parent b06a84af80
commit 3ba3dd28f3
5 changed files with 527 additions and 1 deletions
+51
View File
@@ -1132,6 +1132,57 @@ phased plan in the Session-57 conversation / BUILD-STATE Next section).
`zmdnczhtdxcddsxzttub`), so local scripts hitting Supabase need an explicit
`SUPABASE_URL=` override. Prod + the MCP connection are fine.
## Settlement outage + challenger scoreboard (Session 77 — non-obvious)
- **`.in('id', [...])` IS A URL, NOT A QUERY.** PostgREST puts filters in the
URL: 500 UUIDs = an 18,499-char request that the fetch layer rejects with
`TypeError: fetch failed`. This silently killed `settleLedger` for two days
(2026-08-01/02) — it refetched rows by id, destructured `const { data: rows }`
with NO error binding, so rows was null, the loop never ran, and it returned
`{settled:0,voided:0,unrecoverable:0,pending:0}`, byte-identical to a healthy
"nothing to settle". 1,444 rows sat with `settle_attempts=0`. NEVER send an
unbounded id list; `ID_FILTER_CHUNK` (100) is the guard, and settleLedger now
selects every column it needs in ONE query.
- **It was VOLUME-TRIGGERED, which is why it hid.** Daily volume ran 20260 rows
for weeks; 2026-08-01 was the first day past `SETTLE_FETCH_LIMIT` (500). If a
pipeline "works for weeks then stops", suspect a threshold that volume just
crossed, not a code change.
- **The ops alarm was blind to it BY CONSTRUCTION.** The zero-settle watchdog
reads settleLedger's own return values, so `pending: 0` told it the backlog was
empty. An alarm that trusts the return value of the thing it watches cannot see
that thing fail silently — the signal must come from OUTSIDE (a direct
`game_date < today AND outcome IS NULL` count).
- **Settled n went 493 → 1,741 the moment it was fixed.** Anything reading
"n-blocked" across MULTIPLE independent challengers at once is a pipeline
symptom, not a sampling fact. Count settled rows before believing it.
- **`specs/challenger-scoreboard.md` is the board.** Nothing promoted: arch-v1
Δ0.0000 CI[0.005,+0.005] on 1,741 (and it MOVED 76% of rows by 2.5pp mean —
active movement carrying zero information), contact-v1 +0.0008 inconclusive,
proj-v1.1 ladder 0.0301 CI excluding zero = reliably WORSE. matchup/tb-v1/
hits-v1 genuinely pending (rows dated 08-02+, settle after ET midnight).
- **arch-v1/contact-v1 need no replay** — they wrote p_win at grade time into
their own columns, so scoring them is a TRUE prospective holdout. Only a
challenger that did not exist at grade time (hits-v1) needs a point-in-time
replay. Don't conflate the two kinds of evidence.
- **Score a nudge on the rows it MOVED**, not on all rows — otherwise the
unmoved rows are the champion measured against itself and dilute any real
effect toward zero. `scripts/challenger-scoreboard.js` does both slices.
## The 429 is odds-api, NOT PropLine (Session 77 — non-obvious)
- **`specs/odds-429-diagnosis.md`.** MEASURED: PropLine 5/3,000 daily (0.17%);
odds-api 478/500 MONTHLY, `allowed:false` (tracker blocks at 95%). One snapshot
= ONE PropLine call per sport (all markets comma-joined) — there is no
per-prop/per-book fan-out and no request-pattern problem to optimize.
- **The 429 text is the BACKUP's.** `oddsService.getOdds` falls through silently
when PropLine returns null/empty, then odds-api's quota gate throws
`429 "Odds data temporarily unavailable"`. So an EMPTY PropLine slate is
indistinguishable from an outage, and the error names the wrong provider. Open
order — don't read a 429 as "PropLine exhausted" without checking
`GET /api/internal/quota`.
- **BOOK BREADTH INVARIANT: we never discard books.** All books are KEPT and
SHOWN (`DISPLAY_BOOKS = MODEL REFERENCE DFS`). The ONLY selectivity is that
DFS pick'em is excluded from PRICING/consensus (`EXCLUDED_FROM_PRICING`) — a
fixed-payout shaded number is not a market price. Never "clean up" breadth.
## Active Skills
- vyndr-voice (all user-facing output)
- prop-analysis (grading methodology)