Scoreboard every challenger; diagnose the 429 as odds-api, not PropLine

PROMOTE-THE-EARNED. Nothing was promoted, because nothing earned it -- not
because the bar was held high. Measured on the same bar that refuted hits-v1:
own rows only, direction-aligned, paired bootstrap, promote only on a CI
excluding zero.

  arch-v1        n=1741  delta 0.0000  CI[-0.0050,+0.0054]  inconclusive
  contact-v1     n=1055  delta +0.0008 CI[-0.0052,+0.0069]  inconclusive
  proj-v1.1      n=1664  delta -0.0301 CI[-0.0543,-0.0060]  reliably WORSE
  matchup/tb-v1/hits-v1  n=0  genuinely pending (rows dated 08-02+)

arch-v1 is the interesting one: it MOVED 76% of rows by 2.5 points on average
and resolution is identical to the champion to four decimals, on the moved
rows too. That is active movement carrying no information -- a finding, not a
pending verdict.

These are true prospective holdouts: arch-v1 and contact-v1 wrote p_win at
grade time into their own columns before the game. Nothing recomputed.

THE 429, read-only. The premise was that we re-pull the full picture every
slot and blow the quota. Measured: PropLine is at 5 calls of 3,000/day --
0.17%. One snapshot is ONE PropLine call per sport, all markets comma-joined.
There is no request-pattern problem, so a change-based pull cannot fix it and
no tier upgrade is needed.

The 429 is odds-api: 478/500 MONTHLY, blocked at 95%. oddsService falls
through silently when PropLine returns empty, and the backup's quota gate
throws the error -- so an empty slate is indistinguishable from an outage and
the message names the wrong provider. Flagged for its own order.

Could NOT verify PropLine movement endpoints: docs are auth-gated and the keys
are production-only. Not asserted either way. The movement-as-data argument
stands on its own merits and should be justified that way, not as a quota fix
it isn't.

Book-breadth invariant written down: we never discard books. All are kept and
shown (DISPLAY_BOOKS = MODEL + REFERENCE + DFS); DFS pick'em is excluded from
PRICING only, because a fixed-payout shaded number is not a market price.
Verified this is already what bookRoles.js does.

Champion byte-identical; every challenger stays wired.
4,159 tests green (332 suites); web build exit 0.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W1sivYNqY2TS5ftykmHBU9
This commit is contained in:
Kev
2026-08-02 22:07:43 -04:00
parent b06a84af80
commit 3ba3dd28f3
5 changed files with 527 additions and 1 deletions
+31 -1
View File
@@ -1,7 +1,37 @@
# VYNDR — Build State
## Last Updated
2026-08-02
2026-08-03
## Session 77 (2026-08-03) — Settlement was dead for two days; scoreboard now readable ✅
Specs: `specs/challenger-scoreboard.md`, `specs/odds-429-diagnosis.md`.
4,159 tests / 332 suites green, web build exit 0.
- **THE FIND.** Three challenger axes read exactly ZERO settled rows. Not low —
zero, on games played days earlier, with `settle_attempts = 0`. `settleLedger`
refetched rows via `.in('id', ids)`; 500 UUIDs = an 18,499-char URL the fetch
layer rejects, and the result was destructured with no error binding, so it
returned all-zeros indistinguishable from a clean "nothing to settle".
Volume-triggered: 2026-08-01 was the first day past the 500-row limit.
The zero-settle ops alarm reads those same return values and was blind to it.
- **FIXED + DRAINED.** One query, all columns, no id list; failed fetches surface.
`captureClosing` chunked at 100 (same defect family). 1,444 rows from 08-01
settled (1,376 hit/miss + 68 void, 0 remaining). Settled n **493 → 1,741**.
- **SCOREBOARD — nothing promoted, nothing earned it.** arch-v1 n=1,741 Δ0.0000
CI[0.0050,+0.0054] (moves 76% of rows by 2.5pp mean = active movement carrying
no information); contact-v1 n=1,055 +0.0008 inconclusive; proj-v1.1 ladder
n=1,664 **0.0301 CI[0.0543,0.0060] = reliably WORSE**. matchup/tb-v1/hits-v1
STILL PENDING (rows dated 08-02+, settle after ET midnight). Champion
byte-identical; all challengers stay wired.
- **429 DIAGNOSED (read-only) — premise refuted with numbers.** PropLine 5/3,000
daily (0.17%); the 429 is **odds-api at 478/500 monthly, blocked at 95%**,
surfacing whenever PropLine returns empty. One snapshot = ONE PropLine call per
sport. Change-based pull is NOT the fix and no tier upgrade is needed. Could
NOT verify PropLine movement endpoints (auth-gated docs, prod-only keys) — not
asserted. Book-breadth invariant recorded: we never discard books; DFS is
excluded from PRICING only.
- **Next:** the silent PropLine fall-through (empty slate must not report the
backup's 429); diagnose the projection family's INPUTS (two independent
measurements now say it trails the champion).
## Session 76 (2026-08-02) — hits-v1: a challenger built, measured, and REFUTED ✅
Spec: `specs/hits-v1-binomial.md`. 4,156 tests / 332 suites green, web build exit 0.
+51
View File
@@ -1132,6 +1132,57 @@ phased plan in the Session-57 conversation / BUILD-STATE Next section).
`zmdnczhtdxcddsxzttub`), so local scripts hitting Supabase need an explicit
`SUPABASE_URL=` override. Prod + the MCP connection are fine.
## Settlement outage + challenger scoreboard (Session 77 — non-obvious)
- **`.in('id', [...])` IS A URL, NOT A QUERY.** PostgREST puts filters in the
URL: 500 UUIDs = an 18,499-char request that the fetch layer rejects with
`TypeError: fetch failed`. This silently killed `settleLedger` for two days
(2026-08-01/02) — it refetched rows by id, destructured `const { data: rows }`
with NO error binding, so rows was null, the loop never ran, and it returned
`{settled:0,voided:0,unrecoverable:0,pending:0}`, byte-identical to a healthy
"nothing to settle". 1,444 rows sat with `settle_attempts=0`. NEVER send an
unbounded id list; `ID_FILTER_CHUNK` (100) is the guard, and settleLedger now
selects every column it needs in ONE query.
- **It was VOLUME-TRIGGERED, which is why it hid.** Daily volume ran 20260 rows
for weeks; 2026-08-01 was the first day past `SETTLE_FETCH_LIMIT` (500). If a
pipeline "works for weeks then stops", suspect a threshold that volume just
crossed, not a code change.
- **The ops alarm was blind to it BY CONSTRUCTION.** The zero-settle watchdog
reads settleLedger's own return values, so `pending: 0` told it the backlog was
empty. An alarm that trusts the return value of the thing it watches cannot see
that thing fail silently — the signal must come from OUTSIDE (a direct
`game_date < today AND outcome IS NULL` count).
- **Settled n went 493 → 1,741 the moment it was fixed.** Anything reading
"n-blocked" across MULTIPLE independent challengers at once is a pipeline
symptom, not a sampling fact. Count settled rows before believing it.
- **`specs/challenger-scoreboard.md` is the board.** Nothing promoted: arch-v1
Δ0.0000 CI[0.005,+0.005] on 1,741 (and it MOVED 76% of rows by 2.5pp mean —
active movement carrying zero information), contact-v1 +0.0008 inconclusive,
proj-v1.1 ladder 0.0301 CI excluding zero = reliably WORSE. matchup/tb-v1/
hits-v1 genuinely pending (rows dated 08-02+, settle after ET midnight).
- **arch-v1/contact-v1 need no replay** — they wrote p_win at grade time into
their own columns, so scoring them is a TRUE prospective holdout. Only a
challenger that did not exist at grade time (hits-v1) needs a point-in-time
replay. Don't conflate the two kinds of evidence.
- **Score a nudge on the rows it MOVED**, not on all rows — otherwise the
unmoved rows are the champion measured against itself and dilute any real
effect toward zero. `scripts/challenger-scoreboard.js` does both slices.
## The 429 is odds-api, NOT PropLine (Session 77 — non-obvious)
- **`specs/odds-429-diagnosis.md`.** MEASURED: PropLine 5/3,000 daily (0.17%);
odds-api 478/500 MONTHLY, `allowed:false` (tracker blocks at 95%). One snapshot
= ONE PropLine call per sport (all markets comma-joined) — there is no
per-prop/per-book fan-out and no request-pattern problem to optimize.
- **The 429 text is the BACKUP's.** `oddsService.getOdds` falls through silently
when PropLine returns null/empty, then odds-api's quota gate throws
`429 "Odds data temporarily unavailable"`. So an EMPTY PropLine slate is
indistinguishable from an outage, and the error names the wrong provider. Open
order — don't read a 429 as "PropLine exhausted" without checking
`GET /api/internal/quota`.
- **BOOK BREADTH INVARIANT: we never discard books.** All books are KEPT and
SHOWN (`DISPLAY_BOOKS = MODEL REFERENCE DFS`). The ONLY selectivity is that
DFS pick'em is excluded from PRICING/consensus (`EXCLUDED_FROM_PRICING`) — a
fixed-payout shaded number is not a market price. Never "clean up" breadth.
## Active Skills
- vyndr-voice (all user-facing output)
- prop-analysis (grading methodology)
+191
View File
@@ -0,0 +1,191 @@
#!/usr/bin/env node
'use strict';
/**
* challenger-scoreboard every accruing challenger, measured on the same bar.
*
* WHY THIS IS A REAL HOLDOUT AND NOT A BACKTEST. Unlike hits-v1 (which did not
* exist when these rows were graded and therefore needed a point-in-time
* replay), arch-v1 and contact-v1 wrote their probability AT GRADE TIME, into
* their own columns, before the game was played. Nothing here is recomputed.
* These numbers are genuinely out-of-sample the strongest evidence available.
*
* THE BAR IS THE SAME ONE THAT REFUTED hits-v1:
* - the challenger's OWN rows only (a challenger that abstains is not scored
* on the rows it declined averaging those in measures the champion twice)
* - direction handled: p_win and p_win_challenger/p_win_contact are all
* P(GRADED SIDE), so they are already aligned. The projection ladder is
* P(OVER) and IS realigned here.
* - paired bootstrap on matched rows, because both models score the SAME rows
* and treating their errors as independent overstates the uncertainty
* - PROMOTE only when the CI on (challenger champion) excludes zero
*
* CONTAMINATION EXCLUSION: rows whose price/book were stamped from a
* non-takeable book are tagged `quarantine_reason LIKE 'nontakeable_book%'` and
* are excluded their locked price describes a market you could not have bet.
*
* PROVENANCE: results are reported for all rows AND split by `model_version`,
* so if a verdict depends on the older `pre-retention-unknown` era that fact is
* visible rather than buried.
*
* SUPABASE_URL=... node scripts/challenger-scoreboard.js
*/
require('dotenv').config();
const { createClient } = require('@supabase/supabase-js');
const SB_URL = process.env.SUPABASE_URL;
const SB_KEY = process.env.SUPABASE_SERVICE_ROLE_KEY || process.env.SUPABASE_SERVICE_KEY;
const PAGE = 1000;
function corr(xs, ys) {
const n = xs.length;
if (n < 3) return null;
const mx = xs.reduce((a, b) => a + b, 0) / n;
const my = ys.reduce((a, b) => a + b, 0) / n;
let sxy = 0; let sxx = 0; let syy = 0;
for (let i = 0; i < n; i += 1) {
const dx = xs[i] - mx; const dy = ys[i] - my;
sxy += dx * dy; sxx += dx * dx; syy += dy * dy;
}
if (sxx <= 0 || syy <= 0) return null;
return sxy / Math.sqrt(sxx * syy);
}
const r4 = (v) => (v == null ? null : Math.round(v * 10000) / 10000);
const meanOf = (a) => (a.length ? a.reduce((x, y) => x + y, 0) / a.length : null);
const brier = (ps, ys) => (ps.length ? ps.reduce((s, p, i) => s + (p - ys[i]) ** 2, 0) / ps.length : null);
/** Paired bootstrap on the DIFFERENCE of resolutions. Deterministic seed. */
function bootstrapDiff(rows, keyA, keyB, iters = 4000, seed = 20260803) {
if (rows.length < 30) return null;
let s = seed >>> 0;
const rnd = () => { s ^= s << 13; s >>>= 0; s ^= s >>> 17; s ^= s << 5; s >>>= 0; return s / 4294967296; };
const n = rows.length;
const diffs = [];
for (let it = 0; it < iters; it += 1) {
const ys = []; const a = []; const b = [];
for (let i = 0; i < n; i += 1) {
const r = rows[Math.floor(rnd() * n)];
ys.push(r.won); a.push(r[keyA]); b.push(r[keyB]);
}
const ca = corr(a, ys); const cb = corr(b, ys);
if (ca == null || cb == null) continue;
diffs.push(ca - cb);
}
if (diffs.length < 100) return null;
diffs.sort((x, y) => x - y);
const q = (p) => r4(diffs[Math.floor(p * (diffs.length - 1))]);
const point = r4(corr(rows.map((r) => r[keyA]), rows.map((r) => r.won))
- corr(rows.map((r) => r[keyB]), rows.map((r) => r.won)));
const ci = [q(0.025), q(0.975)];
return { point, ci95: ci, p_improves: r4(diffs.filter((d) => d > 0).length / diffs.length),
ci_excludes_zero: ci[0] > 0 || ci[1] < 0 };
}
function score(rows, challKey, label) {
const ys = rows.map((r) => r.won);
const ch = rows.map((r) => r[challKey]);
const cp = rows.map((r) => r.champ);
const bs = bootstrapDiff(rows, challKey, 'champ');
let verdict = 'STILL PENDING';
if (rows.length >= 30 && bs) {
if (bs.ci_excludes_zero && bs.point > 0) verdict = 'PROMOTE';
else if (bs.ci_excludes_zero && bs.point < 0) verdict = 'STAY WIRED (measured worse)';
else verdict = 'STAY WIRED (inconclusive)';
}
return {
challenger: label,
settled_n: rows.length,
base_rate: r4(meanOf(ys)),
resolution_challenger: r4(corr(ch, ys)),
resolution_champion: r4(corr(cp, ys)),
brier_challenger: r4(brier(ch, ys)),
brier_champion: r4(brier(cp, ys)),
delta_vs_champion: bs,
verdict,
};
}
async function fetchAll(sb) {
const out = [];
for (let from = 0; ; from += PAGE) {
const { data, error } = await sb.from('ledger_entries')
.select('id, stat, side, outcome, model_version, quarantine_reason, p_win, p_win_challenger, p_win_contact, challenger_delta, contact_delta, proj_p_over_line, proj_tb_p_over, proj_hits_p_over, challenger_adjustments')
.eq('sport', 'mlb').is('user_id', null)
.in('outcome', ['hit', 'miss'])
.not('p_win', 'is', null)
.range(from, from + PAGE - 1);
if (error) throw error;
if (!data || data.length === 0) break;
out.push(...data);
if (data.length < PAGE) break;
}
return out;
}
async function main() {
if (!SB_URL || !SB_KEY) throw new Error('SUPABASE_URL / service key required');
const sb = createClient(SB_URL, SB_KEY, { auth: { persistSession: false } });
const raw = (await fetchAll(sb))
.filter((r) => !(r.quarantine_reason || '').startsWith('nontakeable_book'));
const base = raw.map((r) => ({
won: r.outcome === 'hit' ? 1 : 0,
champ: Number(r.p_win),
arch: r.p_win_challenger == null ? null : Number(r.p_win_challenger),
contact: r.p_win_contact == null ? null : Number(r.p_win_contact),
// The ladder is P(OVER); realign it to the graded side before comparing.
ladder: r.proj_p_over_line == null ? null
: (String(r.side).toLowerCase() === 'under' ? 1 - Number(r.proj_p_over_line) : Number(r.proj_p_over_line)),
tb: r.proj_tb_p_over == null ? null
: (String(r.side).toLowerCase() === 'under' ? 1 - Number(r.proj_tb_p_over) : Number(r.proj_tb_p_over)),
hits: r.proj_hits_p_over == null ? null
: (String(r.side).toLowerCase() === 'under' ? 1 - Number(r.proj_hits_p_over) : Number(r.proj_hits_p_over)),
stat: r.stat,
// Did the challenger actually MOVE this row? A nudge that leaves p_win
// untouched is the champion wearing a different name, and scoring it on
// those rows measures the champion against itself — which is exactly how a
// real effect gets averaged down to zero.
archMoved: r.challenger_delta != null && Number(r.challenger_delta) !== 0,
contactMoved: r.contact_delta != null && Number(r.contact_delta) !== 0,
era: r.model_version || 'unknown',
axes: new Set(((r.challenger_adjustments) || []).map((a) => a && a.axis).filter(Boolean)),
}));
const withKey = (k, extra = () => true) => base.filter((r) => r[k] != null && Number.isFinite(r[k]) && extra(r));
const board = [
score(withKey('arch'), 'arch', 'arch-v1 (market-relative nudge)'),
score(withKey('contact'), 'contact', 'contact-v1 (season contact quality)'),
score(withKey('ladder'), 'ladder', 'proj-v1.1 ladder (all stats)'),
score(withKey('tb', (r) => r.stat === 'total_bases'), 'tb', 'tb-v1 (total_bases only)'),
score(withKey('hits', (r) => r.stat === 'hits'), 'hits', 'hits-v1 (hits only)'),
];
// THE SHARPEST TEST OF A NUDGE — only the rows it actually moved.
const movedBoard = [
score(withKey('arch', (r) => r.archMoved), 'arch', 'arch-v1 · rows it MOVED only'),
score(withKey('contact', (r) => r.contactMoved), 'contact', 'contact-v1 · rows it MOVED only'),
];
// Per-AXIS: arch-v1 restricted to the rows where that axis actually fired.
const axisBoard = ['environment', 'opportunity', 'matchup'].map((ax) =>
score(withKey('arch', (r) => r.axes.has(ax)), 'arch', `arch-v1 · ${ax} axis rows only`));
// PROVENANCE split — does any verdict depend on the older era?
const eras = [...new Set(base.map((r) => r.era))];
const provenance = eras.map((era) => ({
era,
...score(withKey('arch', (r) => r.era === era), 'arch', `arch-v1 · ${era}`),
}));
console.log(JSON.stringify({
measurement: 'PROSPECTIVE HOLDOUT — challenger values were written at grade time, before the game. No recomputation, no lookahead.',
total_settled_rows: base.length,
board, movedBoard, axisBoard, provenance,
}, null, 2));
process.exit(0);
}
main().catch((e) => { console.error(e); process.exit(1); });
+109
View File
@@ -0,0 +1,109 @@
# CHALLENGER SCOREBOARD — 2026-08-03
> **Nothing was promoted. Nothing earned it yet.** Not because the bar was held
> too high, but because no challenger's confidence interval excludes zero on the
> good side. The champion is byte-identical; every challenger stays wired.
The headline of this session is not the scoreboard. It is that **the scoreboard
was unmeasurable until a two-day-old settlement outage was found and fixed** —
see §1. Settled sample went **493 → 1,741** the moment it was repaired.
---
## 1. Why "n-blocked" was the wrong diagnosis
The order said: don't repeat "n-blocked" without counting. Counting is what found
the real problem.
Three of the four axes read **exactly zero** settled rows — not low, *zero*:
environment 1,496 rows / 0 settled, opportunity 922 / 0, matchup 603 / 0. Rows
whose games had been **played days earlier** and never settled, with
`settle_attempts = 0` — never even attempted.
**Root cause:** `settleLedger` fetched open ids, then refetched full rows via
`.in('id', ids)`. PostgREST puts filters in the URL, so 500 UUIDs became an
**18,499-character request** that the fetch layer rejects with `TypeError: fetch
failed`. The result was destructured as `const { data: rows } = ...` with **no
error binding**, so `rows` came back null, the loop never ran, and the function
returned `{settled:0, voided:0, unrecoverable:0, pending:0}` — byte-identical to
a healthy "nothing to settle."
It hid for two days because it is **volume-triggered**: daily volume ran 20260
rows and settled perfectly for weeks. **2026-08-01 was the first day past the
500-row fetch limit** and settlement died that night. Worse, the zero-settle ops
alarm reads these same return values, so `pending: 0` told the watchdog the
backlog was empty — *the alarm built to catch exactly this could not see it.*
Fixed, deployed, and drained: **1,444 rows from 2026-08-01 settled (1,376
hit/miss + 68 void, 0 remaining).** `captureClosing` carried the same shape one
level down and is now chunked at 100 ids.
## 2. THE SCOREBOARD
Bar: the challenger's **own rows only**, direction-aligned, **paired bootstrap**
(4,000 resamples, deterministic seed) on the difference in resolution, because
both models score the same rows and independent standard errors would overstate
certainty. **PROMOTE requires the CI to exclude zero on the good side.** Same bar
that refuted hits-v1 — no lighter test for a would-be winner.
| challenger | settled n | resolution (chal / champ) | Δ vs champion | CI95 | verdict |
|---|---|---|---|---|---|
| arch-v1 (market-relative nudge) | **1,741** | 0.4599 / 0.4599 | 0.0000 | [0.0050, +0.0054] | **STAY WIRED** (inconclusive) |
| arch-v1 · rows it MOVED only | 1,325 | 0.4886 / 0.4886 | 0.0001 | [0.0064, +0.0061] | **STAY WIRED** (inconclusive) |
| contact-v1 (season contact quality) | **1,055** | 0.4070 / 0.4061 | +0.0008 | [0.0052, +0.0069] | **STAY WIRED** (inconclusive) |
| contact-v1 · rows it MOVED only | 511 | 0.3966 / 0.3948 | +0.0018 | [0.0107, +0.0144] | **STAY WIRED** (inconclusive) |
| proj-v1.1 ladder (all stats) | **1,664** | 0.4339 / 0.4640 | **0.0301** | **[0.0543, 0.0060]** | **STAY WIRED** (measured WORSE) |
| arch-v1 · environment axis rows | 871 | 0.5651 / 0.5679 | 0.0028 | [0.0089, +0.0036] | **STAY WIRED** (inconclusive) |
| arch-v1 · opportunity axis rows | 539 | 0.5311 / 0.5310 | +0.0001 | [0.0091, +0.0090] | **STAY WIRED** (inconclusive) |
| arch-v1 · matchup axis rows | **0** | — | — | — | **STILL PENDING** |
| tb-v1 (total_bases only) | **0** | — | — | — | **STILL PENDING** |
| hits-v1 (hits only) | **0** | — | — | — | **STILL PENDING** (refuted by replay, `specs/hits-v1-binomial.md`) |
**These are true prospective holdouts, not backtests.** arch-v1 and contact-v1
wrote their probability at grade time, into their own columns, before the game
was played. Nothing was recomputed. That is the strongest evidence available and
it is why no replay was needed here.
## 3. What the numbers actually say
**arch-v1 moves a lot and changes nothing.** It moved **1,325 of 1,741 rows
(76%)**, mean absolute move **2.5 points**, max 10.9 — and resolution is
identical to the champion to four decimal places, on the moved rows too. This is
not "too small to detect." It is movement that carries **no information about the
outcome**. A nudge this active with an effect this precisely zero is a finding,
not a pending verdict.
**The projection ladder is reliably worse than the champion.** 0.0301 with a CI
excluding zero, across 1,664 rows and all stats. Combined with hits-v1's refutation
(`specs/hits-v1-binomial.md`), the projection family now has two independent
measurements pointing the same way: it is not the champion's equal on any stat
measured so far. That is an argument for diagnosing its *inputs*, not for shipping
another variant of it.
**Three are genuinely pending, for a legitimate reason now.** matchup, tb-v1 and
hits-v1 all have rows written only on 2026-08-02/03, which settle after ET
midnight. matchup has ~496 rows queued, tb-v1 65, hits-v1 pending its first
snapshot write. They will read within a day or two — and now that settlement
works, they actually will.
## 4. Provenance
All 1,741 arch-v1 rows carry a single `model_version` (`engine1@2026-07-20`). The
older `pre-retention-unknown` rows (320 settled) carry no challenger values at
all, so they cannot influence any verdict. **No verdict here depends on
mixed-provenance rows** — the split was checked, not assumed.
Contamination excluded throughout: `quarantine_reason LIKE 'nontakeable_book%'`.
## 5. Promotion mechanics — specified, deliberately unused
No flip was performed because nothing qualified. When one does, the shape is:
challenger-first (write the promoted value into the served path while the
champion column keeps recording), version-tagged, atomic, with the previous
version one env flag away. Recorded here so a future promotion is a decision,
not an improvisation.
## 6. Reproduce
`SUPABASE_URL=... node scripts/challenger-scoreboard.js` — prints the full board,
the moved-rows-only slice, the per-axis slice and the provenance split.
+145
View File
@@ -0,0 +1,145 @@
# THE 429 — DIAGNOSED, AND IT IS NOT WHAT WE THOUGHT
**Read-only investigation, 2026-08-03.** Nothing was implemented, no tier bought.
> **VERDICT: the change-based pull is NOT the fix, and no tier upgrade is
> needed. There is no request-pattern problem to solve.** PropLine is running at
> **5 calls out of 3,000/day — 0.17% of the free allowance.** The 429 comes from a
> different provider entirely.
---
## 1. BOOK BREADTH — the invariant, written down (and already true in code)
**WE DO NOT DISCARD BOOKS.** All books are KEPT and SHOWN. More breadth is
strictly good for display, line-shopping and consensus. The **only** selectivity
is that **DFS pick'em (prizepicks / underdog / sleeper / dabble) does not vote on
the fair-price CONSENSUS**, because a fixed-payout shaded number is not a market
price. That is one category exclusion from *pricing*, not a reduction in breadth.
Verified in `src/config/bookRoles.js` — this is already exactly what the code
does, and the invariant is recorded so no future "cleanup" undoes it:
```
DISPLAY_BOOKS = MODEL_BOOKS REFERENCE_BOOKS DFS_PLATFORMS ← everything shows
EXCLUDED_FROM_PRICING = DFS_PLATFORMS OFFSHORE_OR_INTL ← pricing only
```
The historical real discarding — `ALLOWED_BOOKS` cutting the slate to 5 keys and
making 64.8% of MLB props invisible — was fixed in Order Zero and must not
return. No language in this repo should describe book handling as "discarding".
## 2. Post-reset status (STEP 1) — the pipeline RECOVERED
```
GET /api/internal/snapshot/status → overdue: false
mlb 2026-08-03T01:00:24Z gradeCount 235
wnba 2026-08-03T01:02:58Z gradeCount 139
```
The 14/19/22 UTC slots on 2026-08-02 all failed; the 01:00 UTC slot after the
reset succeeded. So the failure was time-bounded and self-healing — which already
argued against a structural request-pattern defect.
## 3. What one snapshot actually requests (STEP 2)
Measured in `proplineAdapter.fetchRaw`: **one snapshot = ONE PropLine HTTP call
per sport.** All markets ride the same request, comma-joined (12 markets for MLB).
There is no per-prop, per-book or per-event fan-out.
```
snapshot 1 call × 2 active sports × 5 slots/day ≈ 10 calls/day
intraday refresh every 20 min, slate hours only (12h) ≈ 72 calls/day
public /api/odds cache-aside, 1h TTL bounded, small
── total order-of-10s/day
```
**MEASURED, not estimated** — `GET /api/internal/quota`, 2026-08-03:
| provider | used | limit | period | allowed |
|---|---|---|---|---|
| **propline** | **5** | **3,000** | daily | ✅ true |
| **odds-api** | **478** | **500** | **monthly** | ❌ **false (95.6%)** |
| oddspapi | 0 | 1,000 | monthly | true |
| tank01 | 86 | 1,000 | monthly | true |
**There is no waste to reclaim.** We are using 0.17% of the free PropLine
allowance. Re-pulling "the full picture" every slot costs one request; a
change-based pull would replace one request with one request.
## 4. Where the 429 actually comes from
`oddsService.getOdds` tries PropLine first, and **falls through silently** when it
returns null or an empty array. The 429 is thrown *after* that, by the **odds-api
backup's** quota gate:
```js
const quotaStatus = await quotaTracker.getQuotaStatus('odds-api');
if (!quotaStatus.allowed) {
const error = new Error('Odds data temporarily unavailable. Try again later.');
error.statusCode = 429; // ← THIS is the 429 we saw
```
So the real sequence on 2026-08-02 was:
1. PropLine returned **no props** for that moment (it had 2,995 calls remaining —
this was an empty/failed slate, **not** exhaustion),
2. control fell through to odds-api,
3. odds-api sits at **478/500 for the month** and the tracker blocks at ≥95%,
4. → user-facing `429 Odds data temporarily unavailable`.
**The 429 is the dead backup announcing itself.** It is not a PropLine quota
signal, and tuning PropLine request volume cannot affect it.
### The real defect this exposes
**An empty PropLine slate is indistinguishable from a provider outage.** A
legitimately empty board (off-hours, no props posted yet) produces the identical
429 as a genuine failure, because the fall-through is silent and the error text
belongs to whichever provider happened to be asked last. That is a
truth-in-reporting bug in the odds path — the operator cannot tell "no games" from
"we are broken." Flagged for its own order; **not** fixed here (read-only).
## 5. Change-based capture (STEP 3) — NOT VERIFIED, and moot for quota
**I could not verify whether PropLine supports `/movement`, odds-history, or a
`since=` parameter.** PropLine's docs are auth-gated (HTTP 401) and the API keys
are production-only — not present in the local environment. **I am not going to
assert an endpoint exists or costs less when I could not test it**; "documented ≠
verified" cuts both ways.
What can be said without testing anything:
- **For quota, it is moot.** At 5/3,000 there is nothing to optimise. A
change-based pull would save at most a handful of calls per day out of an
allowance we are not close to touching.
- **The movement-data argument stands on its own, and is the better reason.** A
change-based capture records line movement as `from → to at time T`, which is
genuine model input we currently lack: steam detection, true CLV attribution,
and the market-movement axis. Today `intradayRefreshService.trackHistory`
reconstructs a coarse version of this by diffing full pulls. If PropLine
exposes movement natively it is worth building **as a data feature** — and it
should be justified that way, not as a quota fix it isn't.
## 6. VERDICT (STEP 4), with the numbers
| question | answer |
|---|---|
| Did the pipeline recover post-reset? | **Yes**`overdue: false`, mlb 235 grades at 01:00Z |
| Requests per snapshot | **1 per sport**, all markets in one call |
| Requests/day vs allowance | **~10s/day vs 3,000/day free — 0.17% used** |
| Is the 429 a request-pattern problem? | **No.** Measured, decisively |
| Is the 429 a PropLine quota problem? | **No.** 2,995 calls remaining, `allowed: true` |
| What is the 429? | **odds-api backup at 478/500 monthly, blocked at 95%** |
| Does change-based pull fix it? | **No** — it cannot; wrong provider, and no volume problem |
| Is a tier upgrade needed? | **No.** Not for PropLine. Nothing is constrained |
**What actually deserves an order**, in priority sequence:
1. **The silent fall-through** — an empty PropLine slate must report "no props
available", never the backup's quota 429 (§4).
2. **odds-api at 478/500 monthly** — it is a dead backup for the rest of August.
Either accept it (PropLine is primary and healthy) and stop letting its quota
gate speak for the whole odds path, or budget it deliberately.
3. **Movement capture as a DATA feature** — worth scoping on its own merits, with
PropLine's endpoint support verified first from an environment that has keys.