Quota guard: close the silent odds-api drain + reserve floor for MLB

Diagnosis (why 500/500 went unpaged): the only regular odds-api burner was
futuresService, which called axios DIRECTLY — bypassing the gateway, so it
never hit recordCall (the ONE place the WARN/BLOCK pager fires) and never
respected the 95% block. It only syncFromHeaders, which updated the counter's
number SILENTLY. oddsService (which does go through the gateway) only touches
odds-api when PropLine fails, so recordCall for odds-api effectively never ran.
Result: the counter could reach 100% with neither pager firing.

Fixes (a silent drain is now impossible, not just guarded):
- futuresService routes through gateway.fetch('odds-api', …) → counted, blocked
  at 95%, and reserve-gated. Closes the raw-axios bypass.
- Reserve floor in the gateway: a DISCRETIONARY call (futures/soccer) passes
  reserve=ODDS_API_RESERVE (default 50) and is refused while remaining <= reserve.
  The ESSENTIAL MLB prop-backup passes no reserve and may spend to the 95% block.
  → a futures/soccer drain can NEVER starve MLB's backup path.
- quotaTracker.syncFromHeaders (the AUTHORITATIVE number) now fires the same
  once-per-period WARN/BLOCK alert on a crossing — extracted fireThresholdAlert
  shared with recordCall. The header-only drain now pages.
- POST /api/internal/quota/test-alert (internal-key) test-fires the pager
  end-to-end so ntfy delivery is verifiable on demand.

Also (reality-corrected cadence): WNBA restored to the full grid. 2026-07-15
had two AFTERNOON WNBA games finished before the 22 UTC slot — 14 UTC (10am ET)
is the only slot early enough for a 1pm ET game's props, and on PropLine the
extra slots cost a rounding error. Soccer stays the only trimmed sport (the
real odds-api discipline). Assumption corrected by observed data.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Kev
2026-07-15 18:07:14 -04:00
parent 4cd933d83e
commit 2d413cfe1e
8 changed files with 191 additions and 61 deletions
+16 -3
View File
@@ -43,7 +43,19 @@ class QuotaExhaustedError extends Error {
}
}
async function tryOne(providerId, callbackFn, syncHeadersFrom) {
async function tryOne(providerId, callbackFn, syncHeadersFrom, reserve = 0) {
// Reserve floor (Job 1 / quota guard) — a DISCRETIONARY call (futures,
// soccer outrights) is refused while fewer than `reserve` credits remain, so
// it can never drain the last credits that the ESSENTIAL path (MLB prop
// backup when PropLine fails) depends on. Essential calls pass reserve=0 and
// use the quota down to the normal 95% block. Checked BEFORE the optimistic
// increment so we don't consume-then-refund. Degraded Redis fails open.
if (reserve > 0) {
const pre = await quotaTracker.getQuotaStatus(providerId);
if (pre && !pre.degraded && Number.isFinite(pre.remaining) && pre.remaining <= reserve) {
return { ok: false, reason: `reserve_floor(${pre.remaining}<=${reserve})`, status: pre };
}
}
// Optimistic increment — if the call throws we roll back below.
// recordCall also evaluates the post-increment threshold; if the
// very next call would put us at 95%+, we still execute THIS one
@@ -89,10 +101,11 @@ async function fetch(primaryId, callbackFn, opts = {}) {
sport,
fallbackProviders,
syncHeadersFrom,
reserve = 0,
} = opts;
const attempts = [];
const result = await tryOne(primaryId, callbackFn, syncHeadersFrom);
const result = await tryOne(primaryId, callbackFn, syncHeadersFrom, reserve);
if (result.ok) return result.result;
// Generic adapter error on the primary — propagate, don't shift.
@@ -112,7 +125,7 @@ async function fetch(primaryId, callbackFn, opts = {}) {
: [];
for (const fallbackId of chain) {
const fb = await tryOne(fallbackId, callbackFn, syncHeadersFrom);
const fb = await tryOne(fallbackId, callbackFn, syncHeadersFrom, reserve);
if (fb.ok) {
console.log(`[gateway] primary=${primaryId} blocked; succeeded via fallback=${fallbackId}`);
return fb.result;