Session 20: Provider intelligence — quota tracker, gateway with fallback cascade, admin quota dashboard (1476 tests)
This commit is contained in:
+186
-1
@@ -4,7 +4,192 @@
|
||||
2026-06-12
|
||||
|
||||
## Current Phase
|
||||
SHIP BUILD v19.0 — Sports experience overhaul: player cards, game-card redesign, scan page revamp (Session 19)
|
||||
SHIP BUILD v20.0 — Provider intelligence: quota tracker, gateway, key rotation (Session 20)
|
||||
|
||||
## Session 20 (2026-06-12) — SHIPPED
|
||||
|
||||
Built the data-pipeline backbone: a per-provider quota tracker, a
|
||||
unified gateway that routes through fallback providers when one
|
||||
approaches its limit, and structural visibility into all of it via
|
||||
the admin dashboard. This is the infrastructure that prevents the
|
||||
"odds-api at 0/500 → all sports 503" incident from happening again.
|
||||
|
||||
### PHASE 1 — Provider registry
|
||||
|
||||
`src/config/providers.js` enumerates the six providers VYNDR talks
|
||||
to (the-odds-api, ODDSPAPI, ParlayAPI, Tank01, API-Football,
|
||||
Football-Data.org). Each entry declares envKey, quotaType
|
||||
(`monthly`|`daily`|`per_minute`), quotaLimit, sports, capabilities,
|
||||
and priority. Exports `getProvider`, `listProviderIds`,
|
||||
`getConfiguredProviders`, `getFallbackChain(capability, sport,
|
||||
excludeId)`. Thresholds (WARN 80%, BLOCK 95%) live in the same
|
||||
module so the tracker and gateway can't drift.
|
||||
|
||||
`src/server.js` now logs which providers have keys at boot:
|
||||
`[VYNDR] providers configured (4): odds-api, tank01, api-football,
|
||||
football-data` and warns about any with missing keys.
|
||||
|
||||
### PHASE 2 — Quota tracker
|
||||
|
||||
`src/services/quotaTracker.js` is the Redis-backed counter. Keys:
|
||||
- `quota:{provider}:{period}` → `{used, limit, syncedAt}`
|
||||
- `quota_warned:{provider}:{period}` → dedupe the 80% log line
|
||||
|
||||
Period format is quota-type-driven: `YYYY-MM` for monthly,
|
||||
`YYYY-MM-DD` for daily, `YYYY-MM-DDTHH:MM` for per-minute. UTC so
|
||||
operators in different timezones see one consistent picture.
|
||||
|
||||
API:
|
||||
- `getQuotaStatus(provider)` — read without mutation
|
||||
- `recordCall(provider)` — increment + return new status
|
||||
- `rollback(provider)` — decrement after a failed call
|
||||
- `syncFromHeaders(provider, headers)` — truth-source override
|
||||
from upstream response headers (odds-api returns
|
||||
`x-requests-used` + `x-requests-remaining`)
|
||||
- `getAllQuotaStatuses()` — snapshot for the dashboard
|
||||
- `getTickInterval(pct)` — scheduler step function
|
||||
(<50% → 5min, <80% → 15min, <95% → 30min, ≥95% → null)
|
||||
- `shouldThrottle(provider)` — composite for schedulers
|
||||
|
||||
**Degraded mode** — when Redis is down, the tracker fails OPEN
|
||||
(`allowed: true, degraded: true`) rather than closed. The
|
||||
alternative (degrade closed) would mean a Redis blip blocks every
|
||||
provider call platform-wide, which is worse than the original
|
||||
quota-exhaustion bug.
|
||||
|
||||
21 unit tests cover period keys, recordCall counting,
|
||||
syncFromHeaders truth-source override, the 80% warning dedupe,
|
||||
threshold flips, rollback, getTickInterval steps, and degraded-mode
|
||||
fail-open.
|
||||
|
||||
### PHASE 3 — Provider gateway
|
||||
|
||||
`src/services/providerGateway.js` is the single entry point every
|
||||
external-data call passes through:
|
||||
|
||||
```
|
||||
const result = await gateway.fetch('odds-api', cb, {
|
||||
capability: 'odds',
|
||||
sport: 'nba',
|
||||
fallbackProviders: ['oddspapi'], // optional
|
||||
syncHeadersFrom: (r) => r.headers, // optional
|
||||
});
|
||||
```
|
||||
|
||||
Flow: check quota → invoke callback → on quota block, walk the
|
||||
fallback chain (explicit or capability-derived) → on full
|
||||
exhaustion throw `QuotaExhaustedError` with the attempt log so
|
||||
operators can see what was tried. The callback receives the
|
||||
provider ID it's running under so adapter code can pick the right
|
||||
base URL / API key per fallback.
|
||||
|
||||
**Critical safety property:** only QUOTA failures trigger fallover.
|
||||
A generic upstream error (network blip, 502) propagates from the
|
||||
primary instead of silently shifting the whole platform to the
|
||||
fallback. That mask was the symptom that hid the original outage.
|
||||
|
||||
Wired into `oddsService.fetchEventsFromApi` +
|
||||
`fetchEventOddsFromApi`. The gateway's `syncHeadersFrom`
|
||||
callback pumps `x-requests-used` / `x-requests-remaining` straight
|
||||
into the tracker on every successful odds-api response.
|
||||
|
||||
8 unit tests cover happy path, single-fallback walk,
|
||||
multi-fallback skip, explicit chain override, full exhaustion,
|
||||
adapter-error propagation, and header sync invocation.
|
||||
|
||||
### PHASE 4 — Scheduler hooks
|
||||
|
||||
`getTickInterval(pct)` exposed for any future polling code. Wired
|
||||
into `poller/soccer.js` — each tick checks
|
||||
`quotaTracker.shouldThrottle('football-data')` and skips if quota
|
||||
is exhausted (logs `tick skipped — football-data quota exhausted`).
|
||||
|
||||
**Honest scope flag:** the NBA/WNBA/MLB pollers hit ESPN
|
||||
scoreboards (no quota), so they don't need wiring. The spec
|
||||
implied a generic poller that hits odds-api on a schedule; that
|
||||
poller doesn't exist — odds-api is on-demand-cached at 15min in
|
||||
oddsService. The gateway + recordCall on every odds-api call gives
|
||||
the same effect (per-call quota enforcement) without a separate
|
||||
scheduler.
|
||||
|
||||
### PHASE 5 — Admin integration
|
||||
|
||||
`GET /api/internal/quota` added to `src/routes/internal.js`. Uses
|
||||
the existing `requireInternalAuth({loopbackOnly:false})` gate so
|
||||
the Next.js admin route proxies through with the shared key.
|
||||
|
||||
`web/src/app/api/admin/stats/route.ts` now also fetches the quota
|
||||
snapshot (best-effort, 4s timeout, surfaces missing-key as a note
|
||||
instead of blanking the dashboard).
|
||||
|
||||
`web/src/app/admin/page.tsx` renders a **Provider quotas** table:
|
||||
provider name + period, used/limit + usage bar, quota type
|
||||
(`monthly|daily|/min`), status indicator (`✅ 18%`, `⚠️ 82%`,
|
||||
`❌ BLOCKED 97%`). Bar color tracks the threshold (green < 80,
|
||||
yellow 80-95, red ≥ 95). Table hides when no providers reported.
|
||||
|
||||
3 new integration tests on the `/quota` endpoint: rejects without
|
||||
internal key, returns snapshot when keyed, returns 500 on tracker
|
||||
error.
|
||||
|
||||
### PHASE 6 — Header sync into tracker
|
||||
|
||||
`oddsService.updateQuota` now also lazily-requires the tracker and
|
||||
calls `syncFromHeaders('odds-api', headers)` so the new counter
|
||||
stays current alongside the legacy hash-based quota in Redis. The
|
||||
gateway's `syncHeadersFrom` already does this on each call — the
|
||||
`updateQuota` hook is belt-and-suspenders for any call path that
|
||||
bypasses the gateway in the future.
|
||||
|
||||
### Honest scope flags
|
||||
|
||||
- Only `oddsService` is wired through the gateway. Tank01,
|
||||
API-Football, and Football-Data adapters still call axios
|
||||
directly. They can be migrated by wrapping their existing axios
|
||||
calls in `gateway.fetch(<providerId>, () => axios.get(...), {
|
||||
capability, sport })` — no upstream contract change. Holding
|
||||
off this session to avoid blast radius on stable adapter code;
|
||||
the gateway + tracker stand alone and are ready when needed.
|
||||
- The Provider Quotas tile renders only the providers whose keys
|
||||
are present on the Express side. If a key is set in prod but
|
||||
unset locally, the local admin view will look thinner than
|
||||
prod — by design.
|
||||
- "Smart scheduler" is wired only for the soccer poller (the one
|
||||
poller that does hit a quota'd provider). The other PM2
|
||||
pollers don't need it.
|
||||
|
||||
### Battery
|
||||
|
||||
- Express suite: **114 passed / 1476 tests** (+32 over baseline
|
||||
1444; 21 quotaTracker + 8 providerGateway + 3 /quota
|
||||
integration). Two pre-existing test files needed their redis
|
||||
mocks extended with `cacheGet`/`cacheSet`/`isDegraded` for the
|
||||
gateway path; degraded-mode fail-open preserves their
|
||||
axios-driven assertions.
|
||||
- Web build: **clean** — `/admin` + `/api/admin/stats` register as
|
||||
dynamic; no TS errors.
|
||||
|
||||
### Files changed (Session 20)
|
||||
|
||||
**Created:**
|
||||
- `src/config/providers.js`
|
||||
- `src/services/quotaTracker.js`
|
||||
- `src/services/providerGateway.js`
|
||||
- `tests/unit/quotaTracker.test.js`
|
||||
- `tests/unit/providerGateway.test.js`
|
||||
|
||||
**Modified:**
|
||||
- `src/services/oddsService.js` — gateway wrap + tracker sync
|
||||
- `src/routes/internal.js` — `/api/internal/quota` endpoint
|
||||
- `src/server.js` — startup provider log
|
||||
- `poller/soccer.js` — quota-aware tick
|
||||
- `tests/unit/oddsService.test.js` — mock extension
|
||||
- `tests/integration/odds.test.js` — mock extension
|
||||
- `tests/integration/internalRoutes.test.js` — `/quota` coverage
|
||||
- `web/src/app/api/admin/stats/route.ts` — provider_quotas tile
|
||||
- `web/src/app/admin/page.tsx` — Provider quotas table
|
||||
|
||||
---
|
||||
|
||||
## Session 19 (2026-06-12) — SHIPPED
|
||||
|
||||
|
||||
Reference in New Issue
Block a user