builtbykev
d09a06c054
Product overhaul core — the two transformative, differentiated systems: Self-learning loop (Phase 2): outcomeService settles locked snapshot grades against real MLB Stats API results → hit/miss/push, rolling accuracy by grade tier (30d window). Idempotent, injectable, unit-tested. New GET /api/accuracy + /api/ledger/accuracy + internal settle triggers + cron hook. AccuracyBadge (dashboard/scan/landing) is honest — "LEARNING" below MIN_SAMPLE, never a fake number. Settled HIT/MISS chips overlay the live slate. Real-time layer (Phase 1): Slate silent 60s auto-refresh (no flash, no wipe on transient blips) + "SIGNAL LIVE · UPDATED Xs ago" freshness strip; Ticker LIVE badge that flashes on fresh events. Landing (Phase 3): TopSignals shows tonight's real top-3 A-rated grades + live accuracy — the product shown, not described. Founder pricing: FOUNDER_CODE_EXPIRY default 2026-06-30 → 2026-12-31 (had lapsed, disabling every founder code + the ClaimMeter pitch). That expiry — not a tier change — was the real cause of the 4 stripe test failures. Backend 2255 (4 failing) → 2274 (all green; +19 new, +4 fixed). Web build exit 0. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
77 lines
4.3 KiB
Markdown
77 lines
4.3 KiB
Markdown
# Spec — Self-Learning Loop (Outcome Tracking + Accuracy) — Session 55
|
|
|
|
## Problem
|
|
VYNDR grades props but never checks whether it was right. The intelligence is
|
|
static — no outcome tracking, no accuracy record, no "the system learns" signal.
|
|
This is the #1 trust builder for a product a 98K-follower bettor would stake his
|
|
reputation on: showing real accuracy, including misses.
|
|
|
|
## What we build
|
|
`outcomeService` — settles locked snapshot grades against real game results,
|
|
records hit/miss per graded prop, and aggregates a rolling accuracy record by
|
|
grade tier. Powered by the FREE MLB Stats API game log (the same source the
|
|
grade pipeline already uses); NBA/WNBA degrade to `pending` when the Python
|
|
stats service is offline (matches existing posture). Zero new dependency, zero
|
|
new paid API credits.
|
|
|
|
## Data source
|
|
`mlbStatsAdapter.getPlayerStats(name)` → `{ found, last10: [{ date, opponent, stat:{…} }] }`.
|
|
A prop is "settled" when a game-log row exists for the graded date (presence in
|
|
the log ⇒ the game is FINAL). The actual stat value is read via a settlement
|
|
map (VYNDR `stat_type` → statsapi game-log field), mirroring `MLB_LOG_FIELD`.
|
|
|
|
## Settlement logic
|
|
For each grade in `snapshot:{sport}:latest`:
|
|
- `targetDates` = the UTC date and America/New_York date of `gradedAt.timestamp`
|
|
(covers late-game ET rollover without heavy TZ math).
|
|
- Find the game-log row whose `date` ∈ `targetDates`. None ⇒ `pending` (skip).
|
|
- `actual` = settlement-map value for the graded `stat_type`. Unmappable ⇒ skip.
|
|
- `result`:
|
|
- side `over`: actual > line ⇒ `hit`; actual < line ⇒ `miss`; == ⇒ `push`
|
|
- side `under`: actual < line ⇒ `hit`; actual > line ⇒ `miss`; == ⇒ `push`
|
|
- Idempotent: outcome key = `nameKey(player)|stat|line|side|date`. A prop settles
|
|
once; re-runs never double-count.
|
|
|
|
## Persistence (Redis, no DB migration)
|
|
- `outcomes:{sport}:log` — array of settled outcomes (newest first, cap 1000,
|
|
deduped by key). Each: `{ key, player, stat, line, side, grade, actual, result, date, gradedAt, settledAt }`.
|
|
- `accuracy:{sport}` — `{ sport, updated_at, window_days:30, sample, overall:{hits,total,pushes,pct}, byGrade:{'A+':{…},'A':{…},'B':{…},'C':{…},'D':{…}} }`.
|
|
- `accuracy:overall` — same shape aggregated across sports (dashboard header).
|
|
- `pct` = hits / (hits + misses) over the trailing 30 days; pushes excluded from pct.
|
|
|
|
## Endpoints
|
|
- `GET /api/accuracy` (public, cached 5m) → `{ overall, sports:{mlb,nba,wnba,…}, updated_at }`.
|
|
- `GET /api/ledger/accuracy` (public) → `{ buckets:[{ grade, hits, total, pct }] }`
|
|
(feeds the existing Next proxy `web/src/app/api/ledger/accuracy`).
|
|
- `POST /api/internal/outcomes/:sport` and `/outcomes/all` (internal-key gated) —
|
|
trigger settlement. Same auth as the snapshot trigger.
|
|
- Cron: `settleAllOutcomes()` runs on the snapshot scheduler tick BEFORE grading
|
|
(settle yesterday's completed games, then grade today's).
|
|
|
|
## Frontend
|
|
- Dashboard header accuracy pill: "A-RATED · 68% HIT RATE (30D)" from `/api/accuracy`.
|
|
Self-hides when sample is too small (< MIN_SAMPLE).
|
|
- `GradeResultCard`: "A-rated props hit 68% of the time" line when accuracy for
|
|
that grade tier is available (passed via `gradeAdapter`).
|
|
- Snapshot prop rows: a settled prop shows `✅ HIT (2)` / `❌ MISS (1)` /
|
|
`PUSH` derived from `outcomes:{sport}:log` (via the snapshot read merge).
|
|
|
|
## Acceptance criteria
|
|
1. `settleSnapshot('mlb', deps)` with injected grades + game logs returns settled
|
|
outcomes with correct hit/miss/push for over AND under sides.
|
|
2. Settlement is idempotent — running twice does not double-count.
|
|
3. Accuracy aggregates hits/total/pct by grade tier over the 30-day window.
|
|
4. A prop with no matching game-log row stays `pending` and is excluded.
|
|
5. `GET /api/accuracy` returns the persisted record (empty-safe when cold).
|
|
6. NBA/WNBA (offline stats) degrade to `pending`, never throw.
|
|
7. All deps injectable → tests hit zero network.
|
|
|
|
## Test plan
|
|
`tests/unit/outcomeService.test.js`:
|
|
- hit/miss/push for over + under; unmappable stat skipped; missing game skipped.
|
|
- idempotency (dedupe by key across two runs).
|
|
- accuracy bucketing + pct math + 30-day window filter + MIN_SAMPLE gating.
|
|
- `getAccuracy` cold-cache empty shape.
|
|
`tests/integration/accuracy.test.js`:
|
|
- `GET /api/accuracy` and `GET /api/ledger/accuracy` return valid shapes.
|