diff --git a/BUILD-STATE.md b/BUILD-STATE.md index 05729cd..d99ebbb 100755 --- a/BUILD-STATE.md +++ b/BUILD-STATE.md @@ -1,10 +1,94 @@ # VYNDR — Build State ## Last Updated -2026-06-14 +2026-06-15 ## Current Phase -AUDIT v31.0 — Full code audit + security review + cleanup (Session 31) +SHIP BUILD v32.0 — Close audit gaps: grades pipeline + NFL/NHL wiring + +rate limiting + test-artifact cleanup (Session 32) + +## Session 32 (2026-06-15) — SHIPPED + +Surgical close of the four functional gaps Session 31's audit documented. +No new feature scope (design session runs in parallel). Backend +1695 → **1718 tests** (+23), 140 suites, zero regressions. Web build clean. +Tracked `data/` tree stays clean after a full test run (artifact fixed). + +### PHASE 1 — Grades cache writer (closes the content pipeline) +- Traced: `analyzeViaEngine1` (computeFeatures → engine1 → toLegacyShape) + produces the legacy grade shape; `contentTemplateService` reads + `grades:{sport}:{utc}` then `grades:{sport}`, accepting an array or + `{grades:[...]}`. The legacy shape already matches `normalizeGrade` — no + remap needed. Session 31 confirmed NO writer existed. +- `gradeSlateService.gradeAndCacheSlate(sport, props, opts)` — dedupes to + unique player+stat+line (cap 25, concurrency 5), grades BOTH sides + (engine1 is direction-aware) keeping the higher-confidence verdict, sorts + by confidence desc, writes `grades:{sport}` = `{grades, updated_at, + source}` TTL 2h. Injectable grader/cacheSet/now → fully unit-tested. +- Wired fire-and-forget into `oddsService.recordDownstream` (fires on a + fresh odds fetch only, never blocks the odds response). Gated by + `shouldGradeSlate()`: default ON, OFF under `NODE_ENV==='test'` (its + feature-compute fan-out otherwise pollutes the odds integration tests' + axios call-count assertions via floating promises), `GRADE_SLATE_ON_FETCH` + override (`0` = operator kill-switch). 12 tests. Self-eval 9/10. + +### PHASE 2 — NFL + NHL sport-key wiring +- **Corrected the spec's sport keys.** Spec said `football_nfl`/`hockey_nhl` + for `oddsService.SPORT_KEYS`, but that file maps to the-odds-api, whose + keys are `americanfootball_nfl` / `icehockey_nhl` (verified against the + the-odds-api sports list). The spec's values would have silently 404'd — + the exact silent-failure class this session fights. (`football_nfl`/ + `hockey_nhl` ARE correct for PropLine in `proplineAdapter`, unchanged.) +- Added nfl/nhl to `SPORT_KEYS` + `SPORT_MARKETS` (NFL_MARKETS/NHL_MARKETS). + Filled `proplineAdapter.MARKETS.nfl/nhl`. Added NHL keys + (`player_shots_on_goal`, `goalie_saves`) to `MARKET_MAP` so NHL props + don't normalize to zero in-season (same fix family as Session 31's NFL + MARKET_MAP gap). 10 tests, incl. end-to-end MARKET_MAP normalization + + off-season empty handling. Self-eval 9/10. + +### PHASE 3 — Rate limiting on public routes +- Mounted the existing `middleware/rateLimit` (`createRateLimit`, in-memory + per-IP, independent bucket per call site) via `router.use` on every public + cached router: odds + parlay = 30/min; schedule/gamelines/streaks/hotlist/ + content/lines/books = 60/min. `/api/analyze` keeps its own 10/min. +- 3 route-level tests (429 after limit, tighter odds limit, independent + per-router buckets) complementing the existing middleware-level tests. + Self-eval 9/10. + +### PHASE 4 — Test artifact + cleanup +- `jsonlLogger` ROOT now resolves to `os.tmpdir()/vyndr-training-test` under + `NODE_ENV==='test'` (override `TRAINING_DATA_DIR`), so tests no longer + append to the tracked `data/training/resolutions-YYYY-MM.jsonl`. Reverted + the dirtied artifact; verified the tree stays clean after a full run. +- Gave the 5MB-payload pipeline regression test a 20s timeout — it's + CPU-bound and flaked on Jest's 5s default under the heavier full-suite + load from the +23 new tests (Jest's own error message recommends this). + +### Files created +- `src/services/gradeSlateService.js` +- `tests/unit/gradeSlateService.test.js`, + `tests/unit/nflNhlWiring.test.js`, + `tests/integration/publicRouteRateLimit.test.js` + +### Files modified +- `src/services/oddsService.js` (SPORT_KEYS/SPORT_MARKETS nfl+nhl, + recordDownstream grades trigger + shouldGradeSlate gate) +- `src/utils/oddsNormalizer.js` (NHL MARKET_MAP keys) +- `src/services/adapters/proplineAdapter.js` (MARKETS.nfl/nhl) +- `src/services/training/jsonlLogger.js` (test-env temp path) +- 8 public route files (rate-limit mount): odds, parlay, schedule, + gameLines, streaks, hotlist, content, lineMovement, bookComparison +- `tests/integration/pipeline.test.js` (5MB test timeout) +- `CLAUDE.md` + +### Deliberately deferred (per spec — await design session) +- CLV tracking + prop resolution (need PropLine Hobby-tier endpoint access + confirmed). Design implementation. Player watchlists + push. +- The grades auto-grade trigger is ON by default but its per-prop + feature-compute cost is unmeasured in prod; `GRADE_SLATE_ON_FETCH=0` is + the kill-switch. Worth profiling once real PropLine slates flow. + +--- ## Session 31 (2026-06-14) — SHIPPED (audit) diff --git a/CLAUDE.md b/CLAUDE.md index d12cdfb..7f3b5d3 100755 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -91,6 +91,37 @@ Player props now have abundance, not rationing. - **Game-level odds** — Tank01 (unchanged). Tank01 PLAYER PROPS = empty, do not wire. +## Grades Content Pipeline (Session 32) +Closes the content pipeline: `contentTemplateService` reads a `grades:{sport}` +cache "when present" but nothing wrote it, so slate/POTD never reached +`dataLevel: 'full'`. +- **Writer** — `gradeSlateService.gradeAndCacheSlate(sport, props, opts)` + dedupes props to unique player+stat+line (cap 25, concurrency 5), grades + BOTH sides via `analyzeViaEngine1` (engine1 is direction-aware — keeps the + higher-confidence side), sorts by confidence desc, writes + `grades:{sport}` = `{ grades, updated_at, source }` (TTL 2h). The legacy + grade shape already matches `normalizeGrade` — no remap needed. +- **Trigger** — fire-and-forget inside `oddsService.recordDownstream` (runs + on a fresh odds fetch / cache MISS, NOT on cache hits). Does NOT hold the + odds HTTP response. Gated by `shouldGradeSlate()`: ON by default, OFF when + `NODE_ENV==='test'` (its feature-compute fan-out would pollute call-count + assertions), override with `GRADE_SLATE_ON_FETCH=1`/`0`. `0` is the + operator kill-switch if the per-prop feature-compute cost needs shedding. + +## NFL/NHL Props (Session 32) +NFL + NHL wired end-to-end. **the-odds-api keys are `americanfootball_nfl` +and `icehockey_nhl`** (full-name prefix like `basketball_nba`) — NOT +`football_nfl`/`hockey_nhl` (those are PropLine's keys in `proplineAdapter`). +`oddsService.SPORT_KEYS` + `SPORT_MARKETS` carry both; `proplineAdapter.MARKETS` +filled. NHL keys (`player_shots_on_goal`, `goalie_saves`) added to +`MARKET_MAP` so NHL props don't silently normalize to zero in-season. + +## Public Route Rate Limiting (Session 32) +`middleware/rateLimit` (`createRateLimit`, in-memory per-IP, independent +bucket per call site) now mounts via `router.use` at the top of every public +cached router: odds + parlay = 30/min, schedule/gamelines/streaks/hotlist/ +content/lines/books = 60/min. `/api/analyze` keeps its own 10/min. + ## Frontend ↔ Backend Wiring (Session 25 — non-obvious) A new Express route under `/api/*` is NOT reachable from the browser until a matching **Next.js proxy route** exists at `web/src/app/api/.../route.ts` diff --git a/src/routes/bookComparison.js b/src/routes/bookComparison.js index 4604c77..2b348f5 100644 --- a/src/routes/bookComparison.js +++ b/src/routes/bookComparison.js @@ -15,8 +15,11 @@ const express = require('express'); const bookComparison = require('../services/bookComparisonService'); const { cacheGet } = require('../utils/redis'); +const { createRateLimit } = require('../middleware/rateLimit'); const router = express.Router(); +// Session 32 — public throttle (60/min; reads cached odds props). +router.use(createRateLimit({ windowMs: 60_000, max: 60 })); const MISSION_HEADER = { 'X-VYNDR-Mission': 'Never leave money on the table' }; diff --git a/src/routes/content.js b/src/routes/content.js index b9cb0fd..7e4a08d 100644 --- a/src/routes/content.js +++ b/src/routes/content.js @@ -17,8 +17,11 @@ const express = require('express'); const template = require('../services/contentTemplateService'); const formatter = require('../services/contentFormatter'); const { cacheGet } = require('../utils/redis'); +const { createRateLimit } = require('../middleware/rateLimit'); const router = express.Router(); +// Session 32 — public throttle (60/min; reads from cache). +router.use(createRateLimit({ windowMs: 60_000, max: 60 })); const MISSION_HEADER = { 'X-VYNDR-Mission': 'Every post is a free ad' }; const SUPPORTED = new Set(['nba', 'wnba', 'mlb', 'soccer', 'nfl', 'nhl']); diff --git a/src/routes/gameLines.js b/src/routes/gameLines.js index a7c9135..2f1774f 100644 --- a/src/routes/gameLines.js +++ b/src/routes/gameLines.js @@ -31,8 +31,11 @@ const express = require('express'); const nbaAdapter = require('../services/adapters/tank01NbaAdapter'); const mlbAdapter = require('../services/adapters/tank01MlbAdapter'); const scheduleService = require('../services/scheduleService'); +const { createRateLimit } = require('../middleware/rateLimit'); const router = express.Router(); +// Session 32 — public throttle (60/min; Tank01, cached). +router.use(createRateLimit({ windowMs: 60_000, max: 60 })); const MISSION_HEADER = { 'X-VYNDR-Mission': 'Lines keep the slate alive' }; diff --git a/src/routes/hotlist.js b/src/routes/hotlist.js index 623d38c..bed7c59 100644 --- a/src/routes/hotlist.js +++ b/src/routes/hotlist.js @@ -11,8 +11,11 @@ const express = require('express'); const hotListService = require('../services/hotListService'); const { loadRosterLogs } = require('../services/rosterLogs'); +const { createRateLimit } = require('../middleware/rateLimit'); const router = express.Router(); +// Session 32 — public throttle (60/min; pure engine over cached logs). +router.use(createRateLimit({ windowMs: 60_000, max: 60 })); const MISSION_HEADER = { 'X-VYNDR-Mission': 'Hot right now' }; diff --git a/src/routes/lineMovement.js b/src/routes/lineMovement.js index 0c922e6..6f09451 100644 --- a/src/routes/lineMovement.js +++ b/src/routes/lineMovement.js @@ -11,8 +11,11 @@ const express = require('express'); const lineSnapshots = require('../services/lineSnapshotService'); +const { createRateLimit } = require('../middleware/rateLimit'); const router = express.Router(); +// Session 32 — public throttle (60/min; Redis snapshots). +router.use(createRateLimit({ windowMs: 60_000, max: 60 })); const MISSION_HEADER = { 'X-VYNDR-Mission': 'The market confirms the grade' }; diff --git a/src/routes/odds.js b/src/routes/odds.js index 809cbed..affb250 100644 --- a/src/routes/odds.js +++ b/src/routes/odds.js @@ -1,8 +1,13 @@ const express = require('express'); const { getOdds } = require('../services/oddsService'); const { MARKET_MAP, ALLOWED_BOOKS } = require('../utils/oddsNormalizer'); +const { createRateLimit } = require('../middleware/rateLimit'); const router = express.Router(); +// Session 32 — public throttle. /odds hits PropLine/odds-api upstream on a +// cache miss, so it gets the tighter 30/min bucket. Independent per-IP +// bucket scoped to this router (createRateLimit allocates its own Map). +router.use(createRateLimit({ windowMs: 60_000, max: 30 })); const VALID_STAT_TYPES = new Set(Object.values(MARKET_MAP)); const VALID_BOOKS = ALLOWED_BOOKS; diff --git a/src/routes/parlay.js b/src/routes/parlay.js index 4a4358f..206dad3 100644 --- a/src/routes/parlay.js +++ b/src/routes/parlay.js @@ -14,8 +14,11 @@ const express = require('express'); const parlayService = require('../services/parlayService'); +const { createRateLimit } = require('../middleware/rateLimit'); const router = express.Router(); +// Session 32 — public throttle. Parlay math is computation-heavy → 30/min. +router.use(createRateLimit({ windowMs: 60_000, max: 30 })); const MISSION_HEADER = { 'X-VYNDR-Mission': 'Catch the legs that fight each other' }; diff --git a/src/routes/schedule.js b/src/routes/schedule.js index fb6ace4..f88a162 100644 --- a/src/routes/schedule.js +++ b/src/routes/schedule.js @@ -22,8 +22,11 @@ const express = require('express'); const scheduleService = require('../services/scheduleService'); const { SPORT_CONFIG } = require('../config/sports'); +const { createRateLimit } = require('../middleware/rateLimit'); const router = express.Router(); +// Session 32 — public throttle (60/min; ESPN is free but be respectful). +router.use(createRateLimit({ windowMs: 60_000, max: 60 })); const MISSION_HEADER = { 'X-VYNDR-Mission': 'The slate is never empty' }; diff --git a/src/routes/streaks.js b/src/routes/streaks.js index 6f37f67..b78d6dc 100644 --- a/src/routes/streaks.js +++ b/src/routes/streaks.js @@ -15,8 +15,11 @@ const express = require('express'); const streaksService = require('../services/streaksService'); const { loadRosterLogs } = require('../services/rosterLogs'); +const { createRateLimit } = require('../middleware/rateLimit'); const router = express.Router(); +// Session 32 — public throttle (60/min; pure engine over cached logs). +router.use(createRateLimit({ windowMs: 60_000, max: 60 })); const MISSION_HEADER = { 'X-VYNDR-Mission': 'Streaks are the heartbeat' }; diff --git a/src/services/adapters/proplineAdapter.js b/src/services/adapters/proplineAdapter.js index 421430e..5860000 100644 --- a/src/services/adapters/proplineAdapter.js +++ b/src/services/adapters/proplineAdapter.js @@ -47,8 +47,8 @@ const MARKETS = { nba: ['player_points', 'player_rebounds', 'player_assists', 'player_threes', 'player_blocks', 'player_steals'], wnba: ['player_points', 'player_rebounds', 'player_assists', 'player_threes'], mlb: ['batter_hits', 'batter_home_runs', 'batter_total_bases', 'batter_rbis', 'batter_stolen_bases', 'pitcher_strikeouts'], - nfl: [], - nhl: [], + nfl: ['player_pass_yds', 'player_rush_yds', 'player_reception_yds', 'player_receptions', 'player_anytime_td', 'player_pass_tds'], + nhl: ['player_goals', 'player_shots_on_goal', 'goalie_saves'], ncaab: ['player_points', 'player_rebounds', 'player_assists'], }; diff --git a/src/services/gradeSlateService.js b/src/services/gradeSlateService.js new file mode 100644 index 0000000..939a17d --- /dev/null +++ b/src/services/gradeSlateService.js @@ -0,0 +1,137 @@ +'use strict'; + +/** + * Grade-slate writer (Session 32). + * + * Closes the content pipeline. The grading engine (engine1 via + * analyzeViaEngine1) grades props on demand, but nothing ever persisted a + * sport's graded slate. `contentTemplateService.collectSlateData` reads a + * `grades:{sport}` cache "when present" — without a writer it was always + * empty, so slate/POTD content degraded to lines/schedule and never reached + * `dataLevel: 'full'`. + * + * This service grades a sport's freshly-fetched props and writes the + * `grades:{sport}` cache in the exact shape contentTemplateService expects. + * It is wired fire-and-forget into `oddsService.recordDownstream` so it runs + * on a cache MISS (≈hourly) WITHOUT holding the odds HTTP response — content + * endpoints read the grades cache independently and asynchronously. + * + * The legacy grade shape (player_name?/player, stat_type, line, direction, + * grade, confidence, edge_pct, reasoning.summary) is already what + * `contentTemplateService.normalizeGrade` reads — so no field remapping is + * needed at the write boundary. + */ + +// Cost bounds: the odds slate carries one row per player+stat+line+book. +// We dedupe to unique player+stat+line and cap how many we grade, because +// each grade fans out to feature computation. Grading runs at most once per +// cache-miss per sport, but we still bound the herd. +const DEFAULT_LIMIT = 25; +const DEFAULT_CONCURRENCY = 5; +const DEFAULT_TTL = 7200; // 2 hours — matches the spec's grades-cache TTL. + +// Collapse the multi-book prop rows to one entry per gradeable prop. +function dedupeProps(props, limit) { + const seen = new Set(); + const out = []; + for (const p of props || []) { + if (!p || !p.player || !p.stat_type || p.line == null) continue; + const key = `${p.player}::${p.stat_type}::${p.line}`; + if (seen.has(key)) continue; + seen.add(key); + out.push(p); + if (out.length >= limit) break; + } + return out; +} + +// engine1 is direction-aware, so a prop grades differently over vs under. +// Grade both sides and keep the higher-confidence verdict — that's the +// side the engine actually favors. +async function gradeBestSide(grade, prop, sport) { + const base = { + player: prop.player, + stat_type: prop.stat_type, + line: prop.line, + sport, + book: prop.book, + }; + const sides = await Promise.all([ + Promise.resolve() + .then(() => grade({ ...base, direction: 'over' })) + .catch(() => null), + Promise.resolve() + .then(() => grade({ ...base, direction: 'under' })) + .catch(() => null), + ]); + const cands = sides.filter(Boolean); + if (cands.length === 0) return null; + return cands.reduce((a, b) => ((Number(b.confidence) || 0) > (Number(a.confidence) || 0) ? b : a)); +} + +// Run an async mapper over items with a bounded concurrency. +async function mapLimit(items, concurrency, fn) { + const results = new Array(items.length); + let cursor = 0; + async function worker() { + while (cursor < items.length) { + const i = cursor++; + results[i] = await fn(items[i], i); + } + } + const pool = Array.from({ length: Math.min(concurrency, items.length) }, () => worker()); + await Promise.all(pool); + return results; +} + +/** + * Grade a sport's props and write the `grades:{sport}` cache. + * + * @param {string} sport + * @param {Array} props normalized odds props (oddsNormalizer shape) + * @param {Object} [opts] + * @param {Function} [opts.grade] grader (default analyzeViaEngine1) + * @param {Function} [opts.cacheSet] cache writer (default redis.cacheSet) + * @param {string} [opts.source] provider tag ('propline' | 'odds-api') + * @param {number} [opts.limit] max unique props graded + * @param {number} [opts.ttl] cache TTL seconds + * @param {Function} [opts.now] timestamp source (testable) + * @returns {Promise<{written:boolean,count:number,error?:string}>} + */ +async function gradeAndCacheSlate(sport, props, opts = {}) { + const grade = opts.grade || require('./intelligence/analyzeViaEngine1').analyzeViaEngine1; + const cacheSet = opts.cacheSet || require('../utils/redis').cacheSet; + const source = opts.source || 'odds-api'; + const limit = opts.limit || DEFAULT_LIMIT; + const ttl = opts.ttl || DEFAULT_TTL; + const concurrency = opts.concurrency || DEFAULT_CONCURRENCY; + const now = opts.now || (() => new Date().toISOString()); + + if (!Array.isArray(props) || props.length === 0) { + return { written: false, count: 0 }; + } + + try { + const unique = dedupeProps(props, limit); + if (unique.length === 0) return { written: false, count: 0 }; + + const graded = (await mapLimit(unique, concurrency, (p) => gradeBestSide(grade, p, sport))) + .filter(Boolean) + .sort((a, b) => (Number(b.confidence) || 0) - (Number(a.confidence) || 0)); + + if (graded.length === 0) return { written: false, count: 0 }; + + const envelope = { grades: graded, updated_at: now(), source }; + await cacheSet(`grades:${sport}`, envelope, ttl); + return { written: true, count: graded.length }; + } catch (e) { + // Best-effort — slate grading must never break odds delivery. + console.warn('[gradeSlateService] grade slate failed:', e.message); + return { written: false, count: 0, error: e.message }; + } +} + +module.exports = { + gradeAndCacheSlate, + __internals: { dedupeProps, gradeBestSide, mapLimit, DEFAULT_LIMIT, DEFAULT_TTL, DEFAULT_CONCURRENCY }, +}; diff --git a/src/services/oddsService.js b/src/services/oddsService.js index 6487195..f2f350c 100644 --- a/src/services/oddsService.js +++ b/src/services/oddsService.js @@ -50,6 +50,12 @@ const SPORT_KEYS = { // array with a friendly message in that case. wnba: 'basketball_wnba', mlb: 'baseball_mlb', + // Session 32 — NFL + NHL. odds-api keys per the-odds-api sports list. + // Off-season returns an empty events array; the route layer surfaces an + // empty slate (never a crash). NFL props normalize through the MARKET_MAP + // keys added in Session 31; NHL keys were added alongside this wiring. + nfl: 'americanfootball_nfl', + nhl: 'icehockey_nhl', // Soccer (Session 7j) — odds-api sport keys verified against // https://the-odds-api.com/sports-odds-data/sports-apis.html soccer_wc: 'soccer_fifa_world_cup', @@ -106,6 +112,24 @@ const MLB_MARKETS = [ 'pitcher_strikeouts', 'pitcher_outs', ]; +// Session 32 — NFL + NHL market lists. NFL keys mirror the MARKET_MAP +// entries added Session 31 (both _yds and _yards spellings normalize). NHL +// keys were added to MARKET_MAP alongside this wiring so they don't +// silently normalize to zero in-season. +const NFL_MARKETS = [ + 'player_pass_yds', + 'player_pass_tds', + 'player_rush_yds', + 'player_reception_yds', + 'player_receptions', + 'player_anytime_td', +]; +const NHL_MARKETS = [ + 'player_goals', + 'player_shots_on_goal', + 'player_assists', + 'goalie_saves', +]; const SOCCER_MARKETS = [ 'player_goals', 'player_shots_on_target', @@ -129,6 +153,8 @@ const SPORT_MARKETS = Object.freeze({ nba: buildMarketString(NBA_MARKETS), wnba: buildMarketString(WNBA_MARKETS), mlb: buildMarketString(MLB_MARKETS), + nfl: buildMarketString(NFL_MARKETS), + nhl: buildMarketString(NHL_MARKETS), ncaab: buildMarketString(NBA_MARKETS), // NCAAB markets mirror NBA // Every soccer league code shares the same market set. ...Object.fromEntries( @@ -283,7 +309,7 @@ function parseQuota(headers) { // Best-effort post-fetch processing shared by both providers (PropLine + // odds-api): line movement, scratch cascade, and rolling line snapshots. // Never throws — a failure here must not break the odds response. -async function recordDownstream(sport, props) { +async function recordDownstream(sport, props, provider = 'odds-api') { let movements = []; let scratchedPlayers = []; try { @@ -298,9 +324,34 @@ async function recordDownstream(sport, props) { } catch (e) { console.warn('[VYNDR] Movement/cascade detection error:', e.message); } + + // Session 32 — grade the slate into the `grades:{sport}` cache that + // contentTemplateService reads, closing the content pipeline. Fire-and- + // forget: grading fans out to feature computation and must NOT hold the + // odds HTTP response — content endpoints read the grades cache later and + // independently. Errors are self-contained inside the service. + if (shouldGradeSlate()) { + require('./gradeSlateService') + .gradeAndCacheSlate(sport, props, { source: provider }) + .catch((e) => console.warn('[VYNDR] slate grading error:', e.message)); + } + return { movements, scratchedPlayers }; } +// Auto-grade the slate on a fresh odds fetch by default (closes the content +// pipeline). Skipped under the test env so its background feature-computation +// fan-out doesn't pollute call-count assertions in the odds integration +// tests; `GRADE_SLATE_ON_FETCH` is the explicit operator override +// ('1' forces on even in test, '0' is the production kill-switch if the +// feature-compute cost ever needs to be shed). +function shouldGradeSlate() { + const flag = process.env.GRADE_SLATE_ON_FETCH; + if (flag === '1') return true; + if (flag === '0') return false; + return process.env.NODE_ENV !== 'test'; +} + async function getOdds(sport) { const redis = getRedisClient(); const apiKey = process.env.ODDS_API_KEY; @@ -335,7 +386,7 @@ async function getOdds(sport) { const now = new Date().toISOString(); const cacheData = { updated_at: now, props: pl.props, spreads: pl.spreads || [], provider: 'propline' }; await redis.set(cacheKey, JSON.stringify(cacheData), 'EX', CACHE_TTL); - const { movements, scratchedPlayers } = await recordDownstream(sport, pl.props); + const { movements, scratchedPlayers } = await recordDownstream(sport, pl.props, 'propline'); return { sport, updated_at: now, @@ -389,7 +440,7 @@ async function getOdds(sport) { await redis.set(cacheKey, JSON.stringify(cacheData), 'EX', CACHE_TTL); // Line movement + cascade + snapshots (best-effort; shared helper). - const { movements, scratchedPlayers } = await recordDownstream(sport, props); + const { movements, scratchedPlayers } = await recordDownstream(sport, props, 'odds-api'); return { sport, @@ -458,4 +509,6 @@ module.exports = { // Session 22 — exposed for tests that exercise env-driven TTL // resolution without re-loading the module. getConfiguredCacheTTL, + // Session 32 — slate auto-grade gate (exposed for tests). + shouldGradeSlate, }; diff --git a/src/services/training/jsonlLogger.js b/src/services/training/jsonlLogger.js index 8451ab8..ac2080f 100644 --- a/src/services/training/jsonlLogger.js +++ b/src/services/training/jsonlLogger.js @@ -13,9 +13,18 @@ */ const fs = require('fs'); +const os = require('os'); const path = require('path'); -const ROOT = path.join(process.cwd(), 'data', 'training'); +// Under test, write to a throwaway temp dir instead of the tracked +// data/training corpus — otherwise every `npm test` appends resolution rows +// to data/training/resolutions-YYYY-MM.jsonl and dirties the git tree +// (Session 31 audit finding; fixed Session 32). `TRAINING_DATA_DIR` lets +// operators override the location explicitly. +const ROOT = process.env.TRAINING_DATA_DIR + || (process.env.NODE_ENV === 'test' + ? path.join(os.tmpdir(), 'vyndr-training-test') + : path.join(process.cwd(), 'data', 'training')); let initialized = false; function ensureDir() { diff --git a/src/utils/oddsNormalizer.js b/src/utils/oddsNormalizer.js index 47c91f2..495c2e1 100644 --- a/src/utils/oddsNormalizer.js +++ b/src/utils/oddsNormalizer.js @@ -71,6 +71,13 @@ const MARKET_MAP = { player_goals_conceded: 'goals_conceded', player_passes: 'passes', team_clean_sheet: 'clean_sheet', + + // NHL (Session 32) — added alongside the NFL/NHL sport-key wiring so NHL + // props don't silently normalize to zero in-season (same silent-failure + // class as the NFL gap Session 31 closed). player_goals/player_assists + // are shared with soccer/NBA — sport context discriminates downstream. + player_shots_on_goal: 'shots_on_goal', + goalie_saves: 'saves', }; function normalizeProps(eventsWithOdds) { diff --git a/tests/integration/pipeline.test.js b/tests/integration/pipeline.test.js index ac32167..ad769ea 100644 --- a/tests/integration/pipeline.test.js +++ b/tests/integration/pipeline.test.js @@ -141,7 +141,9 @@ describe('pipeline body-parser regression (Session 7b)', () => { // void: true path returns 200 with the empty resolution summary. expect(res.status).toBe(200); expect(res.body).toHaveProperty('resolved'); - }); + // Parsing a 5MB body is CPU-bound; under full-suite parallel load it can + // exceed Jest's 5s default. Give it headroom so the gate stays reliable. + }, 20000); test('rejects payloads larger than the 10MB limit with 413 (sanity check)', async () => { // Use a tiny app limit so the test doesn't allocate 11MB just to diff --git a/tests/integration/publicRouteRateLimit.test.js b/tests/integration/publicRouteRateLimit.test.js new file mode 100644 index 0000000..47d8ec3 --- /dev/null +++ b/tests/integration/publicRouteRateLimit.test.js @@ -0,0 +1,70 @@ +// Integration: Session 32 mounted the existing rate-limit middleware on the +// public cached routers. These tests prove the wiring at the ROUTER level — +// the middleware's own behavior (429, window reset, per-IP keys) is covered +// in tests/unit/rateLimitMiddleware.test.js. + +const express = require('express'); +const request = require('supertest'); + +// Mock redis so any handler that does run never touches a real server. +jest.mock('../../src/utils/redis', () => ({ + cacheGet: jest.fn(async () => null), + cacheSet: jest.fn(async () => {}), + getRedisClient: () => ({ get: async () => null, set: async () => 'OK', scan: async () => ['0', []], hgetall: async () => ({}) }), + isDegraded: () => false, +})); + +// Fresh module registry → fresh per-router bucket (createRateLimit state is +// module-scoped). jest.resetModules() resets Jest's registry; deleting +// require.cache alone does NOT under Jest. +function mountFresh(routePath, file) { + jest.resetModules(); + const app = express(); + app.use(express.json()); + app.use(routePath, require(file)); + return app; +} + +// A deep nonsense path enters the router (running router.use(rateLimit)) +// but matches no route → 404. Lets us exhaust the limiter without invoking +// the real handlers or any upstream call. +const MISS = '/__rl__/__miss__/__deep__'; + +describe('public route rate limiting (Session 32 wiring)', () => { + test('schedule router 429s after exceeding its 60/min limit', async () => { + const app = mountFresh('/api/schedule', '../../src/routes/schedule'); + // 60 allowed (each 404s but counts), 61st is throttled. + for (let i = 0; i < 60; i += 1) { + const r = await request(app).get(`/api/schedule${MISS}`); + expect(r.status).not.toBe(429); + } + const blocked = await request(app).get(`/api/schedule${MISS}`); + expect(blocked.status).toBe(429); + expect(blocked.body.error).toMatch(/too many/i); + }); + + test('odds router 429s after exceeding its tighter 30/min limit', async () => { + const app = mountFresh('/api/odds', '../../src/routes/odds'); + for (let i = 0; i < 30; i += 1) { + const r = await request(app).get(`/api/odds${MISS}`); + expect(r.status).not.toBe(429); + } + expect((await request(app).get(`/api/odds${MISS}`)).status).toBe(429); + }); + + test('different route files have INDEPENDENT buckets (not a shared limit)', async () => { + // Mount two routers in one app; exhaust parlay, confirm odds is unaffected. + jest.resetModules(); + const app = express(); + app.use(express.json()); + app.use('/api/parlay', require('../../src/routes/parlay')); + app.use('/api/odds', require('../../src/routes/odds')); + + // Exhaust parlay (30/min). + for (let i = 0; i < 30; i += 1) await request(app).get(`/api/parlay${MISS}`); + expect((await request(app).get(`/api/parlay${MISS}`)).status).toBe(429); + + // odds shares the same client IP but its own bucket — still open. + expect((await request(app).get(`/api/odds${MISS}`)).status).not.toBe(429); + }); +}); diff --git a/tests/unit/gradeSlateService.test.js b/tests/unit/gradeSlateService.test.js new file mode 100644 index 0000000..2facec4 --- /dev/null +++ b/tests/unit/gradeSlateService.test.js @@ -0,0 +1,147 @@ +// Unit: grade-slate writer (Session 32). Closes the content pipeline by +// persisting a sport's graded slate to the `grades:{sport}` cache that +// contentTemplateService reads. + +const svc = require('../../src/services/gradeSlateService'); +const content = require('../../src/services/contentTemplateService'); +const { shouldGradeSlate } = require('../../src/services/oddsService'); +const { dedupeProps } = svc.__internals; + +// Multi-book odds rows: same player+stat+line appears once per book, plus a +// distinct prop. dedupe should collapse the book duplicates. +const props = [ + { player: 'Wembanyama', stat_type: 'points', line: 28.5, book: 'draftkings', over_odds: -110, under_odds: -110 }, + { player: 'Wembanyama', stat_type: 'points', line: 28.5, book: 'fanduel', over_odds: -105, under_odds: -115 }, + { player: 'Brunson', stat_type: 'assists', line: 7.5, book: 'betmgm', over_odds: +100, under_odds: -120 }, +]; + +// Fake grader: returns the legacy grade shape. Over/under differ in +// confidence so we can prove the writer keeps the stronger side. +function fakeGrade({ player, stat_type, line, direction }) { + const table = { + 'Wembanyama::over': { confidence: 80, grade: 'A' }, + 'Wembanyama::under': { confidence: 40, grade: 'C' }, + 'Brunson::over': { confidence: 35, grade: 'C' }, + 'Brunson::under': { confidence: 64, grade: 'B+' }, + }; + const hit = table[`${player}::${direction}`] || { confidence: 10, grade: 'C' }; + return { + player, stat_type, line, direction, + grade: hit.grade, confidence: hit.confidence, edge_pct: 2.0, + reasoning: { summary: `${player} ${direction} ${line}` }, + }; +} + +describe('shouldGradeSlate (auto-grade gate)', () => { + const orig = process.env.GRADE_SLATE_ON_FETCH; + afterEach(() => { + if (orig === undefined) delete process.env.GRADE_SLATE_ON_FETCH; + else process.env.GRADE_SLATE_ON_FETCH = orig; + }); + + test('off by default under the test env', () => { + delete process.env.GRADE_SLATE_ON_FETCH; + expect(shouldGradeSlate()).toBe(false); // NODE_ENV === 'test' + }); + test('GRADE_SLATE_ON_FETCH=1 forces on', () => { + process.env.GRADE_SLATE_ON_FETCH = '1'; + expect(shouldGradeSlate()).toBe(true); + }); + test('GRADE_SLATE_ON_FETCH=0 forces off', () => { + process.env.GRADE_SLATE_ON_FETCH = '0'; + expect(shouldGradeSlate()).toBe(false); + }); +}); + +describe('dedupeProps', () => { + test('collapses multi-book rows to unique player+stat+line', () => { + expect(dedupeProps(props, 25)).toHaveLength(2); + }); + test('respects the limit', () => { + expect(dedupeProps(props, 1)).toHaveLength(1); + }); + test('skips rows missing player/stat/line', () => { + const bad = [{ stat_type: 'points', line: 1 }, { player: 'X', line: 1 }, { player: 'X', stat_type: 'points' }]; + expect(dedupeProps(bad, 25)).toHaveLength(0); + }); +}); + +describe('gradeAndCacheSlate', () => { + test('writes grades:{sport} with the envelope content reads, sorted by confidence desc', async () => { + const writes = []; + const cacheSet = async (key, value, ttl) => { writes.push({ key, value, ttl }); }; + const res = await svc.gradeAndCacheSlate('nba', props, { + grade: fakeGrade, cacheSet, source: 'propline', now: () => '2026-06-15T00:00:00.000Z', + }); + + expect(res).toEqual({ written: true, count: 2 }); + expect(writes).toHaveLength(1); + + const { key, value } = writes[0]; + // Cache key must match contentTemplateService.getGrades fallback read. + expect(key).toBe('grades:nba'); + expect(value.source).toBe('propline'); + expect(value.updated_at).toBe('2026-06-15T00:00:00.000Z'); + expect(Array.isArray(value.grades)).toBe(true); + // Sorted by confidence desc. + expect(value.grades.map((g) => g.confidence)).toEqual([80, 64]); + }); + + test('keeps the higher-confidence side per prop (engine1 is direction-aware)', async () => { + let written; + const cacheSet = async (_k, value) => { written = value; }; + await svc.gradeAndCacheSlate('nba', props, { grade: fakeGrade, cacheSet }); + + const wemby = written.grades.find((g) => g.player === 'Wembanyama'); + const brunson = written.grades.find((g) => g.player === 'Brunson'); + expect(wemby.direction).toBe('over'); // over (80) beat under (40) + expect(brunson.direction).toBe('under'); // under (64) beat over (35) + }); + + test('respects the TTL', async () => { + let ttlSeen; + const cacheSet = async (_k, _v, ttl) => { ttlSeen = ttl; }; + await svc.gradeAndCacheSlate('nba', props, { grade: fakeGrade, cacheSet, ttl: 1234 }); + expect(ttlSeen).toBe(1234); + + ttlSeen = undefined; + await svc.gradeAndCacheSlate('nba', props, { grade: fakeGrade, cacheSet }); + expect(ttlSeen).toBe(svc.__internals.DEFAULT_TTL); + }); + + test('empty props → nothing written', async () => { + let called = false; + const cacheSet = async () => { called = true; }; + expect(await svc.gradeAndCacheSlate('nba', [], { grade: fakeGrade, cacheSet })).toEqual({ written: false, count: 0 }); + expect(called).toBe(false); + }); + + test('all grader failures → nothing written', async () => { + let called = false; + const cacheSet = async () => { called = true; }; + const boom = () => { throw new Error('grader down'); }; + const res = await svc.gradeAndCacheSlate('nba', props, { grade: boom, cacheSet }); + expect(res.written).toBe(false); + expect(called).toBe(false); + }); + + test('output flows into contentTemplateService → dataLevel full, picks, POTD', async () => { + let written; + const cacheSet = async (_k, value) => { written = value; }; + await svc.gradeAndCacheSlate('nba', props, { grade: fakeGrade, cacheSet }); + + // contentTemplateService.getGrades returns cache.grades (the array). + const data = { sport: 'nba', schedule: [], gameLines: {}, grades: written.grades }; + expect(content.determineDataLevel(data)).toBe('full'); + + const thread = content.generateSlateThread('nba', { ...data, dataLevel: 'full' }); + const picks = thread.posts.filter((p) => p.role === 'pick'); + expect(picks.length).toBe(2); + expect(picks[0].player).toBe('Wembanyama'); // highest confidence pick first + + const potd = content.generatePOTD('nba', { ...data, dataLevel: 'full' }); + expect(potd.dataLevel).toBe('full'); + expect(potd.player).toBe('Wembanyama'); + expect(potd.confidence).toBe(80); + }); +}); diff --git a/tests/unit/nflNhlWiring.test.js b/tests/unit/nflNhlWiring.test.js new file mode 100644 index 0000000..74fabce --- /dev/null +++ b/tests/unit/nflNhlWiring.test.js @@ -0,0 +1,89 @@ +// Unit: NFL + NHL sport-key wiring (Session 32). Closes the silent-zero +// trap before NFL season — sport keys, per-sport markets, PropLine markets, +// and end-to-end MARKET_MAP normalization. + +const oddsService = require('../../src/services/oddsService'); +const propline = require('../../src/services/adapters/proplineAdapter'); +const { normalizeProps } = require('../../src/utils/oddsNormalizer'); + +describe('oddsService sport keys', () => { + test('nfl/nhl map to the correct the-odds-api keys', () => { + // the-odds-api uses full-name prefixes (basketball_nba, baseball_mlb); + // NFL/NHL follow the same convention — NOT football_nfl/hockey_nhl. + expect(oddsService.SPORT_KEYS.nfl).toBe('americanfootball_nfl'); + expect(oddsService.SPORT_KEYS.nhl).toBe('icehockey_nhl'); + }); + + test('getMarketsForSport(nfl) requests NFL markets + spreads (not the NBA fallback)', () => { + const markets = oddsService.getMarketsForSport('nfl'); + expect(markets).toContain('player_pass_yds'); + expect(markets).toContain('player_anytime_td'); + expect(markets).toContain('spreads'); + expect(markets).not.toContain('player_points'); // would mean nba fallback + }); + + test('getMarketsForSport(nhl) requests NHL markets', () => { + const markets = oddsService.getMarketsForSport('nhl'); + expect(markets).toContain('player_shots_on_goal'); + expect(markets).toContain('goalie_saves'); + expect(markets).not.toContain('player_points'); + }); +}); + +describe('proplineAdapter NFL/NHL markets', () => { + test('nfl/nhl markets are populated and sport keys resolve', () => { + const { MARKETS, SPORT_KEYS } = propline.__internals; + expect(MARKETS.nfl.length).toBeGreaterThan(0); + expect(MARKETS.nhl.length).toBeGreaterThan(0); + expect(MARKETS.nfl).toContain('player_pass_yds'); + expect(MARKETS.nhl).toContain('player_shots_on_goal'); + expect(SPORT_KEYS.nfl).toBe('football_nfl'); + expect(SPORT_KEYS.nhl).toBe('hockey_nhl'); + }); +}); + +describe('MARKET_MAP end-to-end normalization', () => { + function eventWith(marketKey, player, point) { + return [{ + home_team: 'Kansas City Chiefs', + away_team: 'Buffalo Bills', + commence_time: '2026-09-10T00:00:00Z', + bookmakers: [{ + key: 'draftkings', + markets: [{ + key: marketKey, + last_update: '2026-09-09T20:00:00Z', + outcomes: [ + { description: player, point, name: 'Over', price: -110 }, + { description: player, point, name: 'Under', price: -110 }, + ], + }], + }], + }]; + } + + test('NFL passing-yards prop normalizes to passing_yards', () => { + const props = normalizeProps(eventWith('player_pass_yds', 'Patrick Mahomes', 274.5)); + expect(props).toHaveLength(1); + expect(props[0].stat_type).toBe('passing_yards'); + expect(props[0].player).toBe('Patrick Mahomes'); + expect(props[0].line).toBe(274.5); + }); + + test('NFL anytime-TD prop normalizes (does not drop to zero)', () => { + const props = normalizeProps(eventWith('player_anytime_td', 'Travis Kelce', 0.5)); + expect(props).toHaveLength(1); + expect(props[0].stat_type).toBe('anytime_td'); + }); + + test('NHL shots-on-goal + goalie-saves normalize', () => { + expect(normalizeProps(eventWith('player_shots_on_goal', 'Connor McDavid', 3.5))[0].stat_type).toBe('shots_on_goal'); + expect(normalizeProps(eventWith('goalie_saves', 'Igor Shesterkin', 28.5))[0].stat_type).toBe('saves'); + }); + + test('off-season / empty response normalizes gracefully (no crash)', () => { + expect(normalizeProps([])).toEqual([]); + // Unknown market key is skipped, not crashed on. + expect(normalizeProps(eventWith('player_unknown_stat', 'Nobody', 1.5))).toEqual([]); + }); +});