Session 56: Full audit — PropLine + boxscore + pipeline + sport coverage (2289 tests)
Research (verified against live MLB Stats / ESPN / The Odds APIs): - specs/propline-audit.md — every stat_type mapped against our 4-layer pipeline; real MLB boxscore fields; sport coverage status; pipeline gap analysis. - specs/vyndr-roadmap.md — priority-ordered Sessions 57–64 + coverage targets. - scripts/propline-audit.js + specs/audit-data/ (raw capture). Headline bug: oddsNormalizer mapped batter_rbis → 'rbis' while the whole grade/feature/outcome chain keys on 'rbi' — every PropLine RBI prop silently failed to grade AND settle. Fixed (+ regression test). Phase 4 — wired missing MLB stats end-to-end: - PropLine MLB markets 6 → 12 (+runs, walks, doubles, earned_runs, hits_allowed, outs — same request, no extra quota). - doubles/outs/triples added to featureCache + outcomeService MLB_LOG_FIELD and all three grade whitelists (analyze/scan/validation.py). Phase 6 — pipeline resilience: - opsNotify.js: ntfy alerts (never throws, test-disabled). Snapshot success/ stale/failure alerts; retry-once on hard odds error (not on empty slate). - Missed-cron watchdog (mostRecentExpectedSlot/isSnapshotOverdue); status probe now returns `overdue`. Coverage truth: MLB is the only end-to-end-live sport; outcome settlement is MLB-only (WNBA/NBA/soccer never settle) — documented as the #1 roadmap gap. Backend 2276 → 2289 tests (+13). Web build exit 0. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
+46
-3
@@ -4,9 +4,52 @@
|
||||
2026-07-10
|
||||
|
||||
## Current Phase
|
||||
SHIP BUILD v55.0 — Product overhaul: the self-learning loop (outcome tracking +
|
||||
accuracy) + the real-time layer (auto-refresh, freshness, live signals) + landing
|
||||
top-signals preview. Founder-pricing expiry restored.
|
||||
SHIP BUILD v56.0 — The full data audit: PropLine/boxscore/pipeline inventory,
|
||||
rbis→rbi fix, +6 MLB markets, doubles/outs wired end-to-end, ntfy alerting +
|
||||
retry + missed-cron detection, master roadmap.
|
||||
|
||||
## Session 56 (2026-07-10) — SHIPPED ✅ THE FULL AUDIT
|
||||
|
||||
Backend 2276 → **2289 tests** (+13), 197 suites. Web build clean (exit 0).
|
||||
Docs: `specs/propline-audit.md`, `specs/vyndr-roadmap.md`,
|
||||
`specs/audit-data/propline-audit-raw.json`, `scripts/propline-audit.js`.
|
||||
|
||||
### Research (Phases 1–3, 5) — verified against LIVE APIs
|
||||
- Fetched real data: The Odds API `/v4/sports` (in-season now: MLB, WNBA, NFL
|
||||
preseason, NBA summer league, 40+ soccer leagues; NHL dark), a real MLB boxscore
|
||||
(Braves@Pirates 2026-07-09 — every settle field confirmed), ESPN WNBA/soccer
|
||||
scoreboards. PropLine inventory derived from code (no dev keys).
|
||||
- **Headline bug found:** `oddsNormalizer` mapped `batter_rbis → 'rbis'` but the
|
||||
entire grade/feature/outcome chain keys on `'rbi'` → **every PropLine RBI prop
|
||||
silently failed to grade AND settle.** Fixed.
|
||||
- Coverage truth: **MLB is the only end-to-end-live sport.** Outcome settlement is
|
||||
MLB-only (WNBA/NBA/soccer never settle → accuracy is MLB-only). Documented as
|
||||
the #1 roadmap gap.
|
||||
|
||||
### Phase 4 — wired missing MLB stat types (a full vertical slice)
|
||||
- `batter_rbis → rbi` (unblocks a market already requested).
|
||||
- PropLine MLB MARKETS 6 → 12: +batter_runs, +batter_walks, +batter_doubles,
|
||||
+pitcher_earned_runs, +pitcher_hits_allowed, +pitcher_outs (same request, no
|
||||
extra quota; runs/walks/earned_runs/hits_allowed were already supported
|
||||
downstream — just never asked for).
|
||||
- `doubles` + `outs` (+`triples`) added to featureCache + outcomeService
|
||||
MLB_LOG_FIELD AND all three grade whitelists (analyze/scan/validation.py).
|
||||
|
||||
### Phase 6 — pipeline alerting + resilience
|
||||
- **`src/utils/opsNotify.js`** — ntfy.sh push (`vyndr-pipeline-kev2026`), never
|
||||
throws, auto-disabled under test / `PIPELINE_ALERTS=0`. Injectable `fetchImpl`.
|
||||
- `snapshotService.runSnapshot`: success alert ("✅ MLB snapshot: N graded, D
|
||||
deltas, X% accuracy"), stale alert (empty slate), failure alert; **retry-once**
|
||||
on a hard odds error/null (NOT on a legit empty slate — that's off-hours, not a
|
||||
failure).
|
||||
- **Missed-cron watchdog:** `snapshotScheduler.mostRecentExpectedSlot` +
|
||||
`isSnapshotOverdue` (pure, tested); the tick alerts once per missed slot; the
|
||||
status probe (`GET /api/internal/snapshot/status`) now returns `overdue`.
|
||||
|
||||
### Phase 7 — the master roadmap
|
||||
`specs/vyndr-roadmap.md` — priority-ordered Sessions 57–64 (WNBA/NBA settlement →
|
||||
ESPN features → soccer e2e → live calibration → NFL/NBA readiness), stat-type +
|
||||
sport coverage targets, operating invariants.
|
||||
|
||||
## Session 55 (2026-07-10) — SHIPPED ✅ SELF-LEARNING LOOP + REAL-TIME LAYER
|
||||
|
||||
|
||||
@@ -658,6 +658,32 @@ snapshot, locked to the line, and read from cache.
|
||||
+ the ClaimMeter pitch). That expiry lapsing — NOT a tier-limit change — was the
|
||||
cause of the "4 stripe test failures." Operators override via `FOUNDER_CODE_EXPIRY`.
|
||||
|
||||
## Data Audit + Pipeline Resilience (Session 56 — non-obvious)
|
||||
- **`specs/propline-audit.md` + `specs/vyndr-roadmap.md` are the source of truth**
|
||||
for data coverage + the session plan. Re-run `node scripts/propline-audit.js`
|
||||
(writes raw JSON to stdout; live sections need network) to refresh.
|
||||
- **A market only fully works if it's wired at FOUR layers:** requested in
|
||||
`proplineAdapter.MARKETS`, mapped in `oddsNormalizer.MARKET_MAP`, whitelisted in
|
||||
all three grade gates (`routes/analyze.js`, `routes/scan.js`,
|
||||
`python/utils/validation.py`), AND (for MLB) present in `MLB_LOG_FIELD` in BOTH
|
||||
`featureCache` and `outcomeService`. Miss the map → silent zero; miss the log
|
||||
field → no features + no settlement. `batter_rbis` was mapped to `rbis` while
|
||||
everything else keyed on `rbi` — a 4-layer desync that silently killed RBI
|
||||
props. It's `rbi` now; a normalizer test locks it.
|
||||
- **The streaks/hotlist path uses its own `rbis` key** built from raw MLB stats —
|
||||
independent of the odds normalizer. Don't "unify" them; the split is intentional.
|
||||
- **MLB is the ONLY end-to-end-live sport.** Outcome settlement is MLB-only
|
||||
(WNBA/NBA/soccer grades never settle → `accuracy` reflects MLB only). Fixing
|
||||
that (ESPN box-score settle path) is roadmap Session 57.
|
||||
- **`src/utils/opsNotify.js`** pushes pipeline alerts to ntfy (`vyndr-pipeline-
|
||||
kev2026`). It NEVER throws and is auto-disabled under `NODE_ENV==='test'` /
|
||||
`PIPELINE_ALERTS=0` (inject `fetchImpl` to test it). `snapshotService` alerts on
|
||||
success/stale/failure; `snapshotScheduler` has a per-minute missed-cron watchdog
|
||||
(`isSnapshotOverdue`, exposed as `overdue` on the status probe).
|
||||
- **Snapshot retry rule:** retry-once ONLY on a thrown/null odds response (a
|
||||
transient provider blip). A successful-but-empty slate is NOT retried — that's a
|
||||
legit off-hours empty slate, and retrying would waste quota + add 60s latency.
|
||||
|
||||
## Active Skills
|
||||
- vyndr-voice (all user-facing output)
|
||||
- prop-analysis (grading methodology)
|
||||
|
||||
@@ -0,0 +1,114 @@
|
||||
'use strict';
|
||||
|
||||
/**
|
||||
* scripts/propline-audit.js (Session 56) — the data-source audit.
|
||||
*
|
||||
* Inventories what our providers CAN send (from code) and what the FREE
|
||||
* settled-result APIs (MLB Stats, ESPN) actually return, so we can map every
|
||||
* prop stat_type to its box-score field. PropLine live fetch runs only when
|
||||
* PROPLINE_API_KEY_* are present (they aren't in dev) — otherwise we report the
|
||||
* authoritative code inventory (MARKETS × MARKET_MAP).
|
||||
*
|
||||
* Usage: node scripts/propline-audit.js (writes JSON to stdout)
|
||||
*/
|
||||
|
||||
require('dotenv').config({ quiet: true });
|
||||
const propline = require('../src/services/adapters/proplineAdapter');
|
||||
const { MARKET_MAP } = require('../src/utils/oddsNormalizer');
|
||||
|
||||
const out = { generatedAt: new Date().toISOString(), sections: {} };
|
||||
|
||||
async function getJson(url, headers) {
|
||||
const r = await fetch(url, { headers: headers || {}, signal: AbortSignal.timeout(12000) });
|
||||
if (!r.ok) throw new Error(`HTTP ${r.status}`);
|
||||
return r.json();
|
||||
}
|
||||
|
||||
// 1. PropLine code inventory — the markets we REQUEST per sport + the stat_type
|
||||
// each normalizes to (or "UNMAPPED" if MARKET_MAP has no entry → silent zero).
|
||||
function proplineInventory() {
|
||||
const { MARKETS, SPORT_KEYS } = propline.__internals;
|
||||
const map = MARKET_MAP || {};
|
||||
const inv = {};
|
||||
for (const [sport, markets] of Object.entries(MARKETS)) {
|
||||
inv[sport] = {
|
||||
sportKey: SPORT_KEYS[sport],
|
||||
requested: markets.map((m) => ({ market: m, stat_type: map[m] || 'UNMAPPED' })),
|
||||
};
|
||||
}
|
||||
// Also list every MARKET_MAP entry (what we CAN normalize even if not requested).
|
||||
inv._allMappedMarkets = Object.entries(map).map(([m, s]) => ({ market: m, stat_type: s }));
|
||||
inv._hasKeys = propline.hasKeys();
|
||||
return inv;
|
||||
}
|
||||
|
||||
// 2. The Odds API active sports (FREE — /v4/sports does not spend quota).
|
||||
async function oddsApiSports() {
|
||||
const key = process.env.ODDS_API_KEY;
|
||||
if (!key) return { skipped: 'no ODDS_API_KEY' };
|
||||
try {
|
||||
const data = await getJson(`https://api.the-odds-api.com/v4/sports?apiKey=${key}`);
|
||||
return (data || [])
|
||||
.filter((s) => s.active)
|
||||
.map((s) => ({ key: s.key, group: s.group, title: s.title }));
|
||||
} catch (e) { return { error: e.message }; }
|
||||
}
|
||||
|
||||
// 3. MLB Stats API — a recent FINAL game's boxscore field keys + a game-log row.
|
||||
async function mlbBoxscore() {
|
||||
try {
|
||||
// Yesterday's schedule.
|
||||
const d = new Date(Date.now() - 24 * 3600 * 1000).toISOString().slice(0, 10);
|
||||
const sched = await getJson(`https://statsapi.mlb.com/api/v1/schedule?sportId=1&date=${d}`);
|
||||
const games = (sched.dates?.[0]?.games) || [];
|
||||
const final = games.find((g) => g.status?.abstractGameState === 'Final') || games[0];
|
||||
if (!final) return { note: `no games ${d}` };
|
||||
const box = await getJson(`https://statsapi.mlb.com/api/v1/game/${final.gamePk}/boxscore`);
|
||||
// Pull one batter + one pitcher stat object.
|
||||
const sampleTeam = box.teams?.away || box.teams?.home || {};
|
||||
const players = Object.values(sampleTeam.players || {});
|
||||
const batter = players.find((p) => p.stats?.batting && Object.keys(p.stats.batting).length);
|
||||
const pitcher = players.find((p) => p.stats?.pitching && Object.keys(p.stats.pitching).length);
|
||||
return {
|
||||
date: d, gamePk: final.gamePk, matchup: final.teams?.away?.team?.name + ' @ ' + final.teams?.home?.team?.name,
|
||||
battingFields: batter ? Object.keys(batter.stats.batting) : [],
|
||||
pitchingFields: pitcher ? Object.keys(pitcher.stats.pitching) : [],
|
||||
sampleBatting: batter ? batter.stats.batting : null,
|
||||
samplePitching: pitcher ? pitcher.stats.pitching : null,
|
||||
};
|
||||
} catch (e) { return { error: e.message }; }
|
||||
}
|
||||
|
||||
// 4. ESPN WNBA — today's scoreboard status + a completed game's boxscore labels.
|
||||
async function espnWnba() {
|
||||
try {
|
||||
const sb = await getJson('https://site.api.espn.com/apis/site/v2/sports/basketball/wnba/scoreboard');
|
||||
const events = (sb.events || []).map((e) => ({ name: e.name, status: e.status?.type?.description, date: e.date }));
|
||||
const final = (sb.events || []).find((e) => e.status?.type?.completed);
|
||||
let boxLabels = null;
|
||||
if (final) {
|
||||
const summary = await getJson(`https://site.api.espn.com/apis/site/v2/sports/basketball/wnba/summary?event=${final.id}`);
|
||||
const teamStats = summary.boxscore?.players?.[0]?.statistics?.[0];
|
||||
boxLabels = teamStats ? { labels: teamStats.labels, names: teamStats.names } : null;
|
||||
}
|
||||
return { count: events.length, events, completedBoxLabels: boxLabels };
|
||||
} catch (e) { return { error: e.message }; }
|
||||
}
|
||||
|
||||
// 5. ESPN soccer — check a common league scoreboard for activity.
|
||||
async function espnSoccer() {
|
||||
try {
|
||||
const sb = await getJson('https://site.api.espn.com/apis/site/v2/sports/soccer/usa.1/scoreboard');
|
||||
const events = (sb.events || []).map((e) => ({ name: e.name, status: e.status?.type?.description }));
|
||||
return { league: 'usa.1 (MLS)', count: events.length, events: events.slice(0, 6) };
|
||||
} catch (e) { return { error: e.message }; }
|
||||
}
|
||||
|
||||
(async () => {
|
||||
out.sections.proplineInventory = proplineInventory();
|
||||
out.sections.oddsApiActiveSports = await oddsApiSports();
|
||||
out.sections.mlbBoxscore = await mlbBoxscore();
|
||||
out.sections.espnWnba = await espnWnba();
|
||||
out.sections.espnSoccer = await espnSoccer();
|
||||
process.stdout.write(JSON.stringify(out, null, 2) + '\n');
|
||||
})();
|
||||
@@ -0,0 +1,875 @@
|
||||
{
|
||||
"generatedAt": "2026-07-10T20:48:09.307Z",
|
||||
"sections": {
|
||||
"proplineInventory": {
|
||||
"nba": {
|
||||
"sportKey": "basketball_nba",
|
||||
"requested": [
|
||||
{
|
||||
"market": "player_points",
|
||||
"stat_type": "points"
|
||||
},
|
||||
{
|
||||
"market": "player_rebounds",
|
||||
"stat_type": "rebounds"
|
||||
},
|
||||
{
|
||||
"market": "player_assists",
|
||||
"stat_type": "assists"
|
||||
},
|
||||
{
|
||||
"market": "player_threes",
|
||||
"stat_type": "threes"
|
||||
},
|
||||
{
|
||||
"market": "player_blocks",
|
||||
"stat_type": "blocks"
|
||||
},
|
||||
{
|
||||
"market": "player_steals",
|
||||
"stat_type": "steals"
|
||||
}
|
||||
]
|
||||
},
|
||||
"wnba": {
|
||||
"sportKey": "basketball_wnba",
|
||||
"requested": [
|
||||
{
|
||||
"market": "player_points",
|
||||
"stat_type": "points"
|
||||
},
|
||||
{
|
||||
"market": "player_rebounds",
|
||||
"stat_type": "rebounds"
|
||||
},
|
||||
{
|
||||
"market": "player_assists",
|
||||
"stat_type": "assists"
|
||||
},
|
||||
{
|
||||
"market": "player_threes",
|
||||
"stat_type": "threes"
|
||||
}
|
||||
]
|
||||
},
|
||||
"mlb": {
|
||||
"sportKey": "baseball_mlb",
|
||||
"requested": [
|
||||
{
|
||||
"market": "batter_hits",
|
||||
"stat_type": "hits"
|
||||
},
|
||||
{
|
||||
"market": "batter_home_runs",
|
||||
"stat_type": "home_runs"
|
||||
},
|
||||
{
|
||||
"market": "batter_total_bases",
|
||||
"stat_type": "total_bases"
|
||||
},
|
||||
{
|
||||
"market": "batter_rbis",
|
||||
"stat_type": "rbis"
|
||||
},
|
||||
{
|
||||
"market": "batter_stolen_bases",
|
||||
"stat_type": "stolen_bases"
|
||||
},
|
||||
{
|
||||
"market": "pitcher_strikeouts",
|
||||
"stat_type": "strikeouts"
|
||||
}
|
||||
]
|
||||
},
|
||||
"nfl": {
|
||||
"sportKey": "football_nfl",
|
||||
"requested": [
|
||||
{
|
||||
"market": "player_pass_yds",
|
||||
"stat_type": "passing_yards"
|
||||
},
|
||||
{
|
||||
"market": "player_rush_yds",
|
||||
"stat_type": "rushing_yards"
|
||||
},
|
||||
{
|
||||
"market": "player_reception_yds",
|
||||
"stat_type": "receiving_yards"
|
||||
},
|
||||
{
|
||||
"market": "player_receptions",
|
||||
"stat_type": "receptions"
|
||||
},
|
||||
{
|
||||
"market": "player_anytime_td",
|
||||
"stat_type": "anytime_td"
|
||||
},
|
||||
{
|
||||
"market": "player_pass_tds",
|
||||
"stat_type": "pass_tds"
|
||||
}
|
||||
]
|
||||
},
|
||||
"nhl": {
|
||||
"sportKey": "hockey_nhl",
|
||||
"requested": [
|
||||
{
|
||||
"market": "player_goals",
|
||||
"stat_type": "goals"
|
||||
},
|
||||
{
|
||||
"market": "player_shots_on_goal",
|
||||
"stat_type": "shots_on_goal"
|
||||
},
|
||||
{
|
||||
"market": "goalie_saves",
|
||||
"stat_type": "saves"
|
||||
}
|
||||
]
|
||||
},
|
||||
"ncaab": {
|
||||
"sportKey": "basketball_ncaab",
|
||||
"requested": [
|
||||
{
|
||||
"market": "player_points",
|
||||
"stat_type": "points"
|
||||
},
|
||||
{
|
||||
"market": "player_rebounds",
|
||||
"stat_type": "rebounds"
|
||||
},
|
||||
{
|
||||
"market": "player_assists",
|
||||
"stat_type": "assists"
|
||||
}
|
||||
]
|
||||
},
|
||||
"_allMappedMarkets": [
|
||||
{
|
||||
"market": "player_points",
|
||||
"stat_type": "points"
|
||||
},
|
||||
{
|
||||
"market": "player_rebounds",
|
||||
"stat_type": "rebounds"
|
||||
},
|
||||
{
|
||||
"market": "player_assists",
|
||||
"stat_type": "assists"
|
||||
},
|
||||
{
|
||||
"market": "player_threes",
|
||||
"stat_type": "threes"
|
||||
},
|
||||
{
|
||||
"market": "player_blocks",
|
||||
"stat_type": "blocks"
|
||||
},
|
||||
{
|
||||
"market": "player_steals",
|
||||
"stat_type": "steals"
|
||||
},
|
||||
{
|
||||
"market": "player_points_rebounds_assists",
|
||||
"stat_type": "pra"
|
||||
},
|
||||
{
|
||||
"market": "player_turnovers",
|
||||
"stat_type": "turnovers"
|
||||
},
|
||||
{
|
||||
"market": "batter_hits",
|
||||
"stat_type": "hits"
|
||||
},
|
||||
{
|
||||
"market": "batter_home_runs",
|
||||
"stat_type": "home_runs"
|
||||
},
|
||||
{
|
||||
"market": "batter_total_bases",
|
||||
"stat_type": "total_bases"
|
||||
},
|
||||
{
|
||||
"market": "batter_rbis",
|
||||
"stat_type": "rbis"
|
||||
},
|
||||
{
|
||||
"market": "batter_runs",
|
||||
"stat_type": "runs"
|
||||
},
|
||||
{
|
||||
"market": "batter_stolen_bases",
|
||||
"stat_type": "stolen_bases"
|
||||
},
|
||||
{
|
||||
"market": "batter_singles",
|
||||
"stat_type": "singles"
|
||||
},
|
||||
{
|
||||
"market": "batter_doubles",
|
||||
"stat_type": "doubles"
|
||||
},
|
||||
{
|
||||
"market": "batter_walks",
|
||||
"stat_type": "walks"
|
||||
},
|
||||
{
|
||||
"market": "batter_strikeouts",
|
||||
"stat_type": "batter_strikeouts"
|
||||
},
|
||||
{
|
||||
"market": "pitcher_strikeouts",
|
||||
"stat_type": "strikeouts"
|
||||
},
|
||||
{
|
||||
"market": "pitcher_earned_runs",
|
||||
"stat_type": "earned_runs"
|
||||
},
|
||||
{
|
||||
"market": "pitcher_hits_allowed",
|
||||
"stat_type": "hits_allowed"
|
||||
},
|
||||
{
|
||||
"market": "pitcher_outs",
|
||||
"stat_type": "outs"
|
||||
},
|
||||
{
|
||||
"market": "player_pass_yds",
|
||||
"stat_type": "passing_yards"
|
||||
},
|
||||
{
|
||||
"market": "player_pass_yards",
|
||||
"stat_type": "passing_yards"
|
||||
},
|
||||
{
|
||||
"market": "player_pass_tds",
|
||||
"stat_type": "pass_tds"
|
||||
},
|
||||
{
|
||||
"market": "player_pass_completions",
|
||||
"stat_type": "pass_completions"
|
||||
},
|
||||
{
|
||||
"market": "player_pass_attempts",
|
||||
"stat_type": "pass_attempts"
|
||||
},
|
||||
{
|
||||
"market": "player_pass_interceptions",
|
||||
"stat_type": "interceptions"
|
||||
},
|
||||
{
|
||||
"market": "player_rush_yds",
|
||||
"stat_type": "rushing_yards"
|
||||
},
|
||||
{
|
||||
"market": "player_rush_yards",
|
||||
"stat_type": "rushing_yards"
|
||||
},
|
||||
{
|
||||
"market": "player_rush_attempts",
|
||||
"stat_type": "rush_attempts"
|
||||
},
|
||||
{
|
||||
"market": "player_rush_tds",
|
||||
"stat_type": "rush_tds"
|
||||
},
|
||||
{
|
||||
"market": "player_receptions",
|
||||
"stat_type": "receptions"
|
||||
},
|
||||
{
|
||||
"market": "player_reception_yds",
|
||||
"stat_type": "receiving_yards"
|
||||
},
|
||||
{
|
||||
"market": "player_receiving_yards",
|
||||
"stat_type": "receiving_yards"
|
||||
},
|
||||
{
|
||||
"market": "player_reception_tds",
|
||||
"stat_type": "reception_tds"
|
||||
},
|
||||
{
|
||||
"market": "player_anytime_td",
|
||||
"stat_type": "anytime_td"
|
||||
},
|
||||
{
|
||||
"market": "player_kicking_points",
|
||||
"stat_type": "kicking_points"
|
||||
},
|
||||
{
|
||||
"market": "player_goals",
|
||||
"stat_type": "goals"
|
||||
},
|
||||
{
|
||||
"market": "player_shots_on_target",
|
||||
"stat_type": "shots_on_target"
|
||||
},
|
||||
{
|
||||
"market": "player_shots",
|
||||
"stat_type": "shots"
|
||||
},
|
||||
{
|
||||
"market": "player_tackles",
|
||||
"stat_type": "tackles"
|
||||
},
|
||||
{
|
||||
"market": "player_cards",
|
||||
"stat_type": "cards"
|
||||
},
|
||||
{
|
||||
"market": "player_corners",
|
||||
"stat_type": "corners"
|
||||
},
|
||||
{
|
||||
"market": "player_saves",
|
||||
"stat_type": "saves"
|
||||
},
|
||||
{
|
||||
"market": "player_goals_conceded",
|
||||
"stat_type": "goals_conceded"
|
||||
},
|
||||
{
|
||||
"market": "player_passes",
|
||||
"stat_type": "passes"
|
||||
},
|
||||
{
|
||||
"market": "team_clean_sheet",
|
||||
"stat_type": "clean_sheet"
|
||||
},
|
||||
{
|
||||
"market": "player_shots_on_goal",
|
||||
"stat_type": "shots_on_goal"
|
||||
},
|
||||
{
|
||||
"market": "goalie_saves",
|
||||
"stat_type": "saves"
|
||||
}
|
||||
],
|
||||
"_hasKeys": false
|
||||
},
|
||||
"oddsApiActiveSports": [
|
||||
{
|
||||
"key": "americanfootball_cfl",
|
||||
"group": "American Football",
|
||||
"title": "CFL"
|
||||
},
|
||||
{
|
||||
"key": "americanfootball_ncaaf",
|
||||
"group": "American Football",
|
||||
"title": "NCAAF"
|
||||
},
|
||||
{
|
||||
"key": "americanfootball_ncaaf_championship_winner",
|
||||
"group": "American Football",
|
||||
"title": "NCAAF Championship Winner"
|
||||
},
|
||||
{
|
||||
"key": "americanfootball_nfl",
|
||||
"group": "American Football",
|
||||
"title": "NFL"
|
||||
},
|
||||
{
|
||||
"key": "americanfootball_nfl_preseason",
|
||||
"group": "American Football",
|
||||
"title": "NFL Preseason"
|
||||
},
|
||||
{
|
||||
"key": "americanfootball_nfl_super_bowl_winner",
|
||||
"group": "American Football",
|
||||
"title": "NFL Super Bowl Winner"
|
||||
},
|
||||
{
|
||||
"key": "aussierules_afl",
|
||||
"group": "Aussie Rules",
|
||||
"title": "AFL"
|
||||
},
|
||||
{
|
||||
"key": "baseball_milb",
|
||||
"group": "Baseball",
|
||||
"title": "MiLB"
|
||||
},
|
||||
{
|
||||
"key": "baseball_mlb",
|
||||
"group": "Baseball",
|
||||
"title": "MLB"
|
||||
},
|
||||
{
|
||||
"key": "baseball_mlb_world_series_winner",
|
||||
"group": "Baseball",
|
||||
"title": "MLB World Series Winner"
|
||||
},
|
||||
{
|
||||
"key": "baseball_npb",
|
||||
"group": "Baseball",
|
||||
"title": "NPB"
|
||||
},
|
||||
{
|
||||
"key": "basketball_nba_summer_league",
|
||||
"group": "Basketball",
|
||||
"title": "NBA Summer League"
|
||||
},
|
||||
{
|
||||
"key": "basketball_wnba",
|
||||
"group": "Basketball",
|
||||
"title": "WNBA"
|
||||
},
|
||||
{
|
||||
"key": "boxing_boxing",
|
||||
"group": "Boxing",
|
||||
"title": "Boxing"
|
||||
},
|
||||
{
|
||||
"key": "cricket_international_t20",
|
||||
"group": "Cricket",
|
||||
"title": "International Twenty20"
|
||||
},
|
||||
{
|
||||
"key": "cricket_odi",
|
||||
"group": "Cricket",
|
||||
"title": "One Day Internationals"
|
||||
},
|
||||
{
|
||||
"key": "cricket_t20_blast",
|
||||
"group": "Cricket",
|
||||
"title": "T20 Blast"
|
||||
},
|
||||
{
|
||||
"key": "golf_the_open_championship_winner",
|
||||
"group": "Golf",
|
||||
"title": "The Open Winner"
|
||||
},
|
||||
{
|
||||
"key": "lacrosse_pll",
|
||||
"group": "Lacrosse",
|
||||
"title": "PLL"
|
||||
},
|
||||
{
|
||||
"key": "mma_mixed_martial_arts",
|
||||
"group": "Mixed Martial Arts",
|
||||
"title": "MMA"
|
||||
},
|
||||
{
|
||||
"key": "politics_us_presidential_election_winner",
|
||||
"group": "Politics",
|
||||
"title": "US Presidential Elections Winner"
|
||||
},
|
||||
{
|
||||
"key": "rugbyleague_nrl",
|
||||
"group": "Rugby League",
|
||||
"title": "NRL"
|
||||
},
|
||||
{
|
||||
"key": "soccer_argentina_primera_division",
|
||||
"group": "Soccer",
|
||||
"title": "Primera División - Argentina"
|
||||
},
|
||||
{
|
||||
"key": "soccer_austria_bundesliga",
|
||||
"group": "Soccer",
|
||||
"title": "Austrian Football Bundesliga"
|
||||
},
|
||||
{
|
||||
"key": "soccer_brazil_campeonato",
|
||||
"group": "Soccer",
|
||||
"title": "Brazil Série A"
|
||||
},
|
||||
{
|
||||
"key": "soccer_brazil_serie_b",
|
||||
"group": "Soccer",
|
||||
"title": "Brazil Série B"
|
||||
},
|
||||
{
|
||||
"key": "soccer_china_superleague",
|
||||
"group": "Soccer",
|
||||
"title": "Super League - China"
|
||||
},
|
||||
{
|
||||
"key": "soccer_conmebol_copa_libertadores",
|
||||
"group": "Soccer",
|
||||
"title": "Copa Libertadores"
|
||||
},
|
||||
{
|
||||
"key": "soccer_conmebol_copa_sudamericana",
|
||||
"group": "Soccer",
|
||||
"title": "Copa Sudamericana"
|
||||
},
|
||||
{
|
||||
"key": "soccer_denmark_superliga",
|
||||
"group": "Soccer",
|
||||
"title": "Denmark Superliga"
|
||||
},
|
||||
{
|
||||
"key": "soccer_efl_champ",
|
||||
"group": "Soccer",
|
||||
"title": "Championship"
|
||||
},
|
||||
{
|
||||
"key": "soccer_england_efl_cup",
|
||||
"group": "Soccer",
|
||||
"title": "EFL Cup"
|
||||
},
|
||||
{
|
||||
"key": "soccer_england_league1",
|
||||
"group": "Soccer",
|
||||
"title": "League 1"
|
||||
},
|
||||
{
|
||||
"key": "soccer_england_league2",
|
||||
"group": "Soccer",
|
||||
"title": "League 2"
|
||||
},
|
||||
{
|
||||
"key": "soccer_epl",
|
||||
"group": "Soccer",
|
||||
"title": "EPL"
|
||||
},
|
||||
{
|
||||
"key": "soccer_fifa_world_cup",
|
||||
"group": "Soccer",
|
||||
"title": "FIFA World Cup"
|
||||
},
|
||||
{
|
||||
"key": "soccer_fifa_world_cup_winner",
|
||||
"group": "Soccer",
|
||||
"title": "FIFA World Cup Winner"
|
||||
},
|
||||
{
|
||||
"key": "soccer_finland_veikkausliiga",
|
||||
"group": "Soccer",
|
||||
"title": "Veikkausliiga - Finland"
|
||||
},
|
||||
{
|
||||
"key": "soccer_france_ligue_one",
|
||||
"group": "Soccer",
|
||||
"title": "Ligue 1 - France"
|
||||
},
|
||||
{
|
||||
"key": "soccer_germany_bundesliga",
|
||||
"group": "Soccer",
|
||||
"title": "Bundesliga - Germany"
|
||||
},
|
||||
{
|
||||
"key": "soccer_germany_bundesliga2",
|
||||
"group": "Soccer",
|
||||
"title": "Bundesliga 2 - Germany"
|
||||
},
|
||||
{
|
||||
"key": "soccer_germany_dfb_pokal",
|
||||
"group": "Soccer",
|
||||
"title": "DFB-Pokal"
|
||||
},
|
||||
{
|
||||
"key": "soccer_germany_liga3",
|
||||
"group": "Soccer",
|
||||
"title": "3. Liga - Germany"
|
||||
},
|
||||
{
|
||||
"key": "soccer_italy_serie_a",
|
||||
"group": "Soccer",
|
||||
"title": "Serie A - Italy"
|
||||
},
|
||||
{
|
||||
"key": "soccer_korea_kleague1",
|
||||
"group": "Soccer",
|
||||
"title": "K League 1"
|
||||
},
|
||||
{
|
||||
"key": "soccer_league_of_ireland",
|
||||
"group": "Soccer",
|
||||
"title": "League of Ireland"
|
||||
},
|
||||
{
|
||||
"key": "soccer_mexico_ligamx",
|
||||
"group": "Soccer",
|
||||
"title": "Liga MX"
|
||||
},
|
||||
{
|
||||
"key": "soccer_netherlands_eredivisie",
|
||||
"group": "Soccer",
|
||||
"title": "Dutch Eredivisie"
|
||||
},
|
||||
{
|
||||
"key": "soccer_norway_eliteserien",
|
||||
"group": "Soccer",
|
||||
"title": "Eliteserien - Norway"
|
||||
},
|
||||
{
|
||||
"key": "soccer_russia_premier_league",
|
||||
"group": "Soccer",
|
||||
"title": "Premier League - Russia"
|
||||
},
|
||||
{
|
||||
"key": "soccer_spain_la_liga",
|
||||
"group": "Soccer",
|
||||
"title": "La Liga - Spain"
|
||||
},
|
||||
{
|
||||
"key": "soccer_spl",
|
||||
"group": "Soccer",
|
||||
"title": "Premiership - Scotland"
|
||||
},
|
||||
{
|
||||
"key": "soccer_sweden_allsvenskan",
|
||||
"group": "Soccer",
|
||||
"title": "Allsvenskan - Sweden"
|
||||
},
|
||||
{
|
||||
"key": "soccer_sweden_superettan",
|
||||
"group": "Soccer",
|
||||
"title": "Superettan - Sweden"
|
||||
},
|
||||
{
|
||||
"key": "soccer_switzerland_superleague",
|
||||
"group": "Soccer",
|
||||
"title": "Swiss Superleague"
|
||||
},
|
||||
{
|
||||
"key": "soccer_uefa_champs_league_qualification",
|
||||
"group": "Soccer",
|
||||
"title": "UEFA Champions League Qualification"
|
||||
},
|
||||
{
|
||||
"key": "soccer_usa_mls",
|
||||
"group": "Soccer",
|
||||
"title": "MLS"
|
||||
},
|
||||
{
|
||||
"key": "tennis_atp_wimbledon",
|
||||
"group": "Tennis",
|
||||
"title": "ATP Wimbledon"
|
||||
},
|
||||
{
|
||||
"key": "tennis_wta_wimbledon",
|
||||
"group": "Tennis",
|
||||
"title": "WTA Wimbledon"
|
||||
}
|
||||
],
|
||||
"mlbBoxscore": {
|
||||
"date": "2026-07-09",
|
||||
"gamePk": 823359,
|
||||
"matchup": "Atlanta Braves @ Pittsburgh Pirates",
|
||||
"battingFields": [
|
||||
"summary",
|
||||
"gamesPlayed",
|
||||
"flyOuts",
|
||||
"groundOuts",
|
||||
"airOuts",
|
||||
"runs",
|
||||
"doubles",
|
||||
"triples",
|
||||
"homeRuns",
|
||||
"strikeOuts",
|
||||
"baseOnBalls",
|
||||
"intentionalWalks",
|
||||
"hits",
|
||||
"hitByPitch",
|
||||
"atBats",
|
||||
"caughtStealing",
|
||||
"stolenBases",
|
||||
"stolenBasePercentage",
|
||||
"groundIntoDoublePlay",
|
||||
"groundIntoTriplePlay",
|
||||
"plateAppearances",
|
||||
"totalBases",
|
||||
"rbi",
|
||||
"leftOnBase",
|
||||
"sacBunts",
|
||||
"sacFlies",
|
||||
"catchersInterference",
|
||||
"pickoffs",
|
||||
"atBatsPerHomeRun",
|
||||
"popOuts",
|
||||
"lineOuts"
|
||||
],
|
||||
"pitchingFields": [
|
||||
"note",
|
||||
"summary",
|
||||
"gamesPlayed",
|
||||
"gamesStarted",
|
||||
"flyOuts",
|
||||
"groundOuts",
|
||||
"airOuts",
|
||||
"runs",
|
||||
"doubles",
|
||||
"triples",
|
||||
"homeRuns",
|
||||
"strikeOuts",
|
||||
"baseOnBalls",
|
||||
"intentionalWalks",
|
||||
"hits",
|
||||
"hitByPitch",
|
||||
"atBats",
|
||||
"caughtStealing",
|
||||
"stolenBases",
|
||||
"stolenBasePercentage",
|
||||
"numberOfPitches",
|
||||
"inningsPitched",
|
||||
"wins",
|
||||
"losses",
|
||||
"saves",
|
||||
"saveOpportunities",
|
||||
"holds",
|
||||
"blownSaves",
|
||||
"earnedRuns",
|
||||
"battersFaced",
|
||||
"outs",
|
||||
"gamesPitched",
|
||||
"completeGames",
|
||||
"shutouts",
|
||||
"pitchesThrown",
|
||||
"balls",
|
||||
"strikes",
|
||||
"strikePercentage",
|
||||
"hitBatsmen",
|
||||
"balks",
|
||||
"wildPitches",
|
||||
"pickoffs",
|
||||
"rbi",
|
||||
"gamesFinished",
|
||||
"runsScoredPer9",
|
||||
"homeRunsPer9",
|
||||
"inheritedRunners",
|
||||
"inheritedRunnersScored",
|
||||
"catchersInterference",
|
||||
"sacBunts",
|
||||
"sacFlies",
|
||||
"passedBall",
|
||||
"popOuts",
|
||||
"lineOuts"
|
||||
],
|
||||
"sampleBatting": {
|
||||
"summary": "1-6 | K, R, SB",
|
||||
"gamesPlayed": 1,
|
||||
"flyOuts": 1,
|
||||
"groundOuts": 1,
|
||||
"airOuts": 3,
|
||||
"runs": 1,
|
||||
"doubles": 0,
|
||||
"triples": 0,
|
||||
"homeRuns": 0,
|
||||
"strikeOuts": 1,
|
||||
"baseOnBalls": 0,
|
||||
"intentionalWalks": 0,
|
||||
"hits": 1,
|
||||
"hitByPitch": 0,
|
||||
"atBats": 6,
|
||||
"caughtStealing": 0,
|
||||
"stolenBases": 1,
|
||||
"stolenBasePercentage": "1.000",
|
||||
"groundIntoDoublePlay": 0,
|
||||
"groundIntoTriplePlay": 0,
|
||||
"plateAppearances": 6,
|
||||
"totalBases": 1,
|
||||
"rbi": 0,
|
||||
"leftOnBase": 3,
|
||||
"sacBunts": 0,
|
||||
"sacFlies": 0,
|
||||
"catchersInterference": 0,
|
||||
"pickoffs": 0,
|
||||
"atBatsPerHomeRun": "-.--",
|
||||
"popOuts": 0,
|
||||
"lineOuts": 2
|
||||
},
|
||||
"samplePitching": {
|
||||
"note": "(W, 2-0)",
|
||||
"summary": "1.0 IP, 0 ER, 0 K, 0 BB",
|
||||
"gamesPlayed": 1,
|
||||
"gamesStarted": 0,
|
||||
"flyOuts": 1,
|
||||
"groundOuts": 1,
|
||||
"airOuts": 2,
|
||||
"runs": 0,
|
||||
"doubles": 0,
|
||||
"triples": 0,
|
||||
"homeRuns": 0,
|
||||
"strikeOuts": 0,
|
||||
"baseOnBalls": 0,
|
||||
"intentionalWalks": 0,
|
||||
"hits": 0,
|
||||
"hitByPitch": 0,
|
||||
"atBats": 3,
|
||||
"caughtStealing": 0,
|
||||
"stolenBases": 0,
|
||||
"stolenBasePercentage": ".---",
|
||||
"numberOfPitches": 15,
|
||||
"inningsPitched": "1.0",
|
||||
"wins": 1,
|
||||
"losses": 0,
|
||||
"saves": 0,
|
||||
"saveOpportunities": 0,
|
||||
"holds": 0,
|
||||
"blownSaves": 0,
|
||||
"earnedRuns": 0,
|
||||
"battersFaced": 3,
|
||||
"outs": 3,
|
||||
"gamesPitched": 1,
|
||||
"completeGames": 0,
|
||||
"shutouts": 0,
|
||||
"pitchesThrown": 15,
|
||||
"balls": 6,
|
||||
"strikes": 9,
|
||||
"strikePercentage": ".600",
|
||||
"hitBatsmen": 0,
|
||||
"balks": 0,
|
||||
"wildPitches": 0,
|
||||
"pickoffs": 0,
|
||||
"rbi": 0,
|
||||
"gamesFinished": 0,
|
||||
"runsScoredPer9": "0.00",
|
||||
"homeRunsPer9": "0.00",
|
||||
"inheritedRunners": 0,
|
||||
"inheritedRunnersScored": 0,
|
||||
"catchersInterference": 0,
|
||||
"sacBunts": 0,
|
||||
"sacFlies": 0,
|
||||
"passedBall": 0,
|
||||
"popOuts": 1,
|
||||
"lineOuts": 0
|
||||
}
|
||||
},
|
||||
"espnWnba": {
|
||||
"count": 3,
|
||||
"events": [
|
||||
{
|
||||
"name": "Golden State Valkyries at Connecticut Sun",
|
||||
"status": "Scheduled",
|
||||
"date": "2026-07-10T23:30Z"
|
||||
},
|
||||
{
|
||||
"name": "Dallas Wings at Toronto Tempo",
|
||||
"status": "Scheduled",
|
||||
"date": "2026-07-10T23:30Z"
|
||||
},
|
||||
{
|
||||
"name": "Chicago Sky at Los Angeles Sparks",
|
||||
"status": "Scheduled",
|
||||
"date": "2026-07-11T02:00Z"
|
||||
}
|
||||
],
|
||||
"completedBoxLabels": null
|
||||
},
|
||||
"espnSoccer": {
|
||||
"league": "usa.1 (MLS)",
|
||||
"count": 4,
|
||||
"events": [
|
||||
{
|
||||
"name": "Toronto FC at CF Montréal",
|
||||
"status": "Scheduled"
|
||||
},
|
||||
{
|
||||
"name": "Vancouver Whitecaps at Chicago Fire FC",
|
||||
"status": "Scheduled"
|
||||
},
|
||||
{
|
||||
"name": "Sporting Kansas City at St. Louis CITY SC",
|
||||
"status": "Scheduled"
|
||||
},
|
||||
{
|
||||
"name": "Portland Timbers at Seattle Sounders FC",
|
||||
"status": "Scheduled"
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,145 @@
|
||||
# PropLine + Box Score + Pipeline Audit — 2026-07-10 (Session 56)
|
||||
|
||||
Ground-truth inventory of what data we can access, what our pipeline handles, and
|
||||
the gaps. Live sections were fetched against real APIs (MLB Stats, ESPN, The Odds
|
||||
API `/v4/sports`). PropLine has no dev keys, so its inventory is derived from the
|
||||
authoritative code (`proplineAdapter.MARKETS` × `oddsNormalizer.MARKET_MAP`) —
|
||||
that IS the set our pipeline requests + normalizes. Raw capture:
|
||||
`specs/audit-data/propline-audit-raw.json`.
|
||||
|
||||
Baseline: **2276 tests** passing at session start.
|
||||
|
||||
---
|
||||
|
||||
## 0. What is in-season RIGHT NOW (The Odds API `/v4/sports`, live 2026-07-10)
|
||||
|
||||
| Sport | Active key(s) | Notes |
|
||||
|-------|---------------|-------|
|
||||
| MLB | `baseball_mlb` | ✅ full season — the live product |
|
||||
| WNBA | `basketball_wnba` | ✅ in-season (3 games tonight per ESPN) |
|
||||
| NBA | `basketball_nba_summer_league` only | ⚠️ regular NBA off until Oct; summer league live |
|
||||
| NFL | `americanfootball_nfl`, `_nfl_preseason` | ⚠️ preseason lines starting |
|
||||
| Soccer | 40+ leagues (`soccer_epl`, `soccer_usa_mls`, `soccer_fifa_world_cup`, La Liga, Serie A, …) | ✅ year-round via odds-api |
|
||||
| NHL | (absent) | off-season — no lines |
|
||||
|
||||
Takeaway: **MLB + WNBA + soccer are the live opportunity today.** NBA/NFL are
|
||||
pre-season shells; NHL is dark.
|
||||
|
||||
---
|
||||
|
||||
## 1. PropLine / odds inventory — what we REQUEST vs what we CAN normalize
|
||||
|
||||
`proplineAdapter.MARKETS` = the markets we actually ask for. `MARKET_MAP` = every
|
||||
market we can normalize (superset). A market we request but don't map → **silent
|
||||
zero**; a market we map but don't request → **unused capacity** (free to add — same
|
||||
call).
|
||||
|
||||
### MLB
|
||||
| Requested market | → stat_type | Grade whitelist? | Features (l5/l20)? | Outcome settle? |
|
||||
|---|---|---|---|---|
|
||||
| batter_hits | hits | ✅ | ✅ hits | ✅ hits |
|
||||
| batter_home_runs | home_runs | ✅ | ✅ homeRuns | ✅ homeRuns |
|
||||
| batter_total_bases | total_bases | ✅ | ✅ totalBases | ✅ totalBases |
|
||||
| batter_rbis | **rbis** ❌ | ✅ (`rbi`) | ✅ (`rbi`) | ✅ (`rbi`) | ← **BUG: normalizes to `rbis`, whole chain keys on `rbi` → never grades/settles** |
|
||||
| batter_stolen_bases | stolen_bases | ✅ | ✅ stolenBases | ✅ stolenBases |
|
||||
| pitcher_strikeouts | strikeouts | ✅ | ✅ strikeOuts | ✅ strikeOuts |
|
||||
|
||||
**Mapped but NOT requested (free capacity — all supported downstream already):**
|
||||
`batter_runs`→runs, `batter_walks`→walks, `pitcher_earned_runs`→earned_runs,
|
||||
`pitcher_hits_allowed`→hits_allowed. **Mapped, not requested, and needs feature/
|
||||
settle wiring:** `batter_doubles`→doubles, `pitcher_outs`→outs. Confirmed present
|
||||
in the real boxscore (§2).
|
||||
|
||||
### WNBA / NBA
|
||||
Requested: `player_points`, `player_rebounds`, `player_assists`, `player_threes`
|
||||
(+ NBA adds `player_blocks`, `player_steals`). All grade-whitelisted. **Features:
|
||||
NBA/WNBA game logs come from the Python nba_api service (usually OFFLINE in prod)
|
||||
→ l5/l20 often empty; ESPN fallback exists for season avgs only. Outcomes: NOT
|
||||
settled at all (outcomeService is MLB-only).** `player_turnovers`/`player_pra` are
|
||||
mapped but not requested.
|
||||
|
||||
### Soccer
|
||||
**PropLine has NO soccer entry** (`SPORT_KEYS`/`MARKETS` omit it) — soccer props
|
||||
can only come from the odds-api backup. `MARKET_MAP` maps goals/shots/shots_on_
|
||||
target/tackles/cards/corners/saves/passes/clean_sheet. Features: soccer extractor
|
||||
exists; outcomes: NOT settled.
|
||||
|
||||
### NFL / NHL
|
||||
NFL markets mapped + requested (pass/rush/rec yds, receptions, TDs) but the flow
|
||||
isn't graded end-to-end yet. NHL mapped/requested but off-season.
|
||||
|
||||
---
|
||||
|
||||
## 2. MLB box score field map (REAL — Braves @ Pirates, 2026-07-09)
|
||||
|
||||
Batting fields present: `runs, doubles, triples, homeRuns, strikeOuts,
|
||||
baseOnBalls, hits, atBats, stolenBases, totalBases, rbi, sacFlies, …`
|
||||
Pitching fields present: `strikeOuts, baseOnBalls, hits, inningsPitched,
|
||||
earnedRuns, outs, battersFaced, …`
|
||||
|
||||
**OUTCOME_MAP (MLB) — stat_type → game-log/boxscore field:**
|
||||
```
|
||||
hits → hits total_bases → totalBases home_runs → homeRuns
|
||||
rbi → rbi runs → runs walks → baseOnBalls
|
||||
stolen_bases → stolenBases strikeouts → strikeOuts earned_runs → earnedRuns
|
||||
hits_allowed → hits (pitching) innings_pitched → inningsPitched
|
||||
doubles → doubles ⟵ NEW triples → triples ⟵ NEW outs → outs ⟵ NEW
|
||||
batter_strikeouts → strikeOuts ⟵ (batter Ks — roadmap)
|
||||
```
|
||||
Every field above is real in the boxscore/game log → all are settleable today.
|
||||
|
||||
## 2b. ESPN box score (WNBA/soccer — the non-MLB settle path)
|
||||
WNBA settlement source = ESPN summary `boxscore.players[].statistics[]`
|
||||
(`labels`/`names` arrays: PTS/REB/AST/3PM/STL/BLK/TO). No completed game at fetch
|
||||
time (all 3 tonight were Scheduled), so labels weren't captured — wiring the ESPN
|
||||
settle path is a roadmap item (`espnStatsAdapter` already parses this shape for
|
||||
season avgs).
|
||||
|
||||
---
|
||||
|
||||
## 3. Pipeline flow + gaps
|
||||
|
||||
**Current flow:** cron (`SNAPSHOT_CRON=1`, UTC 14,19,22,1,3) → per sport:
|
||||
`settleAllOutcomes()` (S55, MLB only) → `snapshotService.runSnapshot` → PropLine
|
||||
(3-key rotation) / odds-api backup → `gradeAndCacheSlate` → per-player archetype +
|
||||
features → Redis (`snapshot:{sport}:latest`, `grades:{sport}`) → ticker.
|
||||
|
||||
**Gaps found:**
|
||||
1. **RBI silent failure** — `batter_rbis → rbis` mismatch (§1). FIX this session.
|
||||
2. **Under-requesting MLB markets** — only 6 of ~12 supported; runs/walks/doubles/
|
||||
earned_runs/hits_allowed/outs are free in the same call. EXPAND this session.
|
||||
3. **Outcome settlement is MLB-only** — WNBA/NBA/soccer grades NEVER settle, so
|
||||
`accuracy` is MLB-only. Biggest roadmap item (needs the ESPN settle path).
|
||||
4. **No pipeline alerting** — a failed/empty/missed cron run is silent. ADD ntfy
|
||||
this session.
|
||||
5. **No retry** — one PropLine blip marks a sport unavailable for the whole cycle.
|
||||
ADD retry-once this session.
|
||||
6. **No missed-cron detector** — nothing notices if a slot didn't fire. ADD.
|
||||
7. **NBA/WNBA features depend on an offline Python service** — l5/l20 often empty;
|
||||
ESPN gives season avgs only. Roadmap: an ESPN game-log feature path.
|
||||
8. **Soccer has no PropLine source** — odds-api backup only; document + monitor.
|
||||
|
||||
---
|
||||
|
||||
## 4. Sport coverage status (verified 2026-07-10)
|
||||
|
||||
| Sport | Odds props | Grade pipeline | Features | Outcomes | Status |
|
||||
|-------|-----------|----------------|----------|----------|--------|
|
||||
| MLB | ✅ PropLine + odds-api | ✅ | ✅ real (statsapi) | ✅ (S55) | **LIVE** |
|
||||
| WNBA | ✅ odds-api/PropLine | ✅ | ⚠️ Python offline → thin | ❌ not settled | **PARTIAL** |
|
||||
| NBA | ⚠️ summer league only | ✅ | ⚠️ offline | ❌ | **OFF-SEASON** |
|
||||
| Soccer | ✅ odds-api (no PropLine) | ⚠️ extractor exists | ⚠️ | ❌ | **PARTIAL** |
|
||||
| NFL | ⚠️ preseason | ⚠️ mapped, not e2e | ❌ | ❌ | **PRE-SEASON** |
|
||||
| NHL | ❌ off-season | ⚠️ mapped | ❌ | ❌ | **DARK** |
|
||||
|
||||
---
|
||||
|
||||
## 5. Fixes shipped this session (Phase 4/6)
|
||||
- `batter_rbis → rbi` (unblocks a market we already request).
|
||||
- PropLine MLB MARKETS expanded: +batter_runs, +batter_walks, +batter_doubles,
|
||||
+pitcher_earned_runs, +pitcher_hits_allowed, +pitcher_outs.
|
||||
- `doubles` + `outs` wired into featureCache + outcomeService + all three grade
|
||||
whitelists (analyze.js / scan.js / validation.py).
|
||||
- Pipeline alerting (ntfy) + PropLine retry-once + missed-cron detector.
|
||||
|
||||
See `specs/vyndr-roadmap.md` for the priority-ordered plan that follows from this.
|
||||
@@ -0,0 +1,96 @@
|
||||
# VYNDR Master Roadmap — Generated 2026-07-10 (Session 56)
|
||||
|
||||
The foundation document. Derived from the Session-56 data audit
|
||||
(`specs/propline-audit.md`, verified against live MLB Stats / ESPN / Odds APIs).
|
||||
Every session after this references it. Update it as sessions ship.
|
||||
|
||||
---
|
||||
|
||||
## Current coverage (verified 2026-07-10)
|
||||
|
||||
| Sport | Odds props | Grading | Features (l5/l20) | Outcomes settled | Status |
|
||||
|-------|-----------|---------|-------------------|------------------|--------|
|
||||
| MLB | ✅ PropLine + odds-api | ✅ | ✅ real (statsapi) | ✅ (S55, expanded S56) | **LIVE** |
|
||||
| WNBA | ✅ odds-api/PropLine | ✅ | ⚠️ Python offline → thin | ❌ | **PARTIAL** |
|
||||
| Soccer | ✅ odds-api only (no PropLine) | ⚠️ extractor only | ⚠️ | ❌ | **PARTIAL** |
|
||||
| NBA | ⚠️ summer league only | ✅ | ⚠️ offline | ❌ | **OFF-SEASON (Oct)** |
|
||||
| NFL | ⚠️ preseason | ⚠️ mapped, not e2e | ❌ | ❌ | **PRE-SEASON (Sep)** |
|
||||
| NHL | ❌ off-season | ⚠️ mapped | ❌ | ❌ | **DARK (Oct)** |
|
||||
|
||||
**The core product is MLB.** It is the only sport that is live end-to-end
|
||||
(odds → grade → real features → settled accuracy). Everything else is a
|
||||
build-out target.
|
||||
|
||||
---
|
||||
|
||||
## Gap analysis (from the audit)
|
||||
|
||||
1. **Outcome settlement is MLB-only** — WNBA/NBA/soccer grades never settle, so
|
||||
the accuracy record (the S55 trust engine) only reflects MLB. **Highest-value
|
||||
gap.** Needs the ESPN box-score settle path (`espnStatsAdapter` already parses
|
||||
the shape for season avgs).
|
||||
2. **NBA/WNBA features depend on an offline Python nba_api service** — l5/l20 are
|
||||
often empty; only season avgs survive (ESPN fallback). Needs an ESPN game-log
|
||||
feature path so intel populates without the Python service.
|
||||
3. **Soccer has no PropLine source** — odds-api backup only; no `SPORT_KEYS` entry.
|
||||
Soccer grading isn't wired into the snapshot pipeline end-to-end.
|
||||
4. **NFL mapped but not graded end-to-end** — markets normalize, but no feature/
|
||||
outcome path. Wire before September.
|
||||
5. ~~RBI silent-failure (rbis/rbi)~~ — **FIXED S56.**
|
||||
6. ~~Under-requesting MLB markets~~ — **FIXED S56** (6 → 12 markets).
|
||||
7. ~~No pipeline alerting / retry / missed-cron~~ — **FIXED S56.**
|
||||
8. **No live calibration** — the accuracy record exists but doesn't yet feed back
|
||||
into grade confidence (spec 2.3 from S55).
|
||||
9. **`batter_strikeouts` (batter Ks)** mapped but not feature/settle-wired.
|
||||
|
||||
---
|
||||
|
||||
## Session plan (priority-ordered)
|
||||
|
||||
| Session | Focus | Ships | Scope |
|
||||
|---------|-------|-------|-------|
|
||||
| 57 | **WNBA/NBA outcome settlement** | ESPN box-score settle path in outcomeService → accuracy for basketball; WNBA goes fully live | M |
|
||||
| 58 | **ESPN game-log features** | l5/l10/l20 for NBA/WNBA without the Python service → real intel on basketball grade cards | M |
|
||||
| 59 | **Soccer end-to-end** | soccer into the snapshot pipeline (odds-api source), feature extractor wired, ESPN settle | L |
|
||||
| 60 | **Live calibration** | accuracy record feeds grade-confidence adjustment (under/over-confident tiers nudge); "Model health: Calibrated/Learning" indicator | M |
|
||||
| 61 | **NFL pre-season prep** | NFL feature + settle path (ESPN NFL boxscore) so Week 1 (Sep) is live | L |
|
||||
| 62 | **Prop breadth** | `batter_strikeouts`, `triples`, pitcher `walks`; NBA `pra`/`turnovers` requested; WNBA extra markets | S |
|
||||
| 63 | **Accuracy depth** | per-archetype hit rates, per-stat hit rates, player-level record ("VYNDR on Judge: 12-5"); ledger UI | M |
|
||||
| 64 | **NBA regular-season readiness** | verify NBA pipeline for the Oct tip-off; depth-chart/cascade live | M |
|
||||
|
||||
Scope key: S ≈ ½ session, M ≈ 1 session, L ≈ 1–2 sessions.
|
||||
|
||||
---
|
||||
|
||||
## Stat-type coverage target (fully built)
|
||||
|
||||
- **MLB (batters):** hits, total_bases, home_runs, rbi, runs, doubles, triples,
|
||||
walks, stolen_bases, batter_strikeouts. **(pitchers):** strikeouts, earned_runs,
|
||||
hits_allowed, innings_pitched, outs, walks. *(Bold-new this session: doubles,
|
||||
triples, outs, runs, walks requested + settleable.)*
|
||||
- **NBA/WNBA:** points, rebounds, assists, threes, steals, blocks, turnovers, pra.
|
||||
- **Soccer:** goals, shots, shots_on_target, tackles, cards, corners, saves,
|
||||
passes, clean_sheet.
|
||||
- **NFL:** passing/rushing/receiving yards, receptions, pass/rush/rec TDs,
|
||||
anytime_td, interceptions, kicking_points.
|
||||
|
||||
## Sport coverage target
|
||||
|
||||
- **MLB** ✅ live now (settled accuracy).
|
||||
- **WNBA** → live after S57 (settlement) + S58 (features).
|
||||
- **Soccer** → live after S59.
|
||||
- **NFL** → ready for September (S61).
|
||||
- **NBA** → ready for October (S64).
|
||||
- **NHL** → ready for October (fast-follow once NBA path exists; shares ESPN pattern).
|
||||
|
||||
---
|
||||
|
||||
## Operating invariants (do not regress)
|
||||
- Three stat_type whitelists stay in sync: `routes/analyze.js`, `routes/scan.js`,
|
||||
`python/utils/validation.py`.
|
||||
- A requested market MUST have a `MARKET_MAP` entry (else silent zero) AND, if
|
||||
MLB, a `MLB_LOG_FIELD` entry in BOTH `featureCache` and `outcomeService` (else
|
||||
no features / no settlement).
|
||||
- The accuracy pill stays HONEST — "LEARNING" below MIN_SAMPLE, never a faked %.
|
||||
- Pipeline alerts (ntfy) fire on success/failure/stale/overdue; the status probe
|
||||
(`GET /api/internal/snapshot/status`) exposes `overdue`.
|
||||
@@ -61,6 +61,9 @@ const VALID_STAT_TYPES = new Set([
|
||||
'strikeouts', 'hits', 'home_runs', 'rbi', 'total_bases',
|
||||
'walks', 'runs', 'earned_runs', 'innings_pitched',
|
||||
'hits_allowed', 'stolen_bases',
|
||||
// Session 56 audit — settleable against the real boxscore; PropLine now
|
||||
// requests batter_doubles + pitcher_outs. Keep in sync with scan.js + validation.py.
|
||||
'doubles', 'outs',
|
||||
]);
|
||||
|
||||
const VALID_DIRECTIONS = new Set(['over', 'under']);
|
||||
|
||||
@@ -112,7 +112,7 @@ router.post('/snapshot/all', async (req, res) => {
|
||||
*/
|
||||
router.get('/snapshot/status', async (req, res) => {
|
||||
const { cacheGet } = require('../utils/redis');
|
||||
const { HOURS_UTC } = require('../snapshotScheduler');
|
||||
const { HOURS_UTC, isSnapshotOverdue } = require('../snapshotScheduler');
|
||||
const SPORTS = ['mlb', 'nba', 'wnba'];
|
||||
try {
|
||||
const redis_keys = {};
|
||||
@@ -135,10 +135,13 @@ router.get('/snapshot/status', async (req, res) => {
|
||||
}
|
||||
const ticker = await cacheGet('ticker:items');
|
||||
redis_keys['ticker:items'] = !!ticker;
|
||||
// Session 56 — surface the missed-cron signal in the health probe.
|
||||
const mlbTs = last_snapshot.mlb && last_snapshot.mlb.updated_at;
|
||||
return res.json({
|
||||
cron_armed: process.env.SNAPSHOT_CRON === '1',
|
||||
cron_hours_utc: HOURS_UTC,
|
||||
last_snapshot,
|
||||
overdue: isSnapshotOverdue(mlbTs),
|
||||
redis_keys,
|
||||
ticker_count: Array.isArray(ticker) ? ticker.length : 0,
|
||||
});
|
||||
|
||||
@@ -17,6 +17,8 @@ const VALID_STAT_TYPES = new Set([
|
||||
'strikeouts', 'hits', 'home_runs', 'rbi', 'total_bases',
|
||||
'walks', 'runs', 'earned_runs', 'innings_pitched',
|
||||
'hits_allowed', 'stolen_bases',
|
||||
// Session 56 audit — keep in sync with analyze.js + validation.py.
|
||||
'doubles', 'outs',
|
||||
]);
|
||||
const VALID_DIRECTIONS = new Set(['over', 'under']);
|
||||
|
||||
|
||||
@@ -46,7 +46,11 @@ const SPORT_KEYS = {
|
||||
const MARKETS = {
|
||||
nba: ['player_points', 'player_rebounds', 'player_assists', 'player_threes', 'player_blocks', 'player_steals'],
|
||||
wnba: ['player_points', 'player_rebounds', 'player_assists', 'player_threes'],
|
||||
mlb: ['batter_hits', 'batter_home_runs', 'batter_total_bases', 'batter_rbis', 'batter_stolen_bases', 'pitcher_strikeouts'],
|
||||
// Session 56 audit — expanded from 6 to 12 markets. runs/walks/earned_runs/
|
||||
// hits_allowed were already fully supported downstream (whitelist + features +
|
||||
// outcomes) but never requested; doubles/outs are wired this session. All ride
|
||||
// the SAME request (no extra quota) → materially more graded props per slate.
|
||||
mlb: ['batter_hits', 'batter_home_runs', 'batter_total_bases', 'batter_rbis', 'batter_stolen_bases', 'batter_runs', 'batter_walks', 'batter_doubles', 'pitcher_strikeouts', 'pitcher_earned_runs', 'pitcher_hits_allowed', 'pitcher_outs'],
|
||||
nfl: ['player_pass_yds', 'player_rush_yds', 'player_reception_yds', 'player_receptions', 'player_anytime_td', 'player_pass_tds'],
|
||||
nhl: ['player_goals', 'player_shots_on_goal', 'goalie_saves'],
|
||||
ncaab: ['player_points', 'player_rebounds', 'player_assists'],
|
||||
|
||||
@@ -82,6 +82,8 @@ const MLB_LOG_FIELD = {
|
||||
runs: 'runs', stolen_bases: 'stolenBases', walks: 'baseOnBalls',
|
||||
strikeouts: 'strikeOuts', earned_runs: 'earnedRuns', hits_allowed: 'hits',
|
||||
innings_pitched: 'inningsPitched',
|
||||
// Session 56 audit — real boxscore/game-log fields (Braves@Pirates verified).
|
||||
doubles: 'doubles', triples: 'triples', outs: 'outs',
|
||||
};
|
||||
|
||||
function mlbStatValue(statObj, statType) {
|
||||
|
||||
@@ -41,6 +41,8 @@ const MLB_LOG_FIELD = {
|
||||
runs: 'runs', stolen_bases: 'stolenBases', walks: 'baseOnBalls',
|
||||
strikeouts: 'strikeOuts', earned_runs: 'earnedRuns', hits_allowed: 'hits',
|
||||
innings_pitched: 'inningsPitched',
|
||||
// Session 56 audit — confirmed present in the real boxscore/game log.
|
||||
doubles: 'doubles', triples: 'triples', outs: 'outs',
|
||||
};
|
||||
|
||||
function statValue(statObj, statType) {
|
||||
|
||||
@@ -18,7 +18,9 @@ VALID_STAT_TYPES = {
|
||||
'steals', 'blocks', 'turnovers'],
|
||||
'mlb': ['strikeouts', 'hits', 'home_runs', 'rbi', 'total_bases',
|
||||
'walks', 'runs', 'earned_runs', 'innings_pitched',
|
||||
'hits_allowed', 'stolen_bases']
|
||||
'hits_allowed', 'stolen_bases',
|
||||
# Session 56 audit — keep in sync with analyze.js + scan.js.
|
||||
'doubles', 'outs']
|
||||
}
|
||||
|
||||
VALID_SPORTS = ['nba', 'mlb']
|
||||
|
||||
@@ -184,18 +184,36 @@ async function runSnapshot(sport, opts = {}) {
|
||||
cacheSet: opts.cacheSet || require('../utils/redis').cacheSet,
|
||||
now: opts.now || (() => new Date().toISOString()),
|
||||
nowMs: opts.nowMs || (() => Date.now()),
|
||||
// Session 56 — ops alerting (ntfy) + retry-once on a hard odds failure.
|
||||
notify: opts.notify || require('../utils/opsNotify').notify,
|
||||
sleep: opts.sleep || ((ms) => new Promise((r) => setTimeout(r, ms))),
|
||||
retryDelayMs: opts.retryDelayMs != null ? opts.retryDelayMs : 60_000,
|
||||
};
|
||||
const start = deps.nowMs();
|
||||
const ts = deps.now();
|
||||
|
||||
// Session 56 — retry ONCE on a hard failure (thrown error / null response = a
|
||||
// transient PropLine/network blip). A successful-but-empty slate is NOT a
|
||||
// failure (off-hours), so it is not retried — that would waste quota + latency.
|
||||
let odds;
|
||||
try {
|
||||
odds = await deps.getOdds(sp);
|
||||
if (odds == null) throw new Error('null odds response');
|
||||
} catch (e) {
|
||||
return { sport: sp, status: 'error', reason: e.message, gradeCount: 0 };
|
||||
try {
|
||||
await deps.sleep(deps.retryDelayMs);
|
||||
odds = await deps.getOdds(sp);
|
||||
if (odds == null) throw new Error('null odds response (retry)');
|
||||
} catch (e2) {
|
||||
await deps.notify(`❌ ${sp.toUpperCase()} snapshot FAILED: ${e2.message}`, { title: 'VYNDR pipeline', priority: 'high', tags: ['x'] });
|
||||
return { sport: sp, status: 'error', reason: e2.message, gradeCount: 0 };
|
||||
}
|
||||
}
|
||||
const props = (odds && Array.isArray(odds.props)) ? odds.props : [];
|
||||
if (props.length === 0) return { sport: sp, status: 'skipped', reason: 'no props', gradeCount: 0 };
|
||||
if (props.length === 0) {
|
||||
await deps.notify(`⚠️ ${sp.toUpperCase()} snapshot: 0 props (odds unavailable)`, { title: 'VYNDR pipeline', priority: 'low', tags: ['warning'] });
|
||||
return { sport: sp, status: 'skipped', reason: 'no props', gradeCount: 0 };
|
||||
}
|
||||
|
||||
// Grade the slate via the existing service; capture the envelope instead of
|
||||
// letting it write (we re-write an ENRICHED version below).
|
||||
@@ -273,6 +291,18 @@ async function runSnapshot(sport, opts = {}) {
|
||||
const events = generateTickerEvents(sp, enriched, deltas, ts);
|
||||
await pushTickerItems(events, deps);
|
||||
|
||||
// Session 56 — success alert, enriched with the rolling accuracy (if settled).
|
||||
let accPart = '';
|
||||
try {
|
||||
const acc = await deps.cacheGet(`accuracy:${sp}`);
|
||||
const pct = acc && acc.overall && acc.overall.pct;
|
||||
if (pct != null) accPart = `, ${pct}% accuracy (30d)`;
|
||||
} catch { /* accuracy is best-effort in the alert */ }
|
||||
await deps.notify(
|
||||
`✅ ${sp.toUpperCase()} snapshot: ${enriched.length} props graded, ${deltas.length} deltas${accPart}`,
|
||||
{ title: 'VYNDR pipeline', tags: ['white_check_mark'] },
|
||||
);
|
||||
|
||||
return {
|
||||
sport: sp,
|
||||
status: 'ok',
|
||||
|
||||
@@ -17,6 +17,30 @@ const HOURS_UTC = (process.env.SNAPSHOT_HOURS_UTC || '14,19,22,1,3')
|
||||
.map((n) => parseInt(n, 10))
|
||||
.filter((n) => Number.isInteger(n) && n >= 0 && n <= 23);
|
||||
|
||||
// Session 56 — missed-cron detection (pure, testable).
|
||||
// The latest scheduled hour:00 (UTC) at or before `date`. null if none in 48h.
|
||||
function mostRecentExpectedSlot(date, hours = HOURS_UTC) {
|
||||
const d = new Date(date.getTime());
|
||||
d.setUTCMinutes(0, 0, 0);
|
||||
for (let i = 0; i < 48; i += 1) {
|
||||
if (hours.includes(d.getUTCHours())) return new Date(d.getTime());
|
||||
d.setUTCHours(d.getUTCHours() - 1);
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
// True when we're >graceMin past the most recent expected slot AND the last
|
||||
// recorded snapshot predates that slot (i.e. the run was missed). Never fires on
|
||||
// cold boot (no lastSnapshotIso) — we don't cry wolf before the first snapshot.
|
||||
function isSnapshotOverdue(lastSnapshotIso, now = new Date(), hours = HOURS_UTC, graceMin = 30) {
|
||||
const slot = mostRecentExpectedSlot(now, hours);
|
||||
if (!slot) return false;
|
||||
if (now.getTime() - slot.getTime() < graceMin * 60_000) return false;
|
||||
if (!lastSnapshotIso) return false;
|
||||
const last = new Date(lastSnapshotIso).getTime();
|
||||
return Number.isFinite(last) && last < slot.getTime();
|
||||
}
|
||||
|
||||
function startSnapshotScheduler(opts = {}) {
|
||||
if (process.env.SNAPSHOT_CRON !== '1') {
|
||||
// Session 52 — log the disarmed state so container logs make it unambiguous
|
||||
@@ -31,10 +55,33 @@ function startSnapshotScheduler(opts = {}) {
|
||||
// real (now-completed) results BEFORE grading the fresh slate, so the accuracy
|
||||
// record reflects yesterday's games each cycle.
|
||||
const settleAll = opts.settleAllOutcomes || require('./services/outcomeService').settleAllOutcomes;
|
||||
const notify = opts.notify || require('./utils/opsNotify').notify;
|
||||
const cacheGet = opts.cacheGet || require('./utils/redis').cacheGet;
|
||||
const now = opts.now || (() => new Date());
|
||||
let lastFiredSlot = null;
|
||||
let lastOverdueSlot = null;
|
||||
|
||||
// Session 56 — missed-cron watchdog. Runs every minute (independent of the
|
||||
// fire schedule): if a scheduled slot came and went without a snapshot, alert
|
||||
// ONCE per missed slot.
|
||||
const checkOverdue = async () => {
|
||||
try {
|
||||
const d = now();
|
||||
const slot = mostRecentExpectedSlot(d);
|
||||
if (!slot) return;
|
||||
const slotKey = slot.toISOString();
|
||||
if (slotKey === lastOverdueSlot) return; // already alerted for this slot
|
||||
const latest = await cacheGet('snapshot:mlb:latest');
|
||||
const lastTs = latest && latest.updated_at;
|
||||
if (isSnapshotOverdue(lastTs, d)) {
|
||||
lastOverdueSlot = slotKey;
|
||||
await notify(`⚠️ Snapshot OVERDUE — expected ${slot.getUTCHours()}:00 UTC, last was ${lastTs || 'never'}`, { title: 'VYNDR pipeline', priority: 'high', tags: ['warning'] });
|
||||
}
|
||||
} catch { /* watchdog must never throw */ }
|
||||
};
|
||||
|
||||
const tick = async () => {
|
||||
await checkOverdue();
|
||||
const d = now();
|
||||
if (d.getUTCMinutes() !== 0) return;
|
||||
const h = d.getUTCHours();
|
||||
@@ -64,4 +111,4 @@ function startSnapshotScheduler(opts = {}) {
|
||||
return { interval, tick };
|
||||
}
|
||||
|
||||
module.exports = { startSnapshotScheduler, HOURS_UTC };
|
||||
module.exports = { startSnapshotScheduler, HOURS_UTC, mostRecentExpectedSlot, isSnapshotOverdue };
|
||||
|
||||
@@ -24,7 +24,12 @@ const MARKET_MAP = {
|
||||
batter_hits: 'hits',
|
||||
batter_home_runs: 'home_runs',
|
||||
batter_total_bases: 'total_bases',
|
||||
batter_rbis: 'rbis',
|
||||
// Session 56 audit — must be 'rbi' (singular): the grade whitelists
|
||||
// (analyze/scan/validation.py), featureCache MLB_LOG_FIELD, and outcomeService
|
||||
// all key on 'rbi'. Normalizing to 'rbis' silently dropped every PropLine RBI
|
||||
// prop from grading AND settlement. The streaks/hotlist path uses its own
|
||||
// 'rbis' key built from raw MLB stats — independent of this normalizer.
|
||||
batter_rbis: 'rbi',
|
||||
batter_runs: 'runs',
|
||||
batter_stolen_bases: 'stolen_bases',
|
||||
batter_singles: 'singles',
|
||||
|
||||
@@ -0,0 +1,54 @@
|
||||
'use strict';
|
||||
|
||||
/**
|
||||
* opsNotify (Session 56) — pipeline alerting via ntfy.sh.
|
||||
*
|
||||
* Push a one-line operational alert (snapshot success / failure / stale / missed
|
||||
* cron) to an ntfy topic so a silent pipeline never goes unnoticed. Fire-and-
|
||||
* forget: NEVER throws, NEVER blocks the pipeline on a notify failure.
|
||||
*
|
||||
* Config:
|
||||
* NTFY_URL (default https://ntfy.sh)
|
||||
* NTFY_TOPIC (default vyndr-pipeline-kev2026)
|
||||
* PIPELINE_ALERTS=0 → disable entirely
|
||||
* Disabled automatically under NODE_ENV==='test' unless a fetchImpl is injected
|
||||
* (so the unit tests can assert the call without hitting the network).
|
||||
*/
|
||||
|
||||
const NTFY_URL = () => process.env.NTFY_URL || 'https://ntfy.sh';
|
||||
const NTFY_TOPIC = () => process.env.NTFY_TOPIC || 'vyndr-pipeline-kev2026';
|
||||
|
||||
function enabled(opts = {}) {
|
||||
if (opts.fetchImpl) return true; // tests inject → always "enabled"
|
||||
if (process.env.PIPELINE_ALERTS === '0') return false;
|
||||
if (process.env.NODE_ENV === 'test') return false;
|
||||
return true;
|
||||
}
|
||||
|
||||
/**
|
||||
* Send an ops alert. `opts`: { title, priority ('min'|'low'|'default'|'high'|
|
||||
* 'urgent'), tags (string[]), fetchImpl }. Resolves { sent: boolean } — never rejects.
|
||||
*/
|
||||
async function notify(message, opts = {}) {
|
||||
if (!enabled(opts)) return { sent: false, reason: 'disabled' };
|
||||
const doFetch = opts.fetchImpl || fetch;
|
||||
const headers = {};
|
||||
if (opts.title) headers.Title = opts.title;
|
||||
if (opts.priority) headers.Priority = opts.priority;
|
||||
if (Array.isArray(opts.tags) && opts.tags.length) headers.Tags = opts.tags.join(',');
|
||||
try {
|
||||
await doFetch(`${NTFY_URL()}/${NTFY_TOPIC()}`, {
|
||||
method: 'POST',
|
||||
headers,
|
||||
body: String(message == null ? '' : message),
|
||||
signal: typeof AbortSignal !== 'undefined' && AbortSignal.timeout ? AbortSignal.timeout(6000) : undefined,
|
||||
});
|
||||
return { sent: true };
|
||||
} catch (err) {
|
||||
// Alerting must never break the pipeline.
|
||||
if (process.env.NODE_ENV !== 'test') console.warn('[opsNotify] failed:', err.message);
|
||||
return { sent: false, reason: err.message };
|
||||
}
|
||||
}
|
||||
|
||||
module.exports = { notify, __internals: { enabled, NTFY_URL, NTFY_TOPIC } };
|
||||
@@ -121,6 +121,14 @@ describe('oddsNormalizer', () => {
|
||||
}
|
||||
});
|
||||
|
||||
it('Session 56 audit — batter_rbis normalizes to rbi (singular), matching the grade chain', () => {
|
||||
// The whole grade/feature/outcome chain keys on 'rbi'. If this reverts to
|
||||
// 'rbis', PropLine RBI props silently stop grading + settling again.
|
||||
expect(MARKET_MAP.batter_rbis).toBe('rbi');
|
||||
expect(MARKET_MAP.batter_doubles).toBe('doubles');
|
||||
expect(MARKET_MAP.pitcher_outs).toBe('outs');
|
||||
});
|
||||
|
||||
it('exposes the NFL market keys added in the Session 31 audit', () => {
|
||||
// Defensive mapping landed before NFL is fully wired so it can't
|
||||
// repeat the MLB silent-zero bug. Both odds-api `_yds` and the
|
||||
|
||||
@@ -59,6 +59,13 @@ describe('outcomeService — settlement math', () => {
|
||||
expect(statValue({ strikeOuts: 7 }, 'strikeouts')).toBe(7);
|
||||
expect(statValue({ hits: 2 }, 'unknown_stat')).toBeNull();
|
||||
});
|
||||
test('Session 56 — newly-wired MLB fields settle (rbi, doubles, outs)', () => {
|
||||
expect(statValue({ rbi: 2 }, 'rbi')).toBe(2);
|
||||
expect(statValue({ doubles: 1 }, 'doubles')).toBe(1);
|
||||
expect(statValue({ outs: 18 }, 'outs')).toBe(18);
|
||||
expect(statValue({ baseOnBalls: 3 }, 'walks')).toBe(3);
|
||||
expect(statValue({ earnedRuns: 1 }, 'earned_runs')).toBe(1);
|
||||
});
|
||||
test('dateStrings yields both UTC and ET calendar dates', () => {
|
||||
const ds = dateStrings(ISO);
|
||||
expect(ds).toContain('2026-07-10'); // UTC
|
||||
|
||||
@@ -0,0 +1,128 @@
|
||||
'use strict';
|
||||
|
||||
// Session 56 — pipeline alerting (ntfy) + retry-once + missed-cron detection.
|
||||
|
||||
const opsNotify = require('../../src/utils/opsNotify');
|
||||
const snapshot = require('../../src/services/snapshotService');
|
||||
const { mostRecentExpectedSlot, isSnapshotOverdue } = require('../../src/snapshotScheduler');
|
||||
|
||||
function memCache(seed = {}) {
|
||||
const store = { ...seed };
|
||||
return { store, cacheGet: async (k) => (k in store ? store[k] : null), cacheSet: async (k, v) => { store[k] = v; return true; } };
|
||||
}
|
||||
|
||||
describe('opsNotify', () => {
|
||||
test('POSTs to the ntfy topic with title/priority/tags headers', async () => {
|
||||
const calls = [];
|
||||
const fetchImpl = async (url, opts) => { calls.push({ url, opts }); return { ok: true }; };
|
||||
const res = await opsNotify.notify('hello', { title: 'VYNDR', priority: 'high', tags: ['x'], fetchImpl });
|
||||
expect(res.sent).toBe(true);
|
||||
expect(calls[0].url).toMatch(/\/vyndr-pipeline-kev2026$/);
|
||||
expect(calls[0].opts.method).toBe('POST');
|
||||
expect(calls[0].opts.headers.Title).toBe('VYNDR');
|
||||
expect(calls[0].opts.headers.Priority).toBe('high');
|
||||
expect(calls[0].opts.headers.Tags).toBe('x');
|
||||
expect(calls[0].opts.body).toBe('hello');
|
||||
});
|
||||
|
||||
test('never throws on fetch failure', async () => {
|
||||
const fetchImpl = async () => { throw new Error('network down'); };
|
||||
const res = await opsNotify.notify('x', { fetchImpl });
|
||||
expect(res.sent).toBe(false);
|
||||
expect(res.reason).toBe('network down');
|
||||
});
|
||||
|
||||
test('disabled under NODE_ENV=test without an injected fetch', async () => {
|
||||
const res = await opsNotify.notify('x');
|
||||
expect(res.sent).toBe(false);
|
||||
expect(res.reason).toBe('disabled');
|
||||
});
|
||||
});
|
||||
|
||||
describe('runSnapshot — retry + alerts', () => {
|
||||
const baseDeps = (cache) => ({
|
||||
...cache,
|
||||
gradeAndCacheSlate: async (_s, _p, opts) => { opts.cacheSet('grades', { grades: [{ player: 'A', stat_type: 'hits', line: 1.5, direction: 'over', grade: 'A', confidence: 90 }] }); },
|
||||
resolveStats: async () => ({ found: false }),
|
||||
classify: () => ({ primary: null }),
|
||||
now: () => '2026-07-10T15:00:00Z',
|
||||
nowMs: () => 1000,
|
||||
sleep: async () => {}, // no real delay in tests
|
||||
retryDelayMs: 0,
|
||||
});
|
||||
|
||||
test('retries once on a thrown odds error, then succeeds', async () => {
|
||||
const cache = memCache();
|
||||
let n = 0;
|
||||
const notes = [];
|
||||
const deps = {
|
||||
...baseDeps(cache),
|
||||
getOdds: async () => { n += 1; if (n === 1) throw new Error('propline 503'); return { props: [{ player: 'A', stat_type: 'hits', line: 1.5 }], provider: 'propline' }; },
|
||||
notify: async (msg) => { notes.push(msg); return { sent: true }; },
|
||||
};
|
||||
const r = await snapshot.runSnapshot('mlb', deps);
|
||||
expect(n).toBe(2); // one retry
|
||||
expect(r.status).toBe('ok');
|
||||
expect(notes.some((m) => m.includes('✅') && m.includes('MLB'))).toBe(true);
|
||||
});
|
||||
|
||||
test('alerts failure when both attempts throw', async () => {
|
||||
const cache = memCache();
|
||||
const notes = [];
|
||||
const deps = {
|
||||
...baseDeps(cache),
|
||||
getOdds: async () => { throw new Error('down'); },
|
||||
notify: async (msg) => { notes.push(msg); return { sent: true }; },
|
||||
};
|
||||
const r = await snapshot.runSnapshot('mlb', deps);
|
||||
expect(r.status).toBe('error');
|
||||
expect(notes.some((m) => m.includes('❌') && m.includes('FAILED'))).toBe(true);
|
||||
});
|
||||
|
||||
test('alerts stale (low priority) on an empty slate, no retry', async () => {
|
||||
const cache = memCache();
|
||||
let n = 0;
|
||||
const notes = [];
|
||||
const deps = {
|
||||
...baseDeps(cache),
|
||||
getOdds: async () => { n += 1; return { props: [] }; },
|
||||
notify: async (msg) => { notes.push(msg); return { sent: true }; },
|
||||
};
|
||||
const r = await snapshot.runSnapshot('mlb', deps);
|
||||
expect(n).toBe(1); // empty is NOT a failure → no retry
|
||||
expect(r.status).toBe('skipped');
|
||||
expect(notes.some((m) => m.includes('⚠️') && m.includes('0 props'))).toBe(true);
|
||||
});
|
||||
});
|
||||
|
||||
describe('missed-cron detection', () => {
|
||||
const HOURS = [14, 19, 22, 1, 3];
|
||||
test('mostRecentExpectedSlot returns the latest scheduled hour at/before now', () => {
|
||||
const now = new Date('2026-07-10T16:30:00Z');
|
||||
const slot = mostRecentExpectedSlot(now, HOURS);
|
||||
expect(slot.getUTCHours()).toBe(14);
|
||||
expect(slot.getUTCDate()).toBe(10);
|
||||
});
|
||||
|
||||
test('overdue when now is >30m past a slot the last snapshot predates', () => {
|
||||
const now = new Date('2026-07-10T16:30:00Z'); // 2.5h past the 14:00 slot
|
||||
const lastTs = '2026-07-10T10:00:00Z'; // before 14:00 → the 14:00 run was missed
|
||||
expect(isSnapshotOverdue(lastTs, now, HOURS, 30)).toBe(true);
|
||||
});
|
||||
|
||||
test('not overdue within the grace window', () => {
|
||||
const now = new Date('2026-07-10T14:20:00Z'); // only 20m past the slot
|
||||
expect(isSnapshotOverdue('2026-07-10T10:00:00Z', now, HOURS, 30)).toBe(false);
|
||||
});
|
||||
|
||||
test('not overdue when the last snapshot is fresh (ran the slot)', () => {
|
||||
const now = new Date('2026-07-10T16:30:00Z');
|
||||
const lastTs = '2026-07-10T14:01:00Z'; // ran the 14:00 slot
|
||||
expect(isSnapshotOverdue(lastTs, now, HOURS, 30)).toBe(false);
|
||||
});
|
||||
|
||||
test('never overdue on cold boot (no prior snapshot)', () => {
|
||||
const now = new Date('2026-07-10T16:30:00Z');
|
||||
expect(isSnapshotOverdue(null, now, HOURS, 30)).toBe(false);
|
||||
});
|
||||
});
|
||||
+1
-1
File diff suppressed because one or more lines are too long
Reference in New Issue
Block a user