Session 56: Full audit — PropLine + boxscore + pipeline + sport coverage (2289 tests)

Research (verified against live MLB Stats / ESPN / The Odds APIs):
- specs/propline-audit.md — every stat_type mapped against our 4-layer pipeline;
  real MLB boxscore fields; sport coverage status; pipeline gap analysis.
- specs/vyndr-roadmap.md — priority-ordered Sessions 57–64 + coverage targets.
- scripts/propline-audit.js + specs/audit-data/ (raw capture).

Headline bug: oddsNormalizer mapped batter_rbis → 'rbis' while the whole
grade/feature/outcome chain keys on 'rbi' — every PropLine RBI prop silently
failed to grade AND settle. Fixed (+ regression test).

Phase 4 — wired missing MLB stats end-to-end:
- PropLine MLB markets 6 → 12 (+runs, walks, doubles, earned_runs, hits_allowed,
  outs — same request, no extra quota).
- doubles/outs/triples added to featureCache + outcomeService MLB_LOG_FIELD and
  all three grade whitelists (analyze/scan/validation.py).

Phase 6 — pipeline resilience:
- opsNotify.js: ntfy alerts (never throws, test-disabled). Snapshot success/
  stale/failure alerts; retry-once on hard odds error (not on empty slate).
- Missed-cron watchdog (mostRecentExpectedSlot/isSnapshotOverdue); status probe
  now returns `overdue`.

Coverage truth: MLB is the only end-to-end-live sport; outcome settlement is
MLB-only (WNBA/NBA/soccer never settle) — documented as the #1 roadmap gap.

Backend 2276 → 2289 tests (+13). Web build exit 0.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Kev
2026-07-10 17:00:29 -04:00
parent d09a06c054
commit 2ae8a5697e
21 changed files with 1607 additions and 11 deletions
+46 -3
View File
@@ -4,9 +4,52 @@
2026-07-10 2026-07-10
## Current Phase ## Current Phase
SHIP BUILD v55.0 — Product overhaul: the self-learning loop (outcome tracking + SHIP BUILD v56.0 — The full data audit: PropLine/boxscore/pipeline inventory,
accuracy) + the real-time layer (auto-refresh, freshness, live signals) + landing rbis→rbi fix, +6 MLB markets, doubles/outs wired end-to-end, ntfy alerting +
top-signals preview. Founder-pricing expiry restored. retry + missed-cron detection, master roadmap.
## Session 56 (2026-07-10) — SHIPPED ✅ THE FULL AUDIT
Backend 2276 → **2289 tests** (+13), 197 suites. Web build clean (exit 0).
Docs: `specs/propline-audit.md`, `specs/vyndr-roadmap.md`,
`specs/audit-data/propline-audit-raw.json`, `scripts/propline-audit.js`.
### Research (Phases 13, 5) — verified against LIVE APIs
- Fetched real data: The Odds API `/v4/sports` (in-season now: MLB, WNBA, NFL
preseason, NBA summer league, 40+ soccer leagues; NHL dark), a real MLB boxscore
(Braves@Pirates 2026-07-09 — every settle field confirmed), ESPN WNBA/soccer
scoreboards. PropLine inventory derived from code (no dev keys).
- **Headline bug found:** `oddsNormalizer` mapped `batter_rbis → 'rbis'` but the
entire grade/feature/outcome chain keys on `'rbi'` → **every PropLine RBI prop
silently failed to grade AND settle.** Fixed.
- Coverage truth: **MLB is the only end-to-end-live sport.** Outcome settlement is
MLB-only (WNBA/NBA/soccer never settle → accuracy is MLB-only). Documented as
the #1 roadmap gap.
### Phase 4 — wired missing MLB stat types (a full vertical slice)
- `batter_rbis → rbi` (unblocks a market already requested).
- PropLine MLB MARKETS 6 → 12: +batter_runs, +batter_walks, +batter_doubles,
+pitcher_earned_runs, +pitcher_hits_allowed, +pitcher_outs (same request, no
extra quota; runs/walks/earned_runs/hits_allowed were already supported
downstream — just never asked for).
- `doubles` + `outs` (+`triples`) added to featureCache + outcomeService
MLB_LOG_FIELD AND all three grade whitelists (analyze/scan/validation.py).
### Phase 6 — pipeline alerting + resilience
- **`src/utils/opsNotify.js`** — ntfy.sh push (`vyndr-pipeline-kev2026`), never
throws, auto-disabled under test / `PIPELINE_ALERTS=0`. Injectable `fetchImpl`.
- `snapshotService.runSnapshot`: success alert ("✅ MLB snapshot: N graded, D
deltas, X% accuracy"), stale alert (empty slate), failure alert; **retry-once**
on a hard odds error/null (NOT on a legit empty slate — that's off-hours, not a
failure).
- **Missed-cron watchdog:** `snapshotScheduler.mostRecentExpectedSlot` +
`isSnapshotOverdue` (pure, tested); the tick alerts once per missed slot; the
status probe (`GET /api/internal/snapshot/status`) now returns `overdue`.
### Phase 7 — the master roadmap
`specs/vyndr-roadmap.md` — priority-ordered Sessions 5764 (WNBA/NBA settlement →
ESPN features → soccer e2e → live calibration → NFL/NBA readiness), stat-type +
sport coverage targets, operating invariants.
## Session 55 (2026-07-10) — SHIPPED ✅ SELF-LEARNING LOOP + REAL-TIME LAYER ## Session 55 (2026-07-10) — SHIPPED ✅ SELF-LEARNING LOOP + REAL-TIME LAYER
+26
View File
@@ -658,6 +658,32 @@ snapshot, locked to the line, and read from cache.
+ the ClaimMeter pitch). That expiry lapsing — NOT a tier-limit change — was the + the ClaimMeter pitch). That expiry lapsing — NOT a tier-limit change — was the
cause of the "4 stripe test failures." Operators override via `FOUNDER_CODE_EXPIRY`. cause of the "4 stripe test failures." Operators override via `FOUNDER_CODE_EXPIRY`.
## Data Audit + Pipeline Resilience (Session 56 — non-obvious)
- **`specs/propline-audit.md` + `specs/vyndr-roadmap.md` are the source of truth**
for data coverage + the session plan. Re-run `node scripts/propline-audit.js`
(writes raw JSON to stdout; live sections need network) to refresh.
- **A market only fully works if it's wired at FOUR layers:** requested in
`proplineAdapter.MARKETS`, mapped in `oddsNormalizer.MARKET_MAP`, whitelisted in
all three grade gates (`routes/analyze.js`, `routes/scan.js`,
`python/utils/validation.py`), AND (for MLB) present in `MLB_LOG_FIELD` in BOTH
`featureCache` and `outcomeService`. Miss the map → silent zero; miss the log
field → no features + no settlement. `batter_rbis` was mapped to `rbis` while
everything else keyed on `rbi` — a 4-layer desync that silently killed RBI
props. It's `rbi` now; a normalizer test locks it.
- **The streaks/hotlist path uses its own `rbis` key** built from raw MLB stats —
independent of the odds normalizer. Don't "unify" them; the split is intentional.
- **MLB is the ONLY end-to-end-live sport.** Outcome settlement is MLB-only
(WNBA/NBA/soccer grades never settle → `accuracy` reflects MLB only). Fixing
that (ESPN box-score settle path) is roadmap Session 57.
- **`src/utils/opsNotify.js`** pushes pipeline alerts to ntfy (`vyndr-pipeline-
kev2026`). It NEVER throws and is auto-disabled under `NODE_ENV==='test'` /
`PIPELINE_ALERTS=0` (inject `fetchImpl` to test it). `snapshotService` alerts on
success/stale/failure; `snapshotScheduler` has a per-minute missed-cron watchdog
(`isSnapshotOverdue`, exposed as `overdue` on the status probe).
- **Snapshot retry rule:** retry-once ONLY on a thrown/null odds response (a
transient provider blip). A successful-but-empty slate is NOT retried — that's a
legit off-hours empty slate, and retrying would waste quota + add 60s latency.
## Active Skills ## Active Skills
- vyndr-voice (all user-facing output) - vyndr-voice (all user-facing output)
- prop-analysis (grading methodology) - prop-analysis (grading methodology)
+114
View File
@@ -0,0 +1,114 @@
'use strict';
/**
* scripts/propline-audit.js (Session 56) — the data-source audit.
*
* Inventories what our providers CAN send (from code) and what the FREE
* settled-result APIs (MLB Stats, ESPN) actually return, so we can map every
* prop stat_type to its box-score field. PropLine live fetch runs only when
* PROPLINE_API_KEY_* are present (they aren't in dev) — otherwise we report the
* authoritative code inventory (MARKETS × MARKET_MAP).
*
* Usage: node scripts/propline-audit.js (writes JSON to stdout)
*/
require('dotenv').config({ quiet: true });
const propline = require('../src/services/adapters/proplineAdapter');
const { MARKET_MAP } = require('../src/utils/oddsNormalizer');
const out = { generatedAt: new Date().toISOString(), sections: {} };
async function getJson(url, headers) {
const r = await fetch(url, { headers: headers || {}, signal: AbortSignal.timeout(12000) });
if (!r.ok) throw new Error(`HTTP ${r.status}`);
return r.json();
}
// 1. PropLine code inventory — the markets we REQUEST per sport + the stat_type
// each normalizes to (or "UNMAPPED" if MARKET_MAP has no entry → silent zero).
function proplineInventory() {
const { MARKETS, SPORT_KEYS } = propline.__internals;
const map = MARKET_MAP || {};
const inv = {};
for (const [sport, markets] of Object.entries(MARKETS)) {
inv[sport] = {
sportKey: SPORT_KEYS[sport],
requested: markets.map((m) => ({ market: m, stat_type: map[m] || 'UNMAPPED' })),
};
}
// Also list every MARKET_MAP entry (what we CAN normalize even if not requested).
inv._allMappedMarkets = Object.entries(map).map(([m, s]) => ({ market: m, stat_type: s }));
inv._hasKeys = propline.hasKeys();
return inv;
}
// 2. The Odds API active sports (FREE — /v4/sports does not spend quota).
async function oddsApiSports() {
const key = process.env.ODDS_API_KEY;
if (!key) return { skipped: 'no ODDS_API_KEY' };
try {
const data = await getJson(`https://api.the-odds-api.com/v4/sports?apiKey=${key}`);
return (data || [])
.filter((s) => s.active)
.map((s) => ({ key: s.key, group: s.group, title: s.title }));
} catch (e) { return { error: e.message }; }
}
// 3. MLB Stats API — a recent FINAL game's boxscore field keys + a game-log row.
async function mlbBoxscore() {
try {
// Yesterday's schedule.
const d = new Date(Date.now() - 24 * 3600 * 1000).toISOString().slice(0, 10);
const sched = await getJson(`https://statsapi.mlb.com/api/v1/schedule?sportId=1&date=${d}`);
const games = (sched.dates?.[0]?.games) || [];
const final = games.find((g) => g.status?.abstractGameState === 'Final') || games[0];
if (!final) return { note: `no games ${d}` };
const box = await getJson(`https://statsapi.mlb.com/api/v1/game/${final.gamePk}/boxscore`);
// Pull one batter + one pitcher stat object.
const sampleTeam = box.teams?.away || box.teams?.home || {};
const players = Object.values(sampleTeam.players || {});
const batter = players.find((p) => p.stats?.batting && Object.keys(p.stats.batting).length);
const pitcher = players.find((p) => p.stats?.pitching && Object.keys(p.stats.pitching).length);
return {
date: d, gamePk: final.gamePk, matchup: final.teams?.away?.team?.name + ' @ ' + final.teams?.home?.team?.name,
battingFields: batter ? Object.keys(batter.stats.batting) : [],
pitchingFields: pitcher ? Object.keys(pitcher.stats.pitching) : [],
sampleBatting: batter ? batter.stats.batting : null,
samplePitching: pitcher ? pitcher.stats.pitching : null,
};
} catch (e) { return { error: e.message }; }
}
// 4. ESPN WNBA — today's scoreboard status + a completed game's boxscore labels.
async function espnWnba() {
try {
const sb = await getJson('https://site.api.espn.com/apis/site/v2/sports/basketball/wnba/scoreboard');
const events = (sb.events || []).map((e) => ({ name: e.name, status: e.status?.type?.description, date: e.date }));
const final = (sb.events || []).find((e) => e.status?.type?.completed);
let boxLabels = null;
if (final) {
const summary = await getJson(`https://site.api.espn.com/apis/site/v2/sports/basketball/wnba/summary?event=${final.id}`);
const teamStats = summary.boxscore?.players?.[0]?.statistics?.[0];
boxLabels = teamStats ? { labels: teamStats.labels, names: teamStats.names } : null;
}
return { count: events.length, events, completedBoxLabels: boxLabels };
} catch (e) { return { error: e.message }; }
}
// 5. ESPN soccer — check a common league scoreboard for activity.
async function espnSoccer() {
try {
const sb = await getJson('https://site.api.espn.com/apis/site/v2/sports/soccer/usa.1/scoreboard');
const events = (sb.events || []).map((e) => ({ name: e.name, status: e.status?.type?.description }));
return { league: 'usa.1 (MLS)', count: events.length, events: events.slice(0, 6) };
} catch (e) { return { error: e.message }; }
}
(async () => {
out.sections.proplineInventory = proplineInventory();
out.sections.oddsApiActiveSports = await oddsApiSports();
out.sections.mlbBoxscore = await mlbBoxscore();
out.sections.espnWnba = await espnWnba();
out.sections.espnSoccer = await espnSoccer();
process.stdout.write(JSON.stringify(out, null, 2) + '\n');
})();
+875
View File
@@ -0,0 +1,875 @@
{
"generatedAt": "2026-07-10T20:48:09.307Z",
"sections": {
"proplineInventory": {
"nba": {
"sportKey": "basketball_nba",
"requested": [
{
"market": "player_points",
"stat_type": "points"
},
{
"market": "player_rebounds",
"stat_type": "rebounds"
},
{
"market": "player_assists",
"stat_type": "assists"
},
{
"market": "player_threes",
"stat_type": "threes"
},
{
"market": "player_blocks",
"stat_type": "blocks"
},
{
"market": "player_steals",
"stat_type": "steals"
}
]
},
"wnba": {
"sportKey": "basketball_wnba",
"requested": [
{
"market": "player_points",
"stat_type": "points"
},
{
"market": "player_rebounds",
"stat_type": "rebounds"
},
{
"market": "player_assists",
"stat_type": "assists"
},
{
"market": "player_threes",
"stat_type": "threes"
}
]
},
"mlb": {
"sportKey": "baseball_mlb",
"requested": [
{
"market": "batter_hits",
"stat_type": "hits"
},
{
"market": "batter_home_runs",
"stat_type": "home_runs"
},
{
"market": "batter_total_bases",
"stat_type": "total_bases"
},
{
"market": "batter_rbis",
"stat_type": "rbis"
},
{
"market": "batter_stolen_bases",
"stat_type": "stolen_bases"
},
{
"market": "pitcher_strikeouts",
"stat_type": "strikeouts"
}
]
},
"nfl": {
"sportKey": "football_nfl",
"requested": [
{
"market": "player_pass_yds",
"stat_type": "passing_yards"
},
{
"market": "player_rush_yds",
"stat_type": "rushing_yards"
},
{
"market": "player_reception_yds",
"stat_type": "receiving_yards"
},
{
"market": "player_receptions",
"stat_type": "receptions"
},
{
"market": "player_anytime_td",
"stat_type": "anytime_td"
},
{
"market": "player_pass_tds",
"stat_type": "pass_tds"
}
]
},
"nhl": {
"sportKey": "hockey_nhl",
"requested": [
{
"market": "player_goals",
"stat_type": "goals"
},
{
"market": "player_shots_on_goal",
"stat_type": "shots_on_goal"
},
{
"market": "goalie_saves",
"stat_type": "saves"
}
]
},
"ncaab": {
"sportKey": "basketball_ncaab",
"requested": [
{
"market": "player_points",
"stat_type": "points"
},
{
"market": "player_rebounds",
"stat_type": "rebounds"
},
{
"market": "player_assists",
"stat_type": "assists"
}
]
},
"_allMappedMarkets": [
{
"market": "player_points",
"stat_type": "points"
},
{
"market": "player_rebounds",
"stat_type": "rebounds"
},
{
"market": "player_assists",
"stat_type": "assists"
},
{
"market": "player_threes",
"stat_type": "threes"
},
{
"market": "player_blocks",
"stat_type": "blocks"
},
{
"market": "player_steals",
"stat_type": "steals"
},
{
"market": "player_points_rebounds_assists",
"stat_type": "pra"
},
{
"market": "player_turnovers",
"stat_type": "turnovers"
},
{
"market": "batter_hits",
"stat_type": "hits"
},
{
"market": "batter_home_runs",
"stat_type": "home_runs"
},
{
"market": "batter_total_bases",
"stat_type": "total_bases"
},
{
"market": "batter_rbis",
"stat_type": "rbis"
},
{
"market": "batter_runs",
"stat_type": "runs"
},
{
"market": "batter_stolen_bases",
"stat_type": "stolen_bases"
},
{
"market": "batter_singles",
"stat_type": "singles"
},
{
"market": "batter_doubles",
"stat_type": "doubles"
},
{
"market": "batter_walks",
"stat_type": "walks"
},
{
"market": "batter_strikeouts",
"stat_type": "batter_strikeouts"
},
{
"market": "pitcher_strikeouts",
"stat_type": "strikeouts"
},
{
"market": "pitcher_earned_runs",
"stat_type": "earned_runs"
},
{
"market": "pitcher_hits_allowed",
"stat_type": "hits_allowed"
},
{
"market": "pitcher_outs",
"stat_type": "outs"
},
{
"market": "player_pass_yds",
"stat_type": "passing_yards"
},
{
"market": "player_pass_yards",
"stat_type": "passing_yards"
},
{
"market": "player_pass_tds",
"stat_type": "pass_tds"
},
{
"market": "player_pass_completions",
"stat_type": "pass_completions"
},
{
"market": "player_pass_attempts",
"stat_type": "pass_attempts"
},
{
"market": "player_pass_interceptions",
"stat_type": "interceptions"
},
{
"market": "player_rush_yds",
"stat_type": "rushing_yards"
},
{
"market": "player_rush_yards",
"stat_type": "rushing_yards"
},
{
"market": "player_rush_attempts",
"stat_type": "rush_attempts"
},
{
"market": "player_rush_tds",
"stat_type": "rush_tds"
},
{
"market": "player_receptions",
"stat_type": "receptions"
},
{
"market": "player_reception_yds",
"stat_type": "receiving_yards"
},
{
"market": "player_receiving_yards",
"stat_type": "receiving_yards"
},
{
"market": "player_reception_tds",
"stat_type": "reception_tds"
},
{
"market": "player_anytime_td",
"stat_type": "anytime_td"
},
{
"market": "player_kicking_points",
"stat_type": "kicking_points"
},
{
"market": "player_goals",
"stat_type": "goals"
},
{
"market": "player_shots_on_target",
"stat_type": "shots_on_target"
},
{
"market": "player_shots",
"stat_type": "shots"
},
{
"market": "player_tackles",
"stat_type": "tackles"
},
{
"market": "player_cards",
"stat_type": "cards"
},
{
"market": "player_corners",
"stat_type": "corners"
},
{
"market": "player_saves",
"stat_type": "saves"
},
{
"market": "player_goals_conceded",
"stat_type": "goals_conceded"
},
{
"market": "player_passes",
"stat_type": "passes"
},
{
"market": "team_clean_sheet",
"stat_type": "clean_sheet"
},
{
"market": "player_shots_on_goal",
"stat_type": "shots_on_goal"
},
{
"market": "goalie_saves",
"stat_type": "saves"
}
],
"_hasKeys": false
},
"oddsApiActiveSports": [
{
"key": "americanfootball_cfl",
"group": "American Football",
"title": "CFL"
},
{
"key": "americanfootball_ncaaf",
"group": "American Football",
"title": "NCAAF"
},
{
"key": "americanfootball_ncaaf_championship_winner",
"group": "American Football",
"title": "NCAAF Championship Winner"
},
{
"key": "americanfootball_nfl",
"group": "American Football",
"title": "NFL"
},
{
"key": "americanfootball_nfl_preseason",
"group": "American Football",
"title": "NFL Preseason"
},
{
"key": "americanfootball_nfl_super_bowl_winner",
"group": "American Football",
"title": "NFL Super Bowl Winner"
},
{
"key": "aussierules_afl",
"group": "Aussie Rules",
"title": "AFL"
},
{
"key": "baseball_milb",
"group": "Baseball",
"title": "MiLB"
},
{
"key": "baseball_mlb",
"group": "Baseball",
"title": "MLB"
},
{
"key": "baseball_mlb_world_series_winner",
"group": "Baseball",
"title": "MLB World Series Winner"
},
{
"key": "baseball_npb",
"group": "Baseball",
"title": "NPB"
},
{
"key": "basketball_nba_summer_league",
"group": "Basketball",
"title": "NBA Summer League"
},
{
"key": "basketball_wnba",
"group": "Basketball",
"title": "WNBA"
},
{
"key": "boxing_boxing",
"group": "Boxing",
"title": "Boxing"
},
{
"key": "cricket_international_t20",
"group": "Cricket",
"title": "International Twenty20"
},
{
"key": "cricket_odi",
"group": "Cricket",
"title": "One Day Internationals"
},
{
"key": "cricket_t20_blast",
"group": "Cricket",
"title": "T20 Blast"
},
{
"key": "golf_the_open_championship_winner",
"group": "Golf",
"title": "The Open Winner"
},
{
"key": "lacrosse_pll",
"group": "Lacrosse",
"title": "PLL"
},
{
"key": "mma_mixed_martial_arts",
"group": "Mixed Martial Arts",
"title": "MMA"
},
{
"key": "politics_us_presidential_election_winner",
"group": "Politics",
"title": "US Presidential Elections Winner"
},
{
"key": "rugbyleague_nrl",
"group": "Rugby League",
"title": "NRL"
},
{
"key": "soccer_argentina_primera_division",
"group": "Soccer",
"title": "Primera División - Argentina"
},
{
"key": "soccer_austria_bundesliga",
"group": "Soccer",
"title": "Austrian Football Bundesliga"
},
{
"key": "soccer_brazil_campeonato",
"group": "Soccer",
"title": "Brazil Série A"
},
{
"key": "soccer_brazil_serie_b",
"group": "Soccer",
"title": "Brazil Série B"
},
{
"key": "soccer_china_superleague",
"group": "Soccer",
"title": "Super League - China"
},
{
"key": "soccer_conmebol_copa_libertadores",
"group": "Soccer",
"title": "Copa Libertadores"
},
{
"key": "soccer_conmebol_copa_sudamericana",
"group": "Soccer",
"title": "Copa Sudamericana"
},
{
"key": "soccer_denmark_superliga",
"group": "Soccer",
"title": "Denmark Superliga"
},
{
"key": "soccer_efl_champ",
"group": "Soccer",
"title": "Championship"
},
{
"key": "soccer_england_efl_cup",
"group": "Soccer",
"title": "EFL Cup"
},
{
"key": "soccer_england_league1",
"group": "Soccer",
"title": "League 1"
},
{
"key": "soccer_england_league2",
"group": "Soccer",
"title": "League 2"
},
{
"key": "soccer_epl",
"group": "Soccer",
"title": "EPL"
},
{
"key": "soccer_fifa_world_cup",
"group": "Soccer",
"title": "FIFA World Cup"
},
{
"key": "soccer_fifa_world_cup_winner",
"group": "Soccer",
"title": "FIFA World Cup Winner"
},
{
"key": "soccer_finland_veikkausliiga",
"group": "Soccer",
"title": "Veikkausliiga - Finland"
},
{
"key": "soccer_france_ligue_one",
"group": "Soccer",
"title": "Ligue 1 - France"
},
{
"key": "soccer_germany_bundesliga",
"group": "Soccer",
"title": "Bundesliga - Germany"
},
{
"key": "soccer_germany_bundesliga2",
"group": "Soccer",
"title": "Bundesliga 2 - Germany"
},
{
"key": "soccer_germany_dfb_pokal",
"group": "Soccer",
"title": "DFB-Pokal"
},
{
"key": "soccer_germany_liga3",
"group": "Soccer",
"title": "3. Liga - Germany"
},
{
"key": "soccer_italy_serie_a",
"group": "Soccer",
"title": "Serie A - Italy"
},
{
"key": "soccer_korea_kleague1",
"group": "Soccer",
"title": "K League 1"
},
{
"key": "soccer_league_of_ireland",
"group": "Soccer",
"title": "League of Ireland"
},
{
"key": "soccer_mexico_ligamx",
"group": "Soccer",
"title": "Liga MX"
},
{
"key": "soccer_netherlands_eredivisie",
"group": "Soccer",
"title": "Dutch Eredivisie"
},
{
"key": "soccer_norway_eliteserien",
"group": "Soccer",
"title": "Eliteserien - Norway"
},
{
"key": "soccer_russia_premier_league",
"group": "Soccer",
"title": "Premier League - Russia"
},
{
"key": "soccer_spain_la_liga",
"group": "Soccer",
"title": "La Liga - Spain"
},
{
"key": "soccer_spl",
"group": "Soccer",
"title": "Premiership - Scotland"
},
{
"key": "soccer_sweden_allsvenskan",
"group": "Soccer",
"title": "Allsvenskan - Sweden"
},
{
"key": "soccer_sweden_superettan",
"group": "Soccer",
"title": "Superettan - Sweden"
},
{
"key": "soccer_switzerland_superleague",
"group": "Soccer",
"title": "Swiss Superleague"
},
{
"key": "soccer_uefa_champs_league_qualification",
"group": "Soccer",
"title": "UEFA Champions League Qualification"
},
{
"key": "soccer_usa_mls",
"group": "Soccer",
"title": "MLS"
},
{
"key": "tennis_atp_wimbledon",
"group": "Tennis",
"title": "ATP Wimbledon"
},
{
"key": "tennis_wta_wimbledon",
"group": "Tennis",
"title": "WTA Wimbledon"
}
],
"mlbBoxscore": {
"date": "2026-07-09",
"gamePk": 823359,
"matchup": "Atlanta Braves @ Pittsburgh Pirates",
"battingFields": [
"summary",
"gamesPlayed",
"flyOuts",
"groundOuts",
"airOuts",
"runs",
"doubles",
"triples",
"homeRuns",
"strikeOuts",
"baseOnBalls",
"intentionalWalks",
"hits",
"hitByPitch",
"atBats",
"caughtStealing",
"stolenBases",
"stolenBasePercentage",
"groundIntoDoublePlay",
"groundIntoTriplePlay",
"plateAppearances",
"totalBases",
"rbi",
"leftOnBase",
"sacBunts",
"sacFlies",
"catchersInterference",
"pickoffs",
"atBatsPerHomeRun",
"popOuts",
"lineOuts"
],
"pitchingFields": [
"note",
"summary",
"gamesPlayed",
"gamesStarted",
"flyOuts",
"groundOuts",
"airOuts",
"runs",
"doubles",
"triples",
"homeRuns",
"strikeOuts",
"baseOnBalls",
"intentionalWalks",
"hits",
"hitByPitch",
"atBats",
"caughtStealing",
"stolenBases",
"stolenBasePercentage",
"numberOfPitches",
"inningsPitched",
"wins",
"losses",
"saves",
"saveOpportunities",
"holds",
"blownSaves",
"earnedRuns",
"battersFaced",
"outs",
"gamesPitched",
"completeGames",
"shutouts",
"pitchesThrown",
"balls",
"strikes",
"strikePercentage",
"hitBatsmen",
"balks",
"wildPitches",
"pickoffs",
"rbi",
"gamesFinished",
"runsScoredPer9",
"homeRunsPer9",
"inheritedRunners",
"inheritedRunnersScored",
"catchersInterference",
"sacBunts",
"sacFlies",
"passedBall",
"popOuts",
"lineOuts"
],
"sampleBatting": {
"summary": "1-6 | K, R, SB",
"gamesPlayed": 1,
"flyOuts": 1,
"groundOuts": 1,
"airOuts": 3,
"runs": 1,
"doubles": 0,
"triples": 0,
"homeRuns": 0,
"strikeOuts": 1,
"baseOnBalls": 0,
"intentionalWalks": 0,
"hits": 1,
"hitByPitch": 0,
"atBats": 6,
"caughtStealing": 0,
"stolenBases": 1,
"stolenBasePercentage": "1.000",
"groundIntoDoublePlay": 0,
"groundIntoTriplePlay": 0,
"plateAppearances": 6,
"totalBases": 1,
"rbi": 0,
"leftOnBase": 3,
"sacBunts": 0,
"sacFlies": 0,
"catchersInterference": 0,
"pickoffs": 0,
"atBatsPerHomeRun": "-.--",
"popOuts": 0,
"lineOuts": 2
},
"samplePitching": {
"note": "(W, 2-0)",
"summary": "1.0 IP, 0 ER, 0 K, 0 BB",
"gamesPlayed": 1,
"gamesStarted": 0,
"flyOuts": 1,
"groundOuts": 1,
"airOuts": 2,
"runs": 0,
"doubles": 0,
"triples": 0,
"homeRuns": 0,
"strikeOuts": 0,
"baseOnBalls": 0,
"intentionalWalks": 0,
"hits": 0,
"hitByPitch": 0,
"atBats": 3,
"caughtStealing": 0,
"stolenBases": 0,
"stolenBasePercentage": ".---",
"numberOfPitches": 15,
"inningsPitched": "1.0",
"wins": 1,
"losses": 0,
"saves": 0,
"saveOpportunities": 0,
"holds": 0,
"blownSaves": 0,
"earnedRuns": 0,
"battersFaced": 3,
"outs": 3,
"gamesPitched": 1,
"completeGames": 0,
"shutouts": 0,
"pitchesThrown": 15,
"balls": 6,
"strikes": 9,
"strikePercentage": ".600",
"hitBatsmen": 0,
"balks": 0,
"wildPitches": 0,
"pickoffs": 0,
"rbi": 0,
"gamesFinished": 0,
"runsScoredPer9": "0.00",
"homeRunsPer9": "0.00",
"inheritedRunners": 0,
"inheritedRunnersScored": 0,
"catchersInterference": 0,
"sacBunts": 0,
"sacFlies": 0,
"passedBall": 0,
"popOuts": 1,
"lineOuts": 0
}
},
"espnWnba": {
"count": 3,
"events": [
{
"name": "Golden State Valkyries at Connecticut Sun",
"status": "Scheduled",
"date": "2026-07-10T23:30Z"
},
{
"name": "Dallas Wings at Toronto Tempo",
"status": "Scheduled",
"date": "2026-07-10T23:30Z"
},
{
"name": "Chicago Sky at Los Angeles Sparks",
"status": "Scheduled",
"date": "2026-07-11T02:00Z"
}
],
"completedBoxLabels": null
},
"espnSoccer": {
"league": "usa.1 (MLS)",
"count": 4,
"events": [
{
"name": "Toronto FC at CF Montréal",
"status": "Scheduled"
},
{
"name": "Vancouver Whitecaps at Chicago Fire FC",
"status": "Scheduled"
},
{
"name": "Sporting Kansas City at St. Louis CITY SC",
"status": "Scheduled"
},
{
"name": "Portland Timbers at Seattle Sounders FC",
"status": "Scheduled"
}
]
}
}
}
+145
View File
@@ -0,0 +1,145 @@
# PropLine + Box Score + Pipeline Audit — 2026-07-10 (Session 56)
Ground-truth inventory of what data we can access, what our pipeline handles, and
the gaps. Live sections were fetched against real APIs (MLB Stats, ESPN, The Odds
API `/v4/sports`). PropLine has no dev keys, so its inventory is derived from the
authoritative code (`proplineAdapter.MARKETS` × `oddsNormalizer.MARKET_MAP`) —
that IS the set our pipeline requests + normalizes. Raw capture:
`specs/audit-data/propline-audit-raw.json`.
Baseline: **2276 tests** passing at session start.
---
## 0. What is in-season RIGHT NOW (The Odds API `/v4/sports`, live 2026-07-10)
| Sport | Active key(s) | Notes |
|-------|---------------|-------|
| MLB | `baseball_mlb` | ✅ full season — the live product |
| WNBA | `basketball_wnba` | ✅ in-season (3 games tonight per ESPN) |
| NBA | `basketball_nba_summer_league` only | ⚠️ regular NBA off until Oct; summer league live |
| NFL | `americanfootball_nfl`, `_nfl_preseason` | ⚠️ preseason lines starting |
| Soccer | 40+ leagues (`soccer_epl`, `soccer_usa_mls`, `soccer_fifa_world_cup`, La Liga, Serie A, …) | ✅ year-round via odds-api |
| NHL | (absent) | off-season — no lines |
Takeaway: **MLB + WNBA + soccer are the live opportunity today.** NBA/NFL are
pre-season shells; NHL is dark.
---
## 1. PropLine / odds inventory — what we REQUEST vs what we CAN normalize
`proplineAdapter.MARKETS` = the markets we actually ask for. `MARKET_MAP` = every
market we can normalize (superset). A market we request but don't map → **silent
zero**; a market we map but don't request → **unused capacity** (free to add — same
call).
### MLB
| Requested market | → stat_type | Grade whitelist? | Features (l5/l20)? | Outcome settle? |
|---|---|---|---|---|
| batter_hits | hits | ✅ | ✅ hits | ✅ hits |
| batter_home_runs | home_runs | ✅ | ✅ homeRuns | ✅ homeRuns |
| batter_total_bases | total_bases | ✅ | ✅ totalBases | ✅ totalBases |
| batter_rbis | **rbis** ❌ | ✅ (`rbi`) | ✅ (`rbi`) | ✅ (`rbi`) | ← **BUG: normalizes to `rbis`, whole chain keys on `rbi` → never grades/settles** |
| batter_stolen_bases | stolen_bases | ✅ | ✅ stolenBases | ✅ stolenBases |
| pitcher_strikeouts | strikeouts | ✅ | ✅ strikeOuts | ✅ strikeOuts |
**Mapped but NOT requested (free capacity — all supported downstream already):**
`batter_runs`→runs, `batter_walks`→walks, `pitcher_earned_runs`→earned_runs,
`pitcher_hits_allowed`→hits_allowed. **Mapped, not requested, and needs feature/
settle wiring:** `batter_doubles`→doubles, `pitcher_outs`→outs. Confirmed present
in the real boxscore (§2).
### WNBA / NBA
Requested: `player_points`, `player_rebounds`, `player_assists`, `player_threes`
(+ NBA adds `player_blocks`, `player_steals`). All grade-whitelisted. **Features:
NBA/WNBA game logs come from the Python nba_api service (usually OFFLINE in prod)
→ l5/l20 often empty; ESPN fallback exists for season avgs only. Outcomes: NOT
settled at all (outcomeService is MLB-only).** `player_turnovers`/`player_pra` are
mapped but not requested.
### Soccer
**PropLine has NO soccer entry** (`SPORT_KEYS`/`MARKETS` omit it) — soccer props
can only come from the odds-api backup. `MARKET_MAP` maps goals/shots/shots_on_
target/tackles/cards/corners/saves/passes/clean_sheet. Features: soccer extractor
exists; outcomes: NOT settled.
### NFL / NHL
NFL markets mapped + requested (pass/rush/rec yds, receptions, TDs) but the flow
isn't graded end-to-end yet. NHL mapped/requested but off-season.
---
## 2. MLB box score field map (REAL — Braves @ Pirates, 2026-07-09)
Batting fields present: `runs, doubles, triples, homeRuns, strikeOuts,
baseOnBalls, hits, atBats, stolenBases, totalBases, rbi, sacFlies, …`
Pitching fields present: `strikeOuts, baseOnBalls, hits, inningsPitched,
earnedRuns, outs, battersFaced, …`
**OUTCOME_MAP (MLB) — stat_type → game-log/boxscore field:**
```
hits → hits total_bases → totalBases home_runs → homeRuns
rbi → rbi runs → runs walks → baseOnBalls
stolen_bases → stolenBases strikeouts → strikeOuts earned_runs → earnedRuns
hits_allowed → hits (pitching) innings_pitched → inningsPitched
doubles → doubles ⟵ NEW triples → triples ⟵ NEW outs → outs ⟵ NEW
batter_strikeouts → strikeOuts ⟵ (batter Ks — roadmap)
```
Every field above is real in the boxscore/game log → all are settleable today.
## 2b. ESPN box score (WNBA/soccer — the non-MLB settle path)
WNBA settlement source = ESPN summary `boxscore.players[].statistics[]`
(`labels`/`names` arrays: PTS/REB/AST/3PM/STL/BLK/TO). No completed game at fetch
time (all 3 tonight were Scheduled), so labels weren't captured — wiring the ESPN
settle path is a roadmap item (`espnStatsAdapter` already parses this shape for
season avgs).
---
## 3. Pipeline flow + gaps
**Current flow:** cron (`SNAPSHOT_CRON=1`, UTC 14,19,22,1,3) → per sport:
`settleAllOutcomes()` (S55, MLB only) → `snapshotService.runSnapshot` → PropLine
(3-key rotation) / odds-api backup → `gradeAndCacheSlate` → per-player archetype +
features → Redis (`snapshot:{sport}:latest`, `grades:{sport}`) → ticker.
**Gaps found:**
1. **RBI silent failure**`batter_rbis → rbis` mismatch (§1). FIX this session.
2. **Under-requesting MLB markets** — only 6 of ~12 supported; runs/walks/doubles/
earned_runs/hits_allowed/outs are free in the same call. EXPAND this session.
3. **Outcome settlement is MLB-only** — WNBA/NBA/soccer grades NEVER settle, so
`accuracy` is MLB-only. Biggest roadmap item (needs the ESPN settle path).
4. **No pipeline alerting** — a failed/empty/missed cron run is silent. ADD ntfy
this session.
5. **No retry** — one PropLine blip marks a sport unavailable for the whole cycle.
ADD retry-once this session.
6. **No missed-cron detector** — nothing notices if a slot didn't fire. ADD.
7. **NBA/WNBA features depend on an offline Python service** — l5/l20 often empty;
ESPN gives season avgs only. Roadmap: an ESPN game-log feature path.
8. **Soccer has no PropLine source** — odds-api backup only; document + monitor.
---
## 4. Sport coverage status (verified 2026-07-10)
| Sport | Odds props | Grade pipeline | Features | Outcomes | Status |
|-------|-----------|----------------|----------|----------|--------|
| MLB | ✅ PropLine + odds-api | ✅ | ✅ real (statsapi) | ✅ (S55) | **LIVE** |
| WNBA | ✅ odds-api/PropLine | ✅ | ⚠️ Python offline → thin | ❌ not settled | **PARTIAL** |
| NBA | ⚠️ summer league only | ✅ | ⚠️ offline | ❌ | **OFF-SEASON** |
| Soccer | ✅ odds-api (no PropLine) | ⚠️ extractor exists | ⚠️ | ❌ | **PARTIAL** |
| NFL | ⚠️ preseason | ⚠️ mapped, not e2e | ❌ | ❌ | **PRE-SEASON** |
| NHL | ❌ off-season | ⚠️ mapped | ❌ | ❌ | **DARK** |
---
## 5. Fixes shipped this session (Phase 4/6)
- `batter_rbis → rbi` (unblocks a market we already request).
- PropLine MLB MARKETS expanded: +batter_runs, +batter_walks, +batter_doubles,
+pitcher_earned_runs, +pitcher_hits_allowed, +pitcher_outs.
- `doubles` + `outs` wired into featureCache + outcomeService + all three grade
whitelists (analyze.js / scan.js / validation.py).
- Pipeline alerting (ntfy) + PropLine retry-once + missed-cron detector.
See `specs/vyndr-roadmap.md` for the priority-ordered plan that follows from this.
+96
View File
@@ -0,0 +1,96 @@
# VYNDR Master Roadmap — Generated 2026-07-10 (Session 56)
The foundation document. Derived from the Session-56 data audit
(`specs/propline-audit.md`, verified against live MLB Stats / ESPN / Odds APIs).
Every session after this references it. Update it as sessions ship.
---
## Current coverage (verified 2026-07-10)
| Sport | Odds props | Grading | Features (l5/l20) | Outcomes settled | Status |
|-------|-----------|---------|-------------------|------------------|--------|
| MLB | ✅ PropLine + odds-api | ✅ | ✅ real (statsapi) | ✅ (S55, expanded S56) | **LIVE** |
| WNBA | ✅ odds-api/PropLine | ✅ | ⚠️ Python offline → thin | ❌ | **PARTIAL** |
| Soccer | ✅ odds-api only (no PropLine) | ⚠️ extractor only | ⚠️ | ❌ | **PARTIAL** |
| NBA | ⚠️ summer league only | ✅ | ⚠️ offline | ❌ | **OFF-SEASON (Oct)** |
| NFL | ⚠️ preseason | ⚠️ mapped, not e2e | ❌ | ❌ | **PRE-SEASON (Sep)** |
| NHL | ❌ off-season | ⚠️ mapped | ❌ | ❌ | **DARK (Oct)** |
**The core product is MLB.** It is the only sport that is live end-to-end
(odds → grade → real features → settled accuracy). Everything else is a
build-out target.
---
## Gap analysis (from the audit)
1. **Outcome settlement is MLB-only** — WNBA/NBA/soccer grades never settle, so
the accuracy record (the S55 trust engine) only reflects MLB. **Highest-value
gap.** Needs the ESPN box-score settle path (`espnStatsAdapter` already parses
the shape for season avgs).
2. **NBA/WNBA features depend on an offline Python nba_api service** — l5/l20 are
often empty; only season avgs survive (ESPN fallback). Needs an ESPN game-log
feature path so intel populates without the Python service.
3. **Soccer has no PropLine source** — odds-api backup only; no `SPORT_KEYS` entry.
Soccer grading isn't wired into the snapshot pipeline end-to-end.
4. **NFL mapped but not graded end-to-end** — markets normalize, but no feature/
outcome path. Wire before September.
5. ~~RBI silent-failure (rbis/rbi)~~**FIXED S56.**
6. ~~Under-requesting MLB markets~~**FIXED S56** (6 → 12 markets).
7. ~~No pipeline alerting / retry / missed-cron~~**FIXED S56.**
8. **No live calibration** — the accuracy record exists but doesn't yet feed back
into grade confidence (spec 2.3 from S55).
9. **`batter_strikeouts` (batter Ks)** mapped but not feature/settle-wired.
---
## Session plan (priority-ordered)
| Session | Focus | Ships | Scope |
|---------|-------|-------|-------|
| 57 | **WNBA/NBA outcome settlement** | ESPN box-score settle path in outcomeService → accuracy for basketball; WNBA goes fully live | M |
| 58 | **ESPN game-log features** | l5/l10/l20 for NBA/WNBA without the Python service → real intel on basketball grade cards | M |
| 59 | **Soccer end-to-end** | soccer into the snapshot pipeline (odds-api source), feature extractor wired, ESPN settle | L |
| 60 | **Live calibration** | accuracy record feeds grade-confidence adjustment (under/over-confident tiers nudge); "Model health: Calibrated/Learning" indicator | M |
| 61 | **NFL pre-season prep** | NFL feature + settle path (ESPN NFL boxscore) so Week 1 (Sep) is live | L |
| 62 | **Prop breadth** | `batter_strikeouts`, `triples`, pitcher `walks`; NBA `pra`/`turnovers` requested; WNBA extra markets | S |
| 63 | **Accuracy depth** | per-archetype hit rates, per-stat hit rates, player-level record ("VYNDR on Judge: 12-5"); ledger UI | M |
| 64 | **NBA regular-season readiness** | verify NBA pipeline for the Oct tip-off; depth-chart/cascade live | M |
Scope key: S ≈ ½ session, M ≈ 1 session, L ≈ 12 sessions.
---
## Stat-type coverage target (fully built)
- **MLB (batters):** hits, total_bases, home_runs, rbi, runs, doubles, triples,
walks, stolen_bases, batter_strikeouts. **(pitchers):** strikeouts, earned_runs,
hits_allowed, innings_pitched, outs, walks. *(Bold-new this session: doubles,
triples, outs, runs, walks requested + settleable.)*
- **NBA/WNBA:** points, rebounds, assists, threes, steals, blocks, turnovers, pra.
- **Soccer:** goals, shots, shots_on_target, tackles, cards, corners, saves,
passes, clean_sheet.
- **NFL:** passing/rushing/receiving yards, receptions, pass/rush/rec TDs,
anytime_td, interceptions, kicking_points.
## Sport coverage target
- **MLB** ✅ live now (settled accuracy).
- **WNBA** → live after S57 (settlement) + S58 (features).
- **Soccer** → live after S59.
- **NFL** → ready for September (S61).
- **NBA** → ready for October (S64).
- **NHL** → ready for October (fast-follow once NBA path exists; shares ESPN pattern).
---
## Operating invariants (do not regress)
- Three stat_type whitelists stay in sync: `routes/analyze.js`, `routes/scan.js`,
`python/utils/validation.py`.
- A requested market MUST have a `MARKET_MAP` entry (else silent zero) AND, if
MLB, a `MLB_LOG_FIELD` entry in BOTH `featureCache` and `outcomeService` (else
no features / no settlement).
- The accuracy pill stays HONEST — "LEARNING" below MIN_SAMPLE, never a faked %.
- Pipeline alerts (ntfy) fire on success/failure/stale/overdue; the status probe
(`GET /api/internal/snapshot/status`) exposes `overdue`.
+3
View File
@@ -61,6 +61,9 @@ const VALID_STAT_TYPES = new Set([
'strikeouts', 'hits', 'home_runs', 'rbi', 'total_bases', 'strikeouts', 'hits', 'home_runs', 'rbi', 'total_bases',
'walks', 'runs', 'earned_runs', 'innings_pitched', 'walks', 'runs', 'earned_runs', 'innings_pitched',
'hits_allowed', 'stolen_bases', 'hits_allowed', 'stolen_bases',
// Session 56 audit — settleable against the real boxscore; PropLine now
// requests batter_doubles + pitcher_outs. Keep in sync with scan.js + validation.py.
'doubles', 'outs',
]); ]);
const VALID_DIRECTIONS = new Set(['over', 'under']); const VALID_DIRECTIONS = new Set(['over', 'under']);
+4 -1
View File
@@ -112,7 +112,7 @@ router.post('/snapshot/all', async (req, res) => {
*/ */
router.get('/snapshot/status', async (req, res) => { router.get('/snapshot/status', async (req, res) => {
const { cacheGet } = require('../utils/redis'); const { cacheGet } = require('../utils/redis');
const { HOURS_UTC } = require('../snapshotScheduler'); const { HOURS_UTC, isSnapshotOverdue } = require('../snapshotScheduler');
const SPORTS = ['mlb', 'nba', 'wnba']; const SPORTS = ['mlb', 'nba', 'wnba'];
try { try {
const redis_keys = {}; const redis_keys = {};
@@ -135,10 +135,13 @@ router.get('/snapshot/status', async (req, res) => {
} }
const ticker = await cacheGet('ticker:items'); const ticker = await cacheGet('ticker:items');
redis_keys['ticker:items'] = !!ticker; redis_keys['ticker:items'] = !!ticker;
// Session 56 — surface the missed-cron signal in the health probe.
const mlbTs = last_snapshot.mlb && last_snapshot.mlb.updated_at;
return res.json({ return res.json({
cron_armed: process.env.SNAPSHOT_CRON === '1', cron_armed: process.env.SNAPSHOT_CRON === '1',
cron_hours_utc: HOURS_UTC, cron_hours_utc: HOURS_UTC,
last_snapshot, last_snapshot,
overdue: isSnapshotOverdue(mlbTs),
redis_keys, redis_keys,
ticker_count: Array.isArray(ticker) ? ticker.length : 0, ticker_count: Array.isArray(ticker) ? ticker.length : 0,
}); });
+2
View File
@@ -17,6 +17,8 @@ const VALID_STAT_TYPES = new Set([
'strikeouts', 'hits', 'home_runs', 'rbi', 'total_bases', 'strikeouts', 'hits', 'home_runs', 'rbi', 'total_bases',
'walks', 'runs', 'earned_runs', 'innings_pitched', 'walks', 'runs', 'earned_runs', 'innings_pitched',
'hits_allowed', 'stolen_bases', 'hits_allowed', 'stolen_bases',
// Session 56 audit — keep in sync with analyze.js + validation.py.
'doubles', 'outs',
]); ]);
const VALID_DIRECTIONS = new Set(['over', 'under']); const VALID_DIRECTIONS = new Set(['over', 'under']);
+5 -1
View File
@@ -46,7 +46,11 @@ const SPORT_KEYS = {
const MARKETS = { const MARKETS = {
nba: ['player_points', 'player_rebounds', 'player_assists', 'player_threes', 'player_blocks', 'player_steals'], nba: ['player_points', 'player_rebounds', 'player_assists', 'player_threes', 'player_blocks', 'player_steals'],
wnba: ['player_points', 'player_rebounds', 'player_assists', 'player_threes'], wnba: ['player_points', 'player_rebounds', 'player_assists', 'player_threes'],
mlb: ['batter_hits', 'batter_home_runs', 'batter_total_bases', 'batter_rbis', 'batter_stolen_bases', 'pitcher_strikeouts'], // Session 56 audit — expanded from 6 to 12 markets. runs/walks/earned_runs/
// hits_allowed were already fully supported downstream (whitelist + features +
// outcomes) but never requested; doubles/outs are wired this session. All ride
// the SAME request (no extra quota) → materially more graded props per slate.
mlb: ['batter_hits', 'batter_home_runs', 'batter_total_bases', 'batter_rbis', 'batter_stolen_bases', 'batter_runs', 'batter_walks', 'batter_doubles', 'pitcher_strikeouts', 'pitcher_earned_runs', 'pitcher_hits_allowed', 'pitcher_outs'],
nfl: ['player_pass_yds', 'player_rush_yds', 'player_reception_yds', 'player_receptions', 'player_anytime_td', 'player_pass_tds'], nfl: ['player_pass_yds', 'player_rush_yds', 'player_reception_yds', 'player_receptions', 'player_anytime_td', 'player_pass_tds'],
nhl: ['player_goals', 'player_shots_on_goal', 'goalie_saves'], nhl: ['player_goals', 'player_shots_on_goal', 'goalie_saves'],
ncaab: ['player_points', 'player_rebounds', 'player_assists'], ncaab: ['player_points', 'player_rebounds', 'player_assists'],
@@ -82,6 +82,8 @@ const MLB_LOG_FIELD = {
runs: 'runs', stolen_bases: 'stolenBases', walks: 'baseOnBalls', runs: 'runs', stolen_bases: 'stolenBases', walks: 'baseOnBalls',
strikeouts: 'strikeOuts', earned_runs: 'earnedRuns', hits_allowed: 'hits', strikeouts: 'strikeOuts', earned_runs: 'earnedRuns', hits_allowed: 'hits',
innings_pitched: 'inningsPitched', innings_pitched: 'inningsPitched',
// Session 56 audit — real boxscore/game-log fields (Braves@Pirates verified).
doubles: 'doubles', triples: 'triples', outs: 'outs',
}; };
function mlbStatValue(statObj, statType) { function mlbStatValue(statObj, statType) {
+2
View File
@@ -41,6 +41,8 @@ const MLB_LOG_FIELD = {
runs: 'runs', stolen_bases: 'stolenBases', walks: 'baseOnBalls', runs: 'runs', stolen_bases: 'stolenBases', walks: 'baseOnBalls',
strikeouts: 'strikeOuts', earned_runs: 'earnedRuns', hits_allowed: 'hits', strikeouts: 'strikeOuts', earned_runs: 'earnedRuns', hits_allowed: 'hits',
innings_pitched: 'inningsPitched', innings_pitched: 'inningsPitched',
// Session 56 audit — confirmed present in the real boxscore/game log.
doubles: 'doubles', triples: 'triples', outs: 'outs',
}; };
function statValue(statObj, statType) { function statValue(statObj, statType) {
+3 -1
View File
@@ -18,7 +18,9 @@ VALID_STAT_TYPES = {
'steals', 'blocks', 'turnovers'], 'steals', 'blocks', 'turnovers'],
'mlb': ['strikeouts', 'hits', 'home_runs', 'rbi', 'total_bases', 'mlb': ['strikeouts', 'hits', 'home_runs', 'rbi', 'total_bases',
'walks', 'runs', 'earned_runs', 'innings_pitched', 'walks', 'runs', 'earned_runs', 'innings_pitched',
'hits_allowed', 'stolen_bases'] 'hits_allowed', 'stolen_bases',
# Session 56 audit — keep in sync with analyze.js + scan.js.
'doubles', 'outs']
} }
VALID_SPORTS = ['nba', 'mlb'] VALID_SPORTS = ['nba', 'mlb']
+32 -2
View File
@@ -184,18 +184,36 @@ async function runSnapshot(sport, opts = {}) {
cacheSet: opts.cacheSet || require('../utils/redis').cacheSet, cacheSet: opts.cacheSet || require('../utils/redis').cacheSet,
now: opts.now || (() => new Date().toISOString()), now: opts.now || (() => new Date().toISOString()),
nowMs: opts.nowMs || (() => Date.now()), nowMs: opts.nowMs || (() => Date.now()),
// Session 56 — ops alerting (ntfy) + retry-once on a hard odds failure.
notify: opts.notify || require('../utils/opsNotify').notify,
sleep: opts.sleep || ((ms) => new Promise((r) => setTimeout(r, ms))),
retryDelayMs: opts.retryDelayMs != null ? opts.retryDelayMs : 60_000,
}; };
const start = deps.nowMs(); const start = deps.nowMs();
const ts = deps.now(); const ts = deps.now();
// Session 56 — retry ONCE on a hard failure (thrown error / null response = a
// transient PropLine/network blip). A successful-but-empty slate is NOT a
// failure (off-hours), so it is not retried — that would waste quota + latency.
let odds; let odds;
try { try {
odds = await deps.getOdds(sp); odds = await deps.getOdds(sp);
if (odds == null) throw new Error('null odds response');
} catch (e) { } catch (e) {
return { sport: sp, status: 'error', reason: e.message, gradeCount: 0 }; try {
await deps.sleep(deps.retryDelayMs);
odds = await deps.getOdds(sp);
if (odds == null) throw new Error('null odds response (retry)');
} catch (e2) {
await deps.notify(`${sp.toUpperCase()} snapshot FAILED: ${e2.message}`, { title: 'VYNDR pipeline', priority: 'high', tags: ['x'] });
return { sport: sp, status: 'error', reason: e2.message, gradeCount: 0 };
}
} }
const props = (odds && Array.isArray(odds.props)) ? odds.props : []; const props = (odds && Array.isArray(odds.props)) ? odds.props : [];
if (props.length === 0) return { sport: sp, status: 'skipped', reason: 'no props', gradeCount: 0 }; if (props.length === 0) {
await deps.notify(`⚠️ ${sp.toUpperCase()} snapshot: 0 props (odds unavailable)`, { title: 'VYNDR pipeline', priority: 'low', tags: ['warning'] });
return { sport: sp, status: 'skipped', reason: 'no props', gradeCount: 0 };
}
// Grade the slate via the existing service; capture the envelope instead of // Grade the slate via the existing service; capture the envelope instead of
// letting it write (we re-write an ENRICHED version below). // letting it write (we re-write an ENRICHED version below).
@@ -273,6 +291,18 @@ async function runSnapshot(sport, opts = {}) {
const events = generateTickerEvents(sp, enriched, deltas, ts); const events = generateTickerEvents(sp, enriched, deltas, ts);
await pushTickerItems(events, deps); await pushTickerItems(events, deps);
// Session 56 — success alert, enriched with the rolling accuracy (if settled).
let accPart = '';
try {
const acc = await deps.cacheGet(`accuracy:${sp}`);
const pct = acc && acc.overall && acc.overall.pct;
if (pct != null) accPart = `, ${pct}% accuracy (30d)`;
} catch { /* accuracy is best-effort in the alert */ }
await deps.notify(
`${sp.toUpperCase()} snapshot: ${enriched.length} props graded, ${deltas.length} deltas${accPart}`,
{ title: 'VYNDR pipeline', tags: ['white_check_mark'] },
);
return { return {
sport: sp, sport: sp,
status: 'ok', status: 'ok',
+48 -1
View File
@@ -17,6 +17,30 @@ const HOURS_UTC = (process.env.SNAPSHOT_HOURS_UTC || '14,19,22,1,3')
.map((n) => parseInt(n, 10)) .map((n) => parseInt(n, 10))
.filter((n) => Number.isInteger(n) && n >= 0 && n <= 23); .filter((n) => Number.isInteger(n) && n >= 0 && n <= 23);
// Session 56 — missed-cron detection (pure, testable).
// The latest scheduled hour:00 (UTC) at or before `date`. null if none in 48h.
function mostRecentExpectedSlot(date, hours = HOURS_UTC) {
const d = new Date(date.getTime());
d.setUTCMinutes(0, 0, 0);
for (let i = 0; i < 48; i += 1) {
if (hours.includes(d.getUTCHours())) return new Date(d.getTime());
d.setUTCHours(d.getUTCHours() - 1);
}
return null;
}
// True when we're >graceMin past the most recent expected slot AND the last
// recorded snapshot predates that slot (i.e. the run was missed). Never fires on
// cold boot (no lastSnapshotIso) — we don't cry wolf before the first snapshot.
function isSnapshotOverdue(lastSnapshotIso, now = new Date(), hours = HOURS_UTC, graceMin = 30) {
const slot = mostRecentExpectedSlot(now, hours);
if (!slot) return false;
if (now.getTime() - slot.getTime() < graceMin * 60_000) return false;
if (!lastSnapshotIso) return false;
const last = new Date(lastSnapshotIso).getTime();
return Number.isFinite(last) && last < slot.getTime();
}
function startSnapshotScheduler(opts = {}) { function startSnapshotScheduler(opts = {}) {
if (process.env.SNAPSHOT_CRON !== '1') { if (process.env.SNAPSHOT_CRON !== '1') {
// Session 52 — log the disarmed state so container logs make it unambiguous // Session 52 — log the disarmed state so container logs make it unambiguous
@@ -31,10 +55,33 @@ function startSnapshotScheduler(opts = {}) {
// real (now-completed) results BEFORE grading the fresh slate, so the accuracy // real (now-completed) results BEFORE grading the fresh slate, so the accuracy
// record reflects yesterday's games each cycle. // record reflects yesterday's games each cycle.
const settleAll = opts.settleAllOutcomes || require('./services/outcomeService').settleAllOutcomes; const settleAll = opts.settleAllOutcomes || require('./services/outcomeService').settleAllOutcomes;
const notify = opts.notify || require('./utils/opsNotify').notify;
const cacheGet = opts.cacheGet || require('./utils/redis').cacheGet;
const now = opts.now || (() => new Date()); const now = opts.now || (() => new Date());
let lastFiredSlot = null; let lastFiredSlot = null;
let lastOverdueSlot = null;
// Session 56 — missed-cron watchdog. Runs every minute (independent of the
// fire schedule): if a scheduled slot came and went without a snapshot, alert
// ONCE per missed slot.
const checkOverdue = async () => {
try {
const d = now();
const slot = mostRecentExpectedSlot(d);
if (!slot) return;
const slotKey = slot.toISOString();
if (slotKey === lastOverdueSlot) return; // already alerted for this slot
const latest = await cacheGet('snapshot:mlb:latest');
const lastTs = latest && latest.updated_at;
if (isSnapshotOverdue(lastTs, d)) {
lastOverdueSlot = slotKey;
await notify(`⚠️ Snapshot OVERDUE — expected ${slot.getUTCHours()}:00 UTC, last was ${lastTs || 'never'}`, { title: 'VYNDR pipeline', priority: 'high', tags: ['warning'] });
}
} catch { /* watchdog must never throw */ }
};
const tick = async () => { const tick = async () => {
await checkOverdue();
const d = now(); const d = now();
if (d.getUTCMinutes() !== 0) return; if (d.getUTCMinutes() !== 0) return;
const h = d.getUTCHours(); const h = d.getUTCHours();
@@ -64,4 +111,4 @@ function startSnapshotScheduler(opts = {}) {
return { interval, tick }; return { interval, tick };
} }
module.exports = { startSnapshotScheduler, HOURS_UTC }; module.exports = { startSnapshotScheduler, HOURS_UTC, mostRecentExpectedSlot, isSnapshotOverdue };
+6 -1
View File
@@ -24,7 +24,12 @@ const MARKET_MAP = {
batter_hits: 'hits', batter_hits: 'hits',
batter_home_runs: 'home_runs', batter_home_runs: 'home_runs',
batter_total_bases: 'total_bases', batter_total_bases: 'total_bases',
batter_rbis: 'rbis', // Session 56 audit — must be 'rbi' (singular): the grade whitelists
// (analyze/scan/validation.py), featureCache MLB_LOG_FIELD, and outcomeService
// all key on 'rbi'. Normalizing to 'rbis' silently dropped every PropLine RBI
// prop from grading AND settlement. The streaks/hotlist path uses its own
// 'rbis' key built from raw MLB stats — independent of this normalizer.
batter_rbis: 'rbi',
batter_runs: 'runs', batter_runs: 'runs',
batter_stolen_bases: 'stolen_bases', batter_stolen_bases: 'stolen_bases',
batter_singles: 'singles', batter_singles: 'singles',
+54
View File
@@ -0,0 +1,54 @@
'use strict';
/**
* opsNotify (Session 56) — pipeline alerting via ntfy.sh.
*
* Push a one-line operational alert (snapshot success / failure / stale / missed
* cron) to an ntfy topic so a silent pipeline never goes unnoticed. Fire-and-
* forget: NEVER throws, NEVER blocks the pipeline on a notify failure.
*
* Config:
* NTFY_URL (default https://ntfy.sh)
* NTFY_TOPIC (default vyndr-pipeline-kev2026)
* PIPELINE_ALERTS=0 → disable entirely
* Disabled automatically under NODE_ENV==='test' unless a fetchImpl is injected
* (so the unit tests can assert the call without hitting the network).
*/
const NTFY_URL = () => process.env.NTFY_URL || 'https://ntfy.sh';
const NTFY_TOPIC = () => process.env.NTFY_TOPIC || 'vyndr-pipeline-kev2026';
function enabled(opts = {}) {
if (opts.fetchImpl) return true; // tests inject → always "enabled"
if (process.env.PIPELINE_ALERTS === '0') return false;
if (process.env.NODE_ENV === 'test') return false;
return true;
}
/**
* Send an ops alert. `opts`: { title, priority ('min'|'low'|'default'|'high'|
* 'urgent'), tags (string[]), fetchImpl }. Resolves { sent: boolean } — never rejects.
*/
async function notify(message, opts = {}) {
if (!enabled(opts)) return { sent: false, reason: 'disabled' };
const doFetch = opts.fetchImpl || fetch;
const headers = {};
if (opts.title) headers.Title = opts.title;
if (opts.priority) headers.Priority = opts.priority;
if (Array.isArray(opts.tags) && opts.tags.length) headers.Tags = opts.tags.join(',');
try {
await doFetch(`${NTFY_URL()}/${NTFY_TOPIC()}`, {
method: 'POST',
headers,
body: String(message == null ? '' : message),
signal: typeof AbortSignal !== 'undefined' && AbortSignal.timeout ? AbortSignal.timeout(6000) : undefined,
});
return { sent: true };
} catch (err) {
// Alerting must never break the pipeline.
if (process.env.NODE_ENV !== 'test') console.warn('[opsNotify] failed:', err.message);
return { sent: false, reason: err.message };
}
}
module.exports = { notify, __internals: { enabled, NTFY_URL, NTFY_TOPIC } };
+8
View File
@@ -121,6 +121,14 @@ describe('oddsNormalizer', () => {
} }
}); });
it('Session 56 audit — batter_rbis normalizes to rbi (singular), matching the grade chain', () => {
// The whole grade/feature/outcome chain keys on 'rbi'. If this reverts to
// 'rbis', PropLine RBI props silently stop grading + settling again.
expect(MARKET_MAP.batter_rbis).toBe('rbi');
expect(MARKET_MAP.batter_doubles).toBe('doubles');
expect(MARKET_MAP.pitcher_outs).toBe('outs');
});
it('exposes the NFL market keys added in the Session 31 audit', () => { it('exposes the NFL market keys added in the Session 31 audit', () => {
// Defensive mapping landed before NFL is fully wired so it can't // Defensive mapping landed before NFL is fully wired so it can't
// repeat the MLB silent-zero bug. Both odds-api `_yds` and the // repeat the MLB silent-zero bug. Both odds-api `_yds` and the
+7
View File
@@ -59,6 +59,13 @@ describe('outcomeService — settlement math', () => {
expect(statValue({ strikeOuts: 7 }, 'strikeouts')).toBe(7); expect(statValue({ strikeOuts: 7 }, 'strikeouts')).toBe(7);
expect(statValue({ hits: 2 }, 'unknown_stat')).toBeNull(); expect(statValue({ hits: 2 }, 'unknown_stat')).toBeNull();
}); });
test('Session 56 — newly-wired MLB fields settle (rbi, doubles, outs)', () => {
expect(statValue({ rbi: 2 }, 'rbi')).toBe(2);
expect(statValue({ doubles: 1 }, 'doubles')).toBe(1);
expect(statValue({ outs: 18 }, 'outs')).toBe(18);
expect(statValue({ baseOnBalls: 3 }, 'walks')).toBe(3);
expect(statValue({ earnedRuns: 1 }, 'earned_runs')).toBe(1);
});
test('dateStrings yields both UTC and ET calendar dates', () => { test('dateStrings yields both UTC and ET calendar dates', () => {
const ds = dateStrings(ISO); const ds = dateStrings(ISO);
expect(ds).toContain('2026-07-10'); // UTC expect(ds).toContain('2026-07-10'); // UTC
+128
View File
@@ -0,0 +1,128 @@
'use strict';
// Session 56 — pipeline alerting (ntfy) + retry-once + missed-cron detection.
const opsNotify = require('../../src/utils/opsNotify');
const snapshot = require('../../src/services/snapshotService');
const { mostRecentExpectedSlot, isSnapshotOverdue } = require('../../src/snapshotScheduler');
function memCache(seed = {}) {
const store = { ...seed };
return { store, cacheGet: async (k) => (k in store ? store[k] : null), cacheSet: async (k, v) => { store[k] = v; return true; } };
}
describe('opsNotify', () => {
test('POSTs to the ntfy topic with title/priority/tags headers', async () => {
const calls = [];
const fetchImpl = async (url, opts) => { calls.push({ url, opts }); return { ok: true }; };
const res = await opsNotify.notify('hello', { title: 'VYNDR', priority: 'high', tags: ['x'], fetchImpl });
expect(res.sent).toBe(true);
expect(calls[0].url).toMatch(/\/vyndr-pipeline-kev2026$/);
expect(calls[0].opts.method).toBe('POST');
expect(calls[0].opts.headers.Title).toBe('VYNDR');
expect(calls[0].opts.headers.Priority).toBe('high');
expect(calls[0].opts.headers.Tags).toBe('x');
expect(calls[0].opts.body).toBe('hello');
});
test('never throws on fetch failure', async () => {
const fetchImpl = async () => { throw new Error('network down'); };
const res = await opsNotify.notify('x', { fetchImpl });
expect(res.sent).toBe(false);
expect(res.reason).toBe('network down');
});
test('disabled under NODE_ENV=test without an injected fetch', async () => {
const res = await opsNotify.notify('x');
expect(res.sent).toBe(false);
expect(res.reason).toBe('disabled');
});
});
describe('runSnapshot — retry + alerts', () => {
const baseDeps = (cache) => ({
...cache,
gradeAndCacheSlate: async (_s, _p, opts) => { opts.cacheSet('grades', { grades: [{ player: 'A', stat_type: 'hits', line: 1.5, direction: 'over', grade: 'A', confidence: 90 }] }); },
resolveStats: async () => ({ found: false }),
classify: () => ({ primary: null }),
now: () => '2026-07-10T15:00:00Z',
nowMs: () => 1000,
sleep: async () => {}, // no real delay in tests
retryDelayMs: 0,
});
test('retries once on a thrown odds error, then succeeds', async () => {
const cache = memCache();
let n = 0;
const notes = [];
const deps = {
...baseDeps(cache),
getOdds: async () => { n += 1; if (n === 1) throw new Error('propline 503'); return { props: [{ player: 'A', stat_type: 'hits', line: 1.5 }], provider: 'propline' }; },
notify: async (msg) => { notes.push(msg); return { sent: true }; },
};
const r = await snapshot.runSnapshot('mlb', deps);
expect(n).toBe(2); // one retry
expect(r.status).toBe('ok');
expect(notes.some((m) => m.includes('✅') && m.includes('MLB'))).toBe(true);
});
test('alerts failure when both attempts throw', async () => {
const cache = memCache();
const notes = [];
const deps = {
...baseDeps(cache),
getOdds: async () => { throw new Error('down'); },
notify: async (msg) => { notes.push(msg); return { sent: true }; },
};
const r = await snapshot.runSnapshot('mlb', deps);
expect(r.status).toBe('error');
expect(notes.some((m) => m.includes('❌') && m.includes('FAILED'))).toBe(true);
});
test('alerts stale (low priority) on an empty slate, no retry', async () => {
const cache = memCache();
let n = 0;
const notes = [];
const deps = {
...baseDeps(cache),
getOdds: async () => { n += 1; return { props: [] }; },
notify: async (msg) => { notes.push(msg); return { sent: true }; },
};
const r = await snapshot.runSnapshot('mlb', deps);
expect(n).toBe(1); // empty is NOT a failure → no retry
expect(r.status).toBe('skipped');
expect(notes.some((m) => m.includes('⚠️') && m.includes('0 props'))).toBe(true);
});
});
describe('missed-cron detection', () => {
const HOURS = [14, 19, 22, 1, 3];
test('mostRecentExpectedSlot returns the latest scheduled hour at/before now', () => {
const now = new Date('2026-07-10T16:30:00Z');
const slot = mostRecentExpectedSlot(now, HOURS);
expect(slot.getUTCHours()).toBe(14);
expect(slot.getUTCDate()).toBe(10);
});
test('overdue when now is >30m past a slot the last snapshot predates', () => {
const now = new Date('2026-07-10T16:30:00Z'); // 2.5h past the 14:00 slot
const lastTs = '2026-07-10T10:00:00Z'; // before 14:00 → the 14:00 run was missed
expect(isSnapshotOverdue(lastTs, now, HOURS, 30)).toBe(true);
});
test('not overdue within the grace window', () => {
const now = new Date('2026-07-10T14:20:00Z'); // only 20m past the slot
expect(isSnapshotOverdue('2026-07-10T10:00:00Z', now, HOURS, 30)).toBe(false);
});
test('not overdue when the last snapshot is fresh (ran the slot)', () => {
const now = new Date('2026-07-10T16:30:00Z');
const lastTs = '2026-07-10T14:01:00Z'; // ran the 14:00 slot
expect(isSnapshotOverdue(lastTs, now, HOURS, 30)).toBe(false);
});
test('never overdue on cold boot (no prior snapshot)', () => {
const now = new Date('2026-07-10T16:30:00Z');
expect(isSnapshotOverdue(null, now, HOURS, 30)).toBe(false);
});
});
+1 -1
View File
File diff suppressed because one or more lines are too long