Layer 1: Statcast mechanism-data ingestion (backfill + nightly refresh)

The data foundation for the archetype and projection layers, built as the
pattern every sport inherits. Layers 2 and 3 are not touched.

PHASE 0 GATE — both match rates measured live, both 100%. Batters 40/40;
PITCHERS 66/66 across five real rosters (CLE, DET, MIN, NYY, LAD) joined by
MLBAM id against the 713-pitcher Savant feed. Zero honest-absent on identity,
because the join is an integer both systems use natively — and the snapshot
pipeline already stores it per graded row.

SOURCE — five Baseball Savant CSV leaderboards, free and public, pulled with
axios and the CSV parser savantAdapter already runs in prod. pybaseball is
deliberately NOT used: it is an MIT wrapper over these same URLs, and adding it
would reintroduce a Python runtime in a stack where the existing Python service
is already offline. min=1 on every feed, not Savant's default min=q, so the
long tail arrives and OUR minimum-sample gate decides what is thin — explicit
and testable rather than silently dropped upstream.

Measured: 1,354 rows per season (604 batters, 750 pitchers), all five feeds in
about five seconds. Pitcher mechanism includes arm angle, GB/FB/LD, chase and
whiff; batters get exit velo, launch angle, barrel and hard-hit, chase and
z-swing. Handedness rides in free on the movement feed (677 pitchers); batter
handedness stays absent pending a roster join rather than being guessed.

BACKFILL AND REFRESH ARE THE SAME CALL — a full re-pull upserted on
(sport, season, source_id). Idempotent and self-healing: a missed night
self-corrects on the next run, with no incremental who-played bookkeeping to
drift out of sync. At 1,354 rows the simple thing is also the robust one.

HONESTY RULES, each with a test: a metric the feed did not carry is null and
never 0; a thin sample is STORED and flagged rather than dropped or inflated,
because thin and missing are different claims; an unjoined player is stored
with a null player_key and joins later; and if every feed comes back empty the
job REFUSES to write, so a bad night can never blank a good table.

Freshness is treated as a truth property. updated_at on every row, and the
scheduler pages on a failed run AND on silent staleness — a job that stops
being scheduled never produces a failure, so staleness has to alarm on its own.
Never-built is deliberately not stale: different condition, different fix, and
paging on a fresh install teaches the operator to ignore the alarm.

Nightly at STATCAST_HOUR_UTC (default 11 UTC, after every game is final), kill
switch STATCAST=0, and induce-able at POST /api/internal/statcast/refresh with
a freshness probe at /statcast/status — we verify a refresh by running it, not
by waiting for the slot.

Migration 030 applied. Promoted columns for the classification-critical metrics
plus a metrics JSONB carrying every raw field, so Layer 2 can reach something we
did not promote without a re-ingest. Raw per-pitch stays out of Postgres on
purpose: one season is ~0.85 GB against a 500 MB plan ceiling, and it is
re-pullable from the free source if Layer 3 ever needs it.

Pattern documented in docs/MECHANISM-DATA.md for NBA tracking and NFL Next Gen.

Tests 3581 passed / 292 suites, web build exit 0.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VCNgGSt5qvcLxaeQqa7Zpj

src/snapshotScheduler.js

@@ -286,6 +286,34 @@ function startSnapshotScheduler(opts = {}) {
       }
     } catch (e) { console.warn('[harness] nightly run failed:', e.message); }
 
+    // Session 68 — LAYER 1 MECHANISM DATA. Nightly full re-pull of the
+    // ~1,350-row Statcast aggregate set at STATCAST_HOUR_UTC. Backfill and
+    // refresh are the SAME call, upserted on the natural key, so the job is
+    // idempotent and self-healing: a missed night self-corrects on the next
+    // run with no incremental bookkeeping to drift. Kill switch: STATCAST=0.
+    try {
+      const statcastHour = Number(process.env.STATCAST_HOUR_UTC || 11); // ~7 AM ET, after every game is final
+      if (h === statcastHour && process.env.STATCAST !== '0') {
+        const agg = opts.statcastService || require('./services/statcastAggregateService');
+        const res = await agg.refreshSeason({});
+        console.log(`[statcast] nightly refresh — ok=${res.ok} rows=${res.rows ?? 'n/a'} written=${res.written ?? 0} joined=${res.joined ?? 'n/a'} thin=${res.thin ?? 'n/a'}${res.reason ? ` reason: ${res.reason}` : ''}`);
+        if (!res.ok) {
+          await notify(`Statcast refresh failed — ${res.reason || 'unknown'}. Mechanism data is not updating.`, {
+            title: 'VYNDR statcast', priority: 'high', tags: ['rotating_light'],
+          });
+        }
+        // Serving a stale aggregate as current is a quiet fabrication of
+        // currency, so staleness pages on its own — separate from a failed run,
+        // because a silently-not-scheduled job never produces a failure.
+        const fresh = await agg.getFreshness({});
+        if (agg.isStale(fresh)) {
+          await notify(`Statcast aggregates stale — last updated ${fresh.updated_at} (${fresh.age_hours}h). Mechanism data is being served as current.`, {
+            title: 'VYNDR statcast', priority: 'high', tags: ['rotating_light'],
+          });
+        }
+      }
+    } catch (e) { console.warn('[statcast] nightly refresh failed:', e.message); }
+
     // Session 8 — quota check after each snapshot run: odds-api >= 80% alerts
     // once per day (Redis-deduped). Never throws (guarded inside checkQuotaDaily).
     await opsWatch.checkQuotaDaily({ getStatus: getQuotaStatus, cacheGet, cacheSet, notify, now: () => now() });
@@ -324,7 +352,7 @@ function startSnapshotScheduler(opts = {}) {
 
   const interval = setInterval(() => { void tick(); void refreshTick(); }, 60_000);
   if (interval.unref) interval.unref();
-  console.log(`[snapshotScheduler] armed — SNAPSHOT_CRON=${process.env.SNAPSHOT_CRON}, hours=${HOURS_UTC.join(',')} UTC, intraday=${process.env.INTRADAY_REFRESH === '0' ? 'off' : `${REFRESH_MINUTES}m (slate hours)`}`);
+  console.log(`[snapshotScheduler] armed — SNAPSHOT_CRON=${process.env.SNAPSHOT_CRON}, hours=${HOURS_UTC.join(',')} UTC, intraday=${process.env.INTRADAY_REFRESH === '0' ? 'off' : `${REFRESH_MINUTES}m (slate hours)`}, statcast=${process.env.STATCAST === '0' ? 'off' : `${process.env.STATCAST_HOUR_UTC || 11}h UTC`}`);
   // Job 1 — state each sport's cadence at boot so "when does WNBA grade?" is
   // answerable from logs, and warn if a sport is scheduled at an hour the
   // scheduler never fires (it would silently never run).