Layer 1: Statcast mechanism-data ingestion (backfill + nightly refresh)

The data foundation for the archetype and projection layers, built as the
pattern every sport inherits. Layers 2 and 3 are not touched.

PHASE 0 GATE — both match rates measured live, both 100%. Batters 40/40;
PITCHERS 66/66 across five real rosters (CLE, DET, MIN, NYY, LAD) joined by
MLBAM id against the 713-pitcher Savant feed. Zero honest-absent on identity,
because the join is an integer both systems use natively — and the snapshot
pipeline already stores it per graded row.

SOURCE — five Baseball Savant CSV leaderboards, free and public, pulled with
axios and the CSV parser savantAdapter already runs in prod. pybaseball is
deliberately NOT used: it is an MIT wrapper over these same URLs, and adding it
would reintroduce a Python runtime in a stack where the existing Python service
is already offline. min=1 on every feed, not Savant's default min=q, so the
long tail arrives and OUR minimum-sample gate decides what is thin — explicit
and testable rather than silently dropped upstream.

Measured: 1,354 rows per season (604 batters, 750 pitchers), all five feeds in
about five seconds. Pitcher mechanism includes arm angle, GB/FB/LD, chase and
whiff; batters get exit velo, launch angle, barrel and hard-hit, chase and
z-swing. Handedness rides in free on the movement feed (677 pitchers); batter
handedness stays absent pending a roster join rather than being guessed.

BACKFILL AND REFRESH ARE THE SAME CALL — a full re-pull upserted on
(sport, season, source_id). Idempotent and self-healing: a missed night
self-corrects on the next run, with no incremental who-played bookkeeping to
drift out of sync. At 1,354 rows the simple thing is also the robust one.

HONESTY RULES, each with a test: a metric the feed did not carry is null and
never 0; a thin sample is STORED and flagged rather than dropped or inflated,
because thin and missing are different claims; an unjoined player is stored
with a null player_key and joins later; and if every feed comes back empty the
job REFUSES to write, so a bad night can never blank a good table.

Freshness is treated as a truth property. updated_at on every row, and the
scheduler pages on a failed run AND on silent staleness — a job that stops
being scheduled never produces a failure, so staleness has to alarm on its own.
Never-built is deliberately not stale: different condition, different fix, and
paging on a fresh install teaches the operator to ignore the alarm.

Nightly at STATCAST_HOUR_UTC (default 11 UTC, after every game is final), kill
switch STATCAST=0, and induce-able at POST /api/internal/statcast/refresh with
a freshness probe at /statcast/status — we verify a refresh by running it, not
by waiting for the slot.

Migration 030 applied. Promoted columns for the classification-critical metrics
plus a metrics JSONB carrying every raw field, so Layer 2 can reach something we
did not promote without a re-ingest. Raw per-pitch stays out of Postgres on
purpose: one season is ~0.85 GB against a 500 MB plan ceiling, and it is
re-pullable from the free source if Layer 3 ever needs it.

Pattern documented in docs/MECHANISM-DATA.md for NBA tracking and NFL Next Gen.

Tests 3581 passed / 292 suites, web build exit 0.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VCNgGSt5qvcLxaeQqa7Zpj

src/routes/internal.js

@@ -431,6 +431,46 @@ router.post('/newsletter/send', async (req, res) => {
   }
 });
 
+/**
+ * POST /api/internal/statcast/refresh — INDUCE the Layer-1 mechanism refresh.
+ *
+ * Same call the nightly scheduler makes. Exists so the job is verifiable ON
+ * DEMAND: we prove a refresh works by running it and reading the result, never
+ * by waiting for the cron slot. Idempotent — running it twice is a no-op beyond
+ * refreshing values and updated_at.
+ */
+router.post('/statcast/refresh', async (req, res) => {
+  try {
+    const agg = require('../services/statcastAggregateService');
+    const season = req.query.season ? Number(req.query.season) : undefined;
+    const out = await agg.refreshSeason({ season });
+    const fresh = await agg.getFreshness({});
+    return res.status(out.ok ? 200 : 500).json({ ...out, freshness: fresh, stale: agg.isStale(fresh) });
+  } catch (err) {
+    console.error('[internal/statcast]', err.message);
+    return res.status(500).json({ ok: false, error: err.message });
+  }
+});
+
+/** GET /api/internal/statcast/status — freshness probe for the mechanism tier. */
+router.get('/statcast/status', async (req, res) => {
+  try {
+    const agg = require('../services/statcastAggregateService');
+    const fresh = await agg.getFreshness({});
+    return res.json({
+      ...fresh,
+      stale: agg.isStale(fresh),
+      max_age_hours: agg.MAX_AGE_HOURS,
+      min_pa: agg.MIN_PA,
+      min_ip: agg.MIN_IP,
+      cron_hour_utc: Number(process.env.STATCAST_HOUR_UTC || 11),
+      enabled: process.env.STATCAST !== '0',
+    });
+  } catch (err) {
+    return res.status(500).json({ error: err.message });
+  }
+});
+
 router.post('/outcomes/:sport', async (req, res) => {
   const outcomes = require('../services/outcomeService');
   try {

src/services/adapters/statcastAdapter.js

@@ -0,0 +1,217 @@
+'use strict';
+
+/**
+ * STATCAST ADAPTER (Layer 1) — the SOURCE ADAPTER of the mechanism-data pattern.
+ *
+ * Pulls per-player-per-season Statcast aggregates from Baseball Savant. Free,
+ * public, no key, no quota — the same host `savantAdapter` already uses in prod.
+ *
+ * WHY NOT pybaseball: it is an MIT-licensed Python wrapper around these exact
+ * CSV URLs. Adding it would reintroduce a Python runtime dependency in a stack
+ * where the existing Python service is already offline in production (nba_api
+ * degrades to found:false for exactly that reason). We call the endpoints
+ * directly with axios and reuse savantAdapter's quote-aware CSV parser — no new
+ * dependency, one less thing to keep alive.
+ *
+ * FIVE FEEDS (all measured live 2026-07-20, all `csv=true`, all min=1 so the
+ * long tail is included rather than silently dropped by Savant's `min=q`):
+ *   1. batter discipline   604 rows — k/bb/whiff/swing/CHASE/z-swing/sweetspot/
+ *                                     barrel/hard-hit/best-speed
+ *   2. batter batted-ball  600 rows — exit velo (avg/max/ev50), launch angle,
+ *                                     GB/FB-LD, barrels, ev95+
+ *   3. pitcher discipline  713 rows — k/bb/whiff/swing/chase, GB/FB/LD%,
+ *                                     barrel/hard-hit allowed, ARM ANGLE
+ *   4. pitcher batted-ball 749 rows — contact quality allowed
+ *   5. pitch movement      677 rows — per pitch type: velo, break_x/z, induced
+ *                                     break, usage — and PITCH_HAND (handedness)
+ *
+ * HONESTY DOCTRINE (inherited from savantAdapter / espnStatsAdapter):
+ *   - A missing metric is NULL, never 0. `Number(null) === 0` is the fabrication
+ *     trap this whole layer exists to avoid.
+ *   - Any unrecognised shape → that field absent; never a guessed default.
+ *   - Injectable (`opts.fetchImpl`) so tests never touch the network.
+ *
+ * COMMODITY, NOT MOAT: this data is public and every competitor can pull it in
+ * about a second. The edge is Layer 2 (archetype definitions) and Layer 3
+ * (projections) built on top — plus the settled ledger that proves them.
+ */
+
+const axios = require('axios');
+const { __internals: savant } = require('./savantAdapter');
+
+const { parseCsv, numOrNull } = savant;
+
+const HTTP_TIMEOUT_MS = 60_000;
+const DEFAULT_SEASON = Number(process.env.STATCAST_SEASON) || 2026;
+const BASE = 'https://baseballsavant.mlb.com/leaderboard';
+
+/** `min=1` deliberately: Savant's default `min=q` drops every non-qualified
+ *  player, which would silently make a thin-sample player look absent rather
+ *  than thin. We take the long tail and apply our OWN minimum-sample gate at
+ *  the aggregate layer, where it is explicit and testable. */
+const FEEDS = Object.freeze({
+  batter_discipline: (y) => `${BASE}/custom?year=${y}&type=batter&filter=&min=1&selections=pa,k_percent,bb_percent,whiff_percent,swing_percent,oz_swing_percent,z_swing_percent,sweet_spot_percent,barrel_batted_rate,hard_hit_percent,avg_best_speed,avg_hyper_speed&chart=false&x=pa&y=pa&r=no&chartType=beeswarm&sort=pa&sortDir=desc&csv=true`,
+  batter_batted_ball: (y) => `${BASE}/statcast?type=batter&year=${y}&position=&team=&min=1&csv=true`,
+  pitcher_discipline: (y) => `${BASE}/custom?year=${y}&type=pitcher&filter=&min=1&selections=p_formatted_ip,k_percent,bb_percent,whiff_percent,swing_percent,oz_swing_percent,groundballs_percent,flyballs_percent,linedrives_percent,barrel_batted_rate,hard_hit_percent,arm_angle&chart=false&x=p_formatted_ip&y=p_formatted_ip&r=no&chartType=beeswarm&sort=p_formatted_ip&sortDir=desc&csv=true`,
+  pitcher_batted_ball: (y) => `${BASE}/statcast?type=pitcher&year=${y}&position=&team=&min=1&csv=true`,
+  pitch_movement: (y) => `${BASE}/pitch-movement?year=${y}&team=&min=1&pitch_type=&hand=&csv=true`,
+});
+
+async function fetchText(url, opts = {}) {
+  if (opts.fetchImpl) return opts.fetchImpl(url);
+  const res = await axios.get(url, {
+    timeout: HTTP_TIMEOUT_MS,
+    responseType: 'text',
+    headers: { Accept: 'text/csv,*/*' },
+    // Savant occasionally returns a large body; don't let axios truncate.
+    maxContentLength: 64 * 1024 * 1024,
+    transformResponse: [(d) => d],
+  });
+  return typeof res.data === 'string' ? res.data : null;
+}
+
+/** Savant's first column is the literal header `last_name, first_name` — a
+ *  comma INSIDE a quoted header. The value is likewise `"Last, First"`. */
+function flipName(v) {
+  const s = String(v || '').trim();
+  if (!s) return null;
+  const i = s.indexOf(',');
+  if (i < 0) return s;
+  const last = s.slice(0, i).trim();
+  const first = s.slice(i + 1).trim();
+  return first ? `${first} ${last}` : last;
+}
+
+function idOf(row) {
+  const raw = row.player_id ?? row.pitcher_id ?? row.batter_id;
+  const n = Number(raw);
+  return Number.isFinite(n) && n > 0 ? n : null;
+}
+
+/** Row → { id, name, metrics } with EVERY metric strictly parsed (null, not 0). */
+function normalizeRow(row, fields) {
+  const id = idOf(row);
+  if (id == null) return null;
+  const metrics = {};
+  for (const [out, src] of Object.entries(fields)) {
+    const v = numOrNull(row[src]);
+    if (v != null) metrics[out] = v;
+  }
+  return { id, name: flipName(row['last_name, first_name'] || row.last_name), metrics };
+}
+
+const BATTER_DISCIPLINE = {
+  pa: 'pa', k_pct: 'k_percent', bb_pct: 'bb_percent', whiff_pct: 'whiff_percent',
+  swing_pct: 'swing_percent', chase_pct: 'oz_swing_percent', z_swing_pct: 'z_swing_percent',
+  sweet_spot_pct: 'sweet_spot_percent', barrel_pct: 'barrel_batted_rate',
+  hard_hit_pct: 'hard_hit_percent', avg_best_speed: 'avg_best_speed',
+};
+const BATTED_BALL = {
+  bip: 'attempts', avg_launch_angle: 'avg_hit_angle', sweet_spot_pct_bb: 'anglesweetspotpercent',
+  max_exit_velo: 'max_hit_speed', avg_exit_velo: 'avg_hit_speed', ev50: 'ev50',
+  fb_ld_pct: 'fbld', gb_pct_bb: 'gb', max_distance: 'max_distance',
+  avg_distance: 'avg_distance', avg_hr_distance: 'avg_hr_distance',
+  ev95_pct: 'ev95percent', barrels: 'barrels', barrel_pct_bb: 'brl_percent', barrel_per_pa: 'brl_pa',
+};
+const PITCHER_DISCIPLINE = {
+  ip: 'p_formatted_ip', k_pct: 'k_percent', bb_pct: 'bb_percent', whiff_pct: 'whiff_percent',
+  swing_pct: 'swing_percent', chase_pct: 'oz_swing_percent', gb_pct: 'groundballs_percent',
+  fb_pct: 'flyballs_percent', ld_pct: 'linedrives_percent',
+  barrel_pct_allowed: 'barrel_batted_rate', hard_hit_pct_allowed: 'hard_hit_percent',
+  arm_angle: 'arm_angle',
+};
+
+function indexBy(rows, fields) {
+  const out = new Map();
+  for (const r of rows) {
+    const n = normalizeRow(r, fields);
+    if (!n) continue;
+    const prev = out.get(n.id);
+    out.set(n.id, prev ? { ...prev, ...n, metrics: { ...prev.metrics, ...n.metrics } } : n);
+  }
+  return out;
+}
+
+/**
+ * Pitch movement is LONG (one row per pitcher × pitch type). Collapses to a
+ * pitch-mix array per pitcher, sorted by usage. Also the only feed carrying
+ * HANDEDNESS (`pitch_hand`), so it doubles as the pitcher-handedness source.
+ */
+function indexPitchMix(rows) {
+  const out = new Map();
+  for (const r of rows) {
+    const id = idOf(r);
+    if (id == null) continue;
+    const entry = out.get(id) || { id, name: flipName(r['last_name, first_name']), throws: null, pitches: [] };
+    if (!entry.throws && r.pitch_hand) entry.throws = String(r.pitch_hand).trim().toUpperCase() || null;
+    // `pitch_per` arrives as a FRACTION (0.371), unlike every other _pct field
+    // on these feeds. Normalise to a percentage so downstream never has to know
+    // which endpoint a number came from.
+    const rawUsage = numOrNull(r.pitch_per);
+    const usage = rawUsage == null ? null : (rawUsage <= 1 ? Math.round(rawUsage * 1000) / 10 : rawUsage);
+    const type = (r.pitch_type || '').trim();
+    if (type) {
+      entry.pitches.push({
+        type,
+        name: (r.pitch_type_name || '').trim() || null,
+        usage_pct: usage,
+        velo: numOrNull(r.avg_speed),
+        break_z_induced: numOrNull(r.pitcher_break_z_induced),
+        break_x: numOrNull(r.pitcher_break_x),
+        thrown: numOrNull(r.pitches_thrown),
+      });
+    }
+    out.set(id, entry);
+  }
+  for (const e of out.values()) {
+    e.pitches.sort((a, b) => (b.usage_pct ?? 0) - (a.usage_pct ?? 0));
+  }
+  return out;
+}
+
+/**
+ * fetchSeason(season, opts) — pull all five feeds and return normalized indexes.
+ * Any single feed failing degrades to an EMPTY index for that feed (its metrics
+ * become absent) rather than failing the whole pull: partial mechanism data is
+ * honest, a failed night is not a reason to have none.
+ */
+async function fetchSeason(season = DEFAULT_SEASON, opts = {}) {
+  const get = async (name) => {
+    try {
+      const text = await fetchText(FEEDS[name](season), opts);
+      const rows = text ? parseCsv(text) : [];
+      return Array.isArray(rows) ? rows : [];
+    } catch (err) {
+      console.warn(`[statcast] feed ${name} failed:`, err.message);
+      return [];
+    }
+  };
+
+  const [bd, bbb, pd, pbb, mv] = await Promise.all([
+    get('batter_discipline'), get('batter_batted_ball'),
+    get('pitcher_discipline'), get('pitcher_batted_ball'), get('pitch_movement'),
+  ]);
+
+  return {
+    season,
+    batterDiscipline: indexBy(bd, BATTER_DISCIPLINE),
+    batterBattedBall: indexBy(bbb, BATTED_BALL),
+    pitcherDiscipline: indexBy(pd, PITCHER_DISCIPLINE),
+    pitcherBattedBall: indexBy(pbb, BATTED_BALL),
+    pitchMix: indexPitchMix(mv),
+    counts: {
+      batter_discipline: bd.length, batter_batted_ball: bbb.length,
+      pitcher_discipline: pd.length, pitcher_batted_ball: pbb.length, pitch_movement: mv.length,
+    },
+  };
+}
+
+module.exports = {
+  fetchSeason,
+  DEFAULT_SEASON,
+  FEEDS,
+  __internals: {
+    flipName, idOf, normalizeRow, indexBy, indexPitchMix,
+    BATTER_DISCIPLINE, BATTED_BALL, PITCHER_DISCIPLINE, fetchText,
+  },
+};

src/services/statcastAggregateService.js

@@ -0,0 +1,273 @@
+'use strict';
+
+/**
+ * STATCAST AGGREGATE SERVICE (Layer 1) — derivation + join + store.
+ *
+ * The AGGREGATE DERIVATION and IDENTITY BRIDGE boxes of the mechanism-data
+ * pattern (docs/MECHANISM-DATA.md). Pure where it can be, injectable
+ * everywhere, so tests never touch the network or the database.
+ *
+ * BACKFILL and REFRESH are the SAME operation: a full re-pull of the ~1,300-row
+ * aggregate set, upserted on (sport, season, source_id). That makes the nightly
+ * job idempotent and self-healing — a missed night self-corrects on the next
+ * run, with no incremental "who played today" bookkeeping to drift out of sync.
+ * At this volume the simple thing is also the robust thing.
+ *
+ * HONESTY RULES:
+ *   - A metric the feed did not carry is ABSENT (null), never 0.
+ *   - A player below the minimum sample keeps his row but is marked
+ *     `sufficient: false` — thin is not the same as missing, and Layer 2 must
+ *     be able to tell them apart.
+ *   - A player we cannot join to our own roster still stores (player_key null)
+ *     and joins later. Storing him is not a claim about him.
+ *   - `updated_at` is written every run. Freshness is a truth property; the
+ *     staleness alarm reads it.
+ */
+
+const { nameKey } = require('../utils/playerName');
+const statcast = require('./adapters/statcastAdapter');
+
+/** Minimum samples for a mechanism claim. Below these the row is stored but
+ *  flagged insufficient — the explicit gate that prevents a 12-PA sample from
+ *  being classified as confidently as a 400-PA one (the FLEX-at-weight-1.0
+ *  failure, structurally prevented). */
+const MIN_PA = Number(process.env.STATCAST_MIN_PA) || 50;
+const MIN_IP = Number(process.env.STATCAST_MIN_IP) || 10;
+
+const num = (v) => {
+  if (v == null || v === '') return null;
+  const n = typeof v === 'number' ? v : Number(v);
+  return Number.isFinite(n) ? n : null;
+};
+
+/**
+ * buildRows(season, feeds, opts) — PURE. Merges the five feed indexes into one
+ * row per player per role. No I/O.
+ */
+function buildRows(season, feeds, opts = {}) {
+  const sport = opts.sport || 'mlb';
+  const now = opts.now || new Date().toISOString();
+  const rows = [];
+
+  // ---- BATTERS -----------------------------------------------------------
+  const batterIds = new Set([...feeds.batterDiscipline.keys(), ...feeds.batterBattedBall.keys()]);
+  for (const id of batterIds) {
+    const d = feeds.batterDiscipline.get(id);
+    const b = feeds.batterBattedBall.get(id);
+    const m = { ...(b ? b.metrics : {}), ...(d ? d.metrics : {}) };
+    const name = (d && d.name) || (b && b.name) || null;
+    const pa = num(m.pa);
+    rows.push({
+      sport,
+      season,
+      source_id: id,
+      player_key: name ? nameKey(name) : null,
+      player_name: name,
+      role: 'batter',
+      throws: null,
+      bats: null, // roster join fills this; absent otherwise (never guessed)
+      sample_pa: pa,
+      sample_ip: null,
+      sample_bip: num(m.bip),
+      k_pct: num(m.k_pct),
+      bb_pct: num(m.bb_pct),
+      whiff_pct: num(m.whiff_pct),
+      swing_pct: num(m.swing_pct),
+      chase_pct: num(m.chase_pct),
+      barrel_pct: num(m.barrel_pct) ?? num(m.barrel_pct_bb),
+      hard_hit_pct: num(m.hard_hit_pct),
+      avg_exit_velo: num(m.avg_exit_velo),
+      max_exit_velo: num(m.max_exit_velo),
+      avg_launch_angle: num(m.avg_launch_angle),
+      sweet_spot_pct: num(m.sweet_spot_pct) ?? num(m.sweet_spot_pct_bb),
+      ev95_pct: num(m.ev95_pct),
+      arm_angle: null,
+      gb_pct: null,
+      fb_pct: null,
+      ld_pct: null,
+      pitch_mix: null,
+      metrics: m,
+      updated_at: now,
+      _sufficient: pa != null && pa >= MIN_PA,
+    });
+  }
+
+  // ---- PITCHERS ----------------------------------------------------------
+  const pitcherIds = new Set([
+    ...feeds.pitcherDiscipline.keys(),
+    ...feeds.pitcherBattedBall.keys(),
+    ...feeds.pitchMix.keys(),
+  ]);
+  for (const id of pitcherIds) {
+    const d = feeds.pitcherDiscipline.get(id);
+    const b = feeds.pitcherBattedBall.get(id);
+    const mix = feeds.pitchMix.get(id);
+    const m = { ...(b ? b.metrics : {}), ...(d ? d.metrics : {}) };
+    const name = (d && d.name) || (mix && mix.name) || (b && b.name) || null;
+    const ip = num(m.ip);
+    rows.push({
+      sport,
+      season,
+      source_id: id,
+      player_key: name ? nameKey(name) : null,
+      player_name: name,
+      role: 'pitcher',
+      // Handedness comes free on the movement feed — the ONLY feed that carries
+      // it. Absent when a pitcher has thrown too few tracked pitches to appear.
+      throws: (mix && mix.throws) || null,
+      bats: null,
+      sample_pa: null,
+      sample_ip: ip,
+      sample_bip: num(m.bip),
+      k_pct: num(m.k_pct),
+      bb_pct: num(m.bb_pct),
+      whiff_pct: num(m.whiff_pct),
+      swing_pct: num(m.swing_pct),
+      chase_pct: num(m.chase_pct),
+      barrel_pct: num(m.barrel_pct_allowed),
+      hard_hit_pct: num(m.hard_hit_pct_allowed),
+      avg_exit_velo: num(m.avg_exit_velo),
+      max_exit_velo: num(m.max_exit_velo),
+      avg_launch_angle: num(m.avg_launch_angle),
+      sweet_spot_pct: num(m.sweet_spot_pct_bb),
+      ev95_pct: num(m.ev95_pct),
+      arm_angle: num(m.arm_angle),
+      gb_pct: num(m.gb_pct),
+      fb_pct: num(m.fb_pct),
+      ld_pct: num(m.ld_pct),
+      pitch_mix: mix && mix.pitches && mix.pitches.length ? mix.pitches : null,
+      metrics: m,
+      updated_at: now,
+      _sufficient: ip != null && ip >= MIN_IP,
+    });
+  }
+
+  return rows;
+}
+
+/** Strip the internal flag before the row reaches Postgres. */
+function toDbRow(r) {
+  const { _sufficient, ...rest } = r;
+  return rest;
+}
+
+/**
+ * refreshSeason(opts) — THE JOB. Backfill and nightly refresh are the same call.
+ *
+ * opts: { season, sport, supabase, fetchSeason, now, chunkSize }
+ * Returns a summary — never throws on a partial failure; a broken feed degrades
+ * to fewer rows, which is honest, rather than to a wrong number.
+ */
+async function refreshSeason(opts = {}) {
+  const season = opts.season || statcast.DEFAULT_SEASON;
+  const sport = opts.sport || 'mlb';
+  const started = opts.now || new Date().toISOString();
+
+  const fetchSeason = opts.fetchSeason || statcast.fetchSeason;
+  const feeds = await fetchSeason(season, opts);
+  const rows = buildRows(season, feeds, { sport, now: started });
+
+  const summary = {
+    ok: true,
+    sport,
+    season,
+    started_at: started,
+    feed_counts: feeds.counts || null,
+    rows: rows.length,
+    batters: rows.filter((r) => r.role === 'batter').length,
+    pitchers: rows.filter((r) => r.role === 'pitcher').length,
+    sufficient: rows.filter((r) => r._sufficient).length,
+    thin: rows.filter((r) => !r._sufficient).length,
+    joined: rows.filter((r) => r.player_key).length,
+    unjoined: rows.filter((r) => !r.player_key).length,
+    with_handedness: rows.filter((r) => r.throws).length,
+    written: 0,
+  };
+
+  if (!rows.length) {
+    // Every feed empty = the source failed, NOT "there is no mechanism data".
+    // Refuse to write, so a bad night can never blank a good table.
+    summary.ok = false;
+    summary.reason = 'no rows from source — refusing to write';
+    return summary;
+  }
+
+  const sb = opts.supabase || getServiceClient();
+  if (!sb) {
+    summary.ok = false;
+    summary.reason = 'no supabase client (env not configured)';
+    return summary;
+  }
+
+  // Chunked upsert — idempotent on the natural key, so re-running is a no-op
+  // beyond refreshing values and `updated_at`.
+  const chunk = opts.chunkSize || 500;
+  for (let i = 0; i < rows.length; i += chunk) {
+    const batch = rows.slice(i, i + chunk).map(toDbRow);
+    const { error } = await sb
+      .from('statcast_aggregates')
+      .upsert(batch, { onConflict: 'sport,season,source_id' });
+    if (error) {
+      summary.ok = false;
+      summary.reason = `upsert failed: ${error.message}`;
+      return summary;
+    }
+    summary.written += batch.length;
+  }
+
+  return summary;
+}
+
+function getServiceClient() {
+  try {
+    return require('../utils/supabase').getSupabaseServiceClient();
+  } catch {
+    return null;
+  }
+}
+
+/**
+ * getFreshness(opts) — how current is the mechanism tier?
+ * Reads the newest `updated_at`. Used by the staleness alarm and the status
+ * probe. Returns { updated_at, age_hours, rows } or nulls — never a guess.
+ */
+async function getFreshness(opts = {}) {
+  const sb = opts.supabase || getServiceClient();
+  if (!sb) return { updated_at: null, age_hours: null, rows: null };
+  const sport = opts.sport || 'mlb';
+  const { data, error } = await sb
+    .from('statcast_aggregates')
+    .select('updated_at')
+    .eq('sport', sport)
+    .order('updated_at', { ascending: false })
+    .limit(1);
+  if (error || !data || !data.length) return { updated_at: null, age_hours: null, rows: null };
+  const updated = data[0].updated_at;
+  const nowMs = opts.nowMs || Date.now();
+  const ageH = (nowMs - new Date(updated).getTime()) / 3_600_000;
+  return { updated_at: updated, age_hours: Math.round(ageH * 10) / 10 };
+}
+
+/**
+ * isStale(freshness, maxAgeHours) — PURE. The alarm predicate.
+ *
+ * NEVER-REFRESHED (updated_at null) is NOT stale — it is "not built yet", a
+ * different condition with a different fix. Conflating them would page on a
+ * fresh install and teach the operator to ignore the alarm.
+ */
+const MAX_AGE_HOURS = Number(process.env.STATCAST_MAX_AGE_HOURS) || 48;
+function isStale(freshness, maxAgeHours = MAX_AGE_HOURS) {
+  if (!freshness || freshness.updated_at == null) return false;
+  return Number.isFinite(freshness.age_hours) && freshness.age_hours > maxAgeHours;
+}
+
+module.exports = {
+  refreshSeason,
+  buildRows,
+  getFreshness,
+  isStale,
+  MIN_PA,
+  MIN_IP,
+  MAX_AGE_HOURS,
+  __internals: { toDbRow, num, getServiceClient },
+};

src/snapshotScheduler.js

@@ -286,6 +286,34 @@ function startSnapshotScheduler(opts = {}) {
       }
     } catch (e) { console.warn('[harness] nightly run failed:', e.message); }
 
+    // Session 68 — LAYER 1 MECHANISM DATA. Nightly full re-pull of the
+    // ~1,350-row Statcast aggregate set at STATCAST_HOUR_UTC. Backfill and
+    // refresh are the SAME call, upserted on the natural key, so the job is
+    // idempotent and self-healing: a missed night self-corrects on the next
+    // run with no incremental bookkeeping to drift. Kill switch: STATCAST=0.
+    try {
+      const statcastHour = Number(process.env.STATCAST_HOUR_UTC || 11); // ~7 AM ET, after every game is final
+      if (h === statcastHour && process.env.STATCAST !== '0') {
+        const agg = opts.statcastService || require('./services/statcastAggregateService');
+        const res = await agg.refreshSeason({});
+        console.log(`[statcast] nightly refresh — ok=${res.ok} rows=${res.rows ?? 'n/a'} written=${res.written ?? 0} joined=${res.joined ?? 'n/a'} thin=${res.thin ?? 'n/a'}${res.reason ? ` reason: ${res.reason}` : ''}`);
+        if (!res.ok) {
+          await notify(`Statcast refresh failed — ${res.reason || 'unknown'}. Mechanism data is not updating.`, {
+            title: 'VYNDR statcast', priority: 'high', tags: ['rotating_light'],
+          });
+        }
+        // Serving a stale aggregate as current is a quiet fabrication of
+        // currency, so staleness pages on its own — separate from a failed run,
+        // because a silently-not-scheduled job never produces a failure.
+        const fresh = await agg.getFreshness({});
+        if (agg.isStale(fresh)) {
+          await notify(`Statcast aggregates stale — last updated ${fresh.updated_at} (${fresh.age_hours}h). Mechanism data is being served as current.`, {
+            title: 'VYNDR statcast', priority: 'high', tags: ['rotating_light'],
+          });
+        }
+      }
+    } catch (e) { console.warn('[statcast] nightly refresh failed:', e.message); }
+
     // Session 8 — quota check after each snapshot run: odds-api >= 80% alerts
     // once per day (Redis-deduped). Never throws (guarded inside checkQuotaDaily).
     await opsWatch.checkQuotaDaily({ getStatus: getQuotaStatus, cacheGet, cacheSet, notify, now: () => now() });
@@ -324,7 +352,7 @@ function startSnapshotScheduler(opts = {}) {
 
   const interval = setInterval(() => { void tick(); void refreshTick(); }, 60_000);
   if (interval.unref) interval.unref();
-  console.log(`[snapshotScheduler] armed — SNAPSHOT_CRON=${process.env.SNAPSHOT_CRON}, hours=${HOURS_UTC.join(',')} UTC, intraday=${process.env.INTRADAY_REFRESH === '0' ? 'off' : `${REFRESH_MINUTES}m (slate hours)`}`);
+  console.log(`[snapshotScheduler] armed — SNAPSHOT_CRON=${process.env.SNAPSHOT_CRON}, hours=${HOURS_UTC.join(',')} UTC, intraday=${process.env.INTRADAY_REFRESH === '0' ? 'off' : `${REFRESH_MINUTES}m (slate hours)`}, statcast=${process.env.STATCAST === '0' ? 'off' : `${process.env.STATCAST_HOUR_UTC || 11}h UTC`}`);
   // Job 1 — state each sport's cadence at boot so "when does WNBA grade?" is
   // answerable from logs, and warn if a sport is scheduled at an hour the
   // scheduler never fires (it would silently never run).