Layer 1: Statcast mechanism-data ingestion (backfill + nightly refresh)

The data foundation for the archetype and projection layers, built as the
pattern every sport inherits. Layers 2 and 3 are not touched.

PHASE 0 GATE — both match rates measured live, both 100%. Batters 40/40;
PITCHERS 66/66 across five real rosters (CLE, DET, MIN, NYY, LAD) joined by
MLBAM id against the 713-pitcher Savant feed. Zero honest-absent on identity,
because the join is an integer both systems use natively — and the snapshot
pipeline already stores it per graded row.

SOURCE — five Baseball Savant CSV leaderboards, free and public, pulled with
axios and the CSV parser savantAdapter already runs in prod. pybaseball is
deliberately NOT used: it is an MIT wrapper over these same URLs, and adding it
would reintroduce a Python runtime in a stack where the existing Python service
is already offline. min=1 on every feed, not Savant's default min=q, so the
long tail arrives and OUR minimum-sample gate decides what is thin — explicit
and testable rather than silently dropped upstream.

Measured: 1,354 rows per season (604 batters, 750 pitchers), all five feeds in
about five seconds. Pitcher mechanism includes arm angle, GB/FB/LD, chase and
whiff; batters get exit velo, launch angle, barrel and hard-hit, chase and
z-swing. Handedness rides in free on the movement feed (677 pitchers); batter
handedness stays absent pending a roster join rather than being guessed.

BACKFILL AND REFRESH ARE THE SAME CALL — a full re-pull upserted on
(sport, season, source_id). Idempotent and self-healing: a missed night
self-corrects on the next run, with no incremental who-played bookkeeping to
drift out of sync. At 1,354 rows the simple thing is also the robust one.

HONESTY RULES, each with a test: a metric the feed did not carry is null and
never 0; a thin sample is STORED and flagged rather than dropped or inflated,
because thin and missing are different claims; an unjoined player is stored
with a null player_key and joins later; and if every feed comes back empty the
job REFUSES to write, so a bad night can never blank a good table.

Freshness is treated as a truth property. updated_at on every row, and the
scheduler pages on a failed run AND on silent staleness — a job that stops
being scheduled never produces a failure, so staleness has to alarm on its own.
Never-built is deliberately not stale: different condition, different fix, and
paging on a fresh install teaches the operator to ignore the alarm.

Nightly at STATCAST_HOUR_UTC (default 11 UTC, after every game is final), kill
switch STATCAST=0, and induce-able at POST /api/internal/statcast/refresh with
a freshness probe at /statcast/status — we verify a refresh by running it, not
by waiting for the slot.

Migration 030 applied. Promoted columns for the classification-critical metrics
plus a metrics JSONB carrying every raw field, so Layer 2 can reach something we
did not promote without a re-ingest. Raw per-pitch stays out of Postgres on
purpose: one season is ~0.85 GB against a 500 MB plan ceiling, and it is
re-pullable from the free source if Layer 3 ever needs it.

Pattern documented in docs/MECHANISM-DATA.md for NBA tracking and NFL Next Gen.

Tests 3581 passed / 292 suites, web build exit 0.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VCNgGSt5qvcLxaeQqa7Zpj

src/routes/internal.js

@@ -431,6 +431,46 @@ router.post('/newsletter/send', async (req, res) => {
   }
 });
 
+/**
+ * POST /api/internal/statcast/refresh — INDUCE the Layer-1 mechanism refresh.
+ *
+ * Same call the nightly scheduler makes. Exists so the job is verifiable ON
+ * DEMAND: we prove a refresh works by running it and reading the result, never
+ * by waiting for the cron slot. Idempotent — running it twice is a no-op beyond
+ * refreshing values and updated_at.
+ */
+router.post('/statcast/refresh', async (req, res) => {
+  try {
+    const agg = require('../services/statcastAggregateService');
+    const season = req.query.season ? Number(req.query.season) : undefined;
+    const out = await agg.refreshSeason({ season });
+    const fresh = await agg.getFreshness({});
+    return res.status(out.ok ? 200 : 500).json({ ...out, freshness: fresh, stale: agg.isStale(fresh) });
+  } catch (err) {
+    console.error('[internal/statcast]', err.message);
+    return res.status(500).json({ ok: false, error: err.message });
+  }
+});
+
+/** GET /api/internal/statcast/status — freshness probe for the mechanism tier. */
+router.get('/statcast/status', async (req, res) => {
+  try {
+    const agg = require('../services/statcastAggregateService');
+    const fresh = await agg.getFreshness({});
+    return res.json({
+      ...fresh,
+      stale: agg.isStale(fresh),
+      max_age_hours: agg.MAX_AGE_HOURS,
+      min_pa: agg.MIN_PA,
+      min_ip: agg.MIN_IP,
+      cron_hour_utc: Number(process.env.STATCAST_HOUR_UTC || 11),
+      enabled: process.env.STATCAST !== '0',
+    });
+  } catch (err) {
+    return res.status(500).json({ error: err.message });
+  }
+});
+
 router.post('/outcomes/:sport', async (req, res) => {
   const outcomes = require('../services/outcomeService');
   try {