Arm harness on our scheduler + start closing-line capture (capture only)

PART A — HARNESS ARMED ON OUR OWN INFRA. snapshotScheduler now runs the
nightly backtest at HARNESS_HOUR_UTC (default 14), appends to
harness_results, and pages via opsWatch.harnessStaleAlarm — a validator
that stops running looks exactly like one that keeps passing. No external
dependency: the join is plain SQL through the service client and the
harness is a pure function. POST /api/internal/harness/run induces the
same code path on demand, because a scheduled mechanism is verified by
inducing it, never by waiting for a slot.

PART B PHASE 0 — GATE PASSED for what is capturable:
- C4 diagnosed: closing_line is ONE overwritable field with no timestamp
  and no provenance. captureClosing writes the current line and, when a
  prop fails to match, silently leaves the earlier value (= the lock) in
  place — so "captured a real close" is indistinguishable from "never
  updated". It is 92% equal, not 100%: 56 rows DID record movement, so
  the defect is provenance, not the value.
- Feeds: normalized props already carry BOTH raw side prices per book,
  with game_time, and the intraday refresh polls every ~20 min during
  slate hours — so the last observable pre-lock line is available.
- SHARP close: pinnacle is in ALLOWED_BOOKS -> a no-vig reference is
  capturable ("beat the market").
- ODAWA: NOT capturable. 'odawa' exists only as a UI preference option in
  onboarding/settings; it is in no adapter, no ALLOWED_BOOKS, no feed. An
  un-capturable source is a finding, not a gap to paper over.
- JOIN: must drop `line` from the natural key, because a close that MOVED
  off the graded line is the entire point of CLV. Verified safe — all 164
  current identity groups have exactly ONE line per
  (sport, player_key, stat, side, game_date). Zero ambiguity.

PART B PHASE 1 — CAPTURE ONLY, built test-first. The refusal was proven
before the capture logic existed: unbound game_time, doubleheader
ambiguity, a missed pre-lock window, or a one-sided price all record
missed_reason with NO price. A stale or mid-day line substituted for a
close would manufacture a CLV proof from a number that was never the
close.

migration 029 closing_captures: append-only, never overwritten (that is
the provenance C4 lacked), BOTH raw side prices so the existing de-vig
engine can compute a fair closing probability later, sharp vs book line
types kept distinct. Wired into the intraday refresh with a capture-rate
alarm — a missed close is unrecoverable.

NO CLV metric built, as ordered. This starts the clock.

Suite 284/3417 green, build exit 0.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SmNjJAwEnqHPtXbvSZR8kA

src/routes/internal.js

@@ -345,6 +345,49 @@ router.get('/backup/offbox', async (req, res) => {
   }
 });
 
+/**
+ * POST /api/internal/harness/run (Session 64) — induce the nightly harness run
+ * on demand. Scheduled mechanisms are verified by INDUCING their real code
+ * path, never by waiting for a slot to discover whether they work.
+ */
+router.post('/harness/run', async (req, res) => {
+  try {
+    const runner = require('../services/harnessRunner');
+    const out = await runner.runAndRecord();
+    return res.json({ ok: out.ok !== false, ...out, last_run_at: await runner.lastRunAt() });
+  } catch (err) {
+    return res.status(500).json({ ok: false, error: err.message });
+  }
+});
+
+/**
+ * POST /api/internal/closing/capture/:sport (Session 64) — induce one closing
+ * capture pass against the CURRENT live odds, so the mechanism is proven now
+ * rather than discovered tomorrow.
+ */
+router.post('/closing/capture/:sport', async (req, res) => {
+  try {
+    const sp = String(req.params.sport || '').toLowerCase();
+    const closing = require('../services/closingCapture');
+    const odds = await require('../services/oddsService').getOdds(sp);
+    const props = (odds && Array.isArray(odds.props)) ? odds.props : [];
+    await require('../services/gameBinder').attachGameTimes(sp, props, {});
+    const windowMinutes = Number(req.query.window || 0) || undefined;
+    const rows = closing.buildCaptureRows(sp, props, { windowMinutes });
+    const priced = rows.filter((r) => !r.missed_reason).length;
+    const persisted = req.query.dry === '1' ? { skipped: true } : await closing.persist(rows);
+    const reasons = {};
+    for (const r of rows) if (r.missed_reason) reasons[r.missed_reason] = (reasons[r.missed_reason] || 0) + 1;
+    return res.json({
+      ok: true, sport: sp, props: props.length,
+      rows: rows.length, priced, missed: rows.length - priced,
+      missed_reasons: reasons, persisted,
+    });
+  } catch (err) {
+    return res.status(500).json({ ok: false, error: err.message });
+  }
+});
+
 /**
  * POST /api/internal/ledger/settle (Session 58, Phase 1) — settle the
  * persistent ledger (outcome + actual_value + CLV) across every sport.

src/services/closingCapture.js

@@ -0,0 +1,175 @@
+'use strict';
+
+/**
+ * CLOSING-LINE CAPTURE (Session 64) — CAPTURE ONLY. No CLV metric here.
+ *
+ * WHY THIS EXISTS: CLV is currently un-backtestable. `ledger_entries
+ * .closing_line` equals the locked line on 92% of rows — not because lines
+ * never move (56 rows DID move) but because `captureClosing` overwrites a
+ * single field with whatever the current props say and, when a prop fails to
+ * match, silently leaves the earlier value in place. There is no timestamp and
+ * no provenance, so "captured a genuine close" is indistinguishable from
+ * "never successfully updated". That ambiguity is the actual C4 defect.
+ *
+ * This module writes an IMMUTABLE, append-only capture instead: one row per
+ * (prop identity × side × book × capture moment), with BOTH raw side prices so
+ * the existing de-vig engine can compute a fair closing probability later.
+ *
+ * THE JOIN: rows carry (sport, player_key, stat, side, game_date) — the natural
+ * key MINUS `line`, deliberately. A closing line that equals the graded line is
+ * the boring case; the whole point of CLV is the case where it MOVED, so `line`
+ * cannot be part of the key. Verified safe: across current retention, all 164
+ * identity groups have exactly ONE line per (sport, player_key, stat, side,
+ * game_date) — zero ambiguity.
+ *
+ * THE REFUSAL: a prop we cannot legitimately price at the close records
+ * `missed_reason` and NO price. Substituting a stale or mid-day line would
+ * manufacture a CLV proof from a number that was never the close.
+ */
+
+const { nameKey } = require('../utils/playerName');
+
+// How close to first pitch/tip counts as "the close". Our odds refresh runs
+// every ~20 min during slate hours, so this window is sized to catch the last
+// observable line without reaching back into mid-day pricing.
+const WINDOW_MINUTES_BEFORE = Number(process.env.CLOSING_WINDOW_MIN || 45);
+// Sharp/no-vig reference book — "beat the market", distinct from "beat our book".
+const SHARP_BOOKS = new Set(['pinnacle']);
+const CAPTURE_RATE_FLOOR = Number(process.env.CLOSING_CAPTURE_FLOOR || 0.6);
+
+function etDate(iso) {
+  if (!iso) return null;
+  const t = new Date(iso);
+  if (Number.isNaN(t.getTime())) return null;
+  return new Intl.DateTimeFormat('en-CA', {
+    timeZone: 'America/New_York', year: 'numeric', month: '2-digit', day: '2-digit',
+  }).format(t);
+}
+
+function numOrNull(v) {
+  if (v == null || v === '') return null;
+  const n = Number(v);
+  return Number.isFinite(n) ? n : null;
+}
+
+/** A refusal row: identity + why, never a price. */
+function missedRow(sport, p, side, reason, nowIso) {
+  return {
+    sport,
+    player_key: nameKey(p.player),
+    player_name: p.player,
+    stat: p.stat_type,
+    side,
+    game_date: etDate(p.game_time) || null,
+    game_time: p.game_time || null,
+    book: p.book || null,
+    line_type: SHARP_BOOKS.has(String(p.book || '').toLowerCase()) ? 'sharp' : 'book',
+    line: null,
+    over_odds: null,
+    under_odds: null,
+    captured_at: nowIso,
+    missed_reason: reason,
+  };
+}
+
+/**
+ * Build capture rows for a slate at a moment in time. PURE — no I/O, so the
+ * refusal rules are unit-testable and cannot silently depend on a live feed.
+ *
+ * Returns [] for props that are not yet near lock (nothing to record — it is
+ * simply not the close yet). Returns refusal rows for props inside the window
+ * that cannot be priced honestly.
+ */
+function buildCaptureRows(sport, props, opts = {}) {
+  const now = opts.now || new Date();
+  const nowIso = now.toISOString();
+  const windowMs = (opts.windowMinutes ?? WINDOW_MINUTES_BEFORE) * 60_000;
+  const out = [];
+
+  for (const p of props || []) {
+    if (!p || !p.player || !p.stat_type) continue;
+    const sides = ['over', 'under'];
+
+    // No bound game time → we cannot know when the close IS. Record it.
+    if (!p.game_time || !etDate(p.game_time)) {
+      for (const s of sides) out.push(missedRow(sport, p, s, 'unbound_game_time', nowIso));
+      continue;
+    }
+    // Doubleheader: two locks, no game number on the prop → unattributable.
+    if (p.game_ambiguous) {
+      for (const s of sides) out.push(missedRow(sport, p, s, 'doubleheader_ambiguous', nowIso));
+      continue;
+    }
+
+    const lockMs = new Date(p.game_time).getTime();
+    const delta = lockMs - now.getTime();
+    if (delta > windowMs) continue;                    // not the close yet — say nothing
+    if (delta < 0) {                                   // already started
+      for (const s of sides) out.push(missedRow(sport, p, s, 'missed_window', nowIso));
+      continue;
+    }
+
+    const over = numOrNull(p.over_odds);
+    const under = numOrNull(p.under_odds);
+    const line = numOrNull(p.line);
+    // A one-sided price cannot produce a fair (de-vigged) closing probability,
+    // so it is not a usable close. Recorded as such rather than half-stored.
+    if (over == null || under == null || line == null) {
+      for (const s of sides) out.push(missedRow(sport, p, s, 'one_sided_price', nowIso));
+      continue;
+    }
+
+    for (const s of sides) {
+      out.push({
+        ...missedRow(sport, p, s, null, nowIso),
+        line,
+        over_odds: over,
+        under_odds: under,
+      });
+    }
+  }
+  return out;
+}
+
+/** Silent-failure discipline: a capture pass that mostly misses is a broken
+ *  pipe, and a missing close cannot be recovered later. */
+function captureRateAlarm({ eligible = 0, captured = 0 } = {}, opts = {}) {
+  const floor = Number.isFinite(opts.floor) ? opts.floor : CAPTURE_RATE_FLOOR;
+  if (!eligible) return { alarm: false, reason: null, rate: null };
+  const rate = captured / eligible;
+  return {
+    alarm: rate < floor,
+    rate,
+    reason: rate < floor
+      ? `closing capture rate ${Math.round(rate * 100)}% of ${eligible} eligible props (floor ${Math.round(floor * 100)}%) — tonight's closes are unrecoverable`
+      : null,
+  };
+}
+
+/** Persist append-only. Never updates an existing row: a capture is a
+ *  historical fact, and rewriting it would destroy the provenance that C4
+ *  lacked in the first place. */
+async function persist(rows, deps = {}) {
+  const out = { attempted: rows ? rows.length : 0, written: 0, skipped: false, error: null };
+  if (!out.attempted) return out;
+  try {
+    const getClient = deps.getClient || require('../utils/supabase').getSupabaseServiceClient;
+    const sb = getClient();
+    if (!sb) { out.skipped = true; return out; }
+    const CHUNK = 250;
+    for (let i = 0; i < rows.length; i += CHUNK) {
+      const chunk = rows.slice(i, i + CHUNK);
+      const { error } = await sb.from('closing_captures').insert(chunk);
+      if (error) { out.error = error.message; break; }
+      out.written += chunk.length;
+    }
+  } catch (e) {
+    out.error = e && e.message ? e.message : String(e);
+  }
+  return out;
+}
+
+module.exports = {
+  buildCaptureRows, captureRateAlarm, persist,
+  WINDOW_MINUTES_BEFORE, SHARP_BOOKS,
+};

src/services/harnessRunner.js

@@ -0,0 +1,118 @@
+'use strict';
+
+/**
+ * HARNESS RUNNER (Session 64) — joins, runs the backtest, appends the result.
+ *
+ * Runs on OUR scheduler, in our container. No external dependency: the join is
+ * plain SQL through the service client, the harness is a pure function, and the
+ * result lands in `harness_results`.
+ *
+ * THE JOIN: model_snapshots (the model's INPUTS + prediction) → ledger_entries
+ * (the single source of truth for OUTCOMES) on the natural key
+ * (sport, player_key, stat, line, side, game_date). Outcomes are never
+ * denormalized onto snapshots.
+ *
+ * Rows that don't join are EXPECTED: retention stores both sides of every prop
+ * plus refusals, while the ledger keeps only the graded side of graded props.
+ */
+
+const harness = require('./backtestHarness');
+
+const HARNESS_VERSION = 'harness@2026-07-20';
+
+/** Pull joined rows. Read-only. */
+async function fetchJoinedRows(deps = {}) {
+  const getClient = deps.getClient || require('../utils/supabase').getSupabaseServiceClient;
+  const sb = getClient();
+  if (!sb) return { rows: [], skipped: true };
+
+  // Supabase's client cannot express this join, so it runs as two reads and is
+  // joined in memory — the volumes here are hundreds of rows, not millions.
+  const [{ data: snaps }, { data: ledger }] = await Promise.all([
+    sb.from('model_snapshots')
+      .select('sport, model_version, grade, grade_11, p_win, player_key, stat, line, side, game_date, quarantine_reason, refused')
+      .eq('refused', false)
+      .limit(20000),
+    sb.from('ledger_entries')
+      .select('sport, player_key, stat, line, side, game_date, outcome, quarantine_reason')
+      .is('user_id', null)
+      .limit(20000),
+  ]);
+
+  const key = (r) => `${r.sport}|${r.player_key}|${r.stat}|${Number(r.line)}|${r.side}|${r.game_date}`;
+  const byKey = new Map();
+  for (const l of ledger || []) byKey.set(key(l), l);
+
+  const rows = [];
+  for (const s of snaps || []) {
+    const l = byKey.get(key(s));
+    if (!l) continue;                 // expected: unselected side / refusal
+    rows.push({
+      sport: s.sport,
+      model_version: s.model_version,
+      grade: s.grade,
+      grade_11: s.grade_11,
+      p_win: s.p_win,
+      outcome: l.outcome,
+      quarantine_reason: l.quarantine_reason,
+      snap_quarantine: s.quarantine_reason,
+    });
+  }
+  return { rows, skipped: false, joined: rows.length, snapshots: (snaps || []).length };
+}
+
+/**
+ * Run the harness and append the result. Never throws — a failed harness run
+ * must not break the scheduler tick, and the staleness alarm is what surfaces
+ * a run that stops happening.
+ */
+async function runAndRecord(deps = {}) {
+  try {
+    const { rows, skipped, joined, snapshots } = await fetchJoinedRows(deps);
+    if (skipped) return { skipped: true, reason: 'no supabase client' };
+
+    const report = harness.runBacktest(rows, {});
+    const getClient = deps.getClient || require('../utils/supabase').getSupabaseServiceClient;
+    const sb = getClient();
+
+    const record = {
+      verdict: report.verdict,
+      can_validate: report.can_validate,
+      min_sample: report.min_sample,
+      scored: report.counts.scored,
+      excluded: {
+        quarantine: report.counts.excluded_quarantine,
+        terminal: report.counts.excluded_terminal,
+        pending: report.counts.excluded_pending,
+        push: report.counts.excluded_push,
+      },
+      report,
+      harness_version: HARNESS_VERSION,
+    };
+    const { error } = await sb.from('harness_results').insert([record]);
+    return {
+      ok: !error,
+      error: error ? error.message : null,
+      verdict: report.verdict,
+      scored: report.counts.scored,
+      joined,
+      snapshots,
+    };
+  } catch (e) {
+    return { ok: false, error: e && e.message ? e.message : String(e) };
+  }
+}
+
+/** Newest run timestamp, for the staleness alarm. */
+async function lastRunAt(deps = {}) {
+  try {
+    const getClient = deps.getClient || require('../utils/supabase').getSupabaseServiceClient;
+    const sb = getClient();
+    if (!sb) return null;
+    const { data } = await sb.from('harness_results')
+      .select('ran_at').order('ran_at', { ascending: false }).limit(1);
+    return data && data[0] ? data[0].ran_at : null;
+  } catch { return null; }
+}
+
+module.exports = { runAndRecord, fetchJoinedRows, lastRunAt, HARNESS_VERSION };

src/services/intradayRefreshService.js

@@ -211,6 +211,28 @@ async function runIntradayRefresh(sport, opts = {}) {
   await deps.cacheSet(`snapshot:${sp}:latest`, updated, SNAP_TTL);
   await deps.cacheSet(`grades:${sp}`, { grades, updated_at: snap.updated_at, refreshed_at: ts, source: snap.source || 'refresh' }, SNAP_TTL);
   try { await deps.ledger.captureClosing(sp, props); } catch { /* best-effort */ }
+
+  // Session 64 — CLOSING CAPTURE. This refresh already runs every ~20 min
+  // during slate hours, so it is the natural place to observe the last line
+  // before each game locks. Append-only and best-effort: a capture failure must
+  // never break the refresh, but a MISSED close is unrecoverable, so the rate
+  // is alarmed rather than silently tolerated.
+  try {
+    const closing = deps.closingCapture || require('./closingCapture');
+    const rows = closing.buildCaptureRows(sp, props, {});
+    if (rows.length) {
+      const res = await closing.persist(rows);
+      const captured = rows.filter((r) => !r.missed_reason).length;
+      const al = closing.captureRateAlarm({ eligible: rows.length, captured });
+      console.log(`[intraday] closing capture ${sp}: ${captured}/${rows.length} priced, ${res.written} stored${al.alarm ? ' — RATE ALARM' : ''}`);
+      if (al.alarm && deps.notify) {
+        await deps.notify(`Closing capture degraded for ${sp.toUpperCase()} — ${al.reason}`,
+          { title: 'VYNDR closing capture', priority: 'high', tags: ['rotating_light'] });
+      }
+    }
+  } catch (e) {
+    console.warn(`[intraday] closing capture failed for ${sp}:`, e.message);
+  }
   if (moveEvents.length > 0) {
     await deps.pushTickerItems(moveEvents.slice(0, TICKER_MOVE_CAP), deps);
   }

src/snapshotScheduler.js

@@ -265,6 +265,27 @@ function startSnapshotScheduler(opts = {}) {
     } catch (e) {
       console.warn('[snapshot] cron run failed:', e.message);
     }
+    // Session 64 — NIGHTLY BACKTEST HARNESS, on our own scheduler. Runs once
+    // per day at HARNESS_HOUR_UTC and appends to harness_results, so the
+    // calibration trend is visible as retention compounds. An
+    // INSUFFICIENT_HISTORY verdict is expected and correct.
+    try {
+      const harnessHour = Number(process.env.HARNESS_HOUR_UTC || 14);
+      if (h === harnessHour) {
+        const runner = opts.harnessRunner || require('./services/harnessRunner');
+        const res = await runner.runAndRecord();
+        console.log(`[harness] nightly run — verdict=${res.verdict || 'n/a'} scored=${res.scored ?? 'n/a'}${res.error ? ` ERROR: ${res.error}` : ''}`);
+        // A validator that stops running looks exactly like one that keeps
+        // passing — so staleness pages.
+        const stale = opsWatch.harnessStaleAlarm(await runner.lastRunAt(), now());
+        if (stale.alarm) {
+          await notify(`Backtest harness stale — ${stale.reason}. The metric gate is not running.`, {
+            title: 'VYNDR harness', priority: 'high', tags: ['rotating_light'],
+          });
+        }
+      }
+    } catch (e) { console.warn('[harness] nightly run failed:', e.message); }
+
     // Session 8 — quota check after each snapshot run: odds-api >= 80% alerts
     // once per day (Redis-deduped). Never throws (guarded inside checkQuotaDaily).
     await opsWatch.checkQuotaDaily({ getStatus: getQuotaStatus, cacheGet, cacheSet, notify, now: () => now() });