Arm harness on our scheduler + start closing-line capture (capture only)

PART A — HARNESS ARMED ON OUR OWN INFRA. snapshotScheduler now runs the
nightly backtest at HARNESS_HOUR_UTC (default 14), appends to
harness_results, and pages via opsWatch.harnessStaleAlarm — a validator
that stops running looks exactly like one that keeps passing. No external
dependency: the join is plain SQL through the service client and the
harness is a pure function. POST /api/internal/harness/run induces the
same code path on demand, because a scheduled mechanism is verified by
inducing it, never by waiting for a slot.

PART B PHASE 0 — GATE PASSED for what is capturable:
- C4 diagnosed: closing_line is ONE overwritable field with no timestamp
  and no provenance. captureClosing writes the current line and, when a
  prop fails to match, silently leaves the earlier value (= the lock) in
  place — so "captured a real close" is indistinguishable from "never
  updated". It is 92% equal, not 100%: 56 rows DID record movement, so
  the defect is provenance, not the value.
- Feeds: normalized props already carry BOTH raw side prices per book,
  with game_time, and the intraday refresh polls every ~20 min during
  slate hours — so the last observable pre-lock line is available.
- SHARP close: pinnacle is in ALLOWED_BOOKS -> a no-vig reference is
  capturable ("beat the market").
- ODAWA: NOT capturable. 'odawa' exists only as a UI preference option in
  onboarding/settings; it is in no adapter, no ALLOWED_BOOKS, no feed. An
  un-capturable source is a finding, not a gap to paper over.
- JOIN: must drop `line` from the natural key, because a close that MOVED
  off the graded line is the entire point of CLV. Verified safe — all 164
  current identity groups have exactly ONE line per
  (sport, player_key, stat, side, game_date). Zero ambiguity.

PART B PHASE 1 — CAPTURE ONLY, built test-first. The refusal was proven
before the capture logic existed: unbound game_time, doubleheader
ambiguity, a missed pre-lock window, or a one-sided price all record
missed_reason with NO price. A stale or mid-day line substituted for a
close would manufacture a CLV proof from a number that was never the
close.

migration 029 closing_captures: append-only, never overwritten (that is
the provenance C4 lacked), BOTH raw side prices so the existing de-vig
engine can compute a fair closing probability later, sharp vs book line
types kept distinct. Wired into the intraday refresh with a capture-rate
alarm — a missed close is unrecoverable.

NO CLV metric built, as ordered. This starts the clock.

Suite 284/3417 green, build exit 0.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SmNjJAwEnqHPtXbvSZR8kA
This commit is contained in:
Kev
2026-07-20 12:05:16 -04:00
parent e809a0eb3c
commit 77a58e4113
6 changed files with 499 additions and 0 deletions
+43
View File
@@ -345,6 +345,49 @@ router.get('/backup/offbox', async (req, res) => {
}
});
/**
* POST /api/internal/harness/run (Session 64) — induce the nightly harness run
* on demand. Scheduled mechanisms are verified by INDUCING their real code
* path, never by waiting for a slot to discover whether they work.
*/
router.post('/harness/run', async (req, res) => {
try {
const runner = require('../services/harnessRunner');
const out = await runner.runAndRecord();
return res.json({ ok: out.ok !== false, ...out, last_run_at: await runner.lastRunAt() });
} catch (err) {
return res.status(500).json({ ok: false, error: err.message });
}
});
/**
* POST /api/internal/closing/capture/:sport (Session 64) — induce one closing
* capture pass against the CURRENT live odds, so the mechanism is proven now
* rather than discovered tomorrow.
*/
router.post('/closing/capture/:sport', async (req, res) => {
try {
const sp = String(req.params.sport || '').toLowerCase();
const closing = require('../services/closingCapture');
const odds = await require('../services/oddsService').getOdds(sp);
const props = (odds && Array.isArray(odds.props)) ? odds.props : [];
await require('../services/gameBinder').attachGameTimes(sp, props, {});
const windowMinutes = Number(req.query.window || 0) || undefined;
const rows = closing.buildCaptureRows(sp, props, { windowMinutes });
const priced = rows.filter((r) => !r.missed_reason).length;
const persisted = req.query.dry === '1' ? { skipped: true } : await closing.persist(rows);
const reasons = {};
for (const r of rows) if (r.missed_reason) reasons[r.missed_reason] = (reasons[r.missed_reason] || 0) + 1;
return res.json({
ok: true, sport: sp, props: props.length,
rows: rows.length, priced, missed: rows.length - priced,
missed_reasons: reasons, persisted,
});
} catch (err) {
return res.status(500).json({ ok: false, error: err.message });
}
});
/**
* POST /api/internal/ledger/settle (Session 58, Phase 1) — settle the
* persistent ledger (outcome + actual_value + CLV) across every sport.
+175
View File
@@ -0,0 +1,175 @@
'use strict';
/**
* CLOSING-LINE CAPTURE (Session 64) — CAPTURE ONLY. No CLV metric here.
*
* WHY THIS EXISTS: CLV is currently un-backtestable. `ledger_entries
* .closing_line` equals the locked line on 92% of rows — not because lines
* never move (56 rows DID move) but because `captureClosing` overwrites a
* single field with whatever the current props say and, when a prop fails to
* match, silently leaves the earlier value in place. There is no timestamp and
* no provenance, so "captured a genuine close" is indistinguishable from
* "never successfully updated". That ambiguity is the actual C4 defect.
*
* This module writes an IMMUTABLE, append-only capture instead: one row per
* (prop identity × side × book × capture moment), with BOTH raw side prices so
* the existing de-vig engine can compute a fair closing probability later.
*
* THE JOIN: rows carry (sport, player_key, stat, side, game_date) — the natural
* key MINUS `line`, deliberately. A closing line that equals the graded line is
* the boring case; the whole point of CLV is the case where it MOVED, so `line`
* cannot be part of the key. Verified safe: across current retention, all 164
* identity groups have exactly ONE line per (sport, player_key, stat, side,
* game_date) — zero ambiguity.
*
* THE REFUSAL: a prop we cannot legitimately price at the close records
* `missed_reason` and NO price. Substituting a stale or mid-day line would
* manufacture a CLV proof from a number that was never the close.
*/
const { nameKey } = require('../utils/playerName');
// How close to first pitch/tip counts as "the close". Our odds refresh runs
// every ~20 min during slate hours, so this window is sized to catch the last
// observable line without reaching back into mid-day pricing.
const WINDOW_MINUTES_BEFORE = Number(process.env.CLOSING_WINDOW_MIN || 45);
// Sharp/no-vig reference book — "beat the market", distinct from "beat our book".
const SHARP_BOOKS = new Set(['pinnacle']);
const CAPTURE_RATE_FLOOR = Number(process.env.CLOSING_CAPTURE_FLOOR || 0.6);
function etDate(iso) {
if (!iso) return null;
const t = new Date(iso);
if (Number.isNaN(t.getTime())) return null;
return new Intl.DateTimeFormat('en-CA', {
timeZone: 'America/New_York', year: 'numeric', month: '2-digit', day: '2-digit',
}).format(t);
}
function numOrNull(v) {
if (v == null || v === '') return null;
const n = Number(v);
return Number.isFinite(n) ? n : null;
}
/** A refusal row: identity + why, never a price. */
function missedRow(sport, p, side, reason, nowIso) {
return {
sport,
player_key: nameKey(p.player),
player_name: p.player,
stat: p.stat_type,
side,
game_date: etDate(p.game_time) || null,
game_time: p.game_time || null,
book: p.book || null,
line_type: SHARP_BOOKS.has(String(p.book || '').toLowerCase()) ? 'sharp' : 'book',
line: null,
over_odds: null,
under_odds: null,
captured_at: nowIso,
missed_reason: reason,
};
}
/**
* Build capture rows for a slate at a moment in time. PURE — no I/O, so the
* refusal rules are unit-testable and cannot silently depend on a live feed.
*
* Returns [] for props that are not yet near lock (nothing to record — it is
* simply not the close yet). Returns refusal rows for props inside the window
* that cannot be priced honestly.
*/
function buildCaptureRows(sport, props, opts = {}) {
const now = opts.now || new Date();
const nowIso = now.toISOString();
const windowMs = (opts.windowMinutes ?? WINDOW_MINUTES_BEFORE) * 60_000;
const out = [];
for (const p of props || []) {
if (!p || !p.player || !p.stat_type) continue;
const sides = ['over', 'under'];
// No bound game time → we cannot know when the close IS. Record it.
if (!p.game_time || !etDate(p.game_time)) {
for (const s of sides) out.push(missedRow(sport, p, s, 'unbound_game_time', nowIso));
continue;
}
// Doubleheader: two locks, no game number on the prop → unattributable.
if (p.game_ambiguous) {
for (const s of sides) out.push(missedRow(sport, p, s, 'doubleheader_ambiguous', nowIso));
continue;
}
const lockMs = new Date(p.game_time).getTime();
const delta = lockMs - now.getTime();
if (delta > windowMs) continue; // not the close yet — say nothing
if (delta < 0) { // already started
for (const s of sides) out.push(missedRow(sport, p, s, 'missed_window', nowIso));
continue;
}
const over = numOrNull(p.over_odds);
const under = numOrNull(p.under_odds);
const line = numOrNull(p.line);
// A one-sided price cannot produce a fair (de-vigged) closing probability,
// so it is not a usable close. Recorded as such rather than half-stored.
if (over == null || under == null || line == null) {
for (const s of sides) out.push(missedRow(sport, p, s, 'one_sided_price', nowIso));
continue;
}
for (const s of sides) {
out.push({
...missedRow(sport, p, s, null, nowIso),
line,
over_odds: over,
under_odds: under,
});
}
}
return out;
}
/** Silent-failure discipline: a capture pass that mostly misses is a broken
* pipe, and a missing close cannot be recovered later. */
function captureRateAlarm({ eligible = 0, captured = 0 } = {}, opts = {}) {
const floor = Number.isFinite(opts.floor) ? opts.floor : CAPTURE_RATE_FLOOR;
if (!eligible) return { alarm: false, reason: null, rate: null };
const rate = captured / eligible;
return {
alarm: rate < floor,
rate,
reason: rate < floor
? `closing capture rate ${Math.round(rate * 100)}% of ${eligible} eligible props (floor ${Math.round(floor * 100)}%) — tonight's closes are unrecoverable`
: null,
};
}
/** Persist append-only. Never updates an existing row: a capture is a
* historical fact, and rewriting it would destroy the provenance that C4
* lacked in the first place. */
async function persist(rows, deps = {}) {
const out = { attempted: rows ? rows.length : 0, written: 0, skipped: false, error: null };
if (!out.attempted) return out;
try {
const getClient = deps.getClient || require('../utils/supabase').getSupabaseServiceClient;
const sb = getClient();
if (!sb) { out.skipped = true; return out; }
const CHUNK = 250;
for (let i = 0; i < rows.length; i += CHUNK) {
const chunk = rows.slice(i, i + CHUNK);
const { error } = await sb.from('closing_captures').insert(chunk);
if (error) { out.error = error.message; break; }
out.written += chunk.length;
}
} catch (e) {
out.error = e && e.message ? e.message : String(e);
}
return out;
}
module.exports = {
buildCaptureRows, captureRateAlarm, persist,
WINDOW_MINUTES_BEFORE, SHARP_BOOKS,
};
+118
View File
@@ -0,0 +1,118 @@
'use strict';
/**
* HARNESS RUNNER (Session 64) — joins, runs the backtest, appends the result.
*
* Runs on OUR scheduler, in our container. No external dependency: the join is
* plain SQL through the service client, the harness is a pure function, and the
* result lands in `harness_results`.
*
* THE JOIN: model_snapshots (the model's INPUTS + prediction) → ledger_entries
* (the single source of truth for OUTCOMES) on the natural key
* (sport, player_key, stat, line, side, game_date). Outcomes are never
* denormalized onto snapshots.
*
* Rows that don't join are EXPECTED: retention stores both sides of every prop
* plus refusals, while the ledger keeps only the graded side of graded props.
*/
const harness = require('./backtestHarness');
const HARNESS_VERSION = 'harness@2026-07-20';
/** Pull joined rows. Read-only. */
async function fetchJoinedRows(deps = {}) {
const getClient = deps.getClient || require('../utils/supabase').getSupabaseServiceClient;
const sb = getClient();
if (!sb) return { rows: [], skipped: true };
// Supabase's client cannot express this join, so it runs as two reads and is
// joined in memory — the volumes here are hundreds of rows, not millions.
const [{ data: snaps }, { data: ledger }] = await Promise.all([
sb.from('model_snapshots')
.select('sport, model_version, grade, grade_11, p_win, player_key, stat, line, side, game_date, quarantine_reason, refused')
.eq('refused', false)
.limit(20000),
sb.from('ledger_entries')
.select('sport, player_key, stat, line, side, game_date, outcome, quarantine_reason')
.is('user_id', null)
.limit(20000),
]);
const key = (r) => `${r.sport}|${r.player_key}|${r.stat}|${Number(r.line)}|${r.side}|${r.game_date}`;
const byKey = new Map();
for (const l of ledger || []) byKey.set(key(l), l);
const rows = [];
for (const s of snaps || []) {
const l = byKey.get(key(s));
if (!l) continue; // expected: unselected side / refusal
rows.push({
sport: s.sport,
model_version: s.model_version,
grade: s.grade,
grade_11: s.grade_11,
p_win: s.p_win,
outcome: l.outcome,
quarantine_reason: l.quarantine_reason,
snap_quarantine: s.quarantine_reason,
});
}
return { rows, skipped: false, joined: rows.length, snapshots: (snaps || []).length };
}
/**
* Run the harness and append the result. Never throws — a failed harness run
* must not break the scheduler tick, and the staleness alarm is what surfaces
* a run that stops happening.
*/
async function runAndRecord(deps = {}) {
try {
const { rows, skipped, joined, snapshots } = await fetchJoinedRows(deps);
if (skipped) return { skipped: true, reason: 'no supabase client' };
const report = harness.runBacktest(rows, {});
const getClient = deps.getClient || require('../utils/supabase').getSupabaseServiceClient;
const sb = getClient();
const record = {
verdict: report.verdict,
can_validate: report.can_validate,
min_sample: report.min_sample,
scored: report.counts.scored,
excluded: {
quarantine: report.counts.excluded_quarantine,
terminal: report.counts.excluded_terminal,
pending: report.counts.excluded_pending,
push: report.counts.excluded_push,
},
report,
harness_version: HARNESS_VERSION,
};
const { error } = await sb.from('harness_results').insert([record]);
return {
ok: !error,
error: error ? error.message : null,
verdict: report.verdict,
scored: report.counts.scored,
joined,
snapshots,
};
} catch (e) {
return { ok: false, error: e && e.message ? e.message : String(e) };
}
}
/** Newest run timestamp, for the staleness alarm. */
async function lastRunAt(deps = {}) {
try {
const getClient = deps.getClient || require('../utils/supabase').getSupabaseServiceClient;
const sb = getClient();
if (!sb) return null;
const { data } = await sb.from('harness_results')
.select('ran_at').order('ran_at', { ascending: false }).limit(1);
return data && data[0] ? data[0].ran_at : null;
} catch { return null; }
}
module.exports = { runAndRecord, fetchJoinedRows, lastRunAt, HARNESS_VERSION };
+22
View File
@@ -211,6 +211,28 @@ async function runIntradayRefresh(sport, opts = {}) {
await deps.cacheSet(`snapshot:${sp}:latest`, updated, SNAP_TTL);
await deps.cacheSet(`grades:${sp}`, { grades, updated_at: snap.updated_at, refreshed_at: ts, source: snap.source || 'refresh' }, SNAP_TTL);
try { await deps.ledger.captureClosing(sp, props); } catch { /* best-effort */ }
// Session 64 — CLOSING CAPTURE. This refresh already runs every ~20 min
// during slate hours, so it is the natural place to observe the last line
// before each game locks. Append-only and best-effort: a capture failure must
// never break the refresh, but a MISSED close is unrecoverable, so the rate
// is alarmed rather than silently tolerated.
try {
const closing = deps.closingCapture || require('./closingCapture');
const rows = closing.buildCaptureRows(sp, props, {});
if (rows.length) {
const res = await closing.persist(rows);
const captured = rows.filter((r) => !r.missed_reason).length;
const al = closing.captureRateAlarm({ eligible: rows.length, captured });
console.log(`[intraday] closing capture ${sp}: ${captured}/${rows.length} priced, ${res.written} stored${al.alarm ? ' — RATE ALARM' : ''}`);
if (al.alarm && deps.notify) {
await deps.notify(`Closing capture degraded for ${sp.toUpperCase()}${al.reason}`,
{ title: 'VYNDR closing capture', priority: 'high', tags: ['rotating_light'] });
}
}
} catch (e) {
console.warn(`[intraday] closing capture failed for ${sp}:`, e.message);
}
if (moveEvents.length > 0) {
await deps.pushTickerItems(moveEvents.slice(0, TICKER_MOVE_CAP), deps);
}
+21
View File
@@ -265,6 +265,27 @@ function startSnapshotScheduler(opts = {}) {
} catch (e) {
console.warn('[snapshot] cron run failed:', e.message);
}
// Session 64 — NIGHTLY BACKTEST HARNESS, on our own scheduler. Runs once
// per day at HARNESS_HOUR_UTC and appends to harness_results, so the
// calibration trend is visible as retention compounds. An
// INSUFFICIENT_HISTORY verdict is expected and correct.
try {
const harnessHour = Number(process.env.HARNESS_HOUR_UTC || 14);
if (h === harnessHour) {
const runner = opts.harnessRunner || require('./services/harnessRunner');
const res = await runner.runAndRecord();
console.log(`[harness] nightly run — verdict=${res.verdict || 'n/a'} scored=${res.scored ?? 'n/a'}${res.error ? ` ERROR: ${res.error}` : ''}`);
// A validator that stops running looks exactly like one that keeps
// passing — so staleness pages.
const stale = opsWatch.harnessStaleAlarm(await runner.lastRunAt(), now());
if (stale.alarm) {
await notify(`Backtest harness stale — ${stale.reason}. The metric gate is not running.`, {
title: 'VYNDR harness', priority: 'high', tags: ['rotating_light'],
});
}
}
} catch (e) { console.warn('[harness] nightly run failed:', e.message); }
// Session 8 — quota check after each snapshot run: odds-api >= 80% alerts
// once per day (Redis-deduped). Never throws (guarded inside checkQuotaDaily).
await opsWatch.checkQuotaDaily({ getStatus: getQuotaStatus, cacheGet, cacheSet, notify, now: () => now() });
+120
View File
@@ -0,0 +1,120 @@
/**
* Session 64 — CLOSING-LINE CAPTURE (capture only; no CLV metric).
*
* Written BEFORE the capture logic. The first property proven is the refusal:
* a prop we cannot legitimately price at the close must record `missed_close`
* and NO price. Substituting a stale or mid-day line would manufacture a CLV
* proof out of a number that was never the close — the exact failure this
* whole substrate exists to prevent.
*/
const cap = require('../../src/services/closingCapture');
const prop = (o = {}) => ({
player: 'José Ramírez', stat_type: 'hits', line: 1.5,
over_odds: -120, under_odds: 100, book: 'draftkings',
game_time: '2026-07-21T23:10:00Z',
home_team: 'CLE', away_team: 'MIN',
...o,
});
const NOW = new Date('2026-07-21T22:55:00Z'); // 15 min before lock
describe('THE REFUSAL — never fabricate a close', () => {
test('a prop with NO bound game_time records missed_close, no price', () => {
const rows = cap.buildCaptureRows('mlb', [prop({ game_time: null })], { now: NOW });
// BOTH sides are recorded as missed — a refusal is per-side, exactly like a
// capture, so the join finds an explicit answer for either side.
expect(rows).toHaveLength(2);
expect(rows[0].missed_reason).toBe('unbound_game_time');
expect(rows[0].over_odds).toBeNull();
expect(rows[0].under_odds).toBeNull();
expect(rows[0].line).toBeNull();
});
test('a DOUBLEHEADER prop is ambiguous — recorded, never attributed', () => {
const rows = cap.buildCaptureRows('mlb', [prop({ game_ambiguous: true })], { now: NOW });
expect(rows[0].missed_reason).toBe('doubleheader_ambiguous');
expect(rows[0].over_odds).toBeNull();
});
test('a game already LOCKED (past start) records missed_close, not a stale line', () => {
const late = new Date('2026-07-21T23:30:00Z'); // 20 min AFTER first pitch
const rows = cap.buildCaptureRows('mlb', [prop()], { now: late });
expect(rows[0].missed_reason).toBe('missed_window');
expect(rows[0].over_odds).toBeNull();
});
test('a prop far from lock is NOT captured at all (not yet the close)', () => {
const early = new Date('2026-07-21T12:00:00Z'); // 11h before
const rows = cap.buildCaptureRows('mlb', [prop()], { now: early });
expect(rows).toHaveLength(0);
});
test('no capture row ever carries a price without a real close', () => {
const rows = cap.buildCaptureRows('mlb', [
prop({ game_time: null }), prop({ game_ambiguous: true }),
], { now: NOW });
for (const r of rows) {
expect(r.missed_reason).toBeTruthy();
expect(r.over_odds).toBeNull();
expect(r.under_odds).toBeNull();
}
});
});
describe('CAPTURE — inside the pre-lock window', () => {
test('captures BOTH side prices raw (de-vig is computed downstream, not here)', () => {
const [r] = cap.buildCaptureRows('mlb', [prop()], { now: NOW });
expect(r.missed_reason).toBeNull();
expect(r.over_odds).toBe(-120);
expect(r.under_odds).toBe(100);
expect(r.line).toBe(1.5);
});
test('a one-sided price is NOT a usable close — recorded as incomplete', () => {
const [r] = cap.buildCaptureRows('mlb', [prop({ under_odds: null })], { now: NOW });
expect(r.missed_reason).toBe('one_sided_price');
expect(r.over_odds).toBeNull();
});
test('rows carry the natural JOIN KEY minus line (a close moves off the graded line)', () => {
const [r] = cap.buildCaptureRows('mlb', [prop()], { now: NOW });
expect(r.sport).toBe('mlb');
expect(r.player_key).toBe('jose ramirez');
expect(r.stat).toBe('hits');
expect(r.game_date).toBe('2026-07-21');
expect(r).toHaveProperty('side');
});
test('both sides of a prop are captured as separate joinable rows', () => {
const rows = cap.buildCaptureRows('mlb', [prop()], { now: NOW });
expect(rows.map((r) => r.side).sort()).toEqual(['over', 'under']);
});
test('every row is timestamped and immutable-by-construction (append-only)', () => {
const [r] = cap.buildCaptureRows('mlb', [prop()], { now: NOW });
expect(r.captured_at).toBe(NOW.toISOString());
expect(r.book).toBe('draftkings');
});
test('sharp vs book close are distinguished (different CLV questions)', () => {
const rows = cap.buildCaptureRows('mlb', [prop({ book: 'pinnacle' })], { now: NOW });
expect(rows[0].line_type).toBe('sharp');
const rows2 = cap.buildCaptureRows('mlb', [prop({ book: 'draftkings' })], { now: NOW });
expect(rows2[0].line_type).toBe('book');
});
});
describe('capture-rate alarm (silent-failure discipline)', () => {
test('alarms when most in-window props failed to capture', () => {
const r = cap.captureRateAlarm({ eligible: 100, captured: 40, missed: 60 });
expect(r.alarm).toBe(true);
});
test('quiet on a healthy pass', () => {
expect(cap.captureRateAlarm({ eligible: 100, captured: 95, missed: 5 }).alarm).toBe(false);
});
test('nothing eligible never alarms', () => {
expect(cap.captureRateAlarm({ eligible: 0, captured: 0, missed: 0 }).alarm).toBe(false);
});
});