Retention: model_snapshots live + base64 SSH key support

RETENTION (Phase 2, priority zero). History starts compounding tonight.

migration 025 model_snapshots — APPLIED to prod. Append-only, one row per
graded prop PER SIDE PER CYCLE, with a unique index on
(snapshot_id, player_key, stat, line, side) so a retried cycle cannot
duplicate. RLS on, service-role writes only.

What it captures that the ledger never did:
- features jsonb — the model's INPUTS. Without these a backtest can only
  grade our own homework; with them any future model can be replayed
  against the exact conditions this one faced.
- REFUSALS (refused + refusal_reason). The ledger drops them, so a gate
  refusing props that would have WON is invisible — unmeasurable lost
  edge. Captured via a new onGraded hook in gradeSlateService that fires
  with BOTH sides before any filtering.
- grade_11, the pre-collapse grade. The 4-letter map throws away the
  entire live C-/C/C+/B- range.
- model_version + code_sha on every row. ledger_entries mixes pre/post-fix
  grades with no marker and cannot be separated retroactively.
- p_win / ev_pct / fair_odds / takeable / value — none of which any
  permanent store held.

Wiring: analyzeViaEngine1 attaches _features/_grade_11 (underscore =
internal); gradeSlateService fires onGraded then STRIPS them so they never
reach a cache or API payload; snapshotService builds rows and persists
best-effort. Retention reuses the LEDGER's dateET/gameIdFor helpers so
rows share the ledger's natural key exactly — otherwise the settle pass
could never join outcomes onto them. Rows are written BEFORE the empty-
slate early return: a slate that refused everything is exactly the case
worth recording.

CONTRACT HELD: retention is injectable and every path is caught. persist()
returns errors, never throws; a missing Supabase client is SKIPPED, not an
error. A retention failure can never break a snapshot.

BACKUP: backup-db.sh now accepts BACKUP_SSH_KEY as base64 (recommended —
survives env-var newline mangling, which is how injected SSH keys usually
break silently) OR raw PEM, detected by decoding and looking for the PEM
header. Verified both forms detect correctly against a real generated key.

Suite 279/3325 green, build exit 0.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SmNjJAwEnqHPtXbvSZR8kA
This commit is contained in:
Kev
2026-07-19 23:01:12 -04:00
parent 04a09ec1b2
commit d3ffa1b8c2
7 changed files with 506 additions and 5 deletions
+36
View File
@@ -231,6 +231,8 @@ async function runSnapshot(sport, opts = {}) {
// Session 63 — the opponent-rank feed. Injectable so tests never hit ESPN;
// under NODE_ENV=test it defaults to a no-op (the opsNotify precedent) so a
// suite that doesn't know about this dep can never make a live ESPN call.
// Session 64 — model-snapshot retention. Injectable; null disables it.
retention: opts.retention !== undefined ? opts.retention : require('./retentionService'),
refreshTeamStats: opts.refreshTeamStats
|| (process.env.NODE_ENV === 'test'
? async () => null
@@ -278,6 +280,28 @@ async function runSnapshot(sport, opts = {}) {
console.warn(`[snapshot] team stats refresh failed for ${sp} (grading continues):`, e.message);
}
// Session 64 — RETENTION (Phase 2, priority zero). Collect one row per graded
// prop per SIDE — graded AND refused — with the feature vector that produced
// it, so history compounds from tonight and a future model can be replayed
// against the exact conditions this one faced. Refusals are included on
// purpose: the ledger drops them, so a gate refusing props that would have
// won is otherwise invisible.
const retention = deps.retention;
// Reuse the LEDGER's date + game-id helpers so retention rows share the
// ledger's natural key exactly — otherwise the settle pass could never join
// outcomes onto them.
const ledgerInternals = (deps.ledger && deps.ledger.__internals) || require('./ledgerService').__internals;
const retentionGameDate = ledgerInternals.dateET(ts) || ledgerInternals.dateET(new Date().toISOString());
const retentionCtx = {
snapshotId: retention ? retention.newSnapshotId() : null,
capturedAt: ts,
cycleHourUtc: new Date(ts).getUTCHours(),
sport: sp,
gameDate: retentionGameDate,
gameIdFor: (base) => ledgerInternals.gameIdFor(sp, base, retentionGameDate),
};
const collector = retention ? retention.createCollector(retentionCtx) : null;
// Grade the slate via the existing service; capture the envelope instead of
// letting it write (we re-write an ENRICHED version below).
let envelope = null;
@@ -285,7 +309,19 @@ async function runSnapshot(sport, opts = {}) {
source: (odds && odds.provider) || 'odds-api',
now: deps.now,
cacheSet: async (_k, v) => { envelope = v; },
onGraded: collector ? collector.onGraded : undefined,
});
// Persist retention BEFORE the early return on an empty slate — a slate that
// graded nothing but refused everything is exactly the case worth recording.
if (retention && collector && collector.rows.length) {
try {
const r = await retention.persist(collector.rows);
console.log(`[snapshot] retention ${sp}: ${r.written}/${r.attempted} rows${r.skipped ? ' (skipped — no supabase env)' : ''}${r.error ? ` ERROR: ${r.error}` : ''}`);
} catch (e) {
console.warn(`[snapshot] retention write failed for ${sp} (snapshot continues):`, e.message);
}
}
const rawGraded = (envelope && Array.isArray(envelope.grades)) ? envelope.grades : [];
if (rawGraded.length === 0) return { sport: sp, status: 'skipped', reason: 'no grades', gradeCount: 0 };