Retention: model_snapshots live + base64 SSH key support
RETENTION (Phase 2, priority zero). History starts compounding tonight. migration 025 model_snapshots — APPLIED to prod. Append-only, one row per graded prop PER SIDE PER CYCLE, with a unique index on (snapshot_id, player_key, stat, line, side) so a retried cycle cannot duplicate. RLS on, service-role writes only. What it captures that the ledger never did: - features jsonb — the model's INPUTS. Without these a backtest can only grade our own homework; with them any future model can be replayed against the exact conditions this one faced. - REFUSALS (refused + refusal_reason). The ledger drops them, so a gate refusing props that would have WON is invisible — unmeasurable lost edge. Captured via a new onGraded hook in gradeSlateService that fires with BOTH sides before any filtering. - grade_11, the pre-collapse grade. The 4-letter map throws away the entire live C-/C/C+/B- range. - model_version + code_sha on every row. ledger_entries mixes pre/post-fix grades with no marker and cannot be separated retroactively. - p_win / ev_pct / fair_odds / takeable / value — none of which any permanent store held. Wiring: analyzeViaEngine1 attaches _features/_grade_11 (underscore = internal); gradeSlateService fires onGraded then STRIPS them so they never reach a cache or API payload; snapshotService builds rows and persists best-effort. Retention reuses the LEDGER's dateET/gameIdFor helpers so rows share the ledger's natural key exactly — otherwise the settle pass could never join outcomes onto them. Rows are written BEFORE the empty- slate early return: a slate that refused everything is exactly the case worth recording. CONTRACT HELD: retention is injectable and every path is caught. persist() returns errors, never throws; a missing Supabase client is SKIPPED, not an error. A retention failure can never break a snapshot. BACKUP: backup-db.sh now accepts BACKUP_SSH_KEY as base64 (recommended — survives env-var newline mangling, which is how injected SSH keys usually break silently) OR raw PEM, detected by decoding and looking for the PEM header. Verified both forms detect correctly against a real generated key. Suite 279/3325 green, build exit 0. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SmNjJAwEnqHPtXbvSZR8kA
This commit is contained in:
@@ -231,6 +231,8 @@ async function runSnapshot(sport, opts = {}) {
|
||||
// Session 63 — the opponent-rank feed. Injectable so tests never hit ESPN;
|
||||
// under NODE_ENV=test it defaults to a no-op (the opsNotify precedent) so a
|
||||
// suite that doesn't know about this dep can never make a live ESPN call.
|
||||
// Session 64 — model-snapshot retention. Injectable; null disables it.
|
||||
retention: opts.retention !== undefined ? opts.retention : require('./retentionService'),
|
||||
refreshTeamStats: opts.refreshTeamStats
|
||||
|| (process.env.NODE_ENV === 'test'
|
||||
? async () => null
|
||||
@@ -278,6 +280,28 @@ async function runSnapshot(sport, opts = {}) {
|
||||
console.warn(`[snapshot] team stats refresh failed for ${sp} (grading continues):`, e.message);
|
||||
}
|
||||
|
||||
// Session 64 — RETENTION (Phase 2, priority zero). Collect one row per graded
|
||||
// prop per SIDE — graded AND refused — with the feature vector that produced
|
||||
// it, so history compounds from tonight and a future model can be replayed
|
||||
// against the exact conditions this one faced. Refusals are included on
|
||||
// purpose: the ledger drops them, so a gate refusing props that would have
|
||||
// won is otherwise invisible.
|
||||
const retention = deps.retention;
|
||||
// Reuse the LEDGER's date + game-id helpers so retention rows share the
|
||||
// ledger's natural key exactly — otherwise the settle pass could never join
|
||||
// outcomes onto them.
|
||||
const ledgerInternals = (deps.ledger && deps.ledger.__internals) || require('./ledgerService').__internals;
|
||||
const retentionGameDate = ledgerInternals.dateET(ts) || ledgerInternals.dateET(new Date().toISOString());
|
||||
const retentionCtx = {
|
||||
snapshotId: retention ? retention.newSnapshotId() : null,
|
||||
capturedAt: ts,
|
||||
cycleHourUtc: new Date(ts).getUTCHours(),
|
||||
sport: sp,
|
||||
gameDate: retentionGameDate,
|
||||
gameIdFor: (base) => ledgerInternals.gameIdFor(sp, base, retentionGameDate),
|
||||
};
|
||||
const collector = retention ? retention.createCollector(retentionCtx) : null;
|
||||
|
||||
// Grade the slate via the existing service; capture the envelope instead of
|
||||
// letting it write (we re-write an ENRICHED version below).
|
||||
let envelope = null;
|
||||
@@ -285,7 +309,19 @@ async function runSnapshot(sport, opts = {}) {
|
||||
source: (odds && odds.provider) || 'odds-api',
|
||||
now: deps.now,
|
||||
cacheSet: async (_k, v) => { envelope = v; },
|
||||
onGraded: collector ? collector.onGraded : undefined,
|
||||
});
|
||||
|
||||
// Persist retention BEFORE the early return on an empty slate — a slate that
|
||||
// graded nothing but refused everything is exactly the case worth recording.
|
||||
if (retention && collector && collector.rows.length) {
|
||||
try {
|
||||
const r = await retention.persist(collector.rows);
|
||||
console.log(`[snapshot] retention ${sp}: ${r.written}/${r.attempted} rows${r.skipped ? ' (skipped — no supabase env)' : ''}${r.error ? ` ERROR: ${r.error}` : ''}`);
|
||||
} catch (e) {
|
||||
console.warn(`[snapshot] retention write failed for ${sp} (snapshot continues):`, e.message);
|
||||
}
|
||||
}
|
||||
const rawGraded = (envelope && Array.isArray(envelope.grades)) ? envelope.grades : [];
|
||||
if (rawGraded.length === 0) return { sport: sp, status: 'skipped', reason: 'no grades', gradeCount: 0 };
|
||||
|
||||
|
||||
Reference in New Issue
Block a user