builtbykev
04a09ec1b2
(a) WHAT REPLAYABLE HISTORY EXISTS — the headline is confirmed and worse than "6 days". ledger_entries is the ONLY store of model history in the database: 640 public rows, 6 distinct game days (Jul 11/12/16/17/18/19 — 13/14/15 are missing entirely), 2 sports, 215 players, 470 settled, 465 settled WITH odds. Every other candidate is 0 rows: grade_history, line_snapshots, historical_props, closing_lines, resolution_results, accuracy_tracking, model_predictions_extended, engine1_weights, prediction_registry and ~30 more. A data warehouse was designed and never filled. Redis holds no history either (latest/previous at 24h TTL; the outcomes log carries no odds/confidence/projection). The blocking gap is not the day count, it is that NO MODEL INPUTS ARE STORED ANYWHERE. No feature vectors, so we can score the grades we emitted but cannot ask whether a different model would have done better — which is the only question a harness exists to answer, and the exact gate the metrics-engine north star requires. Also missing: p_win/ev_pct/ fair_odds (born tonight, on no column), grade_11 (only the 4-letter collapse is stored, so the entire live C-/C/C+/B- range is unrecoverable), and any model_version, so pre- and post-fix rows are already silently mixed in one table. CLV remains unusable (C4). Settlement gaps surfaced too: Jul 17 MLB 86 graded/57 settled, Jul 18 103/75, and 0 pushes across 470 settled rows — both feed the settlement-correctness audit. Verdict: we cannot meaningfully backtest yet. Retention is priority zero; every night without it is history we can never recover. (b) DESIGN PROPOSAL — model_snapshots in Postgres (not Redis, which is what lost us history twice). One append-only row per graded prop PER CYCLE, capturing market values, model output, outcome (stamped later by the settle pass), and critically a `features` JSONB — the counterfactual enabler. Carries model_version + code_sha so eras never mix, grade_11 so resolution is not thrown away, and refused/refusal_reason because refusals are training data the ledger currently discards entirely. Written from snapshotService (the existing chokepoint), best-effort so a retention failure can never break a snapshot. Volume: ~800 rows/day ~ 292k/year, ~300-600MB/yr of features, which would exceed the Supabase free tier alone — so the proposal keeps full features 90 days and scalars forever. Three open questions for Kev before building: the 90-day policy, whether to backfill the 640 existing rows as scalars-only with explicit null features, and confirming we store refusals. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SmNjJAwEnqHPtXbvSZR8kA