builtbykev/vyndr
1
0
Fork 0
Code Issues Pull Requests Actions 7 Packages Projects Releases Wiki Activity

STATE: retention live + settlement findings + version contamination

Browse Source 
Records model_snapshots as LIVE and verified capturing (100 rows over 2
cycles: MLB 14 graded/36 refused, WNBA 50 graded; features, grade_11,
p_win, ev_pct on 100% of graded rows).

First-ever refusal visibility: juiced_no_edge 18, rare_event_over_below_
line 13, insufficient_data 5 — the MLB gate refused 36 of 50 sides (72%),
now measurable for the first time.

Flags EV as OVERCONFIDENT and not fit to surface: first captured values
include +62.1%/+61%/+56.9%, which real markets do not offer. Cause is the
estimator clamping p_win at PROB_CEIL 0.95 off ~10 games. Hero v2 already
ranks on ev_pct, so it will pick the MOST overconfident read — calibration
must gate this before EV drives anything user-facing.

Logs the two settlement-correctness findings Kev asked to track (zero
pushes across 470 settled rows; ~28 props/day never settling) and the
ledger model-version contamination, with the rule that any backtest off
existing history must treat the 2026-07-19 fix boundary as a hard cutoff.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SmNjJAwEnqHPtXbvSZR8kA
This commit is contained in:
Kev
2026-07-19 23:14:58 -04:00
parent 14f47af74b
commit 971f641d12
1 changed files with 43 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
specs/STATE.md
+43
View File
@@ -1,6 +1,49 @@
# VYNDR — STATE OF THE WORLD
### As of `cc8e478` (main, DEPLOYED + fingerprinted live), 2026-07-20. This file opens every future session. Update it when a train ships.
## 🟢 RETENTION LIVE — model_snapshots capturing (2026-07-20, migration 025 applied)
Phase 2 priority zero. **History now compounds from tonight.** Spec:
`specs/snapshot-retention.md`.
- **Why:** the ONLY model history was `ledger_entries` (640 rows / 6 game days)
and it stores NO model inputs. All ~30 other warehouse tables are empty. We
could score our own grades but could not replay a different model — the only
question a backtest answers.
- **`model_snapshots`** (migration 025, APPLIED): append-only, ONE ROW PER GRADED
PROP PER SIDE PER CYCLE. Unique on `(snapshot_id, player_key, stat, line,
side)` so a retried cycle can't duplicate. RLS on, service-role writes only.
- **VERIFIED CAPTURING (2026-07-20 03:02 UTC, 2 triggered cycles): 100 rows.**
MLB 50 (14 graded / **36 refused**), WNBA 50 (50 graded). `features`,
`grade_11`, `p_win`, `ev_pct` populated on 100% of graded rows; 1 model_version.
- **Refusals are captured — the ledger drops them entirely.** First-ever refusal
breakdown: `juiced_no_edge` 18, `rare_event_over_below_line` 13,
`insufficient_data` 5. **The MLB gate refused 36 of 50 sides (72%)** — whether
that costs edge is now measurable for the first time.
- Contract: retention is best-effort + injectable; `persist()` returns errors and
never throws. **A retention failure can never break a snapshot.**
- 🔶 **EV LOOKS OVERCONFIDENT — do not surface it as truth yet.** First captured
EVs include +62.1%, +61%, +56.9% — implausible for real markets. Cause is the
quantile estimator clamping `p_win` at 0.95 (`PROB_CEIL`) off ~10 games.
**Hero v2 already ranks by highest `ev_pct`, so it will select the MOST
overconfident read.** Calibration (C-cal) must run on the harness before EV
drives anything user-facing.
## 🔶 SETTLEMENT-CORRECTNESS AUDIT — two concrete findings (open, scheduled)
Logged from the Phase 2 history report; do NOT fix inside a retention build.
1. **ZERO pushes across 470 settled rows.** Implausible — hits/TB land exactly on
the number regularly. The exact-number push case is almost certainly
mis-scored (silently counted as hit or miss).
2. **~28 props/day never settle** (Jul 17 MLB 86 graded/57 settled; Jul 18
103/75). Cause undiagnosed.
Both are correctness bugs in the scoring that accuracy, ROI, CLV and every public
record depend on.
## 🔴 MODEL-VERSION CONTAMINATION in ledger_entries (permanent, mitigate by cutoff)
`ledger_entries` mixes pre- and post-2026-07-19-fix grades with **no version
marker**, and the eras cannot be separated retroactively. **Any backtest or
accuracy claim computed off existing ledger history must treat the fix boundary
as a hard cutoff.** `model_snapshots` stamps `model_version` + `code_sha` on every
row so this can never recur.
## 🟢 BACKUP — DURABLE ON-BOX, VERIFIED BY READ-BACK (2026-07-20, deployed)
The highest-severity open item is closed **on-box**. Off-box is a tracked follow-up.
- **Runs as CODE, not a cron somebody must install** (`src/backupScheduler.js`,