diff --git a/specs/STATE.md b/specs/STATE.md index bdc6a73..af92023 100644 --- a/specs/STATE.md +++ b/specs/STATE.md @@ -1,6 +1,120 @@ # VYNDR โ€” STATE OF THE WORLD ### As of `cc8e478` (main, DEPLOYED + fingerprinted live), 2026-07-20. This file opens every future session. Update it when a train ships. +--- + +# ๐Ÿงญ CURRENT STATUS + OPEN ITEMS โ€” orientation block (2026-07-20, ~03:20 UTC) +*Written to orient a fresh session. Ground truth, not optimism. Everything below +was verified live unless explicitly marked UNVERIFIED.* + +## What shipped tonight (all deployed + fingerprinted) + +| # | Shipped | Proof | +|---|---|---| +| 1 | **Probability layer revived** โ€” it was DEAD in prod | `p_win`/`ev_pct`/`model_odds`/`value` on **32/32** live grades (was 0/8) | +| 2 | **Value engine Arc 1** โ€” de-vig, EV, value triplet, hero v2 | live: book โˆ’140 ยท fair โˆ’125 ยท model โˆ’109, `value:false` correctly | +| 3 | **Grade-range work** โ€” L20 symmetry, `refreshTeamStats` wired, consistency CV guard, `confidence_basis`, dead `mlbGrader.js` removed | suite 279/3325; D emitted on real WNBA data locally | +| 4 | **Backup durable ON-BOX** | dump read back: **643 rows in dump == 643 live**, 18 MB | +| 5 | **`model_snapshots` retention LIVE** (migration 025) | **100 rows** captured over 2 cycles, incl. **36 refusals** | +| 6 | ESPN team-stats parser fixed (`buckets is not iterable`, 0/15 โ†’ 15/15) | live refresh | + +### โš ๏ธ Two honest qualifiers on the above +- **A STILL DOES NOT EMIT IN PRODUCTION.** The 11-step grade is now *stored* + (`grade_11`) and A/D are *arithmetically* reachable + locked by tests, but the + ยฑ1.0 opponent factor is still dead because `opp_rank_stat` is underivable from + ESPN (see open item 6). Live boards remain **B/C only**. + **โ†’ The "A-RATED" marketing hold STANDS** (`AccuracyBadge` correctly falls + through to "MODEL ยท X% HIT"; `TopSignals` self-hides). Do not promote A-rated. +- **EV IS OVERCONFIDENT AND UNVALIDATED.** First captured values include + **+62.1% / +61% / +56.9%** โ€” real markets don't offer that. Cause: `p_win` + clamps at `PROB_CEIL` 0.95 off ~10 games. **Hero v2 ranks on `ev_pct`, so it + picks the MOST overconfident read.** Do not surface EV as truth until + calibration runs on the harness. + +## ๐Ÿ”ด OFF-BOX BACKUP โ€” NOT WORKING. Deferred. (real state) + +**On-box is durable and proven. Off-box has never succeeded once.** +Every dump currently lives only on the Hetzner persistent volume โ€” **a single +box-side loss takes the backups with it.** + +Fixed tonight (both real blockers, both now gone): +1. `BACKUP_SSH_KEY` was not base64-decoded โ†’ now auto-detects base64 vs raw PEM. +2. Container had `rsync` but **no `ssh` binary** โ†’ `openssh-client` added to the + Dockerfile. (`Failed to exec ssh` is gone.) + +**Remaining blocker โ€” SSH auth is rejected at the Storage Box.** rsync exits 255. +Verified from the dev box with the private key directly: +``` +debug1: Offering public key: ... SHA256:ZO3j7en9J7wN7EKcbYHfpyNcxTPw266hJxECUWXoMTo +debug1: Authentications that can continue: <-- EMPTY +Permission denied (). +``` +**The server offers NO auth methods at all.** That is not a wrong-key signature โ€” +it is the account refusing all authentication. Port 23 answers with an OpenSSH +banner regardless, which is why SSH can look enabled when it isn't. + +**UNVERIFIED โ€” I did NOT run a Chrome/UI diagnostic and have no data on:** the +Hetzner "SSH support" toggle state, an "External Reachability" setting, whether +the key was added at project level vs the box itself, or any Hetzner incident. +Those are the untested hypotheses, listed in likelihood order: +1. SSH support disabled on the Storage Box (separate checkbox from adding a key). +2. Key added to the wrong scope (project/sub-account vs this box). +3. External reachability disabled for the box. +4. Hetzner-side outage โ€” no evidence either way; not checked. + +**To close:** confirm the panel shows fingerprint +`SHA256:ZO3j7en9J7wN7EKcbYHfpyNcxTPw266hJxECUWXoMTo`, enable SSH, then +`POST /api/internal/backup/run` and look for `off-box push OK`. **The full +scratch-Postgres restore proof is still OWED and is blocked on this** (needs the +dump off the container; docker is available locally to do it immediately after). + +## Open items โ€” with status + +| Item | Status | Note | +|---|---|---| +| **Settlement: 0 pushes / 470 settled** | ๐Ÿ”ด OPEN, unstarted | Implausible โ€” hits/TB land on the number regularly. Exact-number push almost certainly mis-scored as hit or miss. Corrupts every accuracy/ROI number. | +| **~28 props/day never settle** | ๐Ÿ”ด OPEN, unstarted | Jul 17 MLB 86 graded/57 settled; Jul 18 103/75. Cause undiagnosed. | +| **Model-version contamination** | ๐ŸŸ  PERMANENT, mitigate | `ledger_entries` mixes pre/post-2026-07-19-fix grades with no marker; eras cannot be separated retroactively. **Any backtest/accuracy claim off existing ledger history MUST treat the fix boundary as a hard cutoff.** `model_snapshots` stamps `model_version`+`code_sha` so it can't recur. | +| **A-grade unreachable in prod** | ๐Ÿ”ด OPEN | `opp_rank_stat` null; ESPN team endpoint has no defensive metric at all. Marketing hold stands. | +| **EV overconfident** | ๐ŸŸ  OPEN | Needs calibration before it drives any surface. Hero v2 already ranks on it. | +| **`edge_pct` broken scale (U-deg pt 2)** | ๐Ÿ”ด OPEN | 51.5% of ledger rows exceed the sane cap; worst 620. 13 frontend surfaces render it; **nothing renders `ev_pct`**; it's the free-tier hook; it's written to the append-only `edge` column every cron. | +| **CLV broken (C4)** | ๐Ÿ”ด OPEN | `closing_line == locked_line` on ~95% of rows. BEAT CLOSE suppressed. **CLV ledger stays PRIVATE until backtest-proven.** | +| **Consistency CV floor** | ๐ŸŸ  STOPGAP | `CONSISTENCY_MIN_MEAN=4` leaves a ยฑ1.0 dead for MLB low-count stats. Real fix = index-of-dispersion classifier; needs a backtest first. | + +### ๐Ÿ”‘ THREE KEYS TO ROTATE (all currently valid) +1. **Storage Box password** โ€” Kev plans a fresh reset; do it before/with the SSH fix. +2. **`VYNDR_INTERNAL_KEY`** โ€” was pasted in plaintext into a chat transcript + (2026-07-19) and is in the local `.env`. It gates snapshot triggers, backup + runs, settle passes and the newsletter send. **Rotate: Coolify env + local + `.env` together.** +3. **GitHub PAT in the `origin` remote** โ€” plaintext in `.git/config`, leaked + since ~Session 31. Deploys run off `gitea` so it's inert, but it is a live + credential. Revoke on GitHub; the remote can simply be deleted. + +## Next queued work (in order) + +1. **Backtest harness** โ€” reads `model_snapshots`. **Needs ~2 weeks of history**; + as of now it holds ONE night. Its honest first output is *"cannot validate + yet"* and that is the expected, correct result. Build the instrument now, let + data accumulate. +2. **Settlement-correctness audit** โ€” the two bugs above. Highest-value + correctness work: everything downstream trusts hit/miss. +3. **A-grade / opponent-strength sourcing** โ€” MLB is **solved and verified**: + `statsapi` `teams/stats?group=pitching` returns all 30 teams free with + `era/whip/avg/slg/ops/HR9`, and it's *stat-specific*, which beats what we were + fetching. NBA/WNBA need `stats.nba.com` or derived allowed-points. Implement + behind the **source-adapter pattern** (health check that PAGES on empty โ€” + tonight's ESPN crash was invisible for exactly that reason). +4. **Metrics engine** โ€” archetype-baselined proprietary metrics + (`specs/model-train.md` ยง2C). **Gated on the harness**: every metric family + ships only if it backtests better than without it. + +**Governing docs:** `specs/model-train.md` (value engine + phases + north star) ยท +`specs/snapshot-retention.md` (Phase 2 report + schema) ยท +`specs/audit-data/grade-collapse.md` ยท `specs/audit-data/gate-simulation.md`. + +--- + ## ๐ŸŸข RETENTION LIVE โ€” model_snapshots capturing (2026-07-20, migration 025 applied) Phase 2 priority zero. **History now compounds from tonight.** Spec: `specs/snapshot-retention.md`.