FOUNDATION-FIRST re-order, phase 1 (tooling + safety).
BACKUP (highest-severity open item) — INSTALLED, not re-proven.
src/backupScheduler.js runs scripts/backup-db.sh nightly from inside the
API container, armed at boot in server.js. The container already has
SUPABASE_DB_URL, pg_dump and the Supabase route, so deploy == installed:
no host crontab, no Coolify click. Arming is deliberately opt-OUT (armed
whenever SUPABASE_DB_URL exists; BACKUP_CRON=0 kills it) because the S62
design was opt-in and nobody ever opted in — the DB went unbacked every
night for weeks. A failed run pages high-priority ntfy; silence is the
danger with backups.
Durability is the one part still needing a human: the container FS is
ephemeral, so a dump dies on redeploy unless BACKUP_REMOTE (off-box
rsync) or BACKUP_DIR (persistent volume) is set. The scheduler detects
that and pages a WARNING at boot rather than letting an undurable backup
read as "backed up". Runbook rewritten to lead with the code path.
MANUAL REGRADE TRIGGER — scripts/run-snapshot.js, runnable via
docker exec with no VYNDR_INTERNAL_KEY and no new HTTP surface. Runs the
SAME snapshotService.runSnapshot the cron runs (including the team-stats
refresh that powers opp_rank_stat), supports `all` and `--settle`, and
prints the grade/confidence distribution plus p_win/ev_pct presence —
which is the thing you actually want when verifying a grading change.
ACCESS BLOCKER, logged honestly in specs/model-train.md: there is no
VYNDR_INTERNAL_KEY in the local .env and SSH to the box times out from
WSL2, so I can neither curl the internal endpoints (which already exist
from S45) nor docker exec. The trigger is built and correct but only Kev
can run it until a key or SSH access exists. This is the highest-leverage
unblock for phases 2 and 3, which both need on-demand regrade+settle to
verify anything.
Also logged the standing cautions: CLV ledger stays private until
backtest-proven; "self-improving model" is unsupported marketing until
the loop closes; the engine is MLB/WNBA-calibrated and NFL/NBA/soccer
need their own calibration before the hub grades them (scaling gate).
Suite 277/3300 green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SmNjJAwEnqHPtXbvSZR8kA
SUPABASE_DB_URL is set in Coolify on the API service, and this WSL2 box can't
reach db.<ref>.supabase.co — so the backup runs INSIDE the API container, which
has the env + Supabase network. Made that real:
- Dockerfile: install postgresql-client (pg_dump/pg_restore) + rsync + bash in
the runner image.
- backup-db.sh: added an integrity fingerprint on every run — pg_restore --list
must parse the archive AND find ledger_entries, else the run FAILS + pages
(stronger than the size check; catches a corrupt/structureless dump).
- BACKUP-RUNBOOK.md: rewritten for the container-exec reality — host cron does
`docker exec <api> sh /app/scripts/backup-db.sh` (inherits env + network +
pg_dump), or a Coolify Scheduled Task. Full restore-fingerprint steps included.
MECHANISM FINGERPRINT (run locally, docker + pg16): seeded a ledger_entries
table (137 rows) → ran backup-db.sh (dump + validate: 22 archive objects,
ledger_entries present) → pg_restore into a scratch DB → 137 rows restored,
exact match. The dump/validate/restore path is proven end-to-end; it's the same
pg_dump/pg_restore that run in the container against Supabase.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
scripts/backup-db.sh: nightly full-DB pg_dump via the direct connection string,
14-day local rotation, weekly off-box rsync copy, ntfy alert on any failure +
an undersized-dump guard (an empty dump is a silent failure). docs/BACKUP-
RUNBOOK.md: the ONE env var Kev must set (SUPABASE_DB_URL — the direct
db.<ref>.supabase.co:5432 URI, not the pooler), the cron line, the off-box
target (Hetzner Storage Box via rsync, simplest for a Hetzner box), and the
restore FINGERPRINT procedure (pg_restore into a scratch DB + count
ledger_entries — proves it's a real, restorable backup).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
builtbykev