Item 2 — nightly pg_dump backup script + runbook (Supabase free tier has none)
scripts/backup-db.sh: nightly full-DB pg_dump via the direct connection string, 14-day local rotation, weekly off-box rsync copy, ntfy alert on any failure + an undersized-dump guard (an empty dump is a silent failure). docs/BACKUP- RUNBOOK.md: the ONE env var Kev must set (SUPABASE_DB_URL — the direct db.<ref>.supabase.co:5432 URI, not the pooler), the cron line, the off-box target (Hetzner Storage Box via rsync, simplest for a Hetzner box), and the restore FINGERPRINT procedure (pg_restore into a scratch DB + count ledger_entries — proves it's a real, restorable backup). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,61 @@
|
||||
# VYNDR Backup Runbook (security follow-up item 2)
|
||||
|
||||
Supabase free tier has **zero** backups (no scheduled, no PITR). `scripts/backup-db.sh`
|
||||
is the safety net: a nightly full-database `pg_dump`, 14 days kept locally, a
|
||||
weekly copy pushed off-box, ntfy alert on any failure.
|
||||
|
||||
## What Kev needs to set (one env var)
|
||||
|
||||
**`SUPABASE_DB_URL`** — the Supabase **direct** connection string (session mode).
|
||||
Supabase → Project → Settings → Database → **Connection string → URI**, the
|
||||
`db.<ref>.supabase.co:5432` one (NOT the `:6543` transaction pooler — `pg_dump`
|
||||
needs a real session). Paste it in Coolify as an env var; never commit it.
|
||||
|
||||
Optional: `BACKUP_REMOTE` (off-box rsync target for the weekly copy — see below).
|
||||
|
||||
## Install the cron (on the Hetzner box)
|
||||
|
||||
```bash
|
||||
# 1. Ensure the client is present
|
||||
sudo apt-get install -y postgresql-client rsync
|
||||
|
||||
# 2. Nightly at 03:10 UTC. Pass the env the script needs (or source an env file).
|
||||
sudo crontab -e
|
||||
# add:
|
||||
10 3 * * * SUPABASE_DB_URL='postgresql://...' BACKUP_REMOTE='u123456@u123456.your-storagebox.de:vyndr-backups/' /path/to/vyndr/scripts/backup-db.sh >> /var/log/vyndr-backup.log 2>&1
|
||||
```
|
||||
|
||||
(If the cron runs inside the Coolify container instead, the env vars are already
|
||||
present — just schedule `scripts/backup-db.sh`.)
|
||||
|
||||
## Off-box target (simplest reliable pick)
|
||||
|
||||
**Hetzner Storage Box** over `rsync`/SSH — you're already on Hetzner, it's ~€3/mo
|
||||
for 1TB, and needs no extra tooling. Create one, add the box's SSH key to it, set
|
||||
`BACKUP_REMOTE=u<id>@u<id>.your-storagebox.de:vyndr-backups/`. The script pushes
|
||||
the latest dump every Sunday. (Alternative: Backblaze B2 via `rclone` if you'd
|
||||
rather keep it off Hetzner entirely — swap the `rsync` line for `rclone copy`.)
|
||||
|
||||
## Restore / FINGERPRINT (proves it's a real backup, not just a file)
|
||||
|
||||
A dump only counts once a restore of it succeeds. Load one into a scratch DB:
|
||||
|
||||
```bash
|
||||
# spin a throwaway local postgres, restore the newest dump, count a known table
|
||||
docker run -d --name vyndr-restore-test -e POSTGRES_PASSWORD=x -p 55432:5432 postgres:15
|
||||
sleep 5
|
||||
newest=$(ls -t /var/backups/vyndr/vyndr-*.dump | head -1)
|
||||
pg_restore --no-owner --no-privileges -d "postgresql://postgres:x@localhost:55432/postgres" "$newest"
|
||||
psql "postgresql://postgres:x@localhost:55432/postgres" -c "select count(*) from public.ledger_entries;"
|
||||
docker rm -f vyndr-restore-test
|
||||
```
|
||||
|
||||
A non-zero `ledger_entries` count from the restored dump = the backup is real and
|
||||
restorable. Record the date + row count as the fingerprint.
|
||||
|
||||
## Alerting
|
||||
|
||||
Any hard failure (missing env, `pg_dump` error, empty/undersized dump) pages
|
||||
`ntfy` topic `vyndr-backups-kev2026` at urgent priority. The weekly off-box push
|
||||
failing (or `BACKUP_REMOTE` unset) pages at high priority but does not fail the
|
||||
run — the local dump still succeeded. Subscribe the phone to that topic.
|
||||
@@ -0,0 +1,73 @@
|
||||
#!/usr/bin/env bash
|
||||
#
|
||||
# VYNDR nightly database backup (security follow-up item 2).
|
||||
#
|
||||
# Supabase free tier has ZERO backups (no scheduled, no PITR) — the ledger and
|
||||
# everything else have no safety net. This dumps the WHOLE database nightly via
|
||||
# the direct connection string, keeps 14 days locally, pushes a weekly copy
|
||||
# off-box, and pages ntfy on ANY failure. Runs on the Hetzner box via cron.
|
||||
#
|
||||
# REQUIRED env (set on the box / in the container that runs the cron):
|
||||
# SUPABASE_DB_URL the Supabase DIRECT connection string (session mode, the
|
||||
# db.<ref>.supabase.co:5432 URL — NOT the :6543 pooler;
|
||||
# pg_dump needs a real session). Kev pastes this in Coolify.
|
||||
# OPTIONAL env:
|
||||
# BACKUP_DIR local dump dir (default /var/backups/vyndr)
|
||||
# BACKUP_KEEP_DAYS local retention (default 14)
|
||||
# BACKUP_REMOTE off-box rsync target for the weekly copy, e.g.
|
||||
# u123456@u123456.your-storagebox.de:vyndr-backups/
|
||||
# (empty = skip the off-box push; a WARN is paged)
|
||||
# NTFY_URL (default https://ntfy.sh)
|
||||
# NTFY_TOPIC (default vyndr-backups-kev2026)
|
||||
#
|
||||
set -Eeuo pipefail
|
||||
|
||||
BACKUP_DIR="${BACKUP_DIR:-/var/backups/vyndr}"
|
||||
KEEP_DAYS="${BACKUP_KEEP_DAYS:-14}"
|
||||
NTFY_URL="${NTFY_URL:-https://ntfy.sh}"
|
||||
NTFY_TOPIC="${NTFY_TOPIC:-vyndr-backups-kev2026}"
|
||||
STAMP="$(date -u +%Y%m%d-%H%M%S)"
|
||||
DUMP="${BACKUP_DIR}/vyndr-${STAMP}.dump"
|
||||
MIN_BYTES="${BACKUP_MIN_BYTES:-50000}" # a real dump of this DB is far bigger; guards an empty/failed dump
|
||||
|
||||
notify() { # notify <title> <priority> <message>
|
||||
curl -fsS --max-time 15 \
|
||||
-H "Title: ${1}" -H "Priority: ${2}" -H "Tags: floppy_disk" \
|
||||
-d "${3}" "${NTFY_URL}/${NTFY_TOPIC}" >/dev/null 2>&1 || true
|
||||
}
|
||||
|
||||
fail() { notify "VYNDR backup FAILED" "urgent" "${1}"; echo "ERROR: ${1}" >&2; exit 1; }
|
||||
trap 'fail "backup script errored near line ${LINENO}"' ERR
|
||||
|
||||
[ -n "${SUPABASE_DB_URL:-}" ] || fail "SUPABASE_DB_URL is not set — cannot back up"
|
||||
command -v pg_dump >/dev/null 2>&1 || fail "pg_dump not installed (apt-get install postgresql-client)"
|
||||
mkdir -p "${BACKUP_DIR}"
|
||||
|
||||
# 1. Dump the whole DB in custom format (-Fc: compressed, restorable with pg_restore).
|
||||
pg_dump "${SUPABASE_DB_URL}" -Fc --no-owner --no-privileges -f "${DUMP}" \
|
||||
|| fail "pg_dump failed"
|
||||
|
||||
# 2. Sanity: a real dump is not tiny. An empty/near-empty file is a silent failure.
|
||||
SIZE="$(stat -c%s "${DUMP}" 2>/dev/null || echo 0)"
|
||||
[ "${SIZE}" -ge "${MIN_BYTES}" ] || fail "dump is only ${SIZE} bytes (< ${MIN_BYTES}) — treating as a failed backup"
|
||||
|
||||
# 3. Rotate: drop local dumps older than KEEP_DAYS.
|
||||
find "${BACKUP_DIR}" -name 'vyndr-*.dump' -type f -mtime "+${KEEP_DAYS}" -delete || true
|
||||
|
||||
# 4. Weekly off-box copy (Sundays). A single failure of the off-box push is a
|
||||
# WARN, not a hard failure — the local dump still succeeded.
|
||||
if [ "$(date -u +%u)" = "7" ]; then
|
||||
if [ -n "${BACKUP_REMOTE:-}" ]; then
|
||||
if rsync -az --timeout=120 "${DUMP}" "${BACKUP_REMOTE}"; then
|
||||
notify "VYNDR backup OK (+off-box)" "default" "Nightly dump ${STAMP} (${SIZE} bytes) + weekly off-box copy pushed."
|
||||
else
|
||||
notify "VYNDR off-box push FAILED" "high" "Local dump ${STAMP} is fine (${SIZE} bytes) but the weekly off-box rsync failed."
|
||||
fi
|
||||
else
|
||||
notify "VYNDR off-box push SKIPPED" "high" "Local dump ${STAMP} OK but BACKUP_REMOTE is unset — no off-box copy this week."
|
||||
fi
|
||||
else
|
||||
echo "backup ok: ${DUMP} (${SIZE} bytes)"
|
||||
fi
|
||||
|
||||
exit 0
|
||||
Reference in New Issue
Block a user