Drop lock_lines and retire its writer (dead table, dead writer)

lock_lines was built in Session 64 for a staleness audit - join lock-time
per-book lines to closing_captures and ask whether our locked line was
stale-high vs consensus. That audit was never written. ruler-comparison.sql
records why: only 43 settled rows ever joined it with >=2 two-sided books.

Measured before removal: 367,595 rows, 104 MB, zero readers in src/, scripts/
or web/src/ - the only from('lock_lines') was an upsert, every other mention a
comment. Zero dependents: no FK, no view, no trigger. The newest pg_dump held
all 367,595 rows, pg_restore-verified before the drop.

The write is off too, because a dead table that keeps refilling is only half
solved: it was accruing 36,440 rows/day, 10.3 MB/day, 23% of all database
growth, for a question nobody was asking. buildLockRows is kept and still
tested - the logic was never what was wrong, and re-arming is one flag plus
re-creating the table.

DB 510 MB -> 406 MB: 81% of the 500 MB cap, +94 MB headroom, under it for the
first time in months.

Moat and grade untouched.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Kev
2026-08-12 01:18:10 -04:00
parent fb0010222d
commit 0657b71d18
5 changed files with 179 additions and 2 deletions
+19
View File
@@ -3,6 +3,25 @@
## Last Updated ## Last Updated
2026-08-12 2026-08-12
## Reclaim 2 (2026-08-12) — lock_lines dropped, writer retired ✅
4,803 tests / 374 suites, web build exit 0, app healthy. **Moat untouched
(`ledger_entries`, `model_snapshots`); grade path untouched.**
- **DROPPED `lock_lines`**: 367,595 rows / 104 MB, zero readers anywhere, zero
dependents (no FK/view/trigger). Dump held all 367,595 rows, verified first.
- **RETIRED the writer** (`LOCK_LINE_CAPTURE=1` to re-arm; also needs the table
re-created). Was 36,440 rows/day — 23% of all DB growth.
- **DB 510 MB → 406 MB — 81% of cap, +94 MB headroom.** Under the free-tier cap
for the first time in months. Growth ~45 → ~35 MB/day.
- 10 new tests; `buildLockRows` kept and still covered.
### STILL OPEN — the survey's conclusion stands
B1 (`2271f46`) and B2 (`fb00102`) are pushed but **NOT deployed** — prod is on
`f61ec6b`. Until B2 lands, `missed_window` resumes at ~66 MB/day tonight and
eats the 94 MB headroom in ~36 hours. **Deploy is the next action.**
Even with B2 live, the hot floor (~372 MB, +33 MB/day) crosses 500 MB in ~4-5
days. Reclaim 2 bought ~3 days, not a solution. Pro remains the arithmetic
answer; the Roundtable decides.
## Fix B2 (2026-08-12) — halt the closing_captures bleed at source ✅ ## Fix B2 (2026-08-12) — halt the closing_captures bleed at source ✅
4,793 tests / 373 suites, web build exit 0. **No rows deleted. Grade untouched. 4,793 tests / 373 suites, web build exit 0. **No rows deleted. Grade untouched.
Readers untouched. No R2 involved.** Readers untouched. No R2 involved.**
+22
View File
@@ -2141,6 +2141,28 @@ phased plan in the Session-57 conversation / BUILD-STATE Next section).
historical 4,236,398 rows are now STATIC — the cleanup can be scheduled historical 4,236,398 rows are now STATIC — the cleanup can be scheduled
calmly instead of raced, and it no longer regrows after a delete. calmly instead of raced, and it no longer regrows after a delete.
## Reclaim 2 — lock_lines dropped + writer retired (non-obvious)
- **`lock_lines` IS GONE (2026-08-12).** 367,595 rows / 104 MB, **zero readers**
in `src/`, `scripts/` or `web/src/` — the only `from('lock_lines')` was an
`.upsert`, everything else was a comment. It was built (S64) for a staleness
audit — join lock-time per-book lines to `closing_captures` — **that was never
written**; `scripts/ruler-comparison.sql:17` records why (only 43 settled rows
ever joined it with >=2 two-sided books).
- **THE WRITE IS OFF TOO.** A dead table that keeps refilling is half-solved: it
was accruing **36,440 rows/day = 10.3 MB/day = 23% of ALL database growth**.
`lockLineCapture.persist()` now returns `{retired:true}` without touching a
client. Re-arm with `LOCK_LINE_CAPTURE=1` — which ALSO requires re-creating the
table (migration 033); the pg_dump chain holds the history.
- **`buildLockRows` is KEPT and still tested.** The logic was never what was
wrong — it is pure and correct, and the capture is one flag away if anyone
writes the audit. Retiring a writer is not the same as deleting the capability.
- **`retired` and `no supabase env` must read differently in the log** — one is a
decision, the other is a fault. The call site distinguishes them; a test asserts it.
- **DB: 510 MB → 406 MB (102% → 81% of cap, +94 MB headroom).** First time under
the free-tier cap in months. Growth drops ~45 → ~35 MB/day.
- Safety net was exact: the newest pg_dump held **367,595 lock_lines rows —
matching the live count row-for-row**, pg_restore-verified before the drop.
## Active Skills ## Active Skills
- vyndr-voice (all user-facing output) - vyndr-voice (all user-facing output)
- prop-analysis (grading methodology) - prop-analysis (grading methodology)
+29 -1
View File
@@ -29,6 +29,23 @@
* *
* HONEST-ABSENT: a graded prop with only one priced book persists as ONE row. Never a * HONEST-ABSENT: a graded prop with only one priced book persists as ONE row. Never a
* fabricated second book. A book row with neither price is not a line and is skipped. * fabricated second book. A book row with neither price is not a line and is skipped.
*
* ── RETIRED 2026-08-12 (Reclaim 2) — THE AUDIT WAS NEVER WRITTEN ─────────
* This table was built in Session 64 to unblock a staleness check: join lock-time
* per-book lines to `closing_captures` and ask whether our locked line was
* stale-high vs consensus. That join was never written. Measured before removal:
* 367,595 rows, 104 MB, **zero readers** anywhere in src/, scripts/ or web/src/ —
* the only mentions were this file's own comments and one comment line in
* `scripts/ruler-comparison.sql` explaining why the reconstruction was abandoned
* (only 43 settled rows ever joined it with >=2 two-sided books).
*
* It was still accruing 36,440 rows/day — 10.3 MB/day, 23% of all database
* growth — for a question nobody was asking. `buildLockRows` is KEPT and still
* tested: it is pure, correct, and the moment someone writes that audit the
* capture can be re-armed with one flag. What is switched off is the WRITE.
*
* Set LOCK_LINE_CAPTURE=1 to re-arm. Doing so requires re-creating the table
* (migration 033) — it was dropped, and the pg_dump chain holds the history.
*/ */
const { nameKey, normalizeName } = require('../utils/playerName'); const { nameKey, normalizeName } = require('../utils/playerName');
@@ -106,8 +123,19 @@ function buildLockRows(sport, props, gradedKeys, opts = {}) {
* Persist append-only. Idempotent via the UNIQUE key (a snapshot retry never * Persist append-only. Idempotent via the UNIQUE key (a snapshot retry never
* double-inserts). Best-effort — a persistence failure never breaks the snapshot. * double-inserts). Best-effort — a persistence failure never breaks the snapshot.
*/ */
/** Re-arm the capture (also needs the table re-created — see the header). */
const captureArmed = () => process.env.LOCK_LINE_CAPTURE === '1';
async function persist(rows, deps = {}) { async function persist(rows, deps = {}) {
const out = { attempted: rows ? rows.length : 0, written: 0, skipped: false, error: null }; const out = { attempted: rows ? rows.length : 0, written: 0, skipped: false, error: null };
// RETIRED: the table is dropped and nothing reads it. Returning early keeps
// every caller's contract intact (a best-effort leaf that reports what it did)
// without touching a table that no longer exists.
if (!(deps.armed ?? captureArmed())) {
out.skipped = true;
out.retired = true;
return out;
}
if (!out.attempted) return out; if (!out.attempted) return out;
try { try {
const getClient = deps.getClient || require('../utils/supabase').getSupabaseServiceClient; const getClient = deps.getClient || require('../utils/supabase').getSupabaseServiceClient;
@@ -129,4 +157,4 @@ async function persist(rows, deps = {}) {
return out; return out;
} }
module.exports = { buildLockRows, persist, lockKey, SHARP_BOOKS }; module.exports = { buildLockRows, persist, lockKey, SHARP_BOOKS, captureArmed };
+5 -1
View File
@@ -691,7 +691,11 @@ async function runSnapshot(sport, opts = {}) {
const gradedKeys = new Set(enriched.map((g) => `${norm(g.player || g.player_name)}|${String(g.stat_type || g.stat || '').toLowerCase()}`)); const gradedKeys = new Set(enriched.map((g) => `${norm(g.player || g.player_name)}|${String(g.stat_type || g.stat || '').toLowerCase()}`));
const lockRows = deps.lockLineCapture.buildLockRows(sp, props, gradedKeys, { lockedAt: ts }); const lockRows = deps.lockLineCapture.buildLockRows(sp, props, gradedKeys, { lockedAt: ts });
const lr = await deps.lockLineCapture.persist(lockRows); const lr = await deps.lockLineCapture.persist(lockRows);
console.log(`[lock-lines] ${sp}: ${lr.written}/${lr.attempted} multi-book rows persisted at lock${lr.skipped ? ' (skipped — no supabase env)' : ''}${lr.error ? ` ERROR: ${lr.error}` : ''}`); // `retired` and `no supabase env` are different states and must read
// differently — a retired capture is a decision, a missing env is a fault.
console.log(`[lock-lines] ${sp}: ${lr.retired
? `RETIRED (table dropped 2026-08-12, zero readers; ${lr.attempted} rows not written)`
: `${lr.written}/${lr.attempted} multi-book rows persisted at lock${lr.skipped ? ' (skipped — no supabase env)' : ''}`}${lr.error ? ` ERROR: ${lr.error}` : ''}`);
} catch (e) { } catch (e) {
console.warn(`[lock-lines] ${sp} persist failed (measurement-only, snapshot continues):`, e.message); console.warn(`[lock-lines] ${sp} persist failed (measurement-only, snapshot continues):`, e.message);
} }
+104
View File
@@ -0,0 +1,104 @@
'use strict';
/**
* Reclaim 2 — the lock_lines writer is retired.
*
* The table was built for a staleness audit that was never written: 367,595
* rows, 104 MB, zero readers in src/, scripts/ or web/src/, still accruing
* 36,440 rows/day (23% of all database growth) for a question nobody asked.
*
* A dead table that keeps refilling is only half-solved, so the WRITE is off
* too. `buildLockRows` is deliberately kept and still tested: it is pure and
* correct, and re-arming is one flag plus re-creating the table.
*/
const llc = require('../../src/services/lockLineCapture');
const PROPS = [
{ player: 'Aaron Judge', stat_type: 'hits', book: 'draftkings', line: 0.5, over_odds: -120, under_odds: 100, game_time: '2026-08-12T23:05:00Z' },
{ player: 'Aaron Judge', stat_type: 'hits', book: 'pinnacle', line: 0.5, over_odds: -115, under_odds: -105, game_time: '2026-08-12T23:05:00Z' },
];
const KEYS = new Set([llc.lockKey('Aaron Judge', 'hits')]);
describe('the writer is retired', () => {
it('persist() writes NOTHING and says so', async () => {
let opened = false;
const out = await llc.persist([{ sport: 'mlb' }], { getClient: () => { opened = true; return {}; } });
expect(out.written).toBe(0);
expect(out.retired).toBe(true);
expect(out.skipped).toBe(true);
expect(opened).toBe(false); // never even reaches for a client
});
it('reports what it DID NOT write, so the drop stays visible in the log', async () => {
const rows = llc.buildLockRows('mlb', PROPS, KEYS, { lockedAt: 'T0' });
const out = await llc.persist(rows, {});
expect(out.attempted).toBe(rows.length);
expect(out.written).toBe(0);
});
it('re-arms behind the flag', async () => {
const chunks = [];
const out = await llc.persist([{ sport: 'mlb', player_key: 'a' }], {
armed: true,
getClient: () => ({ from: () => ({ upsert: async (c) => { chunks.push(...c); return { error: null }; } }) }),
});
expect(out.written).toBe(1);
expect(out.retired).toBeUndefined();
expect(chunks).toHaveLength(1);
});
it('captureArmed reads the env flag', () => {
const prev = process.env.LOCK_LINE_CAPTURE;
delete process.env.LOCK_LINE_CAPTURE;
expect(llc.captureArmed()).toBe(false);
process.env.LOCK_LINE_CAPTURE = '1';
expect(llc.captureArmed()).toBe(true);
if (prev === undefined) delete process.env.LOCK_LINE_CAPTURE; else process.env.LOCK_LINE_CAPTURE = prev;
});
});
describe('buildLockRows is KEPT and still correct', () => {
it('still builds the rows it always did — the logic is not what was wrong', () => {
const rows = llc.buildLockRows('mlb', PROPS, KEYS, { lockedAt: 'T0' });
expect(rows).toHaveLength(2); // one per book
expect(rows.map((r) => r.book).sort()).toEqual(['draftkings', 'pinnacle']);
expect(rows.find((r) => r.book === 'pinnacle').line_type).toBe('sharp');
expect(rows.every((r) => r.locked_at === 'T0')).toBe(true);
});
it('still skips a book with neither price — honest-absent survives', () => {
const rows = llc.buildLockRows('mlb', [{ ...PROPS[0], over_odds: null, under_odds: null }], KEYS, {});
expect(rows).toHaveLength(0);
});
it('still grades-only: an ungraded prop is not captured', () => {
expect(llc.buildLockRows('mlb', PROPS, new Set(), {})).toHaveLength(0);
});
});
describe('the snapshot pass survives a retired capture', () => {
it('the call site distinguishes RETIRED from a missing-env skip', () => {
const src = require('fs').readFileSync(
require('path').join(__dirname, '..', '..', 'src/services/snapshotService.js'), 'utf8');
expect(src).toMatch(/lr\.retired/);
expect(src).toMatch(/RETIRED \(table dropped/);
});
it('persist never throws, so the snapshot cannot break on it', async () => {
await expect(llc.persist(null, {})).resolves.toMatchObject({ written: 0 });
await expect(llc.persist([], {})).resolves.toMatchObject({ written: 0 });
});
});
describe('nothing reads lock_lines (the precondition for the drop)', () => {
it('no source file selects from it', () => {
const { execSync } = require('child_process');
const root = require('path').join(__dirname, '..', '..');
const hits = execSync(
"grep -rn \"from('lock_lines')\\|FROM lock_lines\" --include='*.js' --include='*.ts' --include='*.tsx' --include='*.sql' src scripts web/src 2>/dev/null | grep -v upsert || true",
{ cwd: root, encoding: 'utf8' },
).trim();
expect(hits).toBe('');
});
});