Drop lock_lines and retire its writer (dead table, dead writer)
lock_lines was built in Session 64 for a staleness audit - join lock-time
per-book lines to closing_captures and ask whether our locked line was
stale-high vs consensus. That audit was never written. ruler-comparison.sql
records why: only 43 settled rows ever joined it with >=2 two-sided books.
Measured before removal: 367,595 rows, 104 MB, zero readers in src/, scripts/
or web/src/ - the only from('lock_lines') was an upsert, every other mention a
comment. Zero dependents: no FK, no view, no trigger. The newest pg_dump held
all 367,595 rows, pg_restore-verified before the drop.
The write is off too, because a dead table that keeps refilling is only half
solved: it was accruing 36,440 rows/day, 10.3 MB/day, 23% of all database
growth, for a question nobody was asking. buildLockRows is kept and still
tested - the logic was never what was wrong, and re-arming is one flag plus
re-creating the table.
DB 510 MB -> 406 MB: 81% of the 500 MB cap, +94 MB headroom, under it for the
first time in months.
Moat and grade untouched.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -3,6 +3,25 @@
|
||||
## Last Updated
|
||||
2026-08-12
|
||||
|
||||
## Reclaim 2 (2026-08-12) — lock_lines dropped, writer retired ✅
|
||||
4,803 tests / 374 suites, web build exit 0, app healthy. **Moat untouched
|
||||
(`ledger_entries`, `model_snapshots`); grade path untouched.**
|
||||
- **DROPPED `lock_lines`**: 367,595 rows / 104 MB, zero readers anywhere, zero
|
||||
dependents (no FK/view/trigger). Dump held all 367,595 rows, verified first.
|
||||
- **RETIRED the writer** (`LOCK_LINE_CAPTURE=1` to re-arm; also needs the table
|
||||
re-created). Was 36,440 rows/day — 23% of all DB growth.
|
||||
- **DB 510 MB → 406 MB — 81% of cap, +94 MB headroom.** Under the free-tier cap
|
||||
for the first time in months. Growth ~45 → ~35 MB/day.
|
||||
- 10 new tests; `buildLockRows` kept and still covered.
|
||||
|
||||
### STILL OPEN — the survey's conclusion stands
|
||||
B1 (`2271f46`) and B2 (`fb00102`) are pushed but **NOT deployed** — prod is on
|
||||
`f61ec6b`. Until B2 lands, `missed_window` resumes at ~66 MB/day tonight and
|
||||
eats the 94 MB headroom in ~36 hours. **Deploy is the next action.**
|
||||
Even with B2 live, the hot floor (~372 MB, +33 MB/day) crosses 500 MB in ~4-5
|
||||
days. Reclaim 2 bought ~3 days, not a solution. Pro remains the arithmetic
|
||||
answer; the Roundtable decides.
|
||||
|
||||
## Fix B2 (2026-08-12) — halt the closing_captures bleed at source ✅
|
||||
4,793 tests / 373 suites, web build exit 0. **No rows deleted. Grade untouched.
|
||||
Readers untouched. No R2 involved.**
|
||||
|
||||
@@ -2141,6 +2141,28 @@ phased plan in the Session-57 conversation / BUILD-STATE Next section).
|
||||
historical 4,236,398 rows are now STATIC — the cleanup can be scheduled
|
||||
calmly instead of raced, and it no longer regrows after a delete.
|
||||
|
||||
## Reclaim 2 — lock_lines dropped + writer retired (non-obvious)
|
||||
- **`lock_lines` IS GONE (2026-08-12).** 367,595 rows / 104 MB, **zero readers**
|
||||
in `src/`, `scripts/` or `web/src/` — the only `from('lock_lines')` was an
|
||||
`.upsert`, everything else was a comment. It was built (S64) for a staleness
|
||||
audit — join lock-time per-book lines to `closing_captures` — **that was never
|
||||
written**; `scripts/ruler-comparison.sql:17` records why (only 43 settled rows
|
||||
ever joined it with >=2 two-sided books).
|
||||
- **THE WRITE IS OFF TOO.** A dead table that keeps refilling is half-solved: it
|
||||
was accruing **36,440 rows/day = 10.3 MB/day = 23% of ALL database growth**.
|
||||
`lockLineCapture.persist()` now returns `{retired:true}` without touching a
|
||||
client. Re-arm with `LOCK_LINE_CAPTURE=1` — which ALSO requires re-creating the
|
||||
table (migration 033); the pg_dump chain holds the history.
|
||||
- **`buildLockRows` is KEPT and still tested.** The logic was never what was
|
||||
wrong — it is pure and correct, and the capture is one flag away if anyone
|
||||
writes the audit. Retiring a writer is not the same as deleting the capability.
|
||||
- **`retired` and `no supabase env` must read differently in the log** — one is a
|
||||
decision, the other is a fault. The call site distinguishes them; a test asserts it.
|
||||
- **DB: 510 MB → 406 MB (102% → 81% of cap, +94 MB headroom).** First time under
|
||||
the free-tier cap in months. Growth drops ~45 → ~35 MB/day.
|
||||
- Safety net was exact: the newest pg_dump held **367,595 lock_lines rows —
|
||||
matching the live count row-for-row**, pg_restore-verified before the drop.
|
||||
|
||||
## Active Skills
|
||||
- vyndr-voice (all user-facing output)
|
||||
- prop-analysis (grading methodology)
|
||||
|
||||
@@ -29,6 +29,23 @@
|
||||
*
|
||||
* HONEST-ABSENT: a graded prop with only one priced book persists as ONE row. Never a
|
||||
* fabricated second book. A book row with neither price is not a line and is skipped.
|
||||
*
|
||||
* ── RETIRED 2026-08-12 (Reclaim 2) — THE AUDIT WAS NEVER WRITTEN ─────────
|
||||
* This table was built in Session 64 to unblock a staleness check: join lock-time
|
||||
* per-book lines to `closing_captures` and ask whether our locked line was
|
||||
* stale-high vs consensus. That join was never written. Measured before removal:
|
||||
* 367,595 rows, 104 MB, **zero readers** anywhere in src/, scripts/ or web/src/ —
|
||||
* the only mentions were this file's own comments and one comment line in
|
||||
* `scripts/ruler-comparison.sql` explaining why the reconstruction was abandoned
|
||||
* (only 43 settled rows ever joined it with >=2 two-sided books).
|
||||
*
|
||||
* It was still accruing 36,440 rows/day — 10.3 MB/day, 23% of all database
|
||||
* growth — for a question nobody was asking. `buildLockRows` is KEPT and still
|
||||
* tested: it is pure, correct, and the moment someone writes that audit the
|
||||
* capture can be re-armed with one flag. What is switched off is the WRITE.
|
||||
*
|
||||
* Set LOCK_LINE_CAPTURE=1 to re-arm. Doing so requires re-creating the table
|
||||
* (migration 033) — it was dropped, and the pg_dump chain holds the history.
|
||||
*/
|
||||
|
||||
const { nameKey, normalizeName } = require('../utils/playerName');
|
||||
@@ -106,8 +123,19 @@ function buildLockRows(sport, props, gradedKeys, opts = {}) {
|
||||
* Persist append-only. Idempotent via the UNIQUE key (a snapshot retry never
|
||||
* double-inserts). Best-effort — a persistence failure never breaks the snapshot.
|
||||
*/
|
||||
/** Re-arm the capture (also needs the table re-created — see the header). */
|
||||
const captureArmed = () => process.env.LOCK_LINE_CAPTURE === '1';
|
||||
|
||||
async function persist(rows, deps = {}) {
|
||||
const out = { attempted: rows ? rows.length : 0, written: 0, skipped: false, error: null };
|
||||
// RETIRED: the table is dropped and nothing reads it. Returning early keeps
|
||||
// every caller's contract intact (a best-effort leaf that reports what it did)
|
||||
// without touching a table that no longer exists.
|
||||
if (!(deps.armed ?? captureArmed())) {
|
||||
out.skipped = true;
|
||||
out.retired = true;
|
||||
return out;
|
||||
}
|
||||
if (!out.attempted) return out;
|
||||
try {
|
||||
const getClient = deps.getClient || require('../utils/supabase').getSupabaseServiceClient;
|
||||
@@ -129,4 +157,4 @@ async function persist(rows, deps = {}) {
|
||||
return out;
|
||||
}
|
||||
|
||||
module.exports = { buildLockRows, persist, lockKey, SHARP_BOOKS };
|
||||
module.exports = { buildLockRows, persist, lockKey, SHARP_BOOKS, captureArmed };
|
||||
|
||||
@@ -691,7 +691,11 @@ async function runSnapshot(sport, opts = {}) {
|
||||
const gradedKeys = new Set(enriched.map((g) => `${norm(g.player || g.player_name)}|${String(g.stat_type || g.stat || '').toLowerCase()}`));
|
||||
const lockRows = deps.lockLineCapture.buildLockRows(sp, props, gradedKeys, { lockedAt: ts });
|
||||
const lr = await deps.lockLineCapture.persist(lockRows);
|
||||
console.log(`[lock-lines] ${sp}: ${lr.written}/${lr.attempted} multi-book rows persisted at lock${lr.skipped ? ' (skipped — no supabase env)' : ''}${lr.error ? ` ERROR: ${lr.error}` : ''}`);
|
||||
// `retired` and `no supabase env` are different states and must read
|
||||
// differently — a retired capture is a decision, a missing env is a fault.
|
||||
console.log(`[lock-lines] ${sp}: ${lr.retired
|
||||
? `RETIRED (table dropped 2026-08-12, zero readers; ${lr.attempted} rows not written)`
|
||||
: `${lr.written}/${lr.attempted} multi-book rows persisted at lock${lr.skipped ? ' (skipped — no supabase env)' : ''}`}${lr.error ? ` ERROR: ${lr.error}` : ''}`);
|
||||
} catch (e) {
|
||||
console.warn(`[lock-lines] ${sp} persist failed (measurement-only, snapshot continues):`, e.message);
|
||||
}
|
||||
|
||||
@@ -0,0 +1,104 @@
|
||||
'use strict';
|
||||
|
||||
/**
|
||||
* Reclaim 2 — the lock_lines writer is retired.
|
||||
*
|
||||
* The table was built for a staleness audit that was never written: 367,595
|
||||
* rows, 104 MB, zero readers in src/, scripts/ or web/src/, still accruing
|
||||
* 36,440 rows/day (23% of all database growth) for a question nobody asked.
|
||||
*
|
||||
* A dead table that keeps refilling is only half-solved, so the WRITE is off
|
||||
* too. `buildLockRows` is deliberately kept and still tested: it is pure and
|
||||
* correct, and re-arming is one flag plus re-creating the table.
|
||||
*/
|
||||
|
||||
const llc = require('../../src/services/lockLineCapture');
|
||||
|
||||
const PROPS = [
|
||||
{ player: 'Aaron Judge', stat_type: 'hits', book: 'draftkings', line: 0.5, over_odds: -120, under_odds: 100, game_time: '2026-08-12T23:05:00Z' },
|
||||
{ player: 'Aaron Judge', stat_type: 'hits', book: 'pinnacle', line: 0.5, over_odds: -115, under_odds: -105, game_time: '2026-08-12T23:05:00Z' },
|
||||
];
|
||||
const KEYS = new Set([llc.lockKey('Aaron Judge', 'hits')]);
|
||||
|
||||
describe('the writer is retired', () => {
|
||||
it('persist() writes NOTHING and says so', async () => {
|
||||
let opened = false;
|
||||
const out = await llc.persist([{ sport: 'mlb' }], { getClient: () => { opened = true; return {}; } });
|
||||
expect(out.written).toBe(0);
|
||||
expect(out.retired).toBe(true);
|
||||
expect(out.skipped).toBe(true);
|
||||
expect(opened).toBe(false); // never even reaches for a client
|
||||
});
|
||||
|
||||
it('reports what it DID NOT write, so the drop stays visible in the log', async () => {
|
||||
const rows = llc.buildLockRows('mlb', PROPS, KEYS, { lockedAt: 'T0' });
|
||||
const out = await llc.persist(rows, {});
|
||||
expect(out.attempted).toBe(rows.length);
|
||||
expect(out.written).toBe(0);
|
||||
});
|
||||
|
||||
it('re-arms behind the flag', async () => {
|
||||
const chunks = [];
|
||||
const out = await llc.persist([{ sport: 'mlb', player_key: 'a' }], {
|
||||
armed: true,
|
||||
getClient: () => ({ from: () => ({ upsert: async (c) => { chunks.push(...c); return { error: null }; } }) }),
|
||||
});
|
||||
expect(out.written).toBe(1);
|
||||
expect(out.retired).toBeUndefined();
|
||||
expect(chunks).toHaveLength(1);
|
||||
});
|
||||
|
||||
it('captureArmed reads the env flag', () => {
|
||||
const prev = process.env.LOCK_LINE_CAPTURE;
|
||||
delete process.env.LOCK_LINE_CAPTURE;
|
||||
expect(llc.captureArmed()).toBe(false);
|
||||
process.env.LOCK_LINE_CAPTURE = '1';
|
||||
expect(llc.captureArmed()).toBe(true);
|
||||
if (prev === undefined) delete process.env.LOCK_LINE_CAPTURE; else process.env.LOCK_LINE_CAPTURE = prev;
|
||||
});
|
||||
});
|
||||
|
||||
describe('buildLockRows is KEPT and still correct', () => {
|
||||
it('still builds the rows it always did — the logic is not what was wrong', () => {
|
||||
const rows = llc.buildLockRows('mlb', PROPS, KEYS, { lockedAt: 'T0' });
|
||||
expect(rows).toHaveLength(2); // one per book
|
||||
expect(rows.map((r) => r.book).sort()).toEqual(['draftkings', 'pinnacle']);
|
||||
expect(rows.find((r) => r.book === 'pinnacle').line_type).toBe('sharp');
|
||||
expect(rows.every((r) => r.locked_at === 'T0')).toBe(true);
|
||||
});
|
||||
|
||||
it('still skips a book with neither price — honest-absent survives', () => {
|
||||
const rows = llc.buildLockRows('mlb', [{ ...PROPS[0], over_odds: null, under_odds: null }], KEYS, {});
|
||||
expect(rows).toHaveLength(0);
|
||||
});
|
||||
|
||||
it('still grades-only: an ungraded prop is not captured', () => {
|
||||
expect(llc.buildLockRows('mlb', PROPS, new Set(), {})).toHaveLength(0);
|
||||
});
|
||||
});
|
||||
|
||||
describe('the snapshot pass survives a retired capture', () => {
|
||||
it('the call site distinguishes RETIRED from a missing-env skip', () => {
|
||||
const src = require('fs').readFileSync(
|
||||
require('path').join(__dirname, '..', '..', 'src/services/snapshotService.js'), 'utf8');
|
||||
expect(src).toMatch(/lr\.retired/);
|
||||
expect(src).toMatch(/RETIRED \(table dropped/);
|
||||
});
|
||||
|
||||
it('persist never throws, so the snapshot cannot break on it', async () => {
|
||||
await expect(llc.persist(null, {})).resolves.toMatchObject({ written: 0 });
|
||||
await expect(llc.persist([], {})).resolves.toMatchObject({ written: 0 });
|
||||
});
|
||||
});
|
||||
|
||||
describe('nothing reads lock_lines (the precondition for the drop)', () => {
|
||||
it('no source file selects from it', () => {
|
||||
const { execSync } = require('child_process');
|
||||
const root = require('path').join(__dirname, '..', '..');
|
||||
const hits = execSync(
|
||||
"grep -rn \"from('lock_lines')\\|FROM lock_lines\" --include='*.js' --include='*.ts' --include='*.tsx' --include='*.sql' src scripts web/src 2>/dev/null | grep -v upsert || true",
|
||||
{ cwd: root, encoding: 'utf8' },
|
||||
).trim();
|
||||
expect(hits).toBe('');
|
||||
});
|
||||
});
|
||||
Reference in New Issue
Block a user