CLV instrument repair: fix attachClosingProb read + recoverable market_unavailable

The closing_prob funnel collapsed 100k priced captures -> 59 usable. Root cause
(VERIFIED against prod, join key is PERFECT with 0 mismatches):
- attachClosingProb read closing_captures with .limit(50000) and NO ORDER BY on
  a 730k-row table that is 86% refusal rows -> saw ~7% for MLB, missed most
  priced closes and declared 200+ rows closeless that HAD a capture.
- market_unavailable_reason was write-once/terminal, so a row wrongly declared
  (truncated read / premature declaration before the capture was visible) could
  never recover even once its genuine capture existed. 298 rows (204 MLB + 94
  WNBA) were stuck this way.

Fix (CLV computation only — no grade/locked_odds/outcome touched):
- Read ONLY priced captures (missed_reason IS NULL, both odds NOT NULL), scoped
  to the candidate rows' game_dates -> small AND complete, no arbitrary truncation.
- Drop the market_unavailable exclusion from candidates; make it a re-checkable
  absence: a genuine close now UPGRADES the row (writes closing_prob, clears the
  verdict). closing_prob stays write-once (first true close wins). No capture +
  past game -> still declared absent (honest). No churn on already-absent rows.
- New internal trigger POST /api/internal/ledger/attach-closing[/:sport] for
  backfill + verification (scheduler already runs attach per tick).

Recovers ~312 usable closes (59 -> ~371), MLB included. Capture itself was
healthy all along (94.9% MLB / 95.8% WNBA per-prop coverage). Full suite 3835
green (17/17 instrument tests incl. 2 new recovery cases), web build exit 0.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VsztNChZ7vEvSR61AuMhD1
This commit is contained in:
Kev
2026-07-28 18:59:12 -04:00
parent afb56b144b
commit 6552281661
3 changed files with 91 additions and 12 deletions
+39 -12
View File
@@ -342,11 +342,17 @@ async function attachClosingProb(sport, opts = {}) {
const sb = opts.sb || defaultClient();
const sp = String(sport || '').toLowerCase();
// Candidates: any locked model-record row still lacking a real close. We do
// NOT exclude rows already stamped market_unavailable_reason — that verdict is
// a RE-CHECKABLE absence, not terminal. A capture we couldn't see before (the
// read was truncated) or that landed after a premature declaration must be
// able to upgrade the row. `closing_prob` itself stays write-once (the filter
// below), so the FIRST true close still wins and is never rewritten.
const { data: rows, error } = await sb.from('ledger_entries')
.select('id, player_key, stat, side, game_date')
.select('id, player_key, stat, side, game_date, market_unavailable_reason')
.is('user_id', null).eq('sport', sp)
.is('closing_prob', null).is('market_unavailable_reason', null)
.limit(opts.limit || 2000);
.is('closing_prob', null)
.limit(opts.limit || 5000);
if (error) return { updated: 0, error: error.message };
if (!rows || !rows.length) return { updated: 0, absent: 0 };
@@ -355,10 +361,20 @@ async function attachClosingProb(sport, opts = {}) {
// the grade-time fair price uses, which is what makes lock and close
// comparable at all. Asking this table for a `fair_prob` column is a bug: it
// has none, and every row then looks closeless.
//
// READ FIX (CLV instrument repair): the table is ~86% refusal rows, and the
// old `.limit(50000)` with no ORDER BY read an arbitrary slice — for MLB
// (730k rows) it saw ~7% and declared 200+ rows closeless that HAD a priced
// capture. Read only PRICED captures, scoped to the candidate rows' game
// dates, so the set is small AND complete.
const { devigTwoWay } = require('../utils/devig');
const { data: caps } = await sb.from('closing_captures')
const dates = [...new Set(rows.map((r) => r.game_date).filter(Boolean))];
let capsQuery = sb.from('closing_captures')
.select('player_key, stat, side, game_date, over_odds, under_odds, captured_at, missed_reason')
.eq('sport', sp).limit(50000);
.eq('sport', sp).is('missed_reason', null)
.not('over_odds', 'is', null).not('under_odds', 'is', null);
if (dates.length) capsQuery = capsQuery.in('game_date', dates);
const { data: caps } = await capsQuery.limit(opts.capLimit || 200000);
// Latest usable capture per identity = the TRUE close.
const best = new Map();
@@ -378,17 +394,28 @@ async function attachClosingProb(sport, opts = {}) {
// rows market-unavailable would be premature absence — as dishonest in the
// other direction as imputing one. Only a past game can be declared closeless.
const cutoff = opts.beforeDate || todayET();
let recovered = 0;
for (const r of rows) {
const hit = best.get(`${r.player_key}|${r.stat}|${r.side}|${r.game_date}`);
if (!hit && String(r.game_date) >= String(cutoff)) continue; // still capturable
const patch = hit
? { closing_prob: hit.fair_prob, closing_captured_at: hit.captured_at }
: { market_unavailable_reason: 'no_usable_close' };
const { error: e } = await sb.from('ledger_entries').update(patch).eq('id', r.id);
if (hit) {
// A real close — write it and CLEAR any prior (premature/truncation-bug)
// market-unavailable verdict. This is the recovery path: a row wrongly
// declared closeless is upgraded the moment its genuine capture is seen.
const patch = { closing_prob: hit.fair_prob, closing_captured_at: hit.captured_at, market_unavailable_reason: null };
const { error: e } = await sb.from('ledger_entries').update(patch).eq('id', r.id);
if (e) continue;
updated += 1;
if (r.market_unavailable_reason) recovered += 1;
continue;
}
// No usable close found for this row.
if (String(r.game_date) >= String(cutoff)) continue; // future/today — a close can still arrive
if (r.market_unavailable_reason) continue; // already honestly declared absent — leave it
const { error: e } = await sb.from('ledger_entries').update({ market_unavailable_reason: 'no_usable_close' }).eq('id', r.id);
if (e) continue;
if (hit) updated += 1; else absent += 1;
absent += 1;
}
return { updated, absent, candidates: rows.length };
return { updated, absent, recovered, candidates: rows.length };
}
/**