The closing_prob funnel collapsed 100k priced captures -> 59 usable. Root cause
(VERIFIED against prod, join key is PERFECT with 0 mismatches):
- attachClosingProb read closing_captures with .limit(50000) and NO ORDER BY on
a 730k-row table that is 86% refusal rows -> saw ~7% for MLB, missed most
priced closes and declared 200+ rows closeless that HAD a capture.
- market_unavailable_reason was write-once/terminal, so a row wrongly declared
(truncated read / premature declaration before the capture was visible) could
never recover even once its genuine capture existed. 298 rows (204 MLB + 94
WNBA) were stuck this way.
Fix (CLV computation only — no grade/locked_odds/outcome touched):
- Read ONLY priced captures (missed_reason IS NULL, both odds NOT NULL), scoped
to the candidate rows' game_dates -> small AND complete, no arbitrary truncation.
- Drop the market_unavailable exclusion from candidates; make it a re-checkable
absence: a genuine close now UPGRADES the row (writes closing_prob, clears the
verdict). closing_prob stays write-once (first true close wins). No capture +
past game -> still declared absent (honest). No churn on already-absent rows.
- New internal trigger POST /api/internal/ledger/attach-closing[/:sport] for
backfill + verification (scheduler already runs attach per tick).
Recovers ~312 usable closes (59 -> ~371), MLB included. Capture itself was
healthy all along (94.9% MLB / 95.8% WNBA per-prop coverage). Full suite 3835
green (17/17 instrument tests incl. 2 new recovery cases), web build exit 0.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VsztNChZ7vEvSR61AuMhD1
Caught by inducing on real rows. The first attach ran and marked 642 rows
market-unavailable while attaching ZERO closes — because it selected a
`fair_prob` column from closing_captures, which has none. That table stores
over_odds and under_odds deliberately (Session 64) so the de-vig can run later
against the same engine the grade-time fair price uses; asking it for a
probability returns nothing and makes every row look closeless.
The de-vig now runs here, via devigTwoWay, which is what makes lock and close
comparable at all. A one-sided capture yields no fair probability and is
correctly not a close.
Repair checked rather than assumed: the 642 markings turn out to be CORRECT —
every one is a game from before closing capture existed on 2026-07-20, so those
rows genuinely have no close and the absence is true. Zero capture-era rows were
wrongly marked. The bug would have mis-marked every future row, which is what
the fix prevents.
Two tests added: the de-vig path with real prices, and a source assertion that
the query never again asks closing_captures for a column it does not have.
Tests 3616 passed / 294 suites.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VCNgGSt5qvcLxaeQqa7Zpj
Step 0 found we have been flying without one. p_win lives only in
model_snapshots, which has 1,000 rows and ZERO settled outcomes; the closing
line lives only in closing_captures, which carries no link to a result; and
ledger_entries, the row that actually settles, carries no probability at all.
So "is the projection calibrated" and "does it beat the market" have never been
answerable — the entire measurable universe was 35 rows recovered by a lossy
in-memory join.
PHASE 0 — closing coverage verified BEFORE reuse, because an instrument built
on a partial close measures a biased subset. closing_captures holds 70,254 rows
of which 13,364 are usable, and the 56,890 refusals are candidates we never
graded plus one-sided prices — not refusals of our props. Coverage on graded
props since capture started is 83/83, 100%. Safe to reuse, with the honest
caveat that capture only began 2026-07-20.
THE FOUR-TUPLE NOW LANDS ON ONE ROW. ledger_entries gains p_win, fair_prob_lock,
archetype_vector and projection_locked_at at LOCK time, and closing_prob plus
closing_captured_at from the append-only capture store. The join is the whole
point: calibration is p_win against outcome, market-comparison is p_win against
the close, and both become plain SQL on one record instead of a join that
silently drops 90% of the rows.
p_win and the archetype vector are IMMUTABLE — written once at lock via the
existing ignoreDuplicates upsert, never re-derived at settle. A re-derivation
would measure a projection we never made.
The archetype is stored as the VECTOR, not the label. "Did archetype-awareness
help?" can only be answered against the axes that were live at grade time, and
a single text column cannot express a blend. A grade with no archetype stores
null rather than an empty object.
HONEST-ABSENT BOTH WAYS. A past game with no usable capture is marked
market_unavailable_reason and never given an imputed line; calibration still
scores on those rows, only market-comparison is absent. And a game that has not
started yet is NOT declared closeless — a close can still arrive, and premature
absence is as dishonest as imputation in the other direction.
One bug caught before it shipped: the scheduler hook iterated a SPORTS
identifier that does not exist in that scope. Inside its try/catch it would have
thrown ReferenceError every tick and silently never run — the instrument would
have looked wired and captured nothing. Now iterates cadence.ALL_SPORTS.
The baseline accrues FORWARD. Historical p_win and closes are gone, discarded
before this existed. Calibration and market-comparison stay honest-absent until
volume accrues.
Tests 3614 passed / 294 suites, web build exit 0. Migration 033 applied.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VCNgGSt5qvcLxaeQqa7Zpj
builtbykev