930d526b0109c9357ccbcb8386d94ee2994e6229
6 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 builtbykev
|
930d526b01 |
Lineage productization: a live graph, an observer that doesn't trust it, and a surface that owns only what it knows
The authority review found there was nothing to switch: lineage is a
write-only graph with no living permanent writer and no product consumer.
Authority theater would have been a switch on a consumer that does not exist,
reading a store that is not being written. So: make the graph live, measure it
from outside, and expose the one category it alone owns.
WRITE-PATH FAILURE SEMANTICS, TRACED FIRST. persist(:857) -> the authoritative
cacheSet snapshot:latest(:1404) -> the lineage gate(:1426) -> ledger(:1473).
Lineage failure cannot fail base retention (earlier, separate upsert), cannot
fail publication (the product write precedes the gate), cannot fail the Ledger
(lineageIndex defaults null; the ledger has its own guard), and cannot create a
new partial row (blankLineage() first, atomicity sweep after the catch). The
isolation this tranche needed already existed; only a mode was missing.
THREE MODES, ONE EVALUATOR. `lineageWriteMode` distinguishes OFF /
CANARY_LEASED / PERSISTENT_SHADOW, and the write gate and the status surface
both read it, so they cannot disagree. The canary is CONSULTED, never
converted: its <=4h absolute expiry, its dynamic evaluation and its
fail-closed parse are untouched.
AN AMBIGUOUS CONFIGURATION FAILS CLOSED. If a sport is named by both persistent
mode and an active lease, the two instructions disagree about WHEN WRITING
STOPS — the lease says 22:45, persistent says never. The dangerous reading is
the quiet one: an operator sets a bounded lease believing writing will stop
while persistent keeps it going. We cannot know which they meant, so that sport
writes nothing until the configuration says one thing. The sport allowlist is
the canary's own, so persistent mode can never widen past it.
THE OBSERVER MAY NOT ASK THE WRITER HOW IT DID. settleLedger returned
{settled:0,pending:0} — byte-identical to a healthy "nothing to settle" — while
1,444 rows sat unprocessed, and the watchdog believed it. So `lineageCoverage`
reads durable retained state only, and THE DENOMINATOR MAY NOT CONSULT
lineage_action: eligibility is "the row was published AND a natural key is
derivable from its own identity columns", neither of which the lineage path
writes. If expectation were derived from whether lineage exists, coverage would
be 100% by construction and the metric would be decoration. Zero-expected and
zero-written are kept as different answers.
THE LEGACY BOUNDARY IS OBSERVED, NOT DECLARED. `publication_id` is stamped only
by commitPublication, so the row itself says whether lineage ran. Verified on
production: 5,353 rows carry it — 4,234 complete actions plus exactly the 1,119
historical partial rows — and zero actions exist without one. No epoch constant
is invented; a date would have been a guess about when the writer was on.
publication_id NULL -> LEGACY_UNVERIFIED. Stamped but incomplete ->
LINEAGE_UNAVAILABLE, which is the honest answer for the 1,119 and is never
quietly rewritten as legacy.
THE GRADE-SHIFT BADGE IS UNTOUCHED. revised_from_grade answers "did the letter
change"; lineage answers "which published claim superseded which". Different
questions, and a test now fails if either route learns the word lineage.
Suite 397/5,496/0 · tsc 0 · 15/15 teeth.
TWO OF MY OWN TESTS WERE VACUOUS AND A TOOTH FOUND IT. Tooth 2 came back green
because the isolation tests asserted the slate was published — true whether or
not the exception propagated — while never reaching the lineage gate at all:
the fake grader never fired `onGraded`, so the collector stayed empty and
persistedRows stayed null. Fixed by firing the hook and counting the commit.
A green teeth run means the test is missing.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CQJeAG8vcDoL5zkiaJyVb8
|
||
|
builtbykev
|
4aca33deb6 |
One human, one semantic identity — MLB participant convergence
The collision autopsy left two unrepaired defects, running in OPPOSITE
directions, and `outbound_collision_count` can only ever see one of them.
UNDER-COLLAPSE. Dedupe keys on `mlb:<personId>` when the participant is
proven and on the RAW PROVIDER SPELLING when it is not. Mickey Gasper
(681508) is on Boston's 40-man and not on its active roster, so an
active-only index could not identify him and every book's spelling of him
survived dedupe as its own proposition — retention was the first layer to
notice, far too late, and could only discard the loser.
SPLIT. The mirror image, and invisible to the collision metric because it
makes MORE identities, not fewer: Leo Jiménez (677870) is published as both
"Leo Jiménez" and "Leonardo Jimenez", so one human became two semantic
players in one game. Measured across the 15 MLB cohort slices since the
canonical-participant repair, this is a recurring class, not one case:
cam/cameron smith (5 slices), mitch/mitchell bratt, zac/zachary thornton,
leo/leonardo jimenez.
THE REPAIR READS MLB'S OWN RECORD. `hydrate=person` on the roster call the
pipeline already makes returns firstName / useName / useLastName, so the
legitimate name forms for a human come from the league rather than from an
alias table. An alias table is a list of the mistakes we happened to notice.
`nickName` is DELIBERATELY EXCLUDED: over 821 people it produced 14
ambiguous keys, because MLB's nickname field carries bare surnames and
shared clubhouse names — `nameKey('Smitty Smith')` is one string for both
Burch Smith and Will Smith. The four forms kept produce ZERO ambiguity.
Canonical participant reach widens to the 40-man; TEAM EVIDENCE still reads
the ACTIVE roster alone, so event admission and the impossible-binding
refusal are unchanged. Identity still fails closed: a name matching more
than one person in the event resolves to nobody.
CONTINUITY, MEASURED BEFORE WRITING ANY CODE. Over the real 19:00 cohort,
208 of 209 player_keys are unchanged and the one that moves is the defect —
`leonardo jimenez` converging onto `leo jimenez`, a key that already exists.
No new lineage family. The natural key contains game_date, so chains never
span dates and a forward change cannot fork a closed one.
DETERMINISTIC REPRESENTATIVE. Which book's payload survives was decided by
position. It is now decided by the existing MODEL_BOOKS declaration order —
reused, not authored; inventing a sportsbook ranking to settle a tiebreak
would be a market judgement smuggled in as a bug fix — with book name and a
content tiebreak. Stable under every input permutation.
TWO GUARDS, BOTH DIRECTIONS. split (one person, many identities) and merge
(one identity, many people). A merge is refused at the same single admission
seam event identity already uses; a split is counted and alerted but does not
cut the board, because it duplicates an identity rather than asserting a
falsehood.
RETENTION REMAINS AN INDEPENDENT CHECK. The old assertion grepped the source
for `player_key: nameKey(player)`. That expression stood in for a PROPERTY,
and a grep verifies a spelling. Replaced with the property itself, asserted
in both modes: when the producer emits two rows for one human, retention
still files them under one identity and still reports the collision.
Replay of the real cohort through the repair: 3,129 offerings, 100%
participants resolved, every one of 207 participants on exactly ONE semantic
key, collision 0, split 0, merge 0.
Suite 396/5,455/0 · tsc 0 · 15/15 teeth. Tooth 12 came back green first
time and that was a coverage hole, not a safe defect: nothing asserted
retention's append-only upsert. It does now.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CQJeAG8vcDoL5zkiaJyVb8
|
||
|
builtbykev
|
cc5bdf5797 |
A canary is temporary: bound activation with a fail-closed lease
`LINEAGE_CANARY_SPORTS=mlb` was parsed once at module load and frozen, so an enabled canary stayed writable for the whole process lifetime. On 2026-08-29 it was left set and the 22:00Z, 01:00Z and 03:00Z scheduled snapshots each wrote lineage overnight with nobody watching. That history happened to be correct. Lineage is append-only evidence, so a defective canary would have written irreversible WRONG evidence exactly as quietly. Correct history was luck, not a safety property. NEW CONTRACT: LINEAGE_CANARY_SPORTS=mlb@2026-08-30T18:00:00Z Absolute UTC instant only -- no duration, no local timezone. A relative "4h" would silently restart on every redeploy, which is the defect being removed. LEGACY `mlb` NO LONGER ACTIVATES ANYTHING. It is INVALID_MISSING_EXPIRY. Leaving it working would have left the defect in place behind a nicer-looking alternative, so this is the load-bearing half of the repair. EXPIRY IS EVALUATED AT THE WRITE GATE, not at startup. `isEnabled(sport, now)` re-reads the clock on every call and `lineageCanaryEnabled` threads it through, so a lease turns itself off with no operator, no restart, no Redis and no network. A design where an expired canary keeps writing until someone restarts is the same failure in a different hat. MAX_LEASE is FOUR HOURS, and the bound is proven rather than chosen. MLB ticks are [14,19,22,1,3] UTC; exhaustively over a minute grid across UTC date boundaries, the shortest span enclosing THREE consecutive ticks is 22:00 -> 01:00 -> 03:00 = five hours. Four hours encloses at most two, with an hour of margin. (An earlier note claimed six hours admitted two. It admits three; that claim was false and the test now pins the arithmetic.) The bound is a function of the scheduler, so a test reads the hours from sportCadence and fails if a cadence change invalidates the proof. Everything fails closed: absent, empty, bare sport, comma list, two leases, duplicate @, non-leasable sport, missing timezone, numeric offset, malformed, over-long, already-expired, expiry-equals-now, and an unreadable clock. Configured-but-EXPIRED stays distinguishable from never-configured so automatic containment is auditable. No Redis, no Supabase, no counter, no network in the lease path -- an unreachable dependency must never decide whether lineage may write. Lineage algorithms, cache-date, participant and retention identity untouched. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CQJeAG8vcDoL5zkiaJyVb8 |
||
|
builtbykev
|
7efb04e280 |
Lineage family lookup: bound it to a slate, and say what an action is
TWO DEFECTS, one lookup. SCALE. The family lookup sent 100 natural keys as a PostgREST IN-list. `read_natural_key` has NO pg_stats row at all -- the table's last autoanalyze (2026-08-26) predates the column ever being populated -- so the planner used a default per-value selectivity, estimated 172,409 rows and chose a sequential scan of 344,818: 8.5s, then 57014. At 50 keys the same shape returned in ~357ms. The cliff is a statistics artifact, not a volume one, which is why the repair does not depend on the estimate improving and is not CH=50. `readNaturalKey` builds `sport|game_date|player_key|stat|side|line[|#event]`, so SPORT AND GAME_DATE ARE COMPONENTS OF THE KEY. Two rows sharing a key necessarily share both, and scoping the lookup to the (sport, game_date) pairs present in the requested keys is LOSSLESS BY CONSTRUCTION. One index-backed range per date, walked with safePaginate; cost is bounded by ONE SLATE however long the chronology gets. Measured: 5,000 keys -> 1 scope, and the plan is `Index Scan using model_snapshots_lineage_family_idx, cost 0.28..1.92`. VALIDITY. A row carrying `read_natural_key` is not history: the key is stamped on every candidate BEFORE the lookup, so a failure leaves it on a row that never became an action. Proven this was not cosmetic -- fed the raw rows the old lookup returned, the resolver produced a REVISION with a NULL read_id (an orphaned chain node) and labelled a brand-new Read LEGACY_UNVERIFIED. `isValidLineageAction` states what a completed action IS: all nine fields, in the query and again in code. ATOMICITY. A failed attempt now leaves NO lineage-specific state. `publication_id`/`published_at` are untouched -- the slate really was published, and erasing a true fact to tidy a false one is the wrong repair. Replayed the exact failed 19:00Z cohort through the real resolver, side-effect free: 119 NEW / 379 CHANGED / 621 UNCHANGED -> ORIGIN 119 / REVISION 379 / RECAPTURE 621, 0 wrong parent, 0 wrong ordinal, 0 null read_id, 0 forks -- byte-identical with all 1,119 failed partial rows present. Clean-head parity 1,024/1,024. Migration 050 is CONCURRENTLY + IF NOT EXISTS, drops nothing, rewrites nothing. Lineage stays OFF. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CQJeAG8vcDoL5zkiaJyVb8 |
||
|
builtbykev
|
8c6aef1e12 |
Event admission gate: an unresolved or contradicted game does not earn a Read
A fractured identity keeps two bad records from merging. It does not make an unknown game true. Until now a prop with an unresolved or verified-impossible event still continued into grading under that synthetic key; it no longer does. - event_binding_status contract: RESOLVED / UNRESOLVED / AMBIGUOUS / CONTRADICTED / UNSUPPORTED. CONTRADICTED and UNRESOLVED stay distinct — one means we know the association is wrong, the other that we do not know. - ONE admission gate (gradeSlateService.admitForGrading), before dedupe and before grading. Admission requires RESOLVED *and* a canonical_event_id; a legacy derived game_id can never satisfy it. Rejected props are returned, not discarded, so retention keeps them as evidence. - Roster evidence is now DATE-SCOPED. statsapi honours ?date= and it changes the answer (Joe Mack is on the 2026-08-26 Marlins roster, absent on 2026-04-15). Evidence that does not describe the slate's date can only yield UNRESOLVED, never CONTRADICTED — uncertainty must not become an accusation. - A mis-nested market is REFUSED, never re-bound. Knowing Joe Mack is a Marlin does not license moving a provider record into the Marlins game; that would invent provenance. Measured on the real 2026-08-26 slate: 2,878 props -> 2,874 admitted, 4 rejected (EVENT_PLAYER_TEAM_CONTRADICTION), each player's correct game still resolving. Doubleheader 824514/824478 both remain independently RESOLVED and admitted. No change to probability, projection, side, grade, confidence, ranking, normalization or calibration. Lineage remains disabled. Suite 380/5,061/0 from the release worktree; web tsc exit 0. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CQJeAG8vcDoL5zkiaJyVb8 |
||
|
builtbykev
|
352016790a |
MLB canonical event identity, impossible-binding refusal, event-aware dedupe, publication commit
Release-isolated slice built from
|