One human, one semantic identity — MLB participant convergence
The collision autopsy left two unrepaired defects, running in OPPOSITE
directions, and `outbound_collision_count` can only ever see one of them.
UNDER-COLLAPSE. Dedupe keys on `mlb:<personId>` when the participant is
proven and on the RAW PROVIDER SPELLING when it is not. Mickey Gasper
(681508) is on Boston's 40-man and not on its active roster, so an
active-only index could not identify him and every book's spelling of him
survived dedupe as its own proposition — retention was the first layer to
notice, far too late, and could only discard the loser.
SPLIT. The mirror image, and invisible to the collision metric because it
makes MORE identities, not fewer: Leo Jiménez (677870) is published as both
"Leo Jiménez" and "Leonardo Jimenez", so one human became two semantic
players in one game. Measured across the 15 MLB cohort slices since the
canonical-participant repair, this is a recurring class, not one case:
cam/cameron smith (5 slices), mitch/mitchell bratt, zac/zachary thornton,
leo/leonardo jimenez.
THE REPAIR READS MLB'S OWN RECORD. `hydrate=person` on the roster call the
pipeline already makes returns firstName / useName / useLastName, so the
legitimate name forms for a human come from the league rather than from an
alias table. An alias table is a list of the mistakes we happened to notice.
`nickName` is DELIBERATELY EXCLUDED: over 821 people it produced 14
ambiguous keys, because MLB's nickname field carries bare surnames and
shared clubhouse names — `nameKey('Smitty Smith')` is one string for both
Burch Smith and Will Smith. The four forms kept produce ZERO ambiguity.
Canonical participant reach widens to the 40-man; TEAM EVIDENCE still reads
the ACTIVE roster alone, so event admission and the impossible-binding
refusal are unchanged. Identity still fails closed: a name matching more
than one person in the event resolves to nobody.
CONTINUITY, MEASURED BEFORE WRITING ANY CODE. Over the real 19:00 cohort,
208 of 209 player_keys are unchanged and the one that moves is the defect —
`leonardo jimenez` converging onto `leo jimenez`, a key that already exists.
No new lineage family. The natural key contains game_date, so chains never
span dates and a forward change cannot fork a closed one.
DETERMINISTIC REPRESENTATIVE. Which book's payload survives was decided by
position. It is now decided by the existing MODEL_BOOKS declaration order —
reused, not authored; inventing a sportsbook ranking to settle a tiebreak
would be a market judgement smuggled in as a bug fix — with book name and a
content tiebreak. Stable under every input permutation.
TWO GUARDS, BOTH DIRECTIONS. split (one person, many identities) and merge
(one identity, many people). A merge is refused at the same single admission
seam event identity already uses; a split is counted and alerted but does not
cut the board, because it duplicates an identity rather than asserting a
falsehood.
RETENTION REMAINS AN INDEPENDENT CHECK. The old assertion grepped the source
for `player_key: nameKey(player)`. That expression stood in for a PROPERTY,
and a grep verifies a spelling. Replaced with the property itself, asserted
in both modes: when the producer emits two rows for one human, retention
still files them under one identity and still reports the collision.
Replay of the real cohort through the repair: 3,129 offerings, 100%
participants resolved, every one of 207 participants on exactly ONE semantic
key, collision 0, split 0, merge 0.
Suite 396/5,455/0 · tsc 0 · 15/15 teeth. Tooth 12 came back green first
time and that was a coverage hole, not a safe defect: nothing asserted
retention's append-only upsert. It does now.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CQJeAG8vcDoL5zkiaJyVb8
This commit is contained in:
@@ -427,20 +427,36 @@ async function resolveTeam(abbr, season = DEFAULT_SEASON) {
|
||||
* Active roster for a team id (Session 51) → [{ id, name, position, jersey }].
|
||||
* Cached 6h. [] on failure.
|
||||
*/
|
||||
async function getTeamRoster(teamId, season = DEFAULT_SEASON, asOfDate = null) {
|
||||
async function getTeamRoster(teamId, season = DEFAULT_SEASON, asOfDate = null, rosterType = 'active') {
|
||||
if (!teamId) return [];
|
||||
// `date` scopes the roster to a MOMENT. Without it the answer is "today",
|
||||
// which is the wrong evidence for judging any other slate date. Verified:
|
||||
// Joe Mack appears on the 2026-08-26 Marlins roster and not on 2026-04-15.
|
||||
const d = asOfDate ? `&date=${String(asOfDate).slice(0, 10)}` : '';
|
||||
const url = `${BASE}/teams/${teamId}/roster?rosterType=active&season=${season}${d}`;
|
||||
const data = await fetchWithCache(url, `mlbstats:roster:${teamId}:${season}:${asOfDate || 'now'}`, 6 * 3600);
|
||||
const rt = rosterType === '40Man' ? '40Man' : 'active';
|
||||
// `hydrate=person` expands each roster entry's person object in the SAME
|
||||
// request — firstName / useName / useLastName arrive at no extra API cost.
|
||||
// Those fields are what let one human's several published spellings converge
|
||||
// on one identity without an alias table.
|
||||
const url = `${BASE}/teams/${teamId}/roster?rosterType=${rt}&season=${season}${d}&hydrate=person`;
|
||||
// rosterType is IN the cache key: a cached 'active' answer must never be
|
||||
// served to a caller that asked for the 40-man, and the hydrate widening
|
||||
// means pre-existing cache entries would be missing the new name fields.
|
||||
const data = await fetchWithCache(url, `mlbstats:roster:${teamId}:${season}:${rt}:${asOfDate || 'now'}`, 6 * 3600);
|
||||
const roster = (data && Array.isArray(data.roster)) ? data.roster : [];
|
||||
return roster.map((r) => ({
|
||||
id: r.person?.id ?? null,
|
||||
name: r.person?.fullName ?? null,
|
||||
position: r.position?.abbreviation ?? null,
|
||||
jersey: r.jerseyNumber ?? null,
|
||||
// Raw StatsAPI name parts. Consumed by participantIdentity to build the
|
||||
// legitimate name forms for this person. Absent on a non-hydrated response,
|
||||
// in which case only the full name is usable — degrades, never throws.
|
||||
fullName: r.person?.fullName ?? null,
|
||||
firstName: r.person?.firstName ?? null,
|
||||
lastName: r.person?.lastName ?? null,
|
||||
useName: r.person?.useName ?? null,
|
||||
useLastName: r.person?.useLastName ?? null,
|
||||
})).filter((p) => p.id && p.name);
|
||||
}
|
||||
|
||||
|
||||
@@ -122,6 +122,7 @@ const norm = (v) => String(v || '').toLowerCase().replace(/[^a-z]/g, '');
|
||||
*/
|
||||
const { NAME_TO_ABBR } = require('../environmentContext');
|
||||
const { nameKey } = require('../../utils/playerName');
|
||||
const { participantNameForms } = require('../model/participantIdentity');
|
||||
|
||||
const TEAM_INDEX = (() => {
|
||||
const byFlatName = new Map();
|
||||
@@ -220,10 +221,17 @@ function checkParticipantEvidence(prop, game, playerTeams) {
|
||||
async function buildPlayerTeamIndex(games, deps) {
|
||||
const d = deps || {};
|
||||
const getRoster = d.getTeamRoster;
|
||||
// OPTIONAL second reader for the 40-man. Absent -> `persons` is built from
|
||||
// the active roster alone, i.e. exactly the previous behaviour. A caller that
|
||||
// cannot read the 40-man loses reach, never correctness.
|
||||
const getRoster40 = typeof d.getTeamRoster40 === 'function' ? d.getTeamRoster40 : null;
|
||||
const asOfDate = d.asOfDate || null;
|
||||
const index = new Map();
|
||||
const persons = new Map();
|
||||
const out = { teams: 0, players: 0, failed: 0, as_of_date: asOfDate, date_scoped: false };
|
||||
const out = {
|
||||
teams: 0, players: 0, failed: 0, as_of_date: asOfDate, date_scoped: false,
|
||||
persons: 0, person_rows: 0, roster40_teams: 0, roster40_failed: 0, name_forms: 0,
|
||||
};
|
||||
if (typeof getRoster !== 'function') return { index, persons, stats: out };
|
||||
|
||||
const ids = new Map();
|
||||
@@ -244,24 +252,57 @@ async function buildPlayerTeamIndex(games, deps) {
|
||||
out.teams += 1;
|
||||
const abbr = teamAbbr(name);
|
||||
if (!abbr) continue;
|
||||
|
||||
// TEAM EVIDENCE stays on the ACTIVE roster, byte-identical to before.
|
||||
// `index` feeds evidenceIsDateValid and the impossible-binding refusal;
|
||||
// widening it would change what counts as a contradiction, which is a
|
||||
// different question from who a participant is and is not in scope here.
|
||||
for (const p of roster) {
|
||||
const canonicalName = p && (p.name || p.fullName || p.player_name);
|
||||
const k = nameKey(canonicalName);
|
||||
if (!k) continue;
|
||||
if (!index.has(k)) index.set(k, new Set());
|
||||
index.get(k).add(abbr);
|
||||
// CANONICAL PARTICIPANT. The roster row already carries the StatsAPI
|
||||
// personId; it was discarded here, which is why the pipeline had no stable
|
||||
// player identity and fell back to provider spellings. Candidates are kept
|
||||
// PER TEAM because a bare name key is NOT globally unique — measured on the
|
||||
// real 2026-08-28 league rosters, `max muncy`, `jose fermin` and
|
||||
// `luis garcia` each resolve to TWO different people.
|
||||
out.players += 1;
|
||||
}
|
||||
|
||||
// CANONICAL PARTICIPANT is built over the ACTIVE roster PLUS the 40-man.
|
||||
// Measured on the real 2026-08-30 slate: Mickey Gasper (681508) is on the
|
||||
// Boston 40-man and NOT on the active roster, so an active-only index could
|
||||
// not identify him and every book's spelling of him survived dedupe
|
||||
// separately. Still event-scoped — only the two clubs playing this game —
|
||||
// so a shared name key can never reach across the league.
|
||||
let wide = roster;
|
||||
if (getRoster40) {
|
||||
let extra = null;
|
||||
try { extra = await getRoster40(id, asOfDate); } catch { extra = null; }
|
||||
if (Array.isArray(extra)) { out.roster40_teams += 1; wide = roster.concat(extra); }
|
||||
else out.roster40_failed += 1;
|
||||
}
|
||||
const seenPerson = new Set();
|
||||
for (const p of wide) {
|
||||
const personId = p && (p.id != null ? p.id : p.personId);
|
||||
if (personId != null && abbr) {
|
||||
if (personId == null) continue;
|
||||
const dedupeKey = `${abbr}|${personId}`;
|
||||
if (seenPerson.has(dedupeKey)) continue;
|
||||
seenPerson.add(dedupeKey);
|
||||
const canonicalName = p.name || p.fullName || p.player_name;
|
||||
if (!canonicalName) continue;
|
||||
// Every name form MLB's own record supports for this human. No alias
|
||||
// table: an alias table is a list of the mistakes we happened to notice.
|
||||
const forms = participantNameForms({
|
||||
fullName: p.fullName || canonicalName,
|
||||
firstName: p.firstName,
|
||||
lastName: p.lastName,
|
||||
useName: p.useName,
|
||||
useLastName: p.useLastName,
|
||||
});
|
||||
out.person_rows += 1;
|
||||
for (const k of forms) {
|
||||
if (!persons.has(k)) persons.set(k, []);
|
||||
persons.get(k).push({ personId, canonicalName, abbr });
|
||||
out.name_forms += 1;
|
||||
}
|
||||
out.players += 1;
|
||||
}
|
||||
}
|
||||
out.date_scoped = Boolean(asOfDate) && out.teams > 0;
|
||||
|
||||
@@ -26,7 +26,7 @@
|
||||
// We dedupe to unique player+stat+line and cap how many we grade, because
|
||||
// each grade fans out to feature computation. Grading runs at most once per
|
||||
// cache-miss per sport, but we still bound the herd.
|
||||
const { isModelBook } = require('../config/bookRoles');
|
||||
const { isModelBook, MODEL_BOOKS } = require('../config/bookRoles');
|
||||
|
||||
// RAISED 25 -> 500 on 2026-08-01, on measured cost, not taste.
|
||||
//
|
||||
@@ -119,6 +119,17 @@ function admitForGrading(props, sport) {
|
||||
const reasons = {};
|
||||
for (const p of list) {
|
||||
if (!p) continue;
|
||||
// PARTICIPANT IDENTITY MERGE. One semantic identity standing for two
|
||||
// different MLB humans is the ONE identity failure that can put a wrong
|
||||
// participant into a Read, so it is refused here — at the same single seam
|
||||
// event admission already uses, before dedupe and before grading.
|
||||
if (p.participant_identity_conflict) {
|
||||
p.event_admission = 'REJECTED';
|
||||
p.event_rejection_reason = 'PARTICIPANT_IDENTITY_MERGE';
|
||||
reasons.PARTICIPANT_IDENTITY_MERGE = (reasons.PARTICIPANT_IDENTITY_MERGE || 0) + 1;
|
||||
rejected.push(p);
|
||||
continue;
|
||||
}
|
||||
const status = p.event_binding_status;
|
||||
// A sport with no canonical resolver is unchanged: it never had canonical
|
||||
// identity, and gating it would delete its slate for no safety gain.
|
||||
@@ -208,6 +219,65 @@ function propositionEventKey(p) {
|
||||
return `legacy:${date}:${away}@${home}`;
|
||||
}
|
||||
|
||||
/**
|
||||
* DETERMINISTIC REPRESENTATIVE.
|
||||
*
|
||||
* One semantic proposition is published by many books. Exactly one of those
|
||||
* offerings is retained, and WHICH one must not depend on where it happened to
|
||||
* sit in the provider's response.
|
||||
*
|
||||
* The ordering is `MODEL_BOOKS` read in its own declaration order — REUSED, not
|
||||
* authored. It is not a claim that DraftKings prices better than Pinnacle; it
|
||||
* is the one book ordering this repository already contains, and inventing a
|
||||
* sportsbook ranking to settle a tiebreak would be a market judgement smuggled
|
||||
* in as a bug fix. Book name and a content tiebreak follow, so two offerings
|
||||
* from the same book still resolve to one answer.
|
||||
*/
|
||||
const MODEL_BOOK_ORDER = Object.freeze([...MODEL_BOOKS]);
|
||||
|
||||
function representativeRank(p) {
|
||||
const book = String((p && p.book) || '').toLowerCase();
|
||||
const i = MODEL_BOOK_ORDER.indexOf(book);
|
||||
return [
|
||||
i < 0 ? MODEL_BOOK_ORDER.length : i,
|
||||
book,
|
||||
// Content tiebreak. Never a timestamp and never an array position: both
|
||||
// reintroduce exactly the order-dependence this exists to remove.
|
||||
JSON.stringify([p && p.line, p && p.over_odds, p && p.under_odds]),
|
||||
];
|
||||
}
|
||||
|
||||
function betterRepresentative(a, b) {
|
||||
if (!a) return b;
|
||||
if (!b) return a;
|
||||
const ra = representativeRank(a);
|
||||
const rb = representativeRank(b);
|
||||
for (let i = 0; i < ra.length; i += 1) {
|
||||
if (ra[i] < rb[i]) return a;
|
||||
if (ra[i] > rb[i]) return b;
|
||||
}
|
||||
return a;
|
||||
}
|
||||
|
||||
/**
|
||||
* A SIDE-EFFECT-FREE event key, for the representative pass only.
|
||||
*
|
||||
* `propositionEventKey` INCREMENTS a fracture sequence for a prop whose event
|
||||
* could not be resolved, so calling it twice on the same prop yields two
|
||||
* different keys. A fractured prop is unique by construction and can never
|
||||
* share an identity with anything, so the representative pass simply skips it
|
||||
* (null) rather than re-fracturing it.
|
||||
*/
|
||||
function stableEventKey(p) {
|
||||
if (p && p.canonical_event_id) return p.canonical_event_id;
|
||||
const method = p && p.event_identity_method;
|
||||
if (method && method !== 'UNSUPPORTED_SPORT') return null;
|
||||
const away = String((p && p.away_team) || '').replace(/\s+/g, '');
|
||||
const home = String((p && p.home_team) || '').replace(/\s+/g, '');
|
||||
const date = (p && p.game_date) || '';
|
||||
return `legacy:${date}:${away}@${home}`;
|
||||
}
|
||||
|
||||
function dedupeProps(props, limit, stats) {
|
||||
const seen = new Set();
|
||||
const out = [];
|
||||
@@ -218,6 +288,10 @@ function dedupeProps(props, limit, stats) {
|
||||
const bump = (k) => { if (stats) stats[k] = (stats[k] || 0) + 1; };
|
||||
const list = props || [];
|
||||
let examined = 0;
|
||||
const order = [];
|
||||
// Remembers the key each ADMITTED prop was filed under, so the representative
|
||||
// pass never recomputes a key that has a side effect.
|
||||
const admittedKeyOf = new Map();
|
||||
for (const p of list) {
|
||||
examined += 1;
|
||||
if (!p || !p.player || !p.stat_type || p.line == null) { bump('dropped_invalid_fields'); continue; }
|
||||
@@ -231,16 +305,37 @@ function dedupeProps(props, limit, stats) {
|
||||
const key = `${propositionEventKey(p)}::${who}::${p.stat_type}::${p.line}`;
|
||||
if (seen.has(key)) { bump('duplicate_identity_removed'); continue; }
|
||||
seen.add(key);
|
||||
order.push(key);
|
||||
admittedKeyOf.set(p, key);
|
||||
out.push(p);
|
||||
if (out.length >= limit) { if (stats) stats.capped = true; break; }
|
||||
}
|
||||
// SECOND PASS — representative selection only. The admitted key SET and its
|
||||
// order are already fixed above and are not touched here, so the cap and
|
||||
// every counter keep their existing meaning. This pass exists so that which
|
||||
// book's payload is retained is decided by a rule instead of by position.
|
||||
const best = new Map();
|
||||
const admitted = new Set(order);
|
||||
for (let i = 0; i < list.length; i += 1) {
|
||||
const p = list[i];
|
||||
if (!p || !p.player || !p.stat_type || p.line == null) continue;
|
||||
if (!isModelBook(p.book)) continue;
|
||||
const ev = admittedKeyOf.has(p) ? null : stableEventKey(p);
|
||||
const who = p.mlb_person_id != null ? `mlb:${p.mlb_person_id}` : p.player;
|
||||
const key = admittedKeyOf.has(p)
|
||||
? admittedKeyOf.get(p)
|
||||
: (ev == null ? null : `${ev}::${who}::${p.stat_type}::${p.line}`);
|
||||
if (key == null || !admitted.has(key)) continue;
|
||||
best.set(key, betterRepresentative(best.get(key), p));
|
||||
}
|
||||
const selected = order.map((k) => best.get(k)).filter(Boolean);
|
||||
if (stats) {
|
||||
stats.input_count = list.length;
|
||||
stats.output_count = out.length;
|
||||
stats.model_book_eligible = out.length + (stats.duplicate_identity_removed || 0);
|
||||
stats.output_count = selected.length;
|
||||
stats.model_book_eligible = selected.length + (stats.duplicate_identity_removed || 0);
|
||||
stats.not_examined = list.length - examined;
|
||||
}
|
||||
return out;
|
||||
return selected;
|
||||
}
|
||||
|
||||
// engine1 is direction-aware, so a prop grades differently over vs under.
|
||||
@@ -262,6 +357,12 @@ async function gradeBestSide(grade, prop, sport, opts = {}) {
|
||||
factor_context_resolver: typeof opts.factorContext === 'function' ? opts.factorContext : null,
|
||||
matchup_keys: matchupKeys,
|
||||
player: prop.player,
|
||||
// CANONICAL PARTICIPANT rides with the grade. Retention and the ledger key
|
||||
// the semantic player on this when it is proven, so a book's spelling can
|
||||
// never re-split a human the league has already identified for us. Absent
|
||||
// when unproven, and then everything downstream behaves exactly as before.
|
||||
canonical_player_name: prop.canonical_player_name ?? null,
|
||||
mlb_person_id: prop.mlb_person_id ?? null,
|
||||
stat_type: prop.stat_type,
|
||||
line: prop.line,
|
||||
sport,
|
||||
|
||||
@@ -28,6 +28,20 @@
|
||||
*/
|
||||
|
||||
const { nameKey, normalizeName } = require('../utils/playerName');
|
||||
const { semanticPlayerKey } = require('./model/participantIdentity');
|
||||
|
||||
/**
|
||||
* Semantic identity for a ledger row, read from whichever of the grade record
|
||||
* or the priced prop carries the proven participant. Retention and the ledger
|
||||
* must never disagree about who a proposition is about.
|
||||
*/
|
||||
function ledgerIdentity(g, prop, player) {
|
||||
return semanticPlayerKey({
|
||||
player,
|
||||
mlb_person_id: (g && g.mlb_person_id) ?? (prop && prop.mlb_person_id) ?? null,
|
||||
canonical_player_name: (g && g.canonical_player_name) ?? (prop && prop.canonical_player_name) ?? null,
|
||||
});
|
||||
}
|
||||
const { settleResult, statValue, logRowOnDate } = require('./outcomeService');
|
||||
// The LEDGER takeable standard (floor on the minus side, UNCAPPED plus) — a
|
||||
// DIFFERENT question from valueEngine's -160..+200 promotion band. See
|
||||
@@ -280,7 +294,9 @@ function rowsFromSnapshot(sport, grades, oddsProps, nowIso, lineageIndex) {
|
||||
team,
|
||||
opponent,
|
||||
user_id: null,
|
||||
player_key: nameKey(player),
|
||||
// Same semantic identity as retention. The two stores must never
|
||||
// disagree about who a proposition is about.
|
||||
player_key: ledgerIdentity(g, prop, player).key,
|
||||
player_name: normalizeName(player).display || player,
|
||||
sport: sp,
|
||||
stat,
|
||||
|
||||
@@ -0,0 +1,134 @@
|
||||
/**
|
||||
* PARTICIPANT IDENTITY — one proven MLB human, one semantic identity.
|
||||
*
|
||||
* THREE DISTINCT CONCEPTS, never conflated:
|
||||
*
|
||||
* RAW PROVIDER IDENTITY the spelling a book published. Evidence, never
|
||||
* identity. Preserved verbatim in
|
||||
* `closing_captures.player_name` (per book, per
|
||||
* cycle, undeduped) — this module never mutates it.
|
||||
* CANONICAL PARTICIPANT MLB StatsAPI `personId`. The strongest identity
|
||||
* we can hold, and the ONLY one derived from the
|
||||
* league's own record rather than a vendor's.
|
||||
* SEMANTIC PRODUCT PLAYER `player_key` — what retention, the ledger and
|
||||
* (later) lineage key on. Derived from the
|
||||
* canonical participant's own name WHEN PROVEN,
|
||||
* and from the raw spelling only when it is not.
|
||||
*
|
||||
* The defect this repairs ran in BOTH directions on one real slate:
|
||||
* two spellings of ONE human survived dedupe separately (under-collapse), and
|
||||
* ONE human produced TWO player_keys (split). `collision_count` can only see
|
||||
* the first — a split makes MORE identities, not fewer.
|
||||
*
|
||||
* NAME FORMS ARE READ OFF MLB'S OWN PERSON RECORD. There is no alias table
|
||||
* here and there must never be one: an alias table is a list of the mistakes
|
||||
* we happened to notice.
|
||||
*/
|
||||
|
||||
const { nameKey } = require('../../utils/playerName');
|
||||
|
||||
/**
|
||||
* The name forms a person may legitimately be published under, taken from
|
||||
* StatsAPI's own fields.
|
||||
*
|
||||
* `nickName` is DELIBERATELY EXCLUDED and this is load-bearing. Measured over
|
||||
* 821 people across the 9 games of 2026-08-30, adding `nickName` produced 14
|
||||
* ambiguous keys — MLB's nickname field carries bare surnames ("Jones",
|
||||
* "Wilson") and shared clubhouse names: `nameKey('Smitty Smith')` is the same
|
||||
* string for Burch Smith (572143) and Will Smith (669257). The four forms kept
|
||||
* here produced ZERO ambiguous keys over the same population.
|
||||
*/
|
||||
const NAME_FORM_FIELDS = Object.freeze(['full', 'first_last', 'use_last', 'use_uselast']);
|
||||
|
||||
function participantNameForms(person) {
|
||||
const p = person || {};
|
||||
const last = p.lastName || '';
|
||||
const raw = [
|
||||
p.fullName,
|
||||
`${p.firstName || ''} ${last}`,
|
||||
`${p.useName || ''} ${last}`,
|
||||
`${p.useName || ''} ${p.useLastName || last}`,
|
||||
];
|
||||
const out = new Set();
|
||||
for (const r of raw) {
|
||||
const k = nameKey(r);
|
||||
if (k) out.add(k);
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
/**
|
||||
* The semantic product identity for a proposition.
|
||||
*
|
||||
* PROVEN participant -> derived from the participant's own canonical name, so
|
||||
* every provider spelling of that human converges on one key.
|
||||
* UNPROVEN -> the raw spelling, exactly as before. We never guess: an
|
||||
* unresolved participant keeps behaving the way it always has.
|
||||
*/
|
||||
function semanticPlayerKey(prop) {
|
||||
const p = prop || {};
|
||||
const canonical = p.mlb_person_id != null ? p.canonical_player_name : null;
|
||||
const source = canonical || p.player;
|
||||
return { key: nameKey(source), display_source: source, canonical: !!canonical };
|
||||
}
|
||||
|
||||
/**
|
||||
* STEP 3 INVARIANT, BOTH DIRECTIONS, within one canonical event.
|
||||
*
|
||||
* SPLIT one proven personId -> more than one semantic key. Expected 0.
|
||||
* MERGE one semantic key -> more than one proven personId. Expected 0.
|
||||
*
|
||||
* Rows whose participant is unproven are EXCLUDED from both counts. They carry
|
||||
* no personId, so neither statement can be made about them — counting them
|
||||
* would be asserting something we do not know.
|
||||
*/
|
||||
function auditParticipantIdentity(rows) {
|
||||
const list = Array.isArray(rows) ? rows : [];
|
||||
const byPerson = new Map();
|
||||
const byKey = new Map();
|
||||
let proven = 0;
|
||||
let unproven = 0;
|
||||
for (const r of list) {
|
||||
if (!r) continue;
|
||||
const ev = r.canonical_event_id || null;
|
||||
const pid = r.mlb_person_id;
|
||||
const key = r.player_key;
|
||||
if (pid == null || !ev || !key) { unproven += 1; continue; }
|
||||
proven += 1;
|
||||
const pk = `${ev}|${pid}`;
|
||||
const kk = `${ev}|${key}`;
|
||||
if (!byPerson.has(pk)) byPerson.set(pk, new Set());
|
||||
byPerson.get(pk).add(key);
|
||||
if (!byKey.has(kk)) byKey.set(kk, new Set());
|
||||
byKey.get(kk).add(pid);
|
||||
}
|
||||
const splits = [];
|
||||
for (const [k, s] of byPerson) {
|
||||
if (s.size > 1) {
|
||||
const [event_id, person_id] = k.split('|');
|
||||
splits.push({ event_id, person_id, player_keys: [...s].sort() });
|
||||
}
|
||||
}
|
||||
const merges = [];
|
||||
for (const [k, s] of byKey) {
|
||||
if (s.size > 1) {
|
||||
const i = k.indexOf('|');
|
||||
merges.push({ event_id: k.slice(0, i), player_key: k.slice(i + 1), person_ids: [...s].sort() });
|
||||
}
|
||||
}
|
||||
return {
|
||||
proven_rows: proven,
|
||||
unproven_rows: unproven,
|
||||
participant_identity_split_count: splits.length,
|
||||
participant_identity_merge_count: merges.length,
|
||||
splits,
|
||||
merges,
|
||||
};
|
||||
}
|
||||
|
||||
module.exports = {
|
||||
NAME_FORM_FIELDS,
|
||||
participantNameForms,
|
||||
semanticPlayerKey,
|
||||
auditParticipantIdentity,
|
||||
};
|
||||
@@ -27,6 +27,7 @@
|
||||
|
||||
const crypto = require('crypto');
|
||||
const { normalizeName, nameKey } = require('../utils/playerName');
|
||||
const { semanticPlayerKey } = require('./model/participantIdentity');
|
||||
|
||||
/** ET calendar date of an ISO timestamp (shares gameBinder's rule). */
|
||||
function etDateOf(iso) {
|
||||
@@ -97,6 +98,16 @@ function rowsFromSides(base, sides, ctx = {}) {
|
||||
if (!s) continue;
|
||||
const player = s.player || base.player;
|
||||
if (!player) continue;
|
||||
// SEMANTIC PLAYER IDENTITY. When MLB has proven who this participant is,
|
||||
// the identity comes from the league's own name for them, so every book's
|
||||
// spelling converges. When it has not, the raw spelling is used exactly as
|
||||
// before — we never guess a human into existence. The raw provider string
|
||||
// is untouched here and survives verbatim in closing_captures.
|
||||
const identity = semanticPlayerKey({
|
||||
player,
|
||||
mlb_person_id: s.mlb_person_id ?? base.mlb_person_id ?? null,
|
||||
canonical_player_name: s.canonical_player_name ?? base.canonical_player_name ?? null,
|
||||
});
|
||||
const stat = s.stat_type || base.stat_type;
|
||||
const line = numOrNull(s.line != null ? s.line : base.line);
|
||||
const side = String(s.direction || '').toLowerCase();
|
||||
@@ -117,7 +128,11 @@ function rowsFromSides(base, sides, ctx = {}) {
|
||||
// never the snapshot clock. ctx.gameDate is only a last resort for props
|
||||
// the binder could not tie to a real game.
|
||||
game_date: etDateOf(base && base.game_time) || ctx.gameDate,
|
||||
player_key: nameKey(player),
|
||||
player_key: identity.key,
|
||||
// RAW SOURCE NAME, deliberately. The identity above converges on the
|
||||
// league's record; the NAME on the row stays the provider's, so the row
|
||||
// still carries the representation it was published under. Identity and
|
||||
// provenance are different jobs and this row does both.
|
||||
player_name: normalizeName(player).display || player,
|
||||
team: s.team || base.team || null,
|
||||
opponent: s.opponent || base.opponent || null,
|
||||
|
||||
@@ -30,6 +30,7 @@ const STATS_CONCURRENCY = 5;
|
||||
|
||||
const { nameKey, normalizeName } = require('../utils/playerName');
|
||||
const acq = require('./ops/acquisitionTrace');
|
||||
const participantIdentity = require('./model/participantIdentity');
|
||||
// Session 46 — group/dedupe by the normalized name key so "A.J. Ewing" and
|
||||
// "AJ Ewing" (or "Jazz Chisholm" / "Jazz Chisholm Jr.") collapse to one player.
|
||||
const norm = (s) => nameKey(s);
|
||||
@@ -572,6 +573,13 @@ async function runSnapshot(sport, opts = {}) {
|
||||
try {
|
||||
built = await evid.buildPlayerTeamIndex(games, {
|
||||
getTeamRoster: (id, asOf) => mlbAdapter.getTeamRoster(id, undefined, asOf),
|
||||
// 40-man as well, for CANONICAL PARTICIPANT reach only. Measured:
|
||||
// Mickey Gasper (681508) is on the Boston 40-man and not on the
|
||||
// active roster, so an active-only index could not identify him and
|
||||
// every book's spelling of him survived dedupe as its own prop.
|
||||
// Team evidence (the impossible-binding refusal) still reads the
|
||||
// ACTIVE roster alone and is unchanged.
|
||||
getTeamRoster40: (id, asOf) => mlbAdapter.getTeamRoster(id, undefined, asOf, '40Man'),
|
||||
asOfDate: slateDate,
|
||||
});
|
||||
playerTeams = built.index;
|
||||
@@ -599,6 +607,7 @@ async function runSnapshot(sport, opts = {}) {
|
||||
// unresolved prop simply carries no participant id and behaves exactly
|
||||
// as before. The raw provider name is never overwritten.
|
||||
let participants = 0;
|
||||
let idAudit = null;
|
||||
if (built && built.persons) {
|
||||
for (const pr of props) {
|
||||
const who = evid.resolveParticipant(pr, games, built.persons);
|
||||
@@ -609,6 +618,51 @@ async function runSnapshot(sport, opts = {}) {
|
||||
}
|
||||
}
|
||||
console.log(`[snapshot] canonical participants ${sp}: ${participants}/${props.length}`);
|
||||
// IDENTITY CONVERGENCE GUARDS. `collision_count` can only see one of
|
||||
// the two failure directions: it counts rows that COLLAPSED. A split —
|
||||
// one proven human wearing two semantic identities — makes MORE
|
||||
// identities, so the collision metric is blind to it by construction.
|
||||
// Both directions are asserted here, where the personId is known.
|
||||
try {
|
||||
idAudit = participantIdentity.auditParticipantIdentity(props.map((pr) => ({
|
||||
canonical_event_id: pr.canonical_event_id,
|
||||
mlb_person_id: pr.mlb_person_id,
|
||||
player_key: participantIdentity.semanticPlayerKey(pr).key,
|
||||
})));
|
||||
if (idAudit.participant_identity_merge_count > 0) {
|
||||
// A merge means one semantic identity is standing for two different
|
||||
// humans. That is the one direction that can put a WRONG
|
||||
// participant into a Read, so those propositions do not get graded.
|
||||
// Structurally this should be unreachable — resolveParticipant
|
||||
// already refuses a name key that matches more than one person in
|
||||
// the event — which is exactly why a nonzero value here is worth
|
||||
// stopping for rather than logging.
|
||||
const bad = new Set(idAudit.merges.map((m) => `${m.event_id}|${m.player_key}`));
|
||||
let marked = 0;
|
||||
for (const pr of props) {
|
||||
if (!bad.has(`${pr.canonical_event_id}|${participantIdentity.semanticPlayerKey(pr).key}`)) continue;
|
||||
// MARKED, not filtered. `props` is consumed by several downstream
|
||||
// readers; admitForGrading is the one seam that decides what
|
||||
// reaches the model, so the refusal is expressed there.
|
||||
pr.participant_identity_conflict = true;
|
||||
marked += 1;
|
||||
}
|
||||
console.warn(`[snapshot] participant MERGE for ${sp}: refused ${marked} propositions across `
|
||||
+ `${idAudit.participant_identity_merge_count} merged identities ${JSON.stringify(idAudit.merges)}`);
|
||||
await deps.notify(`Participant identity MERGE for ${sp.toUpperCase()} — ${idAudit.participant_identity_merge_count} semantic identities each matched more than one MLB person. Those propositions were not graded.`, { tag: 'identity' });
|
||||
}
|
||||
if (idAudit.participant_identity_split_count > 0) {
|
||||
// A split duplicates an identity; it does not assert a falsehood,
|
||||
// so the board is not cut. It is still a hard-zero expectation.
|
||||
console.warn(`[snapshot] participant SPLIT for ${sp}: ${idAudit.participant_identity_split_count} `
|
||||
+ `proven humans carry more than one semantic identity ${JSON.stringify(idAudit.splits)}`);
|
||||
await deps.notify(`Participant identity SPLIT for ${sp.toUpperCase()} — ${idAudit.participant_identity_split_count} proven MLB humans produced more than one semantic player identity.`, { tag: 'identity' });
|
||||
}
|
||||
console.log(`[snapshot] participant identity ${sp}: split=${idAudit.participant_identity_split_count} `
|
||||
+ `merge=${idAudit.participant_identity_merge_count} proven_rows=${idAudit.proven_rows} unproven_rows=${idAudit.unproven_rows}`);
|
||||
} catch (eId) {
|
||||
console.warn(`[snapshot] participant identity audit failed for ${sp} (slate continues):`, eId.message);
|
||||
}
|
||||
pgRec.identity({
|
||||
started: true, completed: true, threw: false,
|
||||
schedule_games: games.length, dates_requested: dates.length,
|
||||
@@ -617,6 +671,9 @@ async function runSnapshot(sport, opts = {}) {
|
||||
unsupported: e.unsupported, contradicted: e.impossible,
|
||||
reasons: { ...e.reasons },
|
||||
participants_resolved: participants,
|
||||
participant_identity_split_count: idAudit ? idAudit.participant_identity_split_count : null,
|
||||
participant_identity_merge_count: idAudit ? idAudit.participant_identity_merge_count : null,
|
||||
participant_proven_rows: idAudit ? idAudit.proven_rows : null,
|
||||
});
|
||||
console.log(`[snapshot] event identity ${sp}: ${e.canonical}/${e.total} canonical, `
|
||||
+ `${e.unresolved} unresolved, ${e.impossible} contradicted`
|
||||
|
||||
Reference in New Issue
Block a user