Canonical MLB participant: prove the human, then dedupe
VYNDR had three definitions of "same player": raw p.player (dedupe), utils/normalize.normalizeName (features, NO nicknames), utils/playerName.nameKey (retention, WITH nicknames). Retention was the first layer to notice they disagreed, and all it could do was discard the loser — 28 collisions. The fix is not a better spelling. The event-scoped roster ALREADY carries the StatsAPI personId; buildPlayerTeamIndex was fetching and discarding it. SCOPE IS LOAD-BEARING, and this is the finding that shaped the design. Measured on the real 2026-08-28 league rosters (1,583 rows, 30 teams): a bare name key is NOT globally unique — `max muncy` (ATH 691777 / LAD 571970), `jose fermin` (665877/820862) and `luis garcia` (472610/671277) each resolve to TWO different humans. Team-scoped: 0 ambiguous. Event-scoped across 33 real events including the verified doubleheader date: 0 ambiguous. So resolution is scoped to the two teams actually playing, and FAILS CLOSED — no event, no game, no candidate, or more than one candidate yields null and the prop keeps prior behaviour. FEATURE IDENTITY PARITY, measured on both real cohorts before writing the patch (504 participants): RAW_SUCCESS/CANONICAL_SUCCESS/SAME id 365 RAW_SUCCESS/CANONICAL_SUCCESS/DIFFERENT id 0 <- required 0 RAW_SUCCESS/CANONICAL_FAIL 0 <- required 0 CANONICAL_AMBIGUOUS 0 <- required 0 RAW_FAIL/CANONICAL_SUCCESS 7 <- repair, not regression The seven improvements are exactly the alias class: Michael->Mickey Gasper, AJ->A.J. Ewing, JT->J.T. Realmuto, Mike->Michael Busch, Richard->Richie Palacios. This is why raw spelling could not be trusted: player_id_map holds `mickey gasper` and NOT `michael gasper`, so under the raw name the lookup outcome depended on which alias happened to survive dedupe. Provider arrival order was deciding model input availability. ALL SEVEN COLLISION GROUPS PROVEN SAME_PLAYER_ALIAS against the authoritative roster for the exact event date — one personId each (681715, 699625, 676356, 695491, 673357, 681508, 671739), zero false normalizations, zero unresolved — and all seven collapse to one participant under the new key, so the 14 colliding propositions merge upstream instead of being discarded downstream. FALSE-MERGE SIMULATION on both cohorts: cross-event 0, different-stat 0, different-line 0, false merges 0. Note honestly: the retained rows cannot exhibit the merges themselves, because retention already discarded the losers — so the merge half is proven directly on the alias groups, the safety half on the cohorts. RETENTION IS UNTOUCHED — schema, player_key, conflict identity and the collision counter are all unchanged. collision_count must reach 0 by upstream repair, never by making the counter lenient. A teeth proof injects raw spelling into the retention identity and the suite rejects it. Source provenance preserved: p.player is never overwritten; the participant rides beside it as mlb_person_id + canonical_player_name. Twelve teeth against a green baseline of 117 — dedupe back to raw name (3), feature lookup back to the alias (2), resolver stops blocking ambiguity (1), lookup degraded to raw (1), event scope dropped (1), event/line/stat dropped from the key (2/2/2), provenance destroyed (1), raw spelling in retention identity (2), collision_count accepted nonzero (2), and a guard proving the frozen game-date repair cannot be reverted (10). Restored byte-identically. Teeth #4 (canonical resolving to a DIFFERENT feature id) is covered by the real-data parity measurement rather than an injected branch, because it is a data comparison, not a code path — stated plainly rather than claimed as a test. No hardcoded player names in production code — a test greps for all seven. 4 files, 86 insertions. retentionService, gameBinder, oddsService, acquisitionTrace, probabilityEstimator, bookRoles, playerName, normalize and snapshotScheduler: UNCHANGED. Admission rules, MODEL_BOOK filters and model formulas: 0 changed lines. The game-date repair is intact. 391 suites / 5,339 tests pass. web tsc exit 0. Lineage stays OFF. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CQJeAG8vcDoL5zkiaJyVb8
This commit is contained in:
@@ -0,0 +1,255 @@
|
||||
'use strict';
|
||||
|
||||
/**
|
||||
* CANONICAL MLB PARTICIPANT IDENTITY.
|
||||
*
|
||||
* VYNDR had three definitions of "same player": raw `p.player` (dedupe),
|
||||
* `utils/normalize.normalizeName` (features, NO nickname handling), and
|
||||
* `utils/playerName.nameKey` (retention, WITH nicknames). Retention was the
|
||||
* first layer to notice they disagreed, and all it could do was discard the
|
||||
* loser — 28 collisions in the blocked controlled cohort.
|
||||
*
|
||||
* The fix is not a better spelling. The event-scoped roster already carries the
|
||||
* StatsAPI personId; it was being thrown away. Scope is load-bearing: measured
|
||||
* on the real 2026-08-28 league rosters, `max muncy`, `jose fermin` and
|
||||
* `luis garcia` each resolve to TWO different people, so a bare name key is NOT
|
||||
* globally unique. Within one game's two rosters, all 33 real events tested
|
||||
* (including the verified doubleheader date) resolved uniquely.
|
||||
*/
|
||||
|
||||
const fs = require('fs');
|
||||
const path = require('path');
|
||||
const evid = require('../../src/services/event/eventIdentity');
|
||||
const { nameKey } = require('../../src/utils/playerName');
|
||||
const { normalizeName: featNorm } = require('../../src/utils/normalize');
|
||||
const { __internals } = require('../../src/services/gradeSlateService');
|
||||
const ROOT = path.resolve(__dirname, '..', '..');
|
||||
|
||||
const G = (pk, home, away) => ({ gamePk: pk, home: { team: home }, away: { team: away } });
|
||||
const personsOf = (rows) => {
|
||||
const m = new Map();
|
||||
for (const r of rows) {
|
||||
const k = nameKey(r.canonicalName);
|
||||
if (!m.has(k)) m.set(k, []);
|
||||
m.get(k).push(r);
|
||||
}
|
||||
return m;
|
||||
};
|
||||
const prop = (o = {}) => ({ player: 'P', stat_type: 'hits', line: 0.5, book: 'draftkings', ...o });
|
||||
|
||||
describe('SCOPE IS LOAD-BEARING — a name key is not a player id', () => {
|
||||
// Real league data: two different humans, one normalized key.
|
||||
const persons = personsOf([
|
||||
{ personId: 691777, canonicalName: 'Max Muncy', abbr: 'ATH' },
|
||||
{ personId: 571970, canonicalName: 'Max Muncy', abbr: 'LAD' },
|
||||
]);
|
||||
const games = [G(1, 'Los Angeles Dodgers', 'San Diego Padres'), G(2, 'Athletics', 'Los Angeles Dodgers')];
|
||||
|
||||
test('one team in the event -> resolves to that team’s person', () => {
|
||||
const r = evid.resolveParticipant(prop({ player: 'Max Muncy', canonical_event_id: 'mlb:gamepk:1' }), games, persons);
|
||||
expect(r.personId).toBe(571970);
|
||||
expect(r.teamAbbr).toBe('LAD');
|
||||
});
|
||||
|
||||
test('BOTH Muncys in the same event -> REFUSES, never guesses', () => {
|
||||
expect(evid.resolveParticipant(prop({ player: 'Max Muncy', canonical_event_id: 'mlb:gamepk:2' }), games, persons)).toBeNull();
|
||||
});
|
||||
|
||||
test('no event, no game, or unknown player -> null (fails closed)', () => {
|
||||
expect(evid.resolveParticipant(prop({ player: 'Max Muncy' }), games, persons)).toBeNull();
|
||||
expect(evid.resolveParticipant(prop({ player: 'Max Muncy', canonical_event_id: 'mlb:gamepk:99' }), games, persons)).toBeNull();
|
||||
expect(evid.resolveParticipant(prop({ player: 'Nobody Here', canonical_event_id: 'mlb:gamepk:1' }), games, persons)).toBeNull();
|
||||
});
|
||||
|
||||
test('an empty persons index never resolves', () => {
|
||||
expect(evid.resolveParticipant(prop({ player: 'Max Muncy', canonical_event_id: 'mlb:gamepk:1' }), games, new Map())).toBeNull();
|
||||
});
|
||||
});
|
||||
|
||||
describe('ALIAS CLASSES — generic, with the seven as regression evidence', () => {
|
||||
const CASES = [
|
||||
['accent', 'Heriberto Hernández', 'Heriberto Hernandez', 681715, 'MIA', 'Miami Marlins'],
|
||||
['suffix', 'Luis Robert Jr.', 'Luis Robert', 673357, 'NYM', 'New York Mets'],
|
||||
['case', 'Jonny DeLuca', 'Jonny Deluca', 676356, 'TB', 'Tampa Bay Rays'],
|
||||
['suffix II', 'Michael Harris II', 'Michael Harris', 671739, 'ATL', 'Atlanta Braves'],
|
||||
['nickname', 'Mickey Gasper', 'Michael Gasper', 681508, 'BOS', 'Boston Red Sox'],
|
||||
['formal', 'Joshua Báez', 'Josh Baez', 695491, 'STL', 'St. Louis Cardinals'],
|
||||
['nick+suf', 'Jimmy Crooks', 'Jimmy Crooks III', 699625, 'STL', 'St. Louis Cardinals'],
|
||||
];
|
||||
for (const [label, canonical, alias, personId, abbr, teamName] of CASES) {
|
||||
test(`${label}: "${alias}" resolves to the same person as "${canonical}"`, () => {
|
||||
const persons = personsOf([{ personId, canonicalName: canonical, abbr }]);
|
||||
const games = [G(7, teamName, 'Chicago Cubs')];
|
||||
const a = evid.resolveParticipant(prop({ player: alias, canonical_event_id: 'mlb:gamepk:7' }), games, persons);
|
||||
const c = evid.resolveParticipant(prop({ player: canonical, canonical_event_id: 'mlb:gamepk:7' }), games, persons);
|
||||
expect(a).not.toBeNull();
|
||||
expect(a.personId).toBe(personId);
|
||||
expect(c.personId).toBe(personId);
|
||||
expect(a.canonicalName).toBe(canonical);
|
||||
});
|
||||
}
|
||||
|
||||
test('the fix is GENERIC — no player name is hardcoded in production code', () => {
|
||||
for (const f of ['src/services/event/eventIdentity.js', 'src/services/gradeSlateService.js',
|
||||
'src/services/intelligence/computeFeatures.js', 'src/services/snapshotService.js']) {
|
||||
const src = fs.readFileSync(path.join(ROOT, f), 'utf8')
|
||||
.replace(/\/\*[\s\S]*?\*\//g, '').replace(/(^|[^:])\/\/.*$/gm, '$1');
|
||||
for (const name of ['Gasper', 'Hernández', 'Hernandez', 'Baez', 'Crooks', 'DeLuca', 'Muncy']) {
|
||||
expect(src).not.toContain(name);
|
||||
}
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
describe('DEDUPE BY PARTICIPANT, NOT DISPLAY NAME', () => {
|
||||
const dd = (rows) => __internals.dedupeProps(rows, 1000);
|
||||
const base = { stat_type: 'hits', line: 0.5, book: 'draftkings', canonical_event_id: 'mlb:gamepk:7' };
|
||||
|
||||
test('two aliases of ONE proven participant collapse to ONE proposition', () => {
|
||||
const out = dd([
|
||||
{ ...base, player: 'Michael Gasper', mlb_person_id: 681508 },
|
||||
{ ...base, player: 'Mickey Gasper', mlb_person_id: 681508 },
|
||||
]);
|
||||
expect(out).toHaveLength(1);
|
||||
});
|
||||
|
||||
test('the SAME person in TWO events stays TWO propositions', () => {
|
||||
const out = dd([
|
||||
{ ...base, player: 'Mickey Gasper', mlb_person_id: 681508, canonical_event_id: 'mlb:gamepk:824514' },
|
||||
{ ...base, player: 'Mickey Gasper', mlb_person_id: 681508, canonical_event_id: 'mlb:gamepk:824478' },
|
||||
]);
|
||||
expect(out).toHaveLength(2);
|
||||
});
|
||||
|
||||
test('same person, different STAT stays two', () => {
|
||||
expect(dd([
|
||||
{ ...base, player: 'A', mlb_person_id: 1 },
|
||||
{ ...base, player: 'A', mlb_person_id: 1, stat_type: 'total_bases' },
|
||||
])).toHaveLength(2);
|
||||
});
|
||||
|
||||
test('same person, different LINE stays two', () => {
|
||||
expect(dd([
|
||||
{ ...base, player: 'A', mlb_person_id: 1 },
|
||||
{ ...base, player: 'A', mlb_person_id: 1, line: 1.5 },
|
||||
])).toHaveLength(2);
|
||||
});
|
||||
|
||||
test('DIFFERENT people are never merged, even with one lossy name key', () => {
|
||||
const out = dd([
|
||||
{ ...base, player: 'Max Muncy', mlb_person_id: 691777 },
|
||||
{ ...base, player: 'Max Muncy', mlb_person_id: 571970 },
|
||||
]);
|
||||
expect(out).toHaveLength(2);
|
||||
});
|
||||
|
||||
test('unresolved props fall back to the raw name — prior behaviour exactly', () => {
|
||||
expect(dd([{ ...base, player: 'Michael Gasper' }, { ...base, player: 'Mickey Gasper' }])).toHaveLength(2);
|
||||
expect(dd([{ ...base, player: 'Same Guy' }, { ...base, player: 'Same Guy' }])).toHaveLength(1);
|
||||
});
|
||||
|
||||
test('the dedupe key uses the participant when present, the raw name otherwise', () => {
|
||||
const src = fs.readFileSync(path.join(ROOT, 'src/services/gradeSlateService.js'), 'utf8');
|
||||
expect(src).toMatch(/const who = p\.mlb_person_id != null \? `mlb:\$\{p\.mlb_person_id\}` : p\.player;/);
|
||||
expect(src).toMatch(/\$\{propositionEventKey\(p\)\}::\$\{who\}::\$\{p\.stat_type\}::\$\{p\.line\}/);
|
||||
});
|
||||
});
|
||||
|
||||
describe('FEATURE IDENTITY IS CANONICAL, NOT THE SURVIVING ALIAS', () => {
|
||||
const src = fs.readFileSync(path.join(ROOT, 'src/services/intelligence/computeFeatures.js'), 'utf8');
|
||||
|
||||
test('the lookup consumes the canonical roster name when proven', () => {
|
||||
expect(src).toMatch(/const featurePlayer = rawProp\.canonical_player_name \|\| player;/);
|
||||
expect(src).toMatch(/lookupPlayer\(\{ player: featurePlayer, sport \}\)/);
|
||||
expect(src).toMatch(/getStatRows\(featurePlayer, sport, statType\)/);
|
||||
});
|
||||
|
||||
test('this is exactly why raw spelling could not be trusted', () => {
|
||||
// player_id_map holds `mickey gasper`, NOT `michael gasper` (verified in
|
||||
// production). Under the raw name the lookup outcome depended on which
|
||||
// alias survived dedupe.
|
||||
expect(featNorm('Mickey Gasper')).toBe('mickey gasper');
|
||||
expect(featNorm('Michael Gasper')).toBe('michael gasper');
|
||||
expect(featNorm('Mickey Gasper')).not.toBe(featNorm('Michael Gasper'));
|
||||
// The retention normalizer DOES unify them — that disagreement is the bug.
|
||||
expect(nameKey('Mickey Gasper')).toBe(nameKey('Michael Gasper'));
|
||||
});
|
||||
|
||||
test('no canonical name -> the raw name, unchanged from before', () => {
|
||||
expect(src).toMatch(/rawProp\.canonical_player_name \|\| player/);
|
||||
});
|
||||
});
|
||||
|
||||
describe('SOURCE PROVENANCE IS PRESERVED', () => {
|
||||
test('the raw provider name is never overwritten', () => {
|
||||
const src = fs.readFileSync(path.join(ROOT, 'src/services/snapshotService.js'), 'utf8');
|
||||
const i = src.indexOf('CANONICAL PARTICIPANT. Resolved AFTER event identity');
|
||||
const block = src.slice(i, i + 900);
|
||||
expect(block).toMatch(/pr\.mlb_person_id = who\.personId;/);
|
||||
expect(block).toMatch(/pr\.canonical_player_name = who\.canonicalName;/);
|
||||
expect(block).not.toMatch(/pr\.player\s*=/);
|
||||
});
|
||||
|
||||
test('retention still stores the SOURCE name and its own player_key', () => {
|
||||
const src = fs.readFileSync(path.join(ROOT, 'src/services/retentionService.js'), 'utf8');
|
||||
expect(src).toMatch(/player_key: nameKey\(player\)/);
|
||||
expect(src).toMatch(/player_name: normalizeName\(player\)\.display \|\| player/);
|
||||
});
|
||||
});
|
||||
|
||||
describe('RETENTION REMAINS AN INDEPENDENT CHECK', () => {
|
||||
const src = fs.readFileSync(path.join(ROOT, 'src/services/retentionService.js'), 'utf8');
|
||||
|
||||
test('the conflict identity is unchanged and carries NO raw spelling', () => {
|
||||
const retention = require('../../src/services/retentionService');
|
||||
expect(retention.RETENTION_CONFLICT.split(',')).toEqual(
|
||||
['snapshot_id', 'game_id', 'canonical_event_id', 'player_key', 'stat', 'line', 'side']);
|
||||
expect(retention.RETENTION_CONFLICT).not.toMatch(/player_name|mlb_person_id/);
|
||||
});
|
||||
|
||||
test('the collision counter still exists and still disqualifies', () => {
|
||||
const retention = require('../../src/services/retentionService');
|
||||
const rows = [
|
||||
{ snapshot_id: 's', game_id: 'g', canonical_event_id: 'e', player_key: 'a', stat: 'hits', line: 0.5, side: 'over' },
|
||||
{ snapshot_id: 's', game_id: 'g', canonical_event_id: 'e', player_key: 'a', stat: 'hits', line: 0.5, side: 'over' },
|
||||
];
|
||||
const exp = retention.expectedMaterialization(rows);
|
||||
expect(exp.collision_count).toBe(1);
|
||||
expect(retention.reconcileMaterialization({
|
||||
expected: exp, actualIdentities: exp.identities, transportStatus: retention.TERMINAL.COMPLETE,
|
||||
}).status).toBe(retention.MATERIALIZATION.COLLISION);
|
||||
expect(src).toMatch(/collision_count/);
|
||||
});
|
||||
});
|
||||
|
||||
describe('THE THREE-NORMALIZER CONTRACT', () => {
|
||||
test('semantic participant identity is independent of every name normalizer', () => {
|
||||
const persons = personsOf([{ personId: 681508, canonicalName: 'Mickey Gasper', abbr: 'BOS' }]);
|
||||
const games = [G(7, 'Boston Red Sox', 'New York Yankees')];
|
||||
const a = evid.resolveParticipant(prop({ player: 'Michael Gasper', canonical_event_id: 'mlb:gamepk:7' }), games, persons);
|
||||
const b = evid.resolveParticipant(prop({ player: 'Mickey Gasper', canonical_event_id: 'mlb:gamepk:7' }), games, persons);
|
||||
// Same identity despite the raw strings differing AND the feature
|
||||
// normalizer disagreeing about them.
|
||||
expect(a.personId).toBe(b.personId);
|
||||
expect(featNorm('Michael Gasper')).not.toBe(featNorm('Mickey Gasper'));
|
||||
// And the id is not derived from a name at all.
|
||||
expect(typeof a.personId).toBe('number');
|
||||
});
|
||||
|
||||
test('the normalizers keep their existing narrow purposes', () => {
|
||||
const rs = fs.readFileSync(path.join(ROOT, 'src/services/retentionService.js'), 'utf8');
|
||||
expect(rs).toMatch(/nameKey/);
|
||||
const cf = fs.readFileSync(path.join(ROOT, 'src/services/intelligence/computeFeatures.js'), 'utf8');
|
||||
expect(cf).toMatch(/normalizeName/);
|
||||
// Neither was globally replaced by the other.
|
||||
expect(cf).not.toMatch(/require\(.*playerName.*\)/);
|
||||
});
|
||||
});
|
||||
|
||||
describe('THE GAME-DATE REPAIR IS UNTOUCHED', () => {
|
||||
test('the fast-path date assignment is still present', () => {
|
||||
const src = fs.readFileSync(path.join(ROOT, 'src/services/gameBinder.js'), 'utf8');
|
||||
expect(src).toMatch(/if \(!p\.game_date\) p\.game_date = etFast;/);
|
||||
expect(src).toMatch(/const etFast = p\.game_time \? etDate\(p\.game_time\) : null;/);
|
||||
});
|
||||
});
|
||||
Reference in New Issue
Block a user