A THIRD challenger (after arch-v1, contact-v1), MLB batting v1. Champion is
market-relative P(stat>LINE); proj-v1 is ABSOLUTE — what the hitter will DO —
emitted as a full distribution from which the WHOLE LADDER (P≥1,P≥2,P≥3) derives.
Champion untouched; nothing claimed; the ledger decides per rung, per stat.
- projection/distribution.js — Bayesian Gamma-Poisson → negative-binomial
predictive. Admits over-dispersion; under-dispersion → Poisson approx
(conservative, documented). Uncertainty scales with sample by construction
(r=α): thin → WIDE (real mass on P≥1, honestly thin P≥3), thick → tight.
NEVER abstains — width carries the honesty.
- projection/matchupRead.js — the input the book doesn't use. HONEST FIDELITY:
pitcher repertoire is rich (97% pitch-mix) but hitters have NO pitch-type
performance, so TRUE repertoire-vs-profile is impossible today. This is the
COARSE version (arsenal buckets fastball/sinker/breaking + whiff/hard-hit
tendency × hitter whiff/chase/gb-fb/hard-hit) — beats generic L/R, derived +
documented + TESTED two-sided. A hitter pitch-type feed unlocks the true form.
- projectionChallenger.js — park RELATIVE to the player's own log exposure
(isHome→own park, away→opp park; Phase B's raw-multiply bug solved), recency-
weighted fit, per-factor breakdown (form/park/weather/platoon/matchup — show
your work), full rung set + book-implied per rung. Combined non-form
multiplier bounded.
- Wired after contact-v1, own try, flag PROJ_V1_ENABLED, reusing arch-v1's
already-computed park/weather/platoon (no duplicate env I/O). Own ledger
columns (migration 032, applied to prod): distribution, ladder, point, line,
our-P, book-implied, factor breakdown — measurable per rung/stat after settle.
Phase 0 (prod-verified): venue join via isHome; NB family; uncertainty-as-width;
coarse matchup honest fidelity; no lineup-slot (per-game rate, volume implicit).
Sanity: thin-hot → wide (credible low rung, thin high rung); .300 hitter ≠ 3.0;
matchup two-sided; champion byte-identical. proj-v1 suites 23/23; snapshot/
ledger/siblings 74 green. Forward-only, version-stamped, PROJ_V1_ENABLED kill.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VCNgGSt5qvcLxaeQqa7Zpj
Phase A #2: the champion grade (l5/l20 result-based form) is a HYPOTHESIS that
contact quality predicts better — unmeasured on our props, with zero settled
p_win yet. Swapping l5/l20 (the champion's two heaviest ±1.0 factors) blind
could degrade the core grade undetectably for weeks. So this NOMINATES contact
quality as a second challenger, records what it WOULD project per prop, and lets
the settled ledger decide. Nothing users see changes; the champion is untouched.
- src/services/contactChallenger.js — pure, mirrors challengerProjection. Log-
odds lean (capped, never a re-forecast) from SEASON contact quality vs league
percentiles. Metric→prop mapping is the whole game: barrel_pct→HR,
hard_hit_pct→TB/doubles, k_pct-INVERSE→hits (singles resolve on contact
frequency, not barrels), k_pct→batter K. rbi/runs/walks ABSTAIN (opportunity/
discipline — no clean contact predictor). Honest-absent: thin (<50 PA)/absent/
unmapped/non-batter → p_win_contact NULL (no projection), never a fallback;
"measured but unremarkable" is distinct (equals champion, delta 0).
- Wired in snapshotService AFTER arch-v1, reusing the already-loaded statcast
rows; its own try so a second challenger can't break the pipeline. Reads
g.p_win, never writes it.
- Retained SEPARATELY on the ledger (p_win_contact/contact_delta/
contact_adjustments/contact_version='contact-v1') so each challenger's marginal
contribution is measured independently; ledger_entries.stat gives per-prop-type
segmentation. Migration 031 (applied to prod).
Phase 0 (prod-verified): statcast_aggregates is SEASON cumulative (not rolling),
48h stale now but season-scoped so ~8 PA/600 is negligible; 100% of graded
hitters covered, 92% at ≥50 PA; no xBA/xwOBA in the feed. Forward-only,
version-stamped (contact_version null on pre-nomination rows). Promotion is a
LATER decision on settled evidence, per prop type — never asserted here.
contactChallenger 14/14; snapshot/ledger/arch-v1 suites 80 green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VCNgGSt5qvcLxaeQqa7Zpj
Found by inducing the real job on the server, not by review: the first chunk
wrote, the second failed with 'ON CONFLICT DO UPDATE command cannot affect row
a second time'. A player can legitimately appear in BOTH the batter and the
pitcher feeds — two-way players, position players who pitch, pitchers who bat —
so (sport, season, source_id) collapsed two real profiles into one key and a
single batch hit the same row twice.
Ohtani has a real batter profile and a real pitcher profile. Merging them would
invent one player out of two genuinely different sets of measurements, so role
goes in the primary key rather than one profile winning. Migration 031 applied;
conflict target updated; a two-way case is now a test.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VCNgGSt5qvcLxaeQqa7Zpj
The data foundation for the archetype and projection layers, built as the
pattern every sport inherits. Layers 2 and 3 are not touched.
PHASE 0 GATE — both match rates measured live, both 100%. Batters 40/40;
PITCHERS 66/66 across five real rosters (CLE, DET, MIN, NYY, LAD) joined by
MLBAM id against the 713-pitcher Savant feed. Zero honest-absent on identity,
because the join is an integer both systems use natively — and the snapshot
pipeline already stores it per graded row.
SOURCE — five Baseball Savant CSV leaderboards, free and public, pulled with
axios and the CSV parser savantAdapter already runs in prod. pybaseball is
deliberately NOT used: it is an MIT wrapper over these same URLs, and adding it
would reintroduce a Python runtime in a stack where the existing Python service
is already offline. min=1 on every feed, not Savant's default min=q, so the
long tail arrives and OUR minimum-sample gate decides what is thin — explicit
and testable rather than silently dropped upstream.
Measured: 1,354 rows per season (604 batters, 750 pitchers), all five feeds in
about five seconds. Pitcher mechanism includes arm angle, GB/FB/LD, chase and
whiff; batters get exit velo, launch angle, barrel and hard-hit, chase and
z-swing. Handedness rides in free on the movement feed (677 pitchers); batter
handedness stays absent pending a roster join rather than being guessed.
BACKFILL AND REFRESH ARE THE SAME CALL — a full re-pull upserted on
(sport, season, source_id). Idempotent and self-healing: a missed night
self-corrects on the next run, with no incremental who-played bookkeeping to
drift out of sync. At 1,354 rows the simple thing is also the robust one.
HONESTY RULES, each with a test: a metric the feed did not carry is null and
never 0; a thin sample is STORED and flagged rather than dropped or inflated,
because thin and missing are different claims; an unjoined player is stored
with a null player_key and joins later; and if every feed comes back empty the
job REFUSES to write, so a bad night can never blank a good table.
Freshness is treated as a truth property. updated_at on every row, and the
scheduler pages on a failed run AND on silent staleness — a job that stops
being scheduled never produces a failure, so staleness has to alarm on its own.
Never-built is deliberately not stale: different condition, different fix, and
paging on a fresh install teaches the operator to ignore the alarm.
Nightly at STATCAST_HOUR_UTC (default 11 UTC, after every game is final), kill
switch STATCAST=0, and induce-able at POST /api/internal/statcast/refresh with
a freshness probe at /statcast/status — we verify a refresh by running it, not
by waiting for the slot.
Migration 030 applied. Promoted columns for the classification-critical metrics
plus a metrics JSONB carrying every raw field, so Layer 2 can reach something we
did not promote without a re-ingest. Raw per-pitch stays out of Postgres on
purpose: one season is ~0.85 GB against a 500 MB plan ceiling, and it is
re-pullable from the free source if Layer 3 ever needs it.
Pattern documented in docs/MECHANISM-DATA.md for NBA tracking and NFL Next Gen.
Tests 3581 passed / 292 suites, web build exit 0.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VCNgGSt5qvcLxaeQqa7Zpj
RETENTION (Phase 2, priority zero). History starts compounding tonight.
migration 025 model_snapshots — APPLIED to prod. Append-only, one row per
graded prop PER SIDE PER CYCLE, with a unique index on
(snapshot_id, player_key, stat, line, side) so a retried cycle cannot
duplicate. RLS on, service-role writes only.
What it captures that the ledger never did:
- features jsonb — the model's INPUTS. Without these a backtest can only
grade our own homework; with them any future model can be replayed
against the exact conditions this one faced.
- REFUSALS (refused + refusal_reason). The ledger drops them, so a gate
refusing props that would have WON is invisible — unmeasurable lost
edge. Captured via a new onGraded hook in gradeSlateService that fires
with BOTH sides before any filtering.
- grade_11, the pre-collapse grade. The 4-letter map throws away the
entire live C-/C/C+/B- range.
- model_version + code_sha on every row. ledger_entries mixes pre/post-fix
grades with no marker and cannot be separated retroactively.
- p_win / ev_pct / fair_odds / takeable / value — none of which any
permanent store held.
Wiring: analyzeViaEngine1 attaches _features/_grade_11 (underscore =
internal); gradeSlateService fires onGraded then STRIPS them so they never
reach a cache or API payload; snapshotService builds rows and persists
best-effort. Retention reuses the LEDGER's dateET/gameIdFor helpers so
rows share the ledger's natural key exactly — otherwise the settle pass
could never join outcomes onto them. Rows are written BEFORE the empty-
slate early return: a slate that refused everything is exactly the case
worth recording.
CONTRACT HELD: retention is injectable and every path is caught. persist()
returns errors, never throws; a missing Supabase client is SKIPPED, not an
error. A retention failure can never break a snapshot.
BACKUP: backup-db.sh now accepts BACKUP_SSH_KEY as base64 (recommended —
survives env-var newline mangling, which is how injected SSH keys usually
break silently) OR raw PEM, detected by decoding and looking for the PEM
header. Verified both forms detect correctly against a real generated key.
Suite 279/3325 green, build exit 0.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SmNjJAwEnqHPtXbvSZR8kA
Migration 019: the truth-infrastructure table. Committed BEFORE it runs,
per the Phase 1 GO instructions.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Line movement system:
- Baseline capture on first odds fetch of the day
- Movement detection >= 0.5 points with direction (up/down)
- Sharp money heuristic (sharp_action/public_action/unknown)
- GET /api/movements with player, stat_type, min_movement filters
- Movements included in GET /api/odds/nba live responses
Cascade detection system:
- Scratch detection: player props disappear from 2+ books
- Affected user lookup via scan_sessions + picks
- Parlay re-grade without scratched legs
- cascade_alerts created for affected users
- GET /api/alerts (Analyst/Desk only), PATCH /api/alerts/:id/read
Zero extra Odds API credits — all detection piggybacks on existing fetches.
Migration 002: line_baselines, line_movements, cascade_alerts tables.
30 new tests, 188 total (161 Node.js + 27 Python), all passing.
Phase 2 Core Product COMPLETE.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
builtbykev