258d8a6655d71d166df99bf77f9349f36d7a7f75
4 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 builtbykev
|
258d8a6655 |
The skill engine: built, gated by construction, and Stage A honestly lost
Built src/services/model/ -- the forward, archetype-selected, skill-based projection, as a challenger. The champion is untouched. featureRegistry makes "earn its place or it's out" structural rather than aspirational: CANDIDATE / PROVEN / DEAD per feature per sport, liveFeatures() returns PROVEN only, promotion requires n>=200 with positive lift and a CI excluding zero, and there is deliberately no override argument. It ships with exactly ONE proven feature -- the incumbent counter, because it is the only one with a measurement. A test asserts that with only PROVEN features allowed the projection returns null, so an unproven model cannot reach a user by accident. The three champion adjustment layers are registered DEAD with their reasons so they cannot be silently rebuilt. skillProjection is a PA outcome tree: K and BB combined by log5 odds-ratio against league (both identities unit-tested), then archetype-weighted contact quality against contact allowed, then Binomial(PA, p_hit) mixed over a PA distribution. Archetype is a FEATURE SELECTOR, not a nudge -- BOMBER reads barrels at 0.50 and ground-ball speed at 0.00, GHOST inverts it -- and a test locks that the same hitter read two ways moves more than 0.15. STAGE A: IT LOSES. Out-of-sample on 570 settled hits props with 91.9% opposing-pitcher coverage, resolution 0.0499 against the champion's 0.166, delta -0.116 with CI [-0.189, -0.043]. It is not selective either: its eight most confident picks hit 50%, a lift of -0.065. Not promoted. The gate did its job on its first real test, which is the point of having built it that way. Two false starts, both recorded because they nearly produced a wrong verdict: statcast_aggregates stores PERCENTAGES, so raw rows made bip = 1-29.6-17.1 and refused 568 of 576 -- the honest-absent guards made a units bug loud instead of silent, and the conversion now lives at one chokepoint. And the first run resolved an opposing pitcher for 1 of 570 rows, because ledger team/opponent are NULL, so it would have reported "skill-v1 loses" while measuring a batter-only model with no matchup in it at all. The verdict above is from the corrected run. The loss is real but partial: park was passed as 1.0, handedness and opportunity_drift never fired, PA is season-PA over a constant, and the skill profiles carry no recency at all while the champion has a last-5 term. Also fixed: the Statcast nightly refresh was unreachable code. It sat inside tick() below "if (!HOURS_UTC.includes(h)) return" while testing h === 11, so it had never run once; the aggregates were 13 days stale and both of its alerts were in the same dead branch. It now runs on its own tick, and the test that passed happily throughout -- it only checked the string existed -- is replaced by one that asserts it is not behind the guard. 4,182 tests green (333 suites); web build exit 0. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W1sivYNqY2TS5ftykmHBU9 |
||
|
builtbykev
|
a49959867d |
Statcast: take the full arsenal, not each pitcher's primary pitch
Caught by spot-checking a real row after the backfill landed: Skubal stored with one pitch. The pitch-movement endpoint with an empty pitch_type returns ONE row per pitcher — their primary offering — so 677 rows for ~700 pitchers, and a five-pitch arsenal was being recorded as a one-pitch one. Not a fabrication, but a silent under-representation of the single most important pitcher-mechanism field, which is worse than useless for Layer 2: it would have classified every pitcher as a one-pitch arm. Mix now comes from pitch-arsenal-stats (3,205 rows = pitcher x pitch type) carrying usage%, whiff%, K%, put-away% and run value per 100 for every pitch. Movement still supplies velo, break and handedness, folded onto the primary pitch; a pitcher present only in the movement feed keeps his handedness and his one measured pitch rather than being dropped. Velo on non-primary pitches is null — absent, not guessed. Skubal now stores 5 pitches, throws L, FF first by usage with velo 96.7. Tests 3583 passed / 292 suites. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VCNgGSt5qvcLxaeQqa7Zpj |
||
|
builtbykev
|
a011ae79fe |
Statcast: role belongs in the key (two-way players)
Found by inducing the real job on the server, not by review: the first chunk wrote, the second failed with 'ON CONFLICT DO UPDATE command cannot affect row a second time'. A player can legitimately appear in BOTH the batter and the pitcher feeds — two-way players, position players who pitch, pitchers who bat — so (sport, season, source_id) collapsed two real profiles into one key and a single batch hit the same row twice. Ohtani has a real batter profile and a real pitcher profile. Merging them would invent one player out of two genuinely different sets of measurements, so role goes in the primary key rather than one profile winning. Migration 031 applied; conflict target updated; a two-way case is now a test. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VCNgGSt5qvcLxaeQqa7Zpj |
||
|
builtbykev
|
528cb1a6d0 |
Layer 1: Statcast mechanism-data ingestion (backfill + nightly refresh)
The data foundation for the archetype and projection layers, built as the pattern every sport inherits. Layers 2 and 3 are not touched. PHASE 0 GATE — both match rates measured live, both 100%. Batters 40/40; PITCHERS 66/66 across five real rosters (CLE, DET, MIN, NYY, LAD) joined by MLBAM id against the 713-pitcher Savant feed. Zero honest-absent on identity, because the join is an integer both systems use natively — and the snapshot pipeline already stores it per graded row. SOURCE — five Baseball Savant CSV leaderboards, free and public, pulled with axios and the CSV parser savantAdapter already runs in prod. pybaseball is deliberately NOT used: it is an MIT wrapper over these same URLs, and adding it would reintroduce a Python runtime in a stack where the existing Python service is already offline. min=1 on every feed, not Savant's default min=q, so the long tail arrives and OUR minimum-sample gate decides what is thin — explicit and testable rather than silently dropped upstream. Measured: 1,354 rows per season (604 batters, 750 pitchers), all five feeds in about five seconds. Pitcher mechanism includes arm angle, GB/FB/LD, chase and whiff; batters get exit velo, launch angle, barrel and hard-hit, chase and z-swing. Handedness rides in free on the movement feed (677 pitchers); batter handedness stays absent pending a roster join rather than being guessed. BACKFILL AND REFRESH ARE THE SAME CALL — a full re-pull upserted on (sport, season, source_id). Idempotent and self-healing: a missed night self-corrects on the next run, with no incremental who-played bookkeeping to drift out of sync. At 1,354 rows the simple thing is also the robust one. HONESTY RULES, each with a test: a metric the feed did not carry is null and never 0; a thin sample is STORED and flagged rather than dropped or inflated, because thin and missing are different claims; an unjoined player is stored with a null player_key and joins later; and if every feed comes back empty the job REFUSES to write, so a bad night can never blank a good table. Freshness is treated as a truth property. updated_at on every row, and the scheduler pages on a failed run AND on silent staleness — a job that stops being scheduled never produces a failure, so staleness has to alarm on its own. Never-built is deliberately not stale: different condition, different fix, and paging on a fresh install teaches the operator to ignore the alarm. Nightly at STATCAST_HOUR_UTC (default 11 UTC, after every game is final), kill switch STATCAST=0, and induce-able at POST /api/internal/statcast/refresh with a freshness probe at /statcast/status — we verify a refresh by running it, not by waiting for the slot. Migration 030 applied. Promoted columns for the classification-critical metrics plus a metrics JSONB carrying every raw field, so Layer 2 can reach something we did not promote without a re-ingest. Raw per-pitch stays out of Postgres on purpose: one season is ~0.85 GB against a 500 MB plan ceiling, and it is re-pullable from the free source if Layer 3 ever needs it. Pattern documented in docs/MECHANISM-DATA.md for NBA tracking and NFL Next Gen. Tests 3581 passed / 292 suites, web build exit 0. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VCNgGSt5qvcLxaeQqa7Zpj |