The skill engine: built, gated by construction, and Stage A honestly lost

Built src/services/model/ -- the forward, archetype-selected, skill-based
projection, as a challenger. The champion is untouched.

featureRegistry makes "earn its place or it's out" structural rather than
aspirational: CANDIDATE / PROVEN / DEAD per feature per sport, liveFeatures()
returns PROVEN only, promotion requires n>=200 with positive lift and a CI
excluding zero, and there is deliberately no override argument. It ships with
exactly ONE proven feature -- the incumbent counter, because it is the only
one with a measurement. A test asserts that with only PROVEN features allowed
the projection returns null, so an unproven model cannot reach a user by
accident. The three champion adjustment layers are registered DEAD with their
reasons so they cannot be silently rebuilt.

skillProjection is a PA outcome tree: K and BB combined by log5 odds-ratio
against league (both identities unit-tested), then archetype-weighted contact
quality against contact allowed, then Binomial(PA, p_hit) mixed over a PA
distribution. Archetype is a FEATURE SELECTOR, not a nudge -- BOMBER reads
barrels at 0.50 and ground-ball speed at 0.00, GHOST inverts it -- and a test
locks that the same hitter read two ways moves more than 0.15.

STAGE A: IT LOSES. Out-of-sample on 570 settled hits props with 91.9%
opposing-pitcher coverage, resolution 0.0499 against the champion's 0.166,
delta -0.116 with CI [-0.189, -0.043]. It is not selective either: its eight
most confident picks hit 50%, a lift of -0.065. Not promoted. The gate did its
job on its first real test, which is the point of having built it that way.

Two false starts, both recorded because they nearly produced a wrong verdict:
statcast_aggregates stores PERCENTAGES, so raw rows made bip = 1-29.6-17.1 and
refused 568 of 576 -- the honest-absent guards made a units bug loud instead of
silent, and the conversion now lives at one chokepoint. And the first run
resolved an opposing pitcher for 1 of 570 rows, because ledger team/opponent
are NULL, so it would have reported "skill-v1 loses" while measuring a
batter-only model with no matchup in it at all. The verdict above is from the
corrected run.

The loss is real but partial: park was passed as 1.0, handedness and
opportunity_drift never fired, PA is season-PA over a constant, and the skill
profiles carry no recency at all while the champion has a last-5 term.

Also fixed: the Statcast nightly refresh was unreachable code. It sat inside
tick() below "if (!HOURS_UTC.includes(h)) return" while testing h === 11, so
it had never run once; the aggregates were 13 days stale and both of its
alerts were in the same dead branch. It now runs on its own tick, and the test
that passed happily throughout -- it only checked the string existed -- is
replaced by one that asserts it is not behind the guard.

4,182 tests green (333 suites); web build exit 0.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W1sivYNqY2TS5ftykmHBU9
This commit is contained in:
Kev
2026-08-03 02:20:40 -04:00
parent c551bf0340
commit 258d8a6655
9 changed files with 1439 additions and 30 deletions
+27
View File
@@ -3,6 +3,33 @@
## Last Updated
2026-08-03
## Session 80 (2026-08-03) — The skill engine: built, gated, and Stage A honestly lost ✅
Spec: `specs/skill-engine-architecture.md`. 4,182 tests / 333 suites green, build exit 0.
- **BUILT `src/services/model/`:** `featureRegistry` (CANDIDATE/PROVEN/DEAD per
sport; `liveFeatures()` = PROVEN only; promotion needs n>=200 + positive lift +
CI excluding zero, no override) and `skillProjection` (PA outcome tree, log5
odds-ratio K/BB, archetype-selected contact quality, Binomial over a PA
distribution). 22 tests assert the five disciplines as BEHAVIOUR.
- **The gate works by construction:** with only PROVEN features allowed the
projection returns NULL. Registry ships with ONE proven feature (the counter).
- **STAGE A: skill-v1 LOSES → NOT PROMOTED.** Out-of-sample (profiles frozen
07-21, only later games scored), 570 rows, 91.9% pitcher coverage: resolution
0.0499 vs champion 0.166, delta 0.116 CI [0.189,0.043]. Not selective
either (top-8 hit 50%, lift 0.065).
- **Two false starts caught:** (1) units — statcast stores PERCENTAGES, raw rows
made bip negative and refused 568/576; now one chokepoint `fromStatcastRow`.
(2) an INVALID first verdict — ledger team/opponent are NULL, so the pitcher
resolved for 1 of 570 rows and it was silently measuring a batter-only model.
Fixed via each player's statsapi game log.
- **Not exercised yet (so the loss is real but partial):** park (passed 1.0),
handedness, opportunity_drift, and PA projection is season-PA/103. And the
skill profiles carry NO recency while the champion has last-5.
- **Fixed: Statcast nightly refresh was UNREACHABLE CODE** — inside tick() below
the HOURS_UTC guard while testing h===11. Never ran; 13 days stale; both alerts
in the same dead branch. Now its own tick; test rewritten to catch it.
- **Next:** recency into the skill profile, wire park/handedness/opportunity_drift,
real PA from lineup slot, then re-run Stage A.
## Session 79 (2026-08-03) — Reality assessment vs the FORWARD-PROJECTION objective ✅
Spec: `specs/forward-model-reality-assessment.md`. READ-ONLY (src/web untouched).
- **Finding: the forward model's parts all EXIST and are all wired downstream of