# THE SKILL ENGINE — architecture, and Stage A's honest result

**Built 2026-08-03.** Baseball first. Challenger-only; the champion is untouched.

> **STAGE A VERDICT: skill-v1 LOSES to the frequency counter and is NOT promoted.**
> Out-of-sample on 570 settled hits props with 91.9% opposing-pitcher coverage:
> resolution **0.0499 vs the champion's 0.166**, delta **−0.116, CI
> [−0.189, −0.043]**. The architecture's own rule fired and kept it out of the
> live path. §5 is what was actually exercised and what wasn't — the loss is
> real, but it does not yet condemn the approach, and §6 says why.

---

## 1. What was built

| module | discipline | what it does |
|---|---|---|
| `src/services/model/featureRegistry.js` | **D3** — earn its place | CANDIDATE / PROVEN / DEAD per feature, per sport. `liveFeatures()` returns PROVEN only |
| `src/services/model/skillProjection.js` | **D1 + D2** | PA outcome tree from skill inputs; archetype selects the feature map |
| `tests/unit/skillModel.test.js` | all | 22 tests asserting the disciplines as behaviour |
| `scripts/skill-v1-stagea.js` | **D4** | the out-of-sample proof + selectivity measurement |

## 2. The model (D1 — skill, not results)

A hitter gets N plate appearances; each resolves through a tree:

```
PA ──> strikeout      (batter K   vs  pitcher K,  odds-ratio vs league)
   ──> walk           (batter BB  vs  pitcher BB, odds-ratio vs league)
   ──> ball in play
         └─> hit?     (archetype-weighted contact quality vs contact allowed, park)
```

Rates combine by **log5 / odds ratio** against league — the standard way to ask
"this batter vs THIS pitcher" instead of "vs the average pitcher". Two identities
are unit-tested rather than assumed: league-vs-league returns league, and an
average pitcher leaves the batter's own rate untouched.

Hits then distribute as `Binomial(PA, p_hit_per_PA)`, mixed over a PA
distribution — because PA is not known in advance and pretending it is would be
a fabricated certainty.

## 3. Archetype selects features (D2)

Not a nudge, not a label — it decides **which skill inputs drive this hitter**:

| archetype | barrel | hard-hit | exit velo | GB/speed | power |
|---|---|---|---|---|---|
| BOMBER | **0.50** | 0.35 | 0.15 | **0.00** | 1.25 |
| GHOST | **0.05** | 0.20 | 0.15 | **0.60** | 0.70 |
| TORCH | 0.20 | 0.35 | 0.25 | 0.20 | 1.00 |

Locked by test: the same hitter read as BOMBER vs GHOST moves **>0.15** in
probability, and barrel rate moves a BOMBER more than 3× what it moves a GHOST.
Weights are documented by what the archetype physically does — **not fitted**,
because fitting them on 1,741 rows is curve-fitting, and the registry exists so
they get measured instead.

## 4. The gate is structural (D3)

The registry ships with **one PROVEN feature** — `recent_frequency_prior`, the
incumbent counter, and only because it has the measurement (S78 ablation). Every
skill input is CANDIDATE.

**Locked by test: with only PROVEN features allowed, `projectSkill` returns
null.** An unproven model cannot reach a user by accident — not by convention,
by construction. Promotion requires n ≥ 200, positive lift, and a CI excluding
zero; there is deliberately **no override argument**.

Three champion layers are already registered **DEAD** with their reasons, so they
cannot be silently rebuilt: `champion_opp_rank_adj`, `champion_home_away_adj`
("removing it IMPROVED rbi resolution — actively harmful"),
`champion_consistency_pull`.

## 5. STAGE A — the measurement, and what it actually tested

**Out-of-sample guarantee:** skill profiles were frozen 2026-07-21 (the nightly
refresh had never run — §7). Only rows with `game_date > 2026-07-21` were scored,
so no profile can contain the game it predicts. The bug that broke production
made this one measurement clean.

```
matched rows            570        pitcher coverage  91.9%
base rate               0.5649
resolution   skill-v1   0.0499     champion  0.1660
brier        skill-v1   0.2839     champion  0.2506
mean forecast skill-v1  0.5372     champion  0.6055
delta vs champion       −0.1161    CI95 [−0.1892, −0.0426]   ← excludes zero
```

**Selectivity (D4), measured not claimed:**

| top N by confidence | hit rate | lift over base |
|---|---|---|
| 8 | 0.500 | −0.065 |
| 15 | 0.533 | −0.032 |
| 25 | 0.480 | −0.085 |
| 100 | 0.550 | −0.015 |

**It is not selective yet either** — its most confident picks are not its best
ones. That is the honest read and it is the second reason not to promote.

### Two false starts caught in the build, both worth recording

1. **A units bug.** `statcast_aggregates` stores percentages (0–100), not
   fractions. The first run made `bip = 1 − 29.6 − 17.1` deeply negative and
   **refused 568 of 576 rows.** The honest-absent guards worked exactly as
   designed — a model that refuses is visible, a model that fabricates is not.
   The conversion now lives at one chokepoint (`fromStatcastRow`), and an
   out-of-range percentage is treated as broken (null), never clamped.
2. **An invalid first verdict.** The initial run resolved an opposing pitcher for
   **1 of 570 rows** (0.18%) because `ledger_entries.team`/`opponent` are NULL on
   575 of 576 rows in this window. It would have reported "skill-v1 loses" while
   silently measuring a **batter-profile-only** model with no matchup at all —
   the exact false verdict this architecture exists to prevent. Fixed by
   resolving the opponent from each player's own statsapi game log (91.9%
   coverage). **The verdict in this document is from the corrected run.**

## 6. What the loss does and does not say

**Does say:** this implementation, on these inputs, is worse than the counter at
calling listed lines, and it does not go live.

**Does not say the approach is dead** — several candidate features never fired in
this run, so the test was of a partial model:

| feature | exercised? |
|---|---|
| batter K / BB / barrel / hard-hit / exit velo / launch | **yes** |
| pitcher K / BB / hard-hit allowed | **yes** (91.9% of rows) |
| `park_factor` | **no** — park was passed as 1.0 for every row |
| `handedness_platoon` | **no** — never wired into the tree |
| `opportunity_drift` | **no** — the one feature with measured residual signal |
| `projected_pa` | **crude** — season PA ÷ a constant 103 team games |

And one structural handicap worth naming: the skill profiles are **season
aggregates with no recency at all**, while the champion carries a last-5 term. So
skill-v1 was asked to beat a recency-aware model using a 13-day-old season
average. That is a fair loss to record and an unfair basis for concluding that
skill inputs don't work.

**Adding the pitcher moved resolution 0.0441 → 0.0499** — almost nothing. That is
the most interesting number here, and the honest next question: is the matchup
signal genuinely small at the 0.5 hits line, or is the log5 combine too blunt to
express it?

## 7. Fixed along the way — the Statcast refresh was unreachable code

`snapshotScheduler.tick()` returns for any hour not in `HOURS_UTC`
(14,19,22,1,3); the Statcast block then tested `h === STATCAST_HOUR_UTC`
(default **11**). That branch could never execute — the nightly refresh had never
run once, the aggregates sat frozen for 13 days while every consumer served them
as current, and **both of its alerts were inside the same dead branch**, so
nothing could report it. Same silently-guarded-out shape as the settlement
outage.

It now runs on its **own tick**, independent of the snapshot hours. The old test
asserted only that the string `STATCAST_HOUR_UTC` appeared in the file and passed
happily the entire time; it is replaced with one that asserts the refresh is
**not reachable only via the snapshot-hours guard**.

## 8. Next iteration — ranked by what the measurement actually showed

1. **Recency in the skill profile.** The champion's edge is recency and skill-v1
   has none. Weight recent PA into the rates before concluding anything else.
2. **Wire the unexercised features** — park, handedness, and `opportunity_drift`
   (the one feature with measured residual signal, S78).
3. **Real PA projection** from lineup slot, not season PA ÷ 103.
4. **Then re-run Stage A.** If it still loses with all of that firing, the
   conclusion is much stronger and worth acting on.

**Not recommended:** promoting anything, tuning the archetype weights to fit
1,741 rows, or relaxing the bar. The gate did its job today.