Files
vyndr/specs/arch-v1-axis-audit.md
T
builtbykev 0d43fb7db8 arch-v1 axis audit: env/matchup were dead; environment fixed
Report for the audit + fix already committed. Records the two things worth
carrying forward:

1. The environment axis has NOT yet been observed writing to the ledger,
   and I am not claiming it has. recordPipelineGrades upserts with
   ignoreDuplicates and dedupes on (user_id, player_key, stat, line, side,
   game_id) -- correctly, so a re-run never overwrites the original lock.
   Today's 429 rows predate the fix, so the axis cannot backfill onto them;
   first ledger observation is tomorrow's slate. What IS directly verified
   is the resolver (105/120) and the join key (416/416) -- the two things
   that were actually broken.

2. Matchup is not fixed and is not claimed as fixed. It needs the opposing
   starter and BOTH hands, and the audit shows three separate absences:
   oppPitcherByTeam 0, handById 0, bats 0/120. Fixing the pitcher feed
   without the hands, or the hands without the feed, still produces an axis
   that fires on zero rows.

Also noted: the S59 slate JOIN INVARIANT keys off the same null `team`
field, so it is currently inert too.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QJs13VsyiSKYQP6rj3NNmc
2026-08-01 04:12:28 -04:00

151 lines
5.5 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# arch-v1 AXIS AUDIT — env/matchup WERE DEAD; ENVIRONMENT FIXED
**Date:** 2026-08-01 · champion `p_win`, ranking, calibration and
`opportunity_drift`'s accruing verdict all **untouched**.
**Gates:** 4,077 tests / 326 suites green · `next build` exit 0 · audit run on
prod · resolver verified live.
---
## STEP 0 — THE AUDIT: "already partly live" was half true
**The code is wired. The axes were not firing.**
Across **634 graded prod rows**, `challenger_adjustments` axis distribution:
| axis | rows | % of slate |
|---|---:|---:|
| *(no axis fired)* | 296 | 46.7% |
| **opportunity** *(new, ours)* | 142 | 22.4% |
| power | 80 | 12.6% |
| swing_miss | 69 | 10.9% |
| contact | 56 | 8.8% |
| launch · line_drive · patience · strikeout · contact_allowed · aggression · chase · velocity · ground_ball · wild | 51→1 | — |
| **environment (park/weather)** | **0** | **0%** |
| **matchup (platoon)** | **0** | **0%** |
Every axis that fired is an **archetype trait**. The two condition axes this
order is about fired on **nothing**.
The ledger confirms it from the other side: `env_multiplier`, `env_park_base`,
`env_weather_mod`, `wx_forecast`, `env_weather_state`**all null on 634/634**.
### Root cause, located rather than inferred
`buildContext` works fine — **15 games, 14 with weather, Coors Field composing to
1.241**. The failure is downstream. A drop-off audit against the live snapshot:
| hop | count |
|---|---:|
| grades examined | 120 |
| **with a `team` field** | **0** |
| team resolves to an abbr | 0 |
| abbr matches a game | 0 |
| **environment produced** | **0** |
| with `playerId` | 120 |
| with `bats` | **0** |
| opposing pitcher known | **0** |
**`team` is a key on every stored grade and NULL on 416/416.** The resolver keyed
off the player's roster team, so it could never find a venue — while
`buildContext` sat there with all 30 teams mapped and 14 weather forecasts
resolved and unused. **Nothing was broken in the park or weather code. The join
key was absent.**
*(Side consequence worth noting: the S59 slate JOIN INVARIANT also keys off this
same `team` field, so it is currently inert too.)*
---
## STEP 1 — THE ENVIRONMENT FIX (and it is the more correct join)
**The park and the weather belong to the GAME, not to the player's roster team**
and the game rides on the prop from the odds feed, where a stats-resolve can
legitimately fail.
1. **`gradeSlateService.gradeBestSide`** now carries `home_team`/`away_team` onto
the graded row. The legacy grade shape dropped them.
2. **`environmentContext.contextFor`** joins on the game first
(`team → home_team → away_team`), keeping the roster team as a fallback.
### Verified live
| | before | after |
|---|---:|---:|
| grades carrying `home_team` | 0 / 416 | **416 / 416** |
| **environment resolves** | **0 / 120** | **105 / 120 (87.5%)** |
Coors Field composes to **1.241** (park 1.20 × weather 1.034) the moment it gets
a key.
### One honest limit on what is confirmed
**The axis has not yet been observed writing to the ledger, and I am not claiming
it has.** `recordPipelineGrades` upserts with `ignoreDuplicates: true` and dedupes
on `(user_id, player_key, stat, line, side, game_id)` — correctly, so a re-run
never overwrites the original lock. Today's 429 rows were written **before** the
fix, so the environment axis **cannot** backfill onto them.
```
game_date 2026-08-01 — 429 rows: opportunity axis 142, environment axis 0
```
**First ledger observation: tomorrow's slate.** What *is* directly verified is the
resolver (105/120) and the join key (416/416), which are the two things that were
broken.
---
## MATCHUP / PLATOON — NOT FIXED, AND NOT CLAIMED AS FIXED
It needs the opposing starter and **both** hands. The audit shows **three separate
absences**:
```
oppPitcherByTeam 0 (the probable-pitcher fetch returns nothing)
handById 0 (so no pitcher handedness)
with_bats 0/120 (no hitter handedness on the slate)
```
Per the order's own guard — *one axis at a time* — that is its **own order with
its own diagnosis**, not a second fix smuggled into this one. Fixing the pitcher
feed without the hands, or the hands without the feed, would still produce an
axis that fires on zero rows.
---
## STEP 2 — HOLDOUTS: BOTH AXES ARE n-BLOCKED
| axis | settled rows carrying it | earliest proof |
|---|---:|---|
| `opportunity` | **0** | first settle pass on 2026-08-01 games |
| `environment` | **0** | first ledger write on tomorrow's slate |
Neither can be proven today, and **a holdout filtered to axis-carrying rows cannot
be run when that set is empty**. The committed query
(`scripts/opportunity-axis-holdout.sql`) already filters that way, for the reason
drift established: including untouched rows dilutes the comparison with rows where
challenger ≡ champion **by construction, biasing toward a false positive.**
**Both accrue in parallel on the same harness. Neither promotes until its own
holdout says so.**
---
## WHAT IS UNTOUCHED
Champion `p_win`, the live grade path, ranking, calibration, and
`opportunity_drift`'s accruing verdict. The environment axis writes only to
`p_win_challenger` / `challenger_adjustments`.
## TAGS
**VERIFIED:** env + matchup axes fired on 0/634 prod rows · 13 archetype axes fire
normally · `team` null on 416/416 · `buildContext` healthy (15 games, 14 weather) ·
post-fix environment resolves 105/120 and `home_team` present 416/416 · matchup
blocked by three separate absences.
**NOT YET OBSERVED:** the environment axis writing to the ledger — blocked by
same-day dedupe, first visible on tomorrow's slate.