checkpoint: chain shadow, WNBA possession feed, baseball chain
Backup commit of uncommitted working-tree state found during Legion recon (Tony resurrection, STEP 0). This work existed only on the laptop disk. - chain shadow accrual + probe script (038_chain_shadow.sql) - WNBA possession feed: ESPN adapter, usage service, verify script (039_wnba_player_game.sql) - baseball chain - retention/snapshot service updates, tableKeys, matchupKeys - specs: chain-v1, wnba-possession-feed, wnba-source-survey - unit tests for the above Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QnvJAkC3h5QGmb6dipoiWn
This commit is contained in:
+163
@@ -3,6 +3,169 @@
|
||||
## Last Updated
|
||||
2026-08-12
|
||||
|
||||
## WNBA v1 (2026-08-13) — the possession/usage feed established ✅
|
||||
Spec: `specs/wnba-possession-feed.md`. 4,905 tests / 377 suites green, web build
|
||||
exit 0. **DATA INFRA ONLY — no chainFn, no archetype wiring, no shadow, nothing
|
||||
served. MLB / chain shadow / A8 / CALIBRATION_DEPLOYED untouched (test-asserted).**
|
||||
- **SOURCE ESTABLISHED BEFORE PLUMBING.** Python `nba_api` rejected as a
|
||||
dependency (offline in prod). **ESPN site API** chosen — free, no auth, already
|
||||
the host for WNBA schedules/box/live. Verified live 2026-08-13.
|
||||
- **chainFn inputs: ALL PRESENT.** minutes SERVED; usage / possessions / pace /
|
||||
TS% / eFG% **DERIVED EXACTLY** from box components (`POSS = FGA−OREB+TOV+
|
||||
0.44·FTA`; the 0.44 free-throw-trip coefficient is the only estimated term).
|
||||
**"Derived" ≠ "proxied"** — these are the quantities themselves, recomputed.
|
||||
Game-state for the `redistribute` hook (starter, final_margin) SERVED.
|
||||
MISSING and named: shot location, on/off lineups, opponent DRtg (the last is
|
||||
derivable later from the same rows).
|
||||
- **POINT-IN-TIME IS NATIVE — no `statcast_history` twin needed.** Stores
|
||||
**PER-GAME** rows, not a season aggregate upserted in place, so an as-of
|
||||
profile is `WHERE game_date < asOf` over immutable completed box scores.
|
||||
**Strictly `<`** — a game ON the date may tip after grade time (`isPreGame`).
|
||||
This is the statcast lesson applied at design time instead of retrofitted.
|
||||
- **`wnba_player_game`** (migration 039), key `(game_id, source_id)`, registered
|
||||
in `tableKeys`. Every derived rate stored **beside its components** so it is
|
||||
re-derivable (`factorFreeze` rule for a feed). Scoped to the chainFn — not a
|
||||
stats dump (the `closing_captures` lesson).
|
||||
- **MEASURED, full live season:** 5,085 rows · 256 games · 241 players · 15 teams ·
|
||||
2026-05-01→08-12 · **0 fetch errors**. minutes/possessions/pace/starter/margin
|
||||
5085/5085; usage 5072/5085. ts_pct/efg_pct gaps are **correct refusals** (no
|
||||
attempts ⇒ no percentage), not missing data.
|
||||
- **TWO CONTAMINATING GAMES FOUND AND EXCLUDED:** an exhibition vs **Nigeria**
|
||||
(05-02) and the **All-Star game** (07-25). Filter reads **ESPN's own `/teams`**,
|
||||
not a hardcoded fifteen — this league has expanded twice in three years. An
|
||||
EMPTY teams response filters NOTHING (a failed feed must not masquerade as an
|
||||
honest absence — the `fielding_oaa` lesson). Concrete effect: Natasha Howard
|
||||
37→36 games, usage 21.897→22.057.
|
||||
- **AS-OF VERIFIED ON REAL DATA through the real function:** asOf 06-24 → 18
|
||||
games (usage 24.826), asOf 08-12 → 35 (22.057), asOf 12-31 → 36 (22.027) — the
|
||||
profile MOVES with the date. Same-day game EXCLUDED; thin history REFUSED
|
||||
(null, never a league-average stand-in). Usage is **minutes-weighted** (the
|
||||
lineup-K-rate lesson).
|
||||
- **⛔ NOT INGESTED TO THE DB.** `SUPABASE_DB_PASSWORD` fails pooler auth and the
|
||||
direct host is IPv6-only (unreachable from WSL2). Migration 039 written and
|
||||
UNAPPLIED; `ingestRange` written and has never written a row. All coverage
|
||||
above was measured by running the real parsers + real as-of read **in memory**
|
||||
over the live season — parse/derivation/as-of verified, **DB round-trip not**.
|
||||
Migration 038 also still outstanding.
|
||||
|
||||
## chain v3 (2026-08-12) — E[PA] fixed; artifact vs signal separated ✅
|
||||
Spec: `specs/chain-v1.md` §9. 4,882 tests / 376 suites green, web build exit 0.
|
||||
**Still SHADOW — served byte-identical, `CALIBRATION_DEPLOYED` `[]`, counter
|
||||
serves, A8 / hitsFactors / WNBA untouched.**
|
||||
- **THE ORDER WAS HALF RIGHT, and the right half was a real defect.** The v2
|
||||
shadow passed **no `lineupSlotFor`**, so every hitter ran on
|
||||
`DEFAULT_PA = 4.1` — "a regular" asserted about the leadoff man and the nine
|
||||
hole alike. `rate × opportunity` with opportunity held constant across the
|
||||
lineup. FIXED: `matchupKeys` now carries `batting_order` (one extra column on
|
||||
a read it already does — no new query); **posted slot on 491/553 props (88.8%)**.
|
||||
- **WRONG about the conversion:** `paDistribution` is a mean-preserving two-point
|
||||
mixture and `atLeast(Binomial(n,p),1)` **IS** `1−(1−p)^n` averaged over n — the
|
||||
order's proposed formula is what the code already computed. The defect was the
|
||||
INPUT E[PA], never the structure.
|
||||
- **WRONG about the direction, and this matters:** claimed "5.7pts BELOW on 92%".
|
||||
MEASURED **+4.5pp ABOVE**, below on **33.6%**. There was never a uniform low
|
||||
bias to collapse. A/B on identical rows: real E[PA] mean **+0.0450** / below
|
||||
33.6%; constant 4.1 **+0.0322** / below 39.6%. The fix moved the chain FURTHER
|
||||
above — correct, not a regression (real slots raise E[PA] for the top of the
|
||||
order, and top-of-order hitters dominate the prop board).
|
||||
- **THE DIAGNOSTIC — the answer is BOTH, not the order's either/or.** Over side,
|
||||
n=3,705, quintiles of opposing-pitcher K%:
|
||||
Q1 (soft) **+0.0749** → Q5 (hard) **+0.0256**; spread **+0.0494**,
|
||||
Pearson r = **−0.119**; `below %` **monotone across all five** (27.8 → 30.8 →
|
||||
34.1 → 35.0 → 38.6).
|
||||
- A **difficulty-correlated component** (~+0.049) — the chain conditions.
|
||||
- A **uniform positive offset** (~+0.026) surviving into the HARDEST quintile —
|
||||
does not move with the matchup, so it is not conditioning. Unexplained.
|
||||
Calling the whole result "signal" would bury it.
|
||||
- **THE CAVEAT THAT TRAVELS WITH IT:** the chain reads opposing-pitcher K%
|
||||
directly (log5 in `paOutcome`) and the counter reads nothing about the pitcher,
|
||||
so the correlation is close to mechanical — it proves the WIRING reaches the
|
||||
number, **not** that the adjustment is right. Correctness needs settled
|
||||
outcomes against the stored triple.
|
||||
- **Open item:** the ~+0.026 floor. Untested candidates — the chain is unclamped
|
||||
where the counter clamps `[0.10,0.95]` (chain min 0.016 vs counter 0.050);
|
||||
`LEAGUE.babip = 0.291`; the ±35% BABIP bound.
|
||||
|
||||
## chain v2 (2026-08-12) — the hand split plumbed; the premise corrected ✅
|
||||
Spec: `specs/chain-v1.md` §8. 4,877 tests / 376 suites green, web build exit 0.
|
||||
**Still SHADOW — served payload byte-identical, `CALIBRATION_DEPLOYED` still `[]`,
|
||||
counter still serves, A8 / hitsFactors / WNBA untouched.**
|
||||
- **THE ORDER'S PREMISE WAS WRONG, and checking it first was the work.** Claimed
|
||||
"paOutcome returned null on 596/725, 82% fell back to the seasonal rate".
|
||||
MEASURED: fire rate **9,752/9,792 (99.6%)**, sole refusal reason
|
||||
`no_batter_profile: 40`. `paOutcome`/`hitOnContact` **read no handedness at
|
||||
all** — a hand split cannot change whether they run. And **there is no fallback
|
||||
path**: a refusal DROPS the atom, it never becomes a season rate. (The "425/425"
|
||||
figure is from `7c8ef8b`, the A1–A7 deploy verification, not the chain.)
|
||||
- **The premise was right about what MATTERS, though:** the chain read a hitter's
|
||||
SEASON rates, which already average his platoon split over whatever hands he
|
||||
faced. That is a season read wearing a matchup read's clothes.
|
||||
- **The split now enters at the RATE** (`p_hit_per_pa × platoonRead`), then the
|
||||
binomial over PA — not at the output probability, which would scale a number
|
||||
already through the opportunity term. A test asserts the no-split path is
|
||||
**arithmetically identical to `projectSkill`** so the restructuring cannot
|
||||
become a second model.
|
||||
- **MEASURED: hand split fires on 288/553 unique props (52.1%)**, from 0. Of the
|
||||
265 that don't, **190 (72%) are principled refusals** — `insufficient_split_
|
||||
sample` 130 (the 60-PA floor), `switch_hitter` 60. Only `no_pitcher_hand` 71 is
|
||||
a plumbing gap, and it is the A5 shape: split present, pitcher hand absent.
|
||||
- **Divergence WIDENED**: median |div| 0.082 → **0.095**, over-side lean +2.9pp →
|
||||
**+4.5pp**. Fired rows disagree slightly LESS than refused rows (0.091 vs
|
||||
0.100) — **confounded, not an effect**: 60+PA-both-sides means an established
|
||||
regular, and the counter has more log on him too.
|
||||
- **Counting bug caught in the first draft:** the hand-split rate was counted per
|
||||
LEG while divergence was sliced per BLOCK (a prop's over+under share one
|
||||
block), reporting 48.1% and 55.1% for the same fact. One denominator now, taken
|
||||
from what is persisted.
|
||||
- **Second unreachable seam found:** `hitsFactorContext.build` /
|
||||
`matchupKeys.build` were gated on an inline `getSupabaseServiceClient()`, so the
|
||||
whole factor + hand-split path was untestable — "it is wired" could only rest on
|
||||
reading the code, which is how A5 shipped three factors that never fired. Both
|
||||
now take `deps.supabase ||`.
|
||||
|
||||
## chain v1 (2026-08-12) — the engine made whole, shadowed on MLB ✅
|
||||
Spec: `specs/chain-v1.md`. 4,859 tests / 376 suites green, web build exit 0.
|
||||
**SERVED BY NOTHING — served payload byte-identical, proven with the shadow on
|
||||
and off. `CALIBRATION_DEPLOYED` still `[]`. A8, hitsFactors and WNBA untouched.**
|
||||
- **THE CORE WAS A SHELL OF ITS OWN HEADER.** `chainFn` — the atom→probability
|
||||
slot — was described in the header and **absent from the code** (no parameter,
|
||||
no call site, no export). That is why the "portable core" was not portable:
|
||||
with no slot for that stage, MLB's real work lived in `scripts/`, called by no
|
||||
pipeline. Added, defaulting to identity-on-`p` so every prior caller is
|
||||
unchanged. An unreadable atom is DROPPED and COUNTED, never `p=0`.
|
||||
- **Correlation un-clamped to [-1,1], direction follows the sign.** `|corr|`
|
||||
interpolates from independence toward the **Fréchet bound its sign selects**
|
||||
(+1 → `min(p)`, −1 → `max(0, Σp−(n−1))`). Positive is arithmetically unchanged.
|
||||
Basketball's negative usage-competition case was previously *inexpressible*.
|
||||
- **`redistribute` now reaches chainAcross too.** Reaching only the team read
|
||||
left `selfCheck` comparing post- to pre-redistribution atoms and flagging an
|
||||
inconsistency the model had just manufactured. `prepareAtoms` is exported so
|
||||
both readings are built from identical legs.
|
||||
- **Baseball's chainFn** (`baseballChain.js`) routes `paOutcome` →
|
||||
Binomial(PA, p_hit) over `paDistribution`. Opportunity is a **LOOKUP**
|
||||
(`PA_BY_SLOT`, 4.65→3.85), not a fit — baseball's opportunity is fixed, which
|
||||
is why it is the clean first fill. Hits only; TB's verdict is already muddy.
|
||||
- **First real measurement** (`scripts/chain-shadow-probe.js`, 9,376 graded rows,
|
||||
08-07→08-12): chain fires on **99.6%**; median |divergence| **0.082**; 40.7%
|
||||
differ by ≥0.10, 10.3% by ≥0.20, 12.9% agree inside 0.02. The two-sided signed
|
||||
symmetry is **arithmetic** (both sides sampled); the over-side slice leans
|
||||
**+2.9pp, higher on 60.7%**. Divergence is not merit.
|
||||
- **The triple `(chain_p, counter_p, outcome)`** lands on `model_snapshots.
|
||||
chain_shadow` (migration 038), side-aligned. `selfCheck` is labelled
|
||||
`vacuous: true` — no independent team read exists, so agreement is arithmetic.
|
||||
- **Found + fixed:** `runSnapshot`'s `deps` was an allowlist of 17 keys while 14
|
||||
call sites read unlisted `deps.X` — every one permanently `undefined`, every
|
||||
documented injection seam a comment only. `...opts` spread first.
|
||||
|
||||
### ⛔ BLOCKING PRECONDITION — migration 038 BEFORE this deploys
|
||||
`retentionService.rowsFromSides` now declares `chain_shadow` on **every** row
|
||||
(it must — PostgREST builds a bulk insert from the FIRST row's shape, so a key
|
||||
present on only some rows is dropped for the whole batch). If the column does
|
||||
not exist, **every retention insert fails**, and retention is best-effort, so it
|
||||
fails SILENTLY — the exact "retention writes nothing" shape the S64 ops alarm
|
||||
exists to catch. Apply `supabase/migrations/038_chain_shadow.sql` first. Not
|
||||
applied yet: this branch is uncommitted, pending Roundtable review.
|
||||
|
||||
## Reclaim 2 (2026-08-12) — lock_lines dropped, writer retired ✅
|
||||
4,803 tests / 374 suites, web build exit 0, app healthy. **Moat untouched
|
||||
(`ledger_entries`, `model_snapshots`); grade path untouched.**
|
||||
|
||||
Reference in New Issue
Block a user