494c83cf76
in code PHASE 0 — getStatRows is the single base-rate path, so every branch is audited, plus the feature builders since l20_avg is the season reference projectionFor reads: getStatRows MLB -> estimator base fullLog CORRECT (929fd81) mlbGameLogFeatures l5/l10/l20 last10 = 10 DEFECTIVE espnStatsAdapter.parseGameLog slice(0,20) DEFECTIVE getStatRows NBA/WNBA ESPN branch inherits 20-cap DEFECTIVE via source getStatRows NBA/WNBA python branch getGameLogs(...,20) dormant (offline) pitcherEngine / skillProjection statcast profiles N/A pitcher props via getStatRows MLB fullLog CORRECT settleSource full log (S64) CORRECT THE PITCHER ANSWER IS GOOD NEWS: pitcher props run through the same getStatRows MLB branch, so929fd81repaired them too. There is no separate defective pitcher base-rate path. THE ONE HIDING IN PLAIN SIGHT: mlbGameLogFeatures carries the comment "l20 = all available (the season per-game reference projectionFor needs)" while building from last10 -- so l20_avg was a TEN-GAME AVERAGE WEARING A SEASON LABEL, feeding both the consistency pull inside the estimator and projectionFor, which decides refusals. It survived the previous repair because that fix touched only getStatRows. PHASE 1 — mlbGameLogFeatures now reads fullLog; espnStatsAdapter drops its slice(0,20) cap. ZERO new API calls on both: each widens data already fetched and then discarded, the same shape as the original repair. The python branch is left alone -- the service is offline in prod and fixing it would be speculative. Their before/after resolution is NOT measured, deliberately: the only way to measure today is to reconstruct the repaired forecast over old rows, which is the reconstruction-vs-served trap this order refuses. Code fix now, measurement at accrual. PHASE 2 — MODEL_VERSION bumped to engine1@2026-08-07-fullwindow, so every forward snapshot is self-identifying (retentionService already stamps it; no new plumbing). model/reAuditEligibility.js encodes the rule: isEligible accepts only the repaired marker, assess counts eligible DATES not rows, and ACCRUAL is frozen at calibration 10 / hits-lift 10 / verdict-reaudit 14 / rbi-gate 14. A test locks the invisible case -- a MIXED table of 330 rows with 30 repaired returns eligible_dates 3, not 330 rows of false confidence. Once both generations share a table a naive count would fit a map on a blend of two forecasters. PHASE 3 — the board, each consequence labelled: calibration WITHDRAWN (refits at 10 dates, never on reconstructions); factor verdicts SUSPECT (all measured against a champion worse than a frequency table, direction UNKNOWN, not pre-priced, 14 dates); hits factor lift UN-REMEASURABLE (10 dates, factors still wired and transmitting); rbi lineup-slot RE-QUEUED (14 dates). Pre-registered order: calibration, hits lift, verdict re-audit, rbi gate. Then STOP and accrue. Nothing further can be honestly measured until the board fills with rows the repaired champion produced. Serving-path changes by design for the MLB feature path and NBA/WNBA logs; eleven frozen model modules verified unchanged. p_win never mutated. No Bonferroni slot. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W1sivYNqY2TS5ftykmHBU9
129 lines
6.0 KiB
Markdown
129 lines
6.0 KiB
Markdown
# The window-bug class, hunted — three more paths, and the forward re-audit rule
|
|
|
|
## PHASE 0 — the audit
|
|
|
|
The defect class: **a fixed short window used AS the season/base rate.**
|
|
`getStatRows` is the single path feeding `meta.gameLogs`, which is where
|
|
`estimateProbability` derives its base rate, so every branch of it is a base-rate
|
|
path. The feature builders are the second surface, because `l20_avg` is the
|
|
season reference `projectionFor` reads.
|
|
|
|
| path | window | classification |
|
|
|---|---|---|
|
|
| `getStatRows` MLB → estimator base rate | `fullLog` | **CORRECT** (fixed 929fd81) |
|
|
| `mlbGameLogFeatures` → `l5/l10/l20_avg`, `l10_stddev` | `last10` = **10** | **DEFECTIVE** |
|
|
| `espnStatsAdapter.parseGameLog` → NBA/WNBA logs | `rows.slice(0, 20)` = **20** | **DEFECTIVE** |
|
|
| `getStatRows` NBA/WNBA ESPN branch | inherits the 20-cap | **DEFECTIVE (via source)** |
|
|
| `getStatRows` NBA/WNBA Python branch | `getGameLogs(..., 20)` | DEFECTIVE-but-dormant (service offline in prod) |
|
|
| **pitcher engine** (`pitcherEngine`, `skillProjection`) | reads statcast **profiles**, no game log | **N/A** |
|
|
| pitcher props (strikeouts) via `getStatRows` MLB | `fullLog` | **CORRECT** — fixed by the same change |
|
|
| `settleSource` | already reads the full log (S64) | CORRECT |
|
|
| `playerIntelService`, `streaksService` | display/streak surfaces, not forecasts | N/A |
|
|
|
|
**The pitcher answer matters and is good news:** pitcher props run through the
|
|
same `getStatRows` MLB branch, so `929fd81` repaired them too — there is no
|
|
separate defective pitcher base-rate path.
|
|
|
|
### The one that was hiding in plain sight
|
|
|
|
`mlbGameLogFeatures` carries this comment:
|
|
|
|
> `l20 = all available (the season per-game reference projectionFor needs)`
|
|
|
|
Built from `last10`, **`l20_avg` was a ten-game average wearing a season label** —
|
|
and it feeds both the consistency (cv) pull inside the estimator and
|
|
`projectionFor`, which decides refusals. Same class as the base-rate bug, same
|
|
file, and it survived the previous repair because that fix touched only
|
|
`getStatRows`.
|
|
|
|
---
|
|
|
|
## PHASE 1 — fixes
|
|
|
|
| path | fix | API cost |
|
|
|---|---|---|
|
|
| `mlbGameLogFeatures` | read `fullLog`, fall back to `last10` | **ZERO** — same response |
|
|
| `espnStatsAdapter.parseGameLog` | drop the `slice(0, 20)` cap | **ZERO** — same payload, already parsed |
|
|
| NBA/WNBA Python branch | left as-is | service offline in prod; fixing it would be speculative |
|
|
|
|
**No new API calls anywhere.** Both fixes widen data that was already fetched and
|
|
then discarded — the same shape as the original repair.
|
|
|
|
### Measurement status, stated honestly
|
|
|
|
These are serving changes for the MLB feature path and the NBA/WNBA log path.
|
|
**Their before/after resolution is NOT measured here**, and deliberately: the
|
|
only way to measure it today would be to reconstruct the repaired forecast over
|
|
old rows, which is the reconstruction-vs-served trap this order explicitly
|
|
refuses. They ship as code fixes with the measurement deferred to accrual, which
|
|
is the honest sequencing.
|
|
|
|
---
|
|
|
|
## PHASE 2 — the forward re-audit rule, in code
|
|
|
|
`MODEL_VERSION` is bumped to **`engine1@2026-08-07-fullwindow`**, so every
|
|
snapshot from this commit forward is self-identifying. `retentionService` already
|
|
stamps it onto `model_snapshots`, so no new plumbing was needed.
|
|
|
|
`model/reAuditEligibility.js` encodes the rule:
|
|
|
|
- **`isEligible(row)`** — true only for rows carrying the repaired marker.
|
|
- **`assess(rows)`** — counts eligible **DATES**, not rows, because dates have
|
|
been the binding scarcity in every interval this session.
|
|
- **`ACCRUAL`** (frozen) — pre-stated minimum dates per measurement:
|
|
|
|
| measurement | minimum eligible dates |
|
|
|---|---|
|
|
| calibration re-fit | 10 |
|
|
| hits factor lift | 10 |
|
|
| prior verdict re-audit | 14 |
|
|
| rbi lineup-slot gate | 14 |
|
|
|
|
A test locks the case that would otherwise be invisible: **a MIXED table** of 330
|
|
rows where only 30 carry the new marker returns `eligible_dates: 3`, not 330
|
|
rows' worth of false confidence. Once both generations sit in the same table, a
|
|
naive count would happily fit a map on a blend of two different forecasters.
|
|
|
|
---
|
|
|
|
## PHASE 3 — the honest board
|
|
|
|
**What happened:** the champion computed its season rate over ten games. Found by
|
|
resolution decomposition, not by a test failing. Fixed in two lines. It no longer
|
|
*loses* to a frequency table — it **beats** it CI-confirmed only on total_bases,
|
|
**ties** on rbi and runs, and leads on the hits point estimate.
|
|
|
|
**Consequences, each labelled:**
|
|
|
|
- **CALIBRATION — WITHDRAWN.** `CALIBRATION_DEPLOYED` is empty. Maps were fitted
|
|
on the retired forecast. Re-fits on repaired-champion settled rows. *Waiting on
|
|
accrual: 10 dates.* Not to be refit on reconstructions.
|
|
- **FACTOR VERDICTS — SUSPECT.** Every prior null and every THEATER was measured
|
|
against a champion worse than a frequency table; signal added to noise reads as
|
|
noise. Re-audit on accrued rows. **Direction UNKNOWN** — some may pass, some
|
|
may still fail. Not pre-priced. *Waiting: 14 dates.*
|
|
- **HITS FACTOR LIFT (1.39%) — UN-REMEASURABLE.** Needs rows produced *by* the
|
|
repaired champion. *Waiting: 10 dates.* The factors remain wired and
|
|
transmitting (43f65d3); only the lift number is unquantified.
|
|
- **RBI LINEUP-SLOT — RE-QUEUED.** Lands after the champion is sound and rows
|
|
accrue. *Waiting: 14 dates.*
|
|
|
|
**Pre-registered re-audit order** (each runs only when its bar is met):
|
|
1. Re-fit calibration on repaired-champion rows (10 dates)
|
|
2. Re-measure hits factor lift (10 dates)
|
|
3. Re-audit prior factor verdicts (14 dates)
|
|
4. Run rbi lineup-slot through the two-part gate (14 dates)
|
|
|
|
**Then STOP and accrue.** Nothing further can be honestly measured until the
|
|
board fills with rows the repaired champion produced.
|
|
|
|
---
|
|
|
|
## Invariants
|
|
|
|
No measurement on reconstructions — hard refusal, and it is why Phase 1 ships
|
|
code without numbers. Serving-path changes by design for the MLB feature path and
|
|
NBA/WNBA logs; frozen model modules verified unchanged. `p_win` never mutated. No
|
|
Bonferroni slot — base-rate repair and a bug hunt, not causal factors.
|