Reality assessment: the forward model exists, wired to the wrong side of the pipe

READ-ONLY. src/ and web/ untouched.

Inventoried every forward-model component against the real objective -- a
forward matchup projection, not an edge number. The finding is that all of it
already exists and is already loaded in production, and 100% of it sits
DOWNSTREAM of the grade in challenger columns nothing serves. The served p_win
reads three features and a game log; it has never seen a pitcher.

Inputs are HAVE, not missing: statcast_aggregates carries 1,354 rows (750
pitchers, 604 batters) with exit velo, launch angle, barrel, hard-hit, whiff,
chase, pitch mix, GB/FB, arm angle, and handedness complete on every row. Real
gaps are team defense and catcher/umpire. So Stage A is a plumbing-and-
modelling job, not a data-acquisition job.

Found along the way: the Statcast nightly refresh is unreachable code. tick()
returns for any hour not in HOURS_UTC (14,19,22,1,3) and the refresh block
then tests h === 11, which that guard can never admit. The mechanism data has
been frozen at its 2026-07-21 backfill for 13 days, and the block's own
failure alert sits in the same dead branch -- the identical silently-guarded-
out shape as the settlement outage.

Design shows the counter: every factor label the SIGNAL BREAKDOWN renders is a
restatement of recent frequency (l5_hot_vs_line, l20_over_line, back_to_back,
home_game) plus several structurally-NBA labels (referees, coach pace,
starters out) inside a baseball product. Not one names a pitcher, pitch type,
handedness or park. The card's forward-read slots already exist and go
unfilled -- the surface needs feeding, not redesigning.

On what changes: the prior measurements were outcome-accuracy, not edge, so
the metric was right and the question was narrow. proj-v1.1 and hits-v1 stay
correctly refuted as DISTRIBUTION swaps on thin inputs -- neither tested a
matchup-fed projection. arch-v1 is a market-relative nudge by construction and
is the one component genuinely measured on the wrong axis. AT CEILING is
provisional: measured only against features the champion already reads.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W1sivYNqY2TS5ftykmHBU9
This commit is contained in:
Kev
2026-08-03 00:53:56 -04:00
parent d8bf7765db
commit c551bf0340
3 changed files with 299 additions and 0 deletions
+29
View File
@@ -3,6 +3,35 @@
## Last Updated
2026-08-03
## Session 79 (2026-08-03) — Reality assessment vs the FORWARD-PROJECTION objective ✅
Spec: `specs/forward-model-reality-assessment.md`. READ-ONLY (src/web untouched).
- **Finding: the forward model's parts all EXIST and are all wired downstream of
the grade.** `probabilityEstimator` (the served p_win) reads 3 features + the
game log. Statcast/arsenal/park/weather/platoon/archetype load in
`snapshotService` AFTER grading, into challenger columns nothing serves.
`mlbContext` has zero consumers.
- **Statcast nightly refresh is DEAD CODE by guard** — tick() returns for hours
not in HOURS_UTC (14,19,22,1,3); the block tests h===11. Data frozen at
2026-07-21 (13 days stale); its own failure alert is in the same dead branch.
- **Inputs are HAVE** — 1,354 statcast rows, handedness complete both sides,
pitch mix/velo/break, GB/FB, barrel, exit velo, launch. MISSING: team defense
(OAA/DRS), catcher framing/umpire. PARTIAL: batter GB/FB (in `metrics` JSONB),
lineup slot (role tables 0 rows).
- **Design shows the COUNTER.** Factor labels are all `l5_hot_vs_line`-family
plus NBA leftovers (refs, coach pace). The card's forward-read slots
(archetypeBlend "Why this grade", vyndrIntel.matchup, propDNA) exist and go
unfilled. Needs feeding, not redesign.
- **STAGED DISTANCE:** Stage A (forward baseball model) = ONE real build, ZERO
data acquisitions — assemble hitter profile × pitcher stuff × conditions as the
SPINE with frequency demoted to a prior; risk is sample, not feasibility.
Stage B (calibrated + scouting surface) = short once A exists (clamp/calibration
already diagnosed + swap the factor vocabulary). Stage C (per sport) = blocked
on mechanism data we do not have for NBA/WNBA (ESPN is box scores, Python
service offline) and soccer is odds-api quota-blocked.
- **Verdict re-checks:** proj-v1.1 + hits-v1 stay refuted AS DISTRIBUTION SWAPS
(neither tested a matchup-fed projection); arch-v1 is market-relative by
construction = the one measured on the wrong axis; "AT CEILING" is provisional.
## Session 78 (2026-08-03) — Champion decomposed: the edge is a hit-rate counter ✅
Spec: `specs/champion-input-diagnosis.md`. READ-ONLY (src/web untouched);
4,159 tests green.
+38
View File
@@ -1224,6 +1224,44 @@ phased plan in the Session-57 conversation / BUILD-STATE Next section).
built for exactly this kind of replay was never settled, so ablations must join
outcomes from `ledger_entries` on (player_key, stat, line, side, game_date).
## Forward-model reality check (Session 79 — non-obvious)
- **`specs/forward-model-reality-assessment.md`.** THE OBJECTIVE is a FORWARD
matchup projection (hitter profile × pitcher stuff × park/conditions, read
through archetype), not a market-edge number. Every component that needs
exists AND is loaded in prod — and ALL of it sits DOWNSTREAM of the grade.
- **The served grade sees NONE of it.** `probabilityEstimator` reads exactly
three features (`opp_rank_stat`, `home_away`, `l10_stddev`/`l20_avg`) plus the
game log. statcast rows, arsenal, park, weather, platoon and archetype are all
loaded in `snapshotService` AFTER grading and written to CHALLENGER columns.
`mlbContext` (platoon/handedness) has ZERO consumers — dead code.
- **STATCAST NIGHTLY REFRESH IS UNREACHABLE CODE.** `snapshotScheduler.tick()`
returns at `if (!HOURS_UTC.includes(h)) return` (14,19,22,1,3); the statcast
block then tests `h === STATCAST_HOUR_UTC` (default **11**), which that guard
can never admit. Data frozen at its 2026-07-21 backfill; its failure alert is
inside the same dead branch so it can't warn. Same shape as the settlement
outage — guarded-out code that reports nothing. Set STATCAST_HOUR_UTC to one of
HOURS_UTC or move the block above the guard.
- **Inputs are HAVE, not missing** — `statcast_aggregates` 1,354 rows (750
pitchers / 604 batters): exit velo, launch, barrel, hard-hit, whiff, chase,
pitch_mix, GB/FB, arm angle, and **bats/throws complete on all 1,354**. Gaps
are team DEFENSE (only a coarse `opp_rank_stat`) and catcher framing/umpire.
PARTIAL: batter GB/FB land in the `metrics` JSONB not the typed columns;
lineup-slot tables (`player_role_profiles`, `lineup_role_profiles`) are 0 rows.
- **The card is designed for the forward read; the engine never fills it.** The
factor vocabulary the "SIGNAL BREAKDOWN" renders is entirely counter-restating
(`l5_hot_vs_line`, `l20_over_line`, `back_to_back`, `home_game`) with several
structurally-NBA labels (`ref_foul_high`, `coach_pace_delta`,
`opp_3plus_starters_out`). No signal names a pitcher, pitch type, handedness or
park. Surface needs FEEDING, not redesigning.
- **What the prior verdicts do and don't say.** Resolution = corr(forecast,
outcome) was never a market/edge test — the metric was right, the QUESTION was
narrow ("does challenger out-rank champion?"). proj-v1.1 and hits-v1 remain
correctly refuted AS DISTRIBUTION SWAPS on thin inputs; neither tested a
matchup-fed projection. arch-v1 IS market-relative by construction and is the
one component genuinely measured on the wrong axis — re-test its axes as
forward inputs. "AT CEILING" (runs/walks) is provisional: measured only against
features the champion already reads.
## Active Skills
- vyndr-voice (all user-facing output)
- prop-analysis (grading methodology)
+232
View File
@@ -0,0 +1,232 @@
# REALITY ASSESSMENT — what exists vs the FORWARD PROJECTION objective
**Read-only inventory, 2026-08-03.** Nothing built, nothing changed.
> **THE HEADLINE:** every component the forward model needs **already exists and
> is already loaded in production** — Statcast batter contact quality, pitcher
> stuff and handedness, park, weather, platoon, arsenal classification,
> archetypes. **All of it sits DOWNSTREAM of the grade**, feeding challenger
> columns that are never served. The thing the user actually sees is a hit-rate
> counter that has never seen a pitcher.
>
> The distance to Stage A is **not data acquisition. It is re-plumbing** — plus
> one broken cron and one unreachable code branch found along the way.
---
## 0. First, a correction — because it changes what needs re-checking
The order says "we have measured EDGE for five sessions — the WRONG test."
**That is not what the measurements were, and the distinction matters.**
Resolution as measured = `corr(forecast, outcome)` — *do higher forecasts hit
more often*. That is outcome accuracy at the listed line. It never referenced the
market. The market-relative objects are separate and few: `proj_book_implied`,
`fair_prob`, CLV/dCLV, and **arch-v1, which is by construction a "market-relative
p_win nudge."**
So the metric was right. **What was wrong was the QUESTION.** Every session asked
*"does challenger X out-rank champion Y?"* and never *"can a forward matchup read
be produced at all, and does it call listed lines better than base rate?"* Those
have different answers, because:
- **Resolution parity ≠ product parity.** A frequency counter and a forward model
could both resolve ~0.20 on hits, and only one of them can say *why*.
- Ranking is not the same as **accuracy above base rate**, which is what "accurate
listed-line calls" means. That was only measured once, at the very end of the
last session (the calibration table), and it failed: **+3.5pt global
over-prediction**.
That correction narrows what has to be re-checked (§4) rather than voiding it.
## 1. Component inventory — SPINE / NUDGE / DISCONNECTED (STEP 1)
**The served grade is `gradeSlateService → analyzeViaEngine1 → probabilityEstimator`.
`p_win` reads exactly three features:** `opp_rank_stat`, `home_away`,
`l10_stddev`/`l20_avg`. Everything below is measured against that path.
| component | exists? | what it computes today | role in the SERVED grade |
|---|---|---|---|
| **base + recency frequency** | yes | frequency of (stat > THIS line), 0.6/0.4 with last-5 | **THE SPINE — ~100% of resolution** (measured, `specs/champion-input-diagnosis.md`) |
| **archetypeService** (41 archetypes) | yes | full feature-scoring classifier, primary/secondary/blend | **DISCONNECTED.** Attached at *snapshot* time for display only; `analyzeViaEngine1` explicitly does not set it |
| **matchupRead** (arsenal vs hitter) | yes | sinker/breaking mix, whiff & hard-hit tendency → multiplier | **DISCONNECTED from the grade.** Only inside proj-v1.1, a challenger measured *worse* |
| **park** (`parkBase`, `parkFactors`) | yes | per-venue factors, exposure-relative baseline | **NUDGE, in challengers only** (arch-v1 env axis, proj-v1.1) |
| **weather** (`environmentContext`) | yes | temp / wind / precip → multiplier | **NUDGE, in challengers only** |
| **platoon** (`platoonSplits`, `mlbContext`) | yes | handedness advantage | **DISCONNECTED — `mlbContext` has ZERO consumers.** Dead code |
| **opportunity_drift** | yes | recent AB ÷ season AB/game | **NUDGE** (arch-v1 opportunity axis). The one feature with repeated residual signal — and the axis extracts none of it |
| **statcastAggregateService** | yes | 1,354 rows of real mechanism data | **DISCONNECTED from grading.** Required only by `snapshotScheduler` (writer) and `routes/internal` |
| **contact quality** (contact-v1) | yes | season contact profile | **NUDGE** (challenger column, n=1,055, inconclusive) |
| **pitcher stuff / tendencies** | yes | pitch mix, velo, break, arm angle, GB/FB, whiff | **DISCONNECTED.** Loaded per snapshot as `arsenalById`, used only by proj-v1.1 |
**Not one forward-model component is in the spine. The spine is the counter.**
## 2. Baseball input coverage — the real distance (STEP 2)
`statcast_aggregates`: **1,354 rows** (750 pitchers, 604 batters), real Baseball
Savant, free and unlimited.
| input the forward model needs | status | evidence |
|---|---|---|
| hitter exit velo (avg/max) | **HAVE** | 600/604 batters |
| hitter launch angle | **HAVE** | 600/604 |
| hitter barrel% / hard-hit% | **HAVE** | 600/604 |
| hitter whiff / chase / K / BB | **HAVE** | 602603/604 |
| **hitter batted-ball type (GB/FB/LD)** | **PARTIAL** | `gb_pct`/`fb_pct` are **0/604 for batters** (populated for pitchers only). The batted-ball feed carries `fb_ld_pct`/`gb_pct_bb` — they land in the `metrics` JSONB, not the typed columns |
| **handedness (bats / throws)** | **HAVE** | 1,354/1,354 — both sides, complete |
| pitcher pitch mix + velo + break | **HAVE** | `pitch_mix` 729/750 |
| pitcher GB/FB/LD | **HAVE** | 713/750 |
| pitcher whiff / chase / barrel & hard-hit allowed | **HAVE** | 713/750 |
| pitcher arm angle | **HAVE** | 713/750 |
| park factors | **HAVE** | `parkBase` / `parkFactors`, per venue |
| weather (temp/wind/precip) | **HAVE** | `wx_forecast` on ledger rows; 5,855 feature rows |
| probable pitcher for tonight | **HAVE** | `probablePitchers``mlbStatsAdapter.getScheduleWithPitchers` |
| lineup slot / opportunity | **PARTIAL** | `opportunity_drift` + `ab_per_game` are proxies; `depthChartService` exists but `lineup_role_profiles` / `player_role_profiles` tables are **0 rows** |
| **team defense (OAA / DRS / range)** | **MISSING** | only `opp_rank_stat`, a coarse 0..1 team rank |
| **catcher framing, umpire zone** | **MISSING** | the `ref_*` features are NBA officiating, not MLB umpires |
| **batter-vs-pitcher history** | PARTIAL | `getBatterVsPitcher` exists in the adapter; not in the grade path |
**Verdict on STEP 2: the inputs are overwhelmingly HAVE.** The forward model is
not blocked on data acquisition. Two real gaps (team defense, catcher/umpire) and
two partials (batter batted-ball columns, lineup slot) — none of which block a
first forward model.
### But the mechanism data is 13 days stale, and here is why
`statcast_aggregates.updated_at` max = **2026-07-21**. Today is 2026-08-03.
**The nightly refresh is unreachable code.** In `src/snapshotScheduler.js`:
```
152 const tick = async () => {
158 if (!HOURS_UTC.includes(h)) return; // HOURS_UTC = 14,19,22,1,3
...
323 const statcastHour = Number(process.env.STATCAST_HOUR_UTC || 11);
324 if (h === statcastHour && process.env.STATCAST !== '0') {
```
Line 324 is only reachable when `h ∈ {14,19,22,1,3}`; it tests `h === 11`. **That
branch can never execute at the default.** The data has sat at its Session-68
backfill ever since. Its failure alert (line 330) is inside the same dead branch,
so it has never been able to warn either — the same *silently guarded-out* shape
as the settlement outage. (Caveat: if `STATCAST_HOUR_UTC` were set in prod to one
of the cron hours it would run; the 13-day staleness says it is not.)
## 3. Does the design reflect the forward read? (STEP 3)
**The card has the slots. The engine never fills the matchup ones.**
`GradeResultCard` already supports `archetypeBlend` (with a literal "Why this
grade:" caption), `propDNA` (reliable/volatile), `statContext`, `vyndrIntel`
(form/usage/**matchup**/rest), `signals`, `killConditions`. The surface was
designed for a scouting report.
**What actually reaches it** — the complete factor-label vocabulary the "SIGNAL
BREAKDOWN · N FACTORS" section renders:
```
l5_hot_vs_line l5_cold_vs_line l20_over_line l20_under_line
l20_contradicts_over/under l5_under_friendly consistency_boom_bust
back_to_back rested_2plus heavy_workload_7d home_game
top_opponent_defense weak_opponent_defense away_vs_top5_defense
ref_foul_high/low ref_pace_adjustment coach_pace_delta
opp_2_starters_out opp_3plus_starters_out rookie/veteran_in_playoffs
```
**Every one is a restatement of the counter** — he's been hot against this
number, he's rested, he's home. **Not one names a pitcher, a pitch type, a
handedness edge, a park, or a batted-ball profile.** Several are structurally NBA
(referees, coach pace, starters out, playoffs) inside a baseball product.
**Design verdict: the product communicates the COUNTER, with the counter's own
inputs relabelled as "signals."** The "why" is currently "because he's been doing
it," which is the one thing a forward projection is supposed to replace. The good
news is that the surface does not need redesigning — it needs feeding.
## 4. What prior work changes (STEP 4)
**KEEP — serves the forward model regardless of objective:**
| work | why it survives |
|---|---|
| Settlement fix (S77) | the measurement floor. Without it nothing is testable at all |
| Clamp + calibration diagnosis (S78) | 20.6% of props pinned to a constant, +3.5pt over-prediction. A forward model inherits this bug unless it is fixed — it is in the *output* stage, not the model |
| Champion decomposition (S78) | tells us precisely what the counter contributes (~all of it from base+recency) — that is the **baseline any forward model must beat** |
| Takeability / book-identity work | market-structure hygiene, objective-independent |
| Ledger + retention infrastructure | the proving ground |
| `opportunity_drift` residual finding | a genuine forward input, wrongly implemented |
**RE-CHECK — verdicts that answered the narrow question:**
| verdict | why it needs re-checking |
|---|---|
| **arch-v1 "inconclusive"** | it is explicitly a **market-relative** nudge — the one component genuinely measured on the wrong axis. Its axes (env/matchup/opportunity) should be re-tested as **forward inputs to a projection**, not as nudges to a market-relative probability |
| **proj-v1.1 "reliably worse"** | valid as measured, but it was a *distribution* swap on the same thin inputs (season rate + bounded multiplier). It does not establish that a **matchup-fed** projection is worse — that was never built |
| **hits-v1 "refuted"** | same. It changed the family, not the information. The refutation stands for *that* model and says nothing about a stuff-vs-profile read |
| **"AT CEILING" on runs/walks** | provisional. The ceiling was measured against *features the champion already reads*. It cannot rule out inputs never wired in |
| **Archetype "unmeasurable"** | unchanged — 2 of 41 labels have testable n. Still the honest answer |
**Nothing gets thrown away. Two verdicts (proj-v1.1, hits-v1) are correctly
scoped to what they tested and simply do not answer the forward question.**
## 5. STAGED DISTANCE TO PRODUCT (STEP 5)
### Stage A — a forward baseball model that beats the counter on listed-line accuracy
**BUILT (more than expected):** all five Statcast feeds ingested and typed;
handedness complete on both sides; arsenal classification (`matchupRead`) written
and unit-tested; park + weather + platoon multipliers written; probable pitchers
resolved per game; archetype classifier complete; the settled ledger that proves
or kills it; a champion baseline decomposed to the number.
**NEEDED:**
1. Fix the Statcast cron (one guard) — a forward model on 13-day-old mechanism
data is not a forward model.
2. **The actual new thing: a projection whose SPINE is `hitter profile × pitcher
stuff × park/conditions`, with frequency demoted to a prior.** Every input
exists; none is assembled this way. This is the real build.
3. Wire it into the *grade path* (`analyzeViaEngine1`), not a challenger column —
though it should ride as a challenger first to be proven.
4. Fill the two partials: batter GB/FB into typed columns, lineup slot.
**Honest distance: this is the one genuinely substantial build, and it is a
plumbing-and-modelling job, not a data job.** Main risk is not feasibility but
**sample**: 1,741 settled rows, ~578 on the biggest stat, is thin for validating a
richer model per stat. Expect the verdict to be n-blocked for weeks after it
ships — which is an argument for shipping it soon, not for waiting.
### Stage B — calibrated + productized (the scouting-report surface)
**BUILT:** the entire card (`archetypeBlend` + "Why this grade", `propDNA`,
`statContext`, `vyndrIntel.matchup`, signals, kill conditions), tier gating, the
row grammar, the honesty layer.
**NEEDED:** (a) the clamp/calibration fix — already diagnosed, no new data;
(b) matchup-derived factor labels to replace the `l5_hot_vs_line` vocabulary;
(c) archetype attached at grade time (currently refused there on I/O-cost grounds
that a forward model changes). **Distance: short once Stage A exists** — this is
mostly deleting the counter's vocabulary and passing through what Stage A already
computes.
### Stage C — same method per sport
**BUILT:** the per-sport doctrine, the sport-agnostic Bayesian scaffolding, WNBA
settling end-to-end.
**NEEDED:** a mechanism feed per sport equivalent to Statcast. **This does not
exist for NBA/WNBA in our stack** — `espnStatsAdapter` is box scores, and the
Python `nba_api` service is offline in production. Soccer is quota-blocked on
odds-api (478/500). **Distance: materially further than A or B, and gated on data
we do not currently have.** Baseball is the only sport where the forward model is
buildable today — which is an argument for making it excellent in one sport
first, exactly as the per-sport doctrine says.
---
## 6. The one-sentence answer to "how far"
**Stage A is one real build away and zero data acquisitions away**; Stage B is
mostly wiring plus an already-diagnosed calibration fix; **Stage C is blocked on
per-sport mechanism data that does not exist in the stack yet.** The reason five
sessions of challengers failed is now legible and it is not the modelling — it is
that every one of them was attached *downstream of the grade*, nudging a counter
instead of replacing it.