Reality assessment: the forward model exists, wired to the wrong side of the pipe

READ-ONLY. src/ and web/ untouched.

Inventoried every forward-model component against the real objective -- a
forward matchup projection, not an edge number. The finding is that all of it
already exists and is already loaded in production, and 100% of it sits
DOWNSTREAM of the grade in challenger columns nothing serves. The served p_win
reads three features and a game log; it has never seen a pitcher.

Inputs are HAVE, not missing: statcast_aggregates carries 1,354 rows (750
pitchers, 604 batters) with exit velo, launch angle, barrel, hard-hit, whiff,
chase, pitch mix, GB/FB, arm angle, and handedness complete on every row. Real
gaps are team defense and catcher/umpire. So Stage A is a plumbing-and-
modelling job, not a data-acquisition job.

Found along the way: the Statcast nightly refresh is unreachable code. tick()
returns for any hour not in HOURS_UTC (14,19,22,1,3) and the refresh block
then tests h === 11, which that guard can never admit. The mechanism data has
been frozen at its 2026-07-21 backfill for 13 days, and the block's own
failure alert sits in the same dead branch -- the identical silently-guarded-
out shape as the settlement outage.

Design shows the counter: every factor label the SIGNAL BREAKDOWN renders is a
restatement of recent frequency (l5_hot_vs_line, l20_over_line, back_to_back,
home_game) plus several structurally-NBA labels (referees, coach pace,
starters out) inside a baseball product. Not one names a pitcher, pitch type,
handedness or park. The card's forward-read slots already exist and go
unfilled -- the surface needs feeding, not redesigning.

On what changes: the prior measurements were outcome-accuracy, not edge, so
the metric was right and the question was narrow. proj-v1.1 and hits-v1 stay
correctly refuted as DISTRIBUTION swaps on thin inputs -- neither tested a
matchup-fed projection. arch-v1 is a market-relative nudge by construction and
is the one component genuinely measured on the wrong axis. AT CEILING is
provisional: measured only against features the champion already reads.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W1sivYNqY2TS5ftykmHBU9
This commit is contained in:
Kev
2026-08-03 00:53:56 -04:00
parent d8bf7765db
commit c551bf0340
3 changed files with 299 additions and 0 deletions
+29
View File
@@ -3,6 +3,35 @@
## Last Updated ## Last Updated
2026-08-03 2026-08-03
## Session 79 (2026-08-03) — Reality assessment vs the FORWARD-PROJECTION objective ✅
Spec: `specs/forward-model-reality-assessment.md`. READ-ONLY (src/web untouched).
- **Finding: the forward model's parts all EXIST and are all wired downstream of
the grade.** `probabilityEstimator` (the served p_win) reads 3 features + the
game log. Statcast/arsenal/park/weather/platoon/archetype load in
`snapshotService` AFTER grading, into challenger columns nothing serves.
`mlbContext` has zero consumers.
- **Statcast nightly refresh is DEAD CODE by guard** — tick() returns for hours
not in HOURS_UTC (14,19,22,1,3); the block tests h===11. Data frozen at
2026-07-21 (13 days stale); its own failure alert is in the same dead branch.
- **Inputs are HAVE** — 1,354 statcast rows, handedness complete both sides,
pitch mix/velo/break, GB/FB, barrel, exit velo, launch. MISSING: team defense
(OAA/DRS), catcher framing/umpire. PARTIAL: batter GB/FB (in `metrics` JSONB),
lineup slot (role tables 0 rows).
- **Design shows the COUNTER.** Factor labels are all `l5_hot_vs_line`-family
plus NBA leftovers (refs, coach pace). The card's forward-read slots
(archetypeBlend "Why this grade", vyndrIntel.matchup, propDNA) exist and go
unfilled. Needs feeding, not redesign.
- **STAGED DISTANCE:** Stage A (forward baseball model) = ONE real build, ZERO
data acquisitions — assemble hitter profile × pitcher stuff × conditions as the
SPINE with frequency demoted to a prior; risk is sample, not feasibility.
Stage B (calibrated + scouting surface) = short once A exists (clamp/calibration
already diagnosed + swap the factor vocabulary). Stage C (per sport) = blocked
on mechanism data we do not have for NBA/WNBA (ESPN is box scores, Python
service offline) and soccer is odds-api quota-blocked.
- **Verdict re-checks:** proj-v1.1 + hits-v1 stay refuted AS DISTRIBUTION SWAPS
(neither tested a matchup-fed projection); arch-v1 is market-relative by
construction = the one measured on the wrong axis; "AT CEILING" is provisional.
## Session 78 (2026-08-03) — Champion decomposed: the edge is a hit-rate counter ✅ ## Session 78 (2026-08-03) — Champion decomposed: the edge is a hit-rate counter ✅
Spec: `specs/champion-input-diagnosis.md`. READ-ONLY (src/web untouched); Spec: `specs/champion-input-diagnosis.md`. READ-ONLY (src/web untouched);
4,159 tests green. 4,159 tests green.
+38
View File
@@ -1224,6 +1224,44 @@ phased plan in the Session-57 conversation / BUILD-STATE Next section).
built for exactly this kind of replay was never settled, so ablations must join built for exactly this kind of replay was never settled, so ablations must join
outcomes from `ledger_entries` on (player_key, stat, line, side, game_date). outcomes from `ledger_entries` on (player_key, stat, line, side, game_date).
## Forward-model reality check (Session 79 — non-obvious)
- **`specs/forward-model-reality-assessment.md`.** THE OBJECTIVE is a FORWARD
matchup projection (hitter profile × pitcher stuff × park/conditions, read
through archetype), not a market-edge number. Every component that needs
exists AND is loaded in prod — and ALL of it sits DOWNSTREAM of the grade.
- **The served grade sees NONE of it.** `probabilityEstimator` reads exactly
three features (`opp_rank_stat`, `home_away`, `l10_stddev`/`l20_avg`) plus the
game log. statcast rows, arsenal, park, weather, platoon and archetype are all
loaded in `snapshotService` AFTER grading and written to CHALLENGER columns.
`mlbContext` (platoon/handedness) has ZERO consumers — dead code.
- **STATCAST NIGHTLY REFRESH IS UNREACHABLE CODE.** `snapshotScheduler.tick()`
returns at `if (!HOURS_UTC.includes(h)) return` (14,19,22,1,3); the statcast
block then tests `h === STATCAST_HOUR_UTC` (default **11**), which that guard
can never admit. Data frozen at its 2026-07-21 backfill; its failure alert is
inside the same dead branch so it can't warn. Same shape as the settlement
outage — guarded-out code that reports nothing. Set STATCAST_HOUR_UTC to one of
HOURS_UTC or move the block above the guard.
- **Inputs are HAVE, not missing** — `statcast_aggregates` 1,354 rows (750
pitchers / 604 batters): exit velo, launch, barrel, hard-hit, whiff, chase,
pitch_mix, GB/FB, arm angle, and **bats/throws complete on all 1,354**. Gaps
are team DEFENSE (only a coarse `opp_rank_stat`) and catcher framing/umpire.
PARTIAL: batter GB/FB land in the `metrics` JSONB not the typed columns;
lineup-slot tables (`player_role_profiles`, `lineup_role_profiles`) are 0 rows.
- **The card is designed for the forward read; the engine never fills it.** The
factor vocabulary the "SIGNAL BREAKDOWN" renders is entirely counter-restating
(`l5_hot_vs_line`, `l20_over_line`, `back_to_back`, `home_game`) with several
structurally-NBA labels (`ref_foul_high`, `coach_pace_delta`,
`opp_3plus_starters_out`). No signal names a pitcher, pitch type, handedness or
park. Surface needs FEEDING, not redesigning.
- **What the prior verdicts do and don't say.** Resolution = corr(forecast,
outcome) was never a market/edge test — the metric was right, the QUESTION was
narrow ("does challenger out-rank champion?"). proj-v1.1 and hits-v1 remain
correctly refuted AS DISTRIBUTION SWAPS on thin inputs; neither tested a
matchup-fed projection. arch-v1 IS market-relative by construction and is the
one component genuinely measured on the wrong axis — re-test its axes as
forward inputs. "AT CEILING" (runs/walks) is provisional: measured only against
features the champion already reads.
## Active Skills ## Active Skills
- vyndr-voice (all user-facing output) - vyndr-voice (all user-facing output)
- prop-analysis (grading methodology) - prop-analysis (grading methodology)
+232
View File
@@ -0,0 +1,232 @@
# REALITY ASSESSMENT — what exists vs the FORWARD PROJECTION objective
**Read-only inventory, 2026-08-03.** Nothing built, nothing changed.
> **THE HEADLINE:** every component the forward model needs **already exists and
> is already loaded in production** — Statcast batter contact quality, pitcher
> stuff and handedness, park, weather, platoon, arsenal classification,
> archetypes. **All of it sits DOWNSTREAM of the grade**, feeding challenger
> columns that are never served. The thing the user actually sees is a hit-rate
> counter that has never seen a pitcher.
>
> The distance to Stage A is **not data acquisition. It is re-plumbing** — plus
> one broken cron and one unreachable code branch found along the way.
---
## 0. First, a correction — because it changes what needs re-checking
The order says "we have measured EDGE for five sessions — the WRONG test."
**That is not what the measurements were, and the distinction matters.**
Resolution as measured = `corr(forecast, outcome)` — *do higher forecasts hit
more often*. That is outcome accuracy at the listed line. It never referenced the
market. The market-relative objects are separate and few: `proj_book_implied`,
`fair_prob`, CLV/dCLV, and **arch-v1, which is by construction a "market-relative
p_win nudge."**
So the metric was right. **What was wrong was the QUESTION.** Every session asked
*"does challenger X out-rank champion Y?"* and never *"can a forward matchup read
be produced at all, and does it call listed lines better than base rate?"* Those
have different answers, because:
- **Resolution parity ≠ product parity.** A frequency counter and a forward model
could both resolve ~0.20 on hits, and only one of them can say *why*.
- Ranking is not the same as **accuracy above base rate**, which is what "accurate
listed-line calls" means. That was only measured once, at the very end of the
last session (the calibration table), and it failed: **+3.5pt global
over-prediction**.
That correction narrows what has to be re-checked (§4) rather than voiding it.
## 1. Component inventory — SPINE / NUDGE / DISCONNECTED (STEP 1)
**The served grade is `gradeSlateService → analyzeViaEngine1 → probabilityEstimator`.
`p_win` reads exactly three features:** `opp_rank_stat`, `home_away`,
`l10_stddev`/`l20_avg`. Everything below is measured against that path.
| component | exists? | what it computes today | role in the SERVED grade |
|---|---|---|---|
| **base + recency frequency** | yes | frequency of (stat > THIS line), 0.6/0.4 with last-5 | **THE SPINE — ~100% of resolution** (measured, `specs/champion-input-diagnosis.md`) |
| **archetypeService** (41 archetypes) | yes | full feature-scoring classifier, primary/secondary/blend | **DISCONNECTED.** Attached at *snapshot* time for display only; `analyzeViaEngine1` explicitly does not set it |
| **matchupRead** (arsenal vs hitter) | yes | sinker/breaking mix, whiff & hard-hit tendency → multiplier | **DISCONNECTED from the grade.** Only inside proj-v1.1, a challenger measured *worse* |
| **park** (`parkBase`, `parkFactors`) | yes | per-venue factors, exposure-relative baseline | **NUDGE, in challengers only** (arch-v1 env axis, proj-v1.1) |
| **weather** (`environmentContext`) | yes | temp / wind / precip → multiplier | **NUDGE, in challengers only** |
| **platoon** (`platoonSplits`, `mlbContext`) | yes | handedness advantage | **DISCONNECTED — `mlbContext` has ZERO consumers.** Dead code |
| **opportunity_drift** | yes | recent AB ÷ season AB/game | **NUDGE** (arch-v1 opportunity axis). The one feature with repeated residual signal — and the axis extracts none of it |
| **statcastAggregateService** | yes | 1,354 rows of real mechanism data | **DISCONNECTED from grading.** Required only by `snapshotScheduler` (writer) and `routes/internal` |
| **contact quality** (contact-v1) | yes | season contact profile | **NUDGE** (challenger column, n=1,055, inconclusive) |
| **pitcher stuff / tendencies** | yes | pitch mix, velo, break, arm angle, GB/FB, whiff | **DISCONNECTED.** Loaded per snapshot as `arsenalById`, used only by proj-v1.1 |
**Not one forward-model component is in the spine. The spine is the counter.**
## 2. Baseball input coverage — the real distance (STEP 2)
`statcast_aggregates`: **1,354 rows** (750 pitchers, 604 batters), real Baseball
Savant, free and unlimited.
| input the forward model needs | status | evidence |
|---|---|---|
| hitter exit velo (avg/max) | **HAVE** | 600/604 batters |
| hitter launch angle | **HAVE** | 600/604 |
| hitter barrel% / hard-hit% | **HAVE** | 600/604 |
| hitter whiff / chase / K / BB | **HAVE** | 602603/604 |
| **hitter batted-ball type (GB/FB/LD)** | **PARTIAL** | `gb_pct`/`fb_pct` are **0/604 for batters** (populated for pitchers only). The batted-ball feed carries `fb_ld_pct`/`gb_pct_bb` — they land in the `metrics` JSONB, not the typed columns |
| **handedness (bats / throws)** | **HAVE** | 1,354/1,354 — both sides, complete |
| pitcher pitch mix + velo + break | **HAVE** | `pitch_mix` 729/750 |
| pitcher GB/FB/LD | **HAVE** | 713/750 |
| pitcher whiff / chase / barrel & hard-hit allowed | **HAVE** | 713/750 |
| pitcher arm angle | **HAVE** | 713/750 |
| park factors | **HAVE** | `parkBase` / `parkFactors`, per venue |
| weather (temp/wind/precip) | **HAVE** | `wx_forecast` on ledger rows; 5,855 feature rows |
| probable pitcher for tonight | **HAVE** | `probablePitchers``mlbStatsAdapter.getScheduleWithPitchers` |
| lineup slot / opportunity | **PARTIAL** | `opportunity_drift` + `ab_per_game` are proxies; `depthChartService` exists but `lineup_role_profiles` / `player_role_profiles` tables are **0 rows** |
| **team defense (OAA / DRS / range)** | **MISSING** | only `opp_rank_stat`, a coarse 0..1 team rank |
| **catcher framing, umpire zone** | **MISSING** | the `ref_*` features are NBA officiating, not MLB umpires |
| **batter-vs-pitcher history** | PARTIAL | `getBatterVsPitcher` exists in the adapter; not in the grade path |
**Verdict on STEP 2: the inputs are overwhelmingly HAVE.** The forward model is
not blocked on data acquisition. Two real gaps (team defense, catcher/umpire) and
two partials (batter batted-ball columns, lineup slot) — none of which block a
first forward model.
### But the mechanism data is 13 days stale, and here is why
`statcast_aggregates.updated_at` max = **2026-07-21**. Today is 2026-08-03.
**The nightly refresh is unreachable code.** In `src/snapshotScheduler.js`:
```
152 const tick = async () => {
158 if (!HOURS_UTC.includes(h)) return; // HOURS_UTC = 14,19,22,1,3
...
323 const statcastHour = Number(process.env.STATCAST_HOUR_UTC || 11);
324 if (h === statcastHour && process.env.STATCAST !== '0') {
```
Line 324 is only reachable when `h ∈ {14,19,22,1,3}`; it tests `h === 11`. **That
branch can never execute at the default.** The data has sat at its Session-68
backfill ever since. Its failure alert (line 330) is inside the same dead branch,
so it has never been able to warn either — the same *silently guarded-out* shape
as the settlement outage. (Caveat: if `STATCAST_HOUR_UTC` were set in prod to one
of the cron hours it would run; the 13-day staleness says it is not.)
## 3. Does the design reflect the forward read? (STEP 3)
**The card has the slots. The engine never fills the matchup ones.**
`GradeResultCard` already supports `archetypeBlend` (with a literal "Why this
grade:" caption), `propDNA` (reliable/volatile), `statContext`, `vyndrIntel`
(form/usage/**matchup**/rest), `signals`, `killConditions`. The surface was
designed for a scouting report.
**What actually reaches it** — the complete factor-label vocabulary the "SIGNAL
BREAKDOWN · N FACTORS" section renders:
```
l5_hot_vs_line l5_cold_vs_line l20_over_line l20_under_line
l20_contradicts_over/under l5_under_friendly consistency_boom_bust
back_to_back rested_2plus heavy_workload_7d home_game
top_opponent_defense weak_opponent_defense away_vs_top5_defense
ref_foul_high/low ref_pace_adjustment coach_pace_delta
opp_2_starters_out opp_3plus_starters_out rookie/veteran_in_playoffs
```
**Every one is a restatement of the counter** — he's been hot against this
number, he's rested, he's home. **Not one names a pitcher, a pitch type, a
handedness edge, a park, or a batted-ball profile.** Several are structurally NBA
(referees, coach pace, starters out, playoffs) inside a baseball product.
**Design verdict: the product communicates the COUNTER, with the counter's own
inputs relabelled as "signals."** The "why" is currently "because he's been doing
it," which is the one thing a forward projection is supposed to replace. The good
news is that the surface does not need redesigning — it needs feeding.
## 4. What prior work changes (STEP 4)
**KEEP — serves the forward model regardless of objective:**
| work | why it survives |
|---|---|
| Settlement fix (S77) | the measurement floor. Without it nothing is testable at all |
| Clamp + calibration diagnosis (S78) | 20.6% of props pinned to a constant, +3.5pt over-prediction. A forward model inherits this bug unless it is fixed — it is in the *output* stage, not the model |
| Champion decomposition (S78) | tells us precisely what the counter contributes (~all of it from base+recency) — that is the **baseline any forward model must beat** |
| Takeability / book-identity work | market-structure hygiene, objective-independent |
| Ledger + retention infrastructure | the proving ground |
| `opportunity_drift` residual finding | a genuine forward input, wrongly implemented |
**RE-CHECK — verdicts that answered the narrow question:**
| verdict | why it needs re-checking |
|---|---|
| **arch-v1 "inconclusive"** | it is explicitly a **market-relative** nudge — the one component genuinely measured on the wrong axis. Its axes (env/matchup/opportunity) should be re-tested as **forward inputs to a projection**, not as nudges to a market-relative probability |
| **proj-v1.1 "reliably worse"** | valid as measured, but it was a *distribution* swap on the same thin inputs (season rate + bounded multiplier). It does not establish that a **matchup-fed** projection is worse — that was never built |
| **hits-v1 "refuted"** | same. It changed the family, not the information. The refutation stands for *that* model and says nothing about a stuff-vs-profile read |
| **"AT CEILING" on runs/walks** | provisional. The ceiling was measured against *features the champion already reads*. It cannot rule out inputs never wired in |
| **Archetype "unmeasurable"** | unchanged — 2 of 41 labels have testable n. Still the honest answer |
**Nothing gets thrown away. Two verdicts (proj-v1.1, hits-v1) are correctly
scoped to what they tested and simply do not answer the forward question.**
## 5. STAGED DISTANCE TO PRODUCT (STEP 5)
### Stage A — a forward baseball model that beats the counter on listed-line accuracy
**BUILT (more than expected):** all five Statcast feeds ingested and typed;
handedness complete on both sides; arsenal classification (`matchupRead`) written
and unit-tested; park + weather + platoon multipliers written; probable pitchers
resolved per game; archetype classifier complete; the settled ledger that proves
or kills it; a champion baseline decomposed to the number.
**NEEDED:**
1. Fix the Statcast cron (one guard) — a forward model on 13-day-old mechanism
data is not a forward model.
2. **The actual new thing: a projection whose SPINE is `hitter profile × pitcher
stuff × park/conditions`, with frequency demoted to a prior.** Every input
exists; none is assembled this way. This is the real build.
3. Wire it into the *grade path* (`analyzeViaEngine1`), not a challenger column —
though it should ride as a challenger first to be proven.
4. Fill the two partials: batter GB/FB into typed columns, lineup slot.
**Honest distance: this is the one genuinely substantial build, and it is a
plumbing-and-modelling job, not a data job.** Main risk is not feasibility but
**sample**: 1,741 settled rows, ~578 on the biggest stat, is thin for validating a
richer model per stat. Expect the verdict to be n-blocked for weeks after it
ships — which is an argument for shipping it soon, not for waiting.
### Stage B — calibrated + productized (the scouting-report surface)
**BUILT:** the entire card (`archetypeBlend` + "Why this grade", `propDNA`,
`statContext`, `vyndrIntel.matchup`, signals, kill conditions), tier gating, the
row grammar, the honesty layer.
**NEEDED:** (a) the clamp/calibration fix — already diagnosed, no new data;
(b) matchup-derived factor labels to replace the `l5_hot_vs_line` vocabulary;
(c) archetype attached at grade time (currently refused there on I/O-cost grounds
that a forward model changes). **Distance: short once Stage A exists** — this is
mostly deleting the counter's vocabulary and passing through what Stage A already
computes.
### Stage C — same method per sport
**BUILT:** the per-sport doctrine, the sport-agnostic Bayesian scaffolding, WNBA
settling end-to-end.
**NEEDED:** a mechanism feed per sport equivalent to Statcast. **This does not
exist for NBA/WNBA in our stack** — `espnStatsAdapter` is box scores, and the
Python `nba_api` service is offline in production. Soccer is quota-blocked on
odds-api (478/500). **Distance: materially further than A or B, and gated on data
we do not currently have.** Baseball is the only sport where the forward model is
buildable today — which is an argument for making it excellent in one sport
first, exactly as the per-sport doctrine says.
---
## 6. The one-sentence answer to "how far"
**Stage A is one real build away and zero data acquisitions away**; Stage B is
mostly wiring plus an already-diagnosed calibration fix; **Stage C is blocked on
per-sport mechanism data that does not exist in the stack yet.** The reason five
sessions of challengers failed is now legible and it is not the modelling — it is
that every one of them was attached *downstream of the grade*, nudging a counter
instead of replacing it.