report: model architecture recovery map — the live grade uses 0 of 3 specced layers

Archaeology only; nothing built, reconnected, or promoted.

The champion is two DISCONNECTED estimates: the letter is engine1's additive
factor index (zero references to p_win or any probability in engine1.js), and
p_win is probabilityEstimator's frequencyOver + 5 heuristic layers, computed
after and merely attached. The live grade path never calls the Python service.

The Python three-layer engine is NOT DEPLOYED — no python/pip in the
Dockerfile; app.js only health-checks it. So Layers 1-2 never shipped.
Layer 3 is wired BACKWARDS: grade_thresholds.json maps PROBABILITY->GRADE and
the live JS reads it in reverse to manufacture confidence from an
already-chosen letter. Per-sport market-efficiency scaling is specced-absent.

Consequence stated plainly: every metric audited to date is on the shadow
model, not the specced engine, which has never been measured.

Sport boundary TESTED not asserted: a new sport on the live path is a ~10-file
core edit with four documented silent-failure modes. Per-sport records DO
exist (sports.mlb n=526/62% vs pooled overall n=937/58%, each n>=20 gated),
but /api/accuracy ignores ?sport= and the pooled overall would absorb a new
sport. Park x weather confirmed challenger-only; xwOBA and leash absent.

Recovery map is dependency-ordered with MLB as the reference module.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QJs13VsyiSKYQP6rj3NNmc
This commit is contained in:
Kev
2026-07-30 23:16:34 -04:00
parent f3bf300b36
commit dd98b0b614
3 changed files with 320 additions and 0 deletions
+64
View File
@@ -399,6 +399,70 @@
> (renders the band IN USER-VISIBLE COPY — "Takeable band 160 to +200", so a floor change is a copy
> change too).**
> ## 🗺️ MODEL ARCHITECTURE RECOVERY MAP 2026-07-30 (report-only) → **`specs/model-architecture-recovery-map.md`**
> Archaeology of the specced three-layer engine vs the live grade path. Nothing built,
> reconnected, promoted, or changed. **HEADLINE: the live grade uses 0 of the 3 specced layers —
> every metric this project has audited (calibration, CLV, the skew audit, the takeable floor, the
> champion's p_win→CLV r=0.375) is measured on the SHADOW model, not the specced engine.** Those
> findings stand (the shadow model served every real grade) but none of them are evidence about the
> specced architecture, which has never been measured.
> **0.1 THE CHAMPION = two DISCONNECTED estimates.** The letter is `engine1.gradeProp`:
> `idx = NEUTRAL_INDEX(3); for (f of factors) idx += f.delta` — and **`engine1.js` has ZERO
> references to `p_win` or any probability** (grep count 0). The probability is
> `probabilityEstimator` (`frequencyOver` + 5 heuristic layers, clamp [.10,.95]) computed AFTER and
> merely attached. **p_win never feeds the letter; the letter never feeds p_win.** The live grade path
> makes **no call to the Python service at all**.
> **🔴 THE PYTHON ENGINE IS NOT DEPLOYED.** `Dockerfile` has no python/pip/requirements — the image is
> Node-only; `app.js:92-95` only health-CHECKS it and reports `down`. Layers 1-2 are not "bypassed at
> runtime", they have **never shipped**. `grade_thresholds.json` carries `"shadow_mode": true`.
> **LAYER STATE:** L1 Similarity `python/utils/similarity.py` (101 ln, MIN_SIMILARITY .7, NBA+MLB
> factor weights) **BUILT·NOT WIRED·NOT DEPLOYED** — the live grade skips straight to season/recent
> averages. L2 Bayesian `python/utils/bayesian.py` (320 ln) **BUILT·NOT WIRED·NOT DEPLOYED** — and the
> spec's "sport-agnostic math, per-sport parameters" claim is **TRUE of the built code**
> (`BAYESIAN_WEIGHTS` prior/recent/context for 11 stat types across both sports, skewness,
> data-sufficiency degradation, ±0.15 calibration clamp, a calibration disclaimer). L3
> **BUILT·WIRED BACKWARDS** — `grade_thresholds.json` maps **PROBABILITY→GRADE** (A+ = p .85-1.00);
> the live JS reads it in REVERSE, picking a letter from the factor index then looking up that
> letter's band MIDPOINT to manufacture `confidence`. **That one artifact IS the drift, provable in a
> single file.** Per-sport **market-efficiency scaling = SPECCED-BUT-ABSENT** (one global scale; the
> spec's MLB .55 / NBA-stars .80 exists nowhere; the only `efficiency` in code is *usage*-efficiency).
> **⚠️ The spec's "min 15 instances" is NOT in the code — CANNOT DETERMINE its origin** (code's
> abstention is `similar_games_below: 3`; the only 15 is NBA `min_minutes_per_game`; the founding
> three-layer spec text is not in the repo). **NOTE: STATE.md's own "Layer 1/2/3" references are the
> Session 73-77 ARCHETYPE build train — a different numbering. Do not conflate them.**
> **🔴 SPORT BOUNDARY — TESTED, NOT ASSERTED: it is NOT clean on the live path.** Adding a sport to the
> Python spine costs nothing (the spine is dead); adding one to the LIVE path is a **CORE EDIT across
> ~10+ files** — proplineAdapter.MARKETS · oddsNormalizer.MARKET_MAP (miss → silent ZERO) ·
> oddsService.SPORT_KEYS/SPORT_MARKETS · **three stat_type whitelists** (analyze.js, scan.js,
> validation.py — miss one → silent 400s) · featureCache log-field · outcomeService MLB_LOG_FIELD +
> SPORTS · liveTrackingService LIVE_BOX_FIELD · a projection-emitting extractor (**without it every
> prop REFUSES**) · emptyState/statFilters/teams · and a settled feed or the sport never settles.
> Making "a sport" a real module is itself a prerequisite build.
> **PER-SPORT RECORDS EXIST (2.7 satisfied):** `getModelAggregate` filters on `opts.sport`
> (`ledgerService:809,819`) and `/api/accuracy` carries a real breakdown — live **`sports.mlb` n=526
> 62%** alongside **`overall` n=937 58%**, each with its own `min_sample:20`. Two caveats: the pooled
> `overall` would absorb a new sport's rows, and **`/api/accuracy` IGNORES the `?sport=` query param**
> (identical payload for mlb vs wnba — consumers must read `sports.{sport}`).
> **PARK×WEATHER CONFIRMED CHALLENGER, NOT CHAMPION** — attached in `snapshotService` (S77) onto
> `env_*`/`challenger_*` columns; `ledgerService.js:253` says it outright ("measured, never served"),
> and proj-v1.1, which consumes park/weather/platoon/arsenal, came back **NOT PROVEN** on 2026-07-29.
> MLB champion inputs are ONLY l5/l20 + opp_rank + rest + usage. **xwOBA = 0 files (absent); leash = 0
> (absent);** barrel/hard-hit/launch-angle/exit-velo/statcast/platoon all present but challenger-only.
> **RECOVERY ORDER (dependency-sequenced, full detail in the map):** 0 decide the grading BASIS
> (probability-vs-factor-index — a founder call; note that moving to probability grading legitimately
> changes the letter distribution because it is NEW INFORMATION, categorically unlike rescaling
> thresholds to mint A's) → 1 pick a runtime (**recommend PORTING `bayesian.py` to Node** rather than
> resurrecting a service every live path already routes around) → 2 wire Layer 2 FIRST (its
> prior/recent/context inputs are already available from `featureCache.getStatRows`, no new I/O) →
> 3 reconnect Layer 3 FORWARD (grade FROM probability; kills the `confidence` tautology) → 4 Layer 1
> Similarity (now feasible — S73-77 produce the context store it needs) → 5 per-sport efficiency
> scaling → 6 challenger promotion on ledger evidence only → 7 make "a sport" a real module →
> 8 NFL/CFB. **MLB = the REFERENCE MODULE** (deepest stack + a working settled feed + its own 526-row
> record); the map lists the exact contract a new sport must declare.
> **HONEST PER-SPORT LABELING: mechanism EXISTS, rule must be enforced** — every future sport renders
> `sports.{sport}` and reads ACCRUING until its own n≥20 clears; never `overall`, which would silently
> borrow MLB/WNBA credibility.
- **Redirect EXISTS + WIRED:** `closingCapture.buildCaptureRows``closing_captures` (append-only,
provenance: captured_at/book/line_type/both-prices/missed_reason) via `intradayRefreshService:221`
+ internal endpoint; `ledgerService.attachClosingProb``closing_prob` (de-vigs both raw sides,