report: model architecture recovery map — the live grade uses 0 of 3 specced layers
Archaeology only; nothing built, reconnected, or promoted. The champion is two DISCONNECTED estimates: the letter is engine1's additive factor index (zero references to p_win or any probability in engine1.js), and p_win is probabilityEstimator's frequencyOver + 5 heuristic layers, computed after and merely attached. The live grade path never calls the Python service. The Python three-layer engine is NOT DEPLOYED — no python/pip in the Dockerfile; app.js only health-checks it. So Layers 1-2 never shipped. Layer 3 is wired BACKWARDS: grade_thresholds.json maps PROBABILITY->GRADE and the live JS reads it in reverse to manufacture confidence from an already-chosen letter. Per-sport market-efficiency scaling is specced-absent. Consequence stated plainly: every metric audited to date is on the shadow model, not the specced engine, which has never been measured. Sport boundary TESTED not asserted: a new sport on the live path is a ~10-file core edit with four documented silent-failure modes. Per-sport records DO exist (sports.mlb n=526/62% vs pooled overall n=937/58%, each n>=20 gated), but /api/accuracy ignores ?sport= and the pooled overall would absorb a new sport. Park x weather confirmed challenger-only; xwOBA and leash absent. Recovery map is dependency-ordered with MLB as the reference module. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QJs13VsyiSKYQP6rj3NNmc
This commit is contained in:
@@ -399,6 +399,70 @@
|
||||
> (renders the band IN USER-VISIBLE COPY — "Takeable band −160 to +200", so a floor change is a copy
|
||||
> change too).**
|
||||
|
||||
> ## 🗺️ MODEL ARCHITECTURE RECOVERY MAP 2026-07-30 (report-only) → **`specs/model-architecture-recovery-map.md`**
|
||||
> Archaeology of the specced three-layer engine vs the live grade path. Nothing built,
|
||||
> reconnected, promoted, or changed. **HEADLINE: the live grade uses 0 of the 3 specced layers —
|
||||
> every metric this project has audited (calibration, CLV, the skew audit, the takeable floor, the
|
||||
> champion's p_win→CLV r=0.375) is measured on the SHADOW model, not the specced engine.** Those
|
||||
> findings stand (the shadow model served every real grade) but none of them are evidence about the
|
||||
> specced architecture, which has never been measured.
|
||||
> **0.1 THE CHAMPION = two DISCONNECTED estimates.** The letter is `engine1.gradeProp`:
|
||||
> `idx = NEUTRAL_INDEX(3); for (f of factors) idx += f.delta` — and **`engine1.js` has ZERO
|
||||
> references to `p_win` or any probability** (grep count 0). The probability is
|
||||
> `probabilityEstimator` (`frequencyOver` + 5 heuristic layers, clamp [.10,.95]) computed AFTER and
|
||||
> merely attached. **p_win never feeds the letter; the letter never feeds p_win.** The live grade path
|
||||
> makes **no call to the Python service at all**.
|
||||
> **🔴 THE PYTHON ENGINE IS NOT DEPLOYED.** `Dockerfile` has no python/pip/requirements — the image is
|
||||
> Node-only; `app.js:92-95` only health-CHECKS it and reports `down`. Layers 1-2 are not "bypassed at
|
||||
> runtime", they have **never shipped**. `grade_thresholds.json` carries `"shadow_mode": true`.
|
||||
> **LAYER STATE:** L1 Similarity `python/utils/similarity.py` (101 ln, MIN_SIMILARITY .7, NBA+MLB
|
||||
> factor weights) **BUILT·NOT WIRED·NOT DEPLOYED** — the live grade skips straight to season/recent
|
||||
> averages. L2 Bayesian `python/utils/bayesian.py` (320 ln) **BUILT·NOT WIRED·NOT DEPLOYED** — and the
|
||||
> spec's "sport-agnostic math, per-sport parameters" claim is **TRUE of the built code**
|
||||
> (`BAYESIAN_WEIGHTS` prior/recent/context for 11 stat types across both sports, skewness,
|
||||
> data-sufficiency degradation, ±0.15 calibration clamp, a calibration disclaimer). L3
|
||||
> **BUILT·WIRED BACKWARDS** — `grade_thresholds.json` maps **PROBABILITY→GRADE** (A+ = p .85-1.00);
|
||||
> the live JS reads it in REVERSE, picking a letter from the factor index then looking up that
|
||||
> letter's band MIDPOINT to manufacture `confidence`. **That one artifact IS the drift, provable in a
|
||||
> single file.** Per-sport **market-efficiency scaling = SPECCED-BUT-ABSENT** (one global scale; the
|
||||
> spec's MLB .55 / NBA-stars .80 exists nowhere; the only `efficiency` in code is *usage*-efficiency).
|
||||
> **⚠️ The spec's "min 15 instances" is NOT in the code — CANNOT DETERMINE its origin** (code's
|
||||
> abstention is `similar_games_below: 3`; the only 15 is NBA `min_minutes_per_game`; the founding
|
||||
> three-layer spec text is not in the repo). **NOTE: STATE.md's own "Layer 1/2/3" references are the
|
||||
> Session 73-77 ARCHETYPE build train — a different numbering. Do not conflate them.**
|
||||
> **🔴 SPORT BOUNDARY — TESTED, NOT ASSERTED: it is NOT clean on the live path.** Adding a sport to the
|
||||
> Python spine costs nothing (the spine is dead); adding one to the LIVE path is a **CORE EDIT across
|
||||
> ~10+ files** — proplineAdapter.MARKETS · oddsNormalizer.MARKET_MAP (miss → silent ZERO) ·
|
||||
> oddsService.SPORT_KEYS/SPORT_MARKETS · **three stat_type whitelists** (analyze.js, scan.js,
|
||||
> validation.py — miss one → silent 400s) · featureCache log-field · outcomeService MLB_LOG_FIELD +
|
||||
> SPORTS · liveTrackingService LIVE_BOX_FIELD · a projection-emitting extractor (**without it every
|
||||
> prop REFUSES**) · emptyState/statFilters/teams · and a settled feed or the sport never settles.
|
||||
> Making "a sport" a real module is itself a prerequisite build.
|
||||
> **PER-SPORT RECORDS EXIST (2.7 satisfied):** `getModelAggregate` filters on `opts.sport`
|
||||
> (`ledgerService:809,819`) and `/api/accuracy` carries a real breakdown — live **`sports.mlb` n=526
|
||||
> 62%** alongside **`overall` n=937 58%**, each with its own `min_sample:20`. Two caveats: the pooled
|
||||
> `overall` would absorb a new sport's rows, and **`/api/accuracy` IGNORES the `?sport=` query param**
|
||||
> (identical payload for mlb vs wnba — consumers must read `sports.{sport}`).
|
||||
> **PARK×WEATHER CONFIRMED CHALLENGER, NOT CHAMPION** — attached in `snapshotService` (S77) onto
|
||||
> `env_*`/`challenger_*` columns; `ledgerService.js:253` says it outright ("measured, never served"),
|
||||
> and proj-v1.1, which consumes park/weather/platoon/arsenal, came back **NOT PROVEN** on 2026-07-29.
|
||||
> MLB champion inputs are ONLY l5/l20 + opp_rank + rest + usage. **xwOBA = 0 files (absent); leash = 0
|
||||
> (absent);** barrel/hard-hit/launch-angle/exit-velo/statcast/platoon all present but challenger-only.
|
||||
> **RECOVERY ORDER (dependency-sequenced, full detail in the map):** 0 decide the grading BASIS
|
||||
> (probability-vs-factor-index — a founder call; note that moving to probability grading legitimately
|
||||
> changes the letter distribution because it is NEW INFORMATION, categorically unlike rescaling
|
||||
> thresholds to mint A's) → 1 pick a runtime (**recommend PORTING `bayesian.py` to Node** rather than
|
||||
> resurrecting a service every live path already routes around) → 2 wire Layer 2 FIRST (its
|
||||
> prior/recent/context inputs are already available from `featureCache.getStatRows`, no new I/O) →
|
||||
> 3 reconnect Layer 3 FORWARD (grade FROM probability; kills the `confidence` tautology) → 4 Layer 1
|
||||
> Similarity (now feasible — S73-77 produce the context store it needs) → 5 per-sport efficiency
|
||||
> scaling → 6 challenger promotion on ledger evidence only → 7 make "a sport" a real module →
|
||||
> 8 NFL/CFB. **MLB = the REFERENCE MODULE** (deepest stack + a working settled feed + its own 526-row
|
||||
> record); the map lists the exact contract a new sport must declare.
|
||||
> **HONEST PER-SPORT LABELING: mechanism EXISTS, rule must be enforced** — every future sport renders
|
||||
> `sports.{sport}` and reads ACCRUING until its own n≥20 clears; never `overall`, which would silently
|
||||
> borrow MLB/WNBA credibility.
|
||||
|
||||
- **Redirect EXISTS + WIRED:** `closingCapture.buildCaptureRows`→`closing_captures` (append-only,
|
||||
provenance: captured_at/book/line_type/both-prices/missed_reason) via `intradayRefreshService:221`
|
||||
+ internal endpoint; `ledgerService.attachClosingProb`→`closing_prob` (de-vigs both raw sides,
|
||||
|
||||
Reference in New Issue
Block a user