Files
vyndr/specs/model-architecture-recovery-map.md
builtbykev dd98b0b614 report: model architecture recovery map — the live grade uses 0 of 3 specced layers
Archaeology only; nothing built, reconnected, or promoted.

The champion is two DISCONNECTED estimates: the letter is engine1's additive
factor index (zero references to p_win or any probability in engine1.js), and
p_win is probabilityEstimator's frequencyOver + 5 heuristic layers, computed
after and merely attached. The live grade path never calls the Python service.

The Python three-layer engine is NOT DEPLOYED — no python/pip in the
Dockerfile; app.js only health-checks it. So Layers 1-2 never shipped.
Layer 3 is wired BACKWARDS: grade_thresholds.json maps PROBABILITY->GRADE and
the live JS reads it in reverse to manufacture confidence from an
already-chosen letter. Per-sport market-efficiency scaling is specced-absent.

Consequence stated plainly: every metric audited to date is on the shadow
model, not the specced engine, which has never been measured.

Sport boundary TESTED not asserted: a new sport on the live path is a ~10-file
core edit with four documented silent-failure modes. Per-sport records DO
exist (sports.mlb n=526/62% vs pooled overall n=937/58%, each n>=20 gated),
but /api/accuracy ignores ?sport= and the pooled overall would absorb a new
sport. Park x weather confirmed challenger-only; xwOBA and leash absent.

Recovery map is dependency-ordered with MLB as the reference module.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QJs13VsyiSKYQP6rj3NNmc
2026-07-30 23:16:34 -04:00

14 KiB
Raw Permalink Blame History

VYNDR — MODEL ARCHITECTURE RECOVERY MAP

Report-only archaeology, 2026-07-30. Nothing was built, reconnected, promoted, or changed. This is the map every subsequent reconnection order and every new-sport module builds against.


HEADLINE — the live grade uses approximately NONE of the specced three-layer engine

Everything this project has audited — calibration, CLV, the over-side skew audit, the takeable floor, the champion's p_win→CLV r=0.375 — is measured on the SHADOW model, not on the specced engine. That does not invalidate those findings (the shadow model is what actually served every grade), but it does mean none of them are evidence about the specced architecture, and the specced engine has never been measured at all.


REVIEW ZERO

0.1 What the live champion actually is — VERIFIED

Two disconnected estimates, neither Bayesian nor similarity-based:

The GRADE LETTERengine1.gradeProp (src/services/intelligence/engine1.js:170-174):

let idx = NEUTRAL_INDEX;              // 3 === 'C'
for (const f of factors) idx += f.delta;
idx = clampIndex(Math.round(idx));
return { grade: GRADE_SCALE[idx], confidence: GRADE_TO_CONFIDENCE[GRADE_SCALE[idx]] ?? 0.25 };

An additive integer index over flat ±1.0/±0.5 factor deltas. engine1.js contains ZERO references to p_win or any probability (verified: grep count 0). The letter is computed with no probability input of any kind.

The PROBABILITYprobabilityEstimator.estimateProbability: empirical frequencyOver(values, line) plus five heuristic adjustment layers (last-5 weighted 2x, opponent bump/fade, home/away ±1.5%, volatility pull toward 0.50), clamped [0.10, 0.95]. Computed AFTER the grade and merely attached to the payload.

So: p_win never feeds the letter, and the letter never feeds p_win. They are two independent estimates of the same event that are free to disagree — which is exactly why confidence (a band-midpoint lookup of the already-chosen letter) carries zero information.

It bypasses the three-layer engine completely. Verified: the live grade path (gradeSlateServiceanalyzeViaEngine1engine1 / featureCache) contains no call to the Python service at all.

0.2 Component inventory — every specced piece, tagged

Component File State
Layer 1 Similarity python/utils/similarity.py (101 ln) BUILT · NOT WIRED · NOT DEPLOYED
Layer 2 Bayesian python/utils/bayesian.py (320 ln) BUILT · NOT WIRED · NOT DEPLOYED
Layer 3 grade scale python/data/grade_thresholds.json BUILT · WIRED BACKWARDS (see §3)
Market-efficiency scaling SPECCED-BUT-ABSENT
Shared archetypes python/utils/archetypes.py BUILT · superseded by JS archetypeService
Sport blueprints python/blueprints/mlb.py, nba_context.py BUILT · NOT WIRED · NOT DEPLOYED
Live grade engine intelligence/engine1.js BUILT · WIRED · IS the champion
Live probability intelligence/probabilityEstimator.js BUILT · WIRED · not fed into the grade

🔴 THE PYTHON ENGINE IS NOT DEPLOYED AT ALL. Dockerfile contains no python, pip, or requirements — the deploy image is Node-only. app.js:92-95 merely health-CHECKS PYTHON_SERVICE_URL and reports python: 'down'. Layers 1 and 2 are not "bypassed at runtime"; they have never shipped. grade_thresholds.json even carries "shadow_mode": true.


PHASE 1 — THE THREE LAYERS, ACTUAL STATE

Layer 1 — Similarity: BUILT, NOT WIRED

MIN_SIMILARITY = 0.7; SIMILARITY_FACTORS carries NBA factors (opp DRtg .15, pace .12, rest .08, home/away .06, role match .15, teammate context .10) and MLB factors (pitcher handedness .12, park .10, opponent quality .12, weather .05, day/night .04, batting order .06). The live grade never performs a similarity search — it goes straight to season/recent averages (projectionFor: l5_avg ?? l20_avg ?? {stat}_per_90 ?? xg_per_90).

⚠️ The spec's "min 15 instances" is NOT in the code — CANNOT DETERMINE its origin. The only abstention rule is ABSTENTION_RULES.similar_games_below = 3 (bayesian.py:46); the only 15 is nba_player.min_minutes_per_game. The founding three-layer spec text is not in the repo, so the 15 could not be reconciled. (STATE.md's "Layer 1/2/3" references are the Session 73-77 archetype build train — a DIFFERENT numbering. Do not conflate them.)

Layer 2 — Bayesian: BUILT, NOT WIRED, and genuinely as specced

bayesian.py header: "Shared by NBA and MLB. Per-stat-type weights. Similar game confidence modifier. Skewness parameter. Data sufficiency smooth degradation curve." It carries BAYESIAN_WEIGHTS = prior/recent/context triples for 11 stat types across both sports (e.g. strikeouts .40/.40/.20, hits .30/.45/.25, points .35/.45/.20), plus data-sufficiency minimums per player type, a calibration offset clamped to ±0.15, and a CALIBRATION_DISCLAIMER ("Model in calibration period... Track record begins building now").

VERDICT: the sport-agnostic-math / per-sport-parameters claim is TRUE of the built code. It is simply not connected to anything. The champion's p_win does NOT flow through it.

Layer 3 — Grading: the table exists and is WIRED BACKWARDS

grade_thresholds.json maps PROBABILITY → GRADE (A+ 0.85-1.00, B 0.60-0.65, F 0.00-0.29). In the specced engine the grade IS a function of the probability.

The live JS reads that same table in REVERSE: it picks a letter from the factor index, then looks up that letter's band MIDPOINT to manufacture confidence (engine1.js:31-36, confidence_basis: 'grade_band'). A probability→grade table is being used as a grade→fake-confidence lookup. This single artifact is the drift, provable in one file.

Market-efficiency scaling per sport: SPECCED-BUT-ABSENT. One global 11-band scale, no sport dimension anywhere. The spec's MLB 0.55 / NBA-stars 0.80 does not exist in code; the only efficiency hits are usage-efficiency in nba_context.py/redistribution.py, an unrelated concept.

The gap, plainly

The live grade uses 0 of 3 specced layers. It uses: recent/season averages, an opponent rank, rest, usage — summed as flat ±deltas onto a neutral 'C'. No similarity search, no Bayesian inference, no probability-based grading, no per-sport calibration.


PHASE 2 — THE SPORT-PLUGIN BOUNDARY

Is the shared spine sport-agnostic? In the Python spine, largely yes: bayesian.py is sport-agnostic math parameterised per stat type. One mild leak: similarity.py co-mingles NBA and MLB factor keys in a single SIMILARITY_FACTORS dict rather than per-sport blocks (harmless — absent keys simply do not contribute — but it is not a clean plug point). blueprints/ is a general module directory (coaching, synergy, evolution, resolution…), not a sport-plugin registry; mlb.py and nba_context.py happen to live there.

🔴 TESTED, NOT ASSERTED: what breaks if a sport is added today

Adding a sport to the Python spine costs nothing — because the spine is dead. Adding a sport to the LIVE path is a CORE EDIT across ~10+ files, not a module drop-in:

  1. proplineAdapter.MARKETS — request the markets
  2. oddsNormalizer.MARKET_MAP — miss it and props silently normalize to ZERO
  3. oddsService.SPORT_KEYS + SPORT_MARKETS
  4. THREE stat_type whitelists that must stay in syncroutes/analyze.js, routes/scan.js, python/utils/validation.py (miss one → silent 400s)
  5. featureCache log-field map (features)
  6. outcomeService.MLB_LOG_FIELD + outcomeService.SPORTS (settlement)
  7. liveTrackingService.LIVE_BOX_FIELD (live tracking)
  8. A feature extractor emitting a projection field — without it every prop REFUSES
  9. lib/emptyState.js, config/statFilters.js, web/src/lib/teams.js
  10. A settled-result feed, or the sport never settles (NBA/soccer still do not)

VERDICT: the boundary is NOT clean on the live path. A new sport today is a core edit with at least four documented silent-failure modes. Making a sport a genuine module is itself a prerequisite build, not a given.

Per-sport records — EXIST (2.7 satisfied at the data layer)

getModelAggregate accepts opts.sport and filters both the settled and pending queries (ledgerService.js:809,819). /api/accuracy returns a real per-sport breakdown — live: sports.mlb = n 526, 62%, alongside overall = n 937, 58% — each carrying its own min_sample: 20 gate. So a new sport CAN accrue an independent record with its own accruing→proven state.

⚠️ Two caveats. (a) A pooled overall exists alongside, and a new sport's rows WOULD enter it — so any surface rendering overall would silently blend a new sport's record with MLB/WNBA's. (b) /api/accuracy ignores the ?sport= query param (identical payload for ?sport=mlb and ?sport=wnba); consumers must read sports.{sport} from the full object.


PHASE 3 — THE MODERN-DATA STACK

MLB — what is wired, and to WHAT

Input Present Feeds the CHAMPION?
l5 / l20 averages, opp_rank_stat, rest, usage yes YES — this is the whole champion
park factor (19 files), weather (18) yes NO — challenger only (env_park_base, env_weather_mod)
platoon / handedness (9 / 7 files) yes NO — challenger only
statcast (11 files): barrel%, hard-hit%, launch angle (4 each), exit velo (2) yes NO — challenger only
pitcher arsenal yes NO — proj-v1.1 challenger only
xwOBA 0 files — ABSENT
bullpen (1 file) / leash (0 — ABSENT) partial NO

Park×weather status CONFIRMED: CHALLENGER, NOT CHAMPION. Attached in snapshotService (Session 77) onto env_*/challenger_* ledger columns — ledgerService.js:253 states it outright: "is what served the user; this is measured, never served." And per the completion matrix, proj-v1.1 (which consumes park/weather/platoon/arsenal) was tested 2026-07-29 and came back NOT PROVEN — it lost to the champion. So the deep-data stack is accruing measurement, not serving grades, and has not yet earned promotion.

Incoming sports — input stacks to scope (NOT built)

  • NFL: snap share, target share / route participation, team pace (plays per game), scheme matchup (coverage/front vs role), role change on injury, red-zone usage, weather (outdoor), Vegas team total as context.
  • CFB: tempo, soft-market efficiency (the actual edge thesis — books price CFB props thinner), role concentration (few-star offenses), talent gradient / mismatch, pace, garbage-time risk, limited-history handling (the Bayesian prior matters MOST here).

Both need a settled-result feed wired BEFORE launch, or they will grade and never settle — the NBA/soccer failure mode already live in this repo.


PHASE 4 — THE RECOVERY MAP, IN DEPENDENCY ORDER

# Step Depends on What it takes
0 DECIDE THE GRADING BASIS — probability-based (spec) vs factor-index (live). Everything below hangs on this. A founder decision, not a build. Note: moving to probability-based grading legitimately CHANGES the letter distribution because it is NEW INFORMATION — that is categorically different from rescaling thresholds to mint A's (permanently forbidden).
1 Choose a runtime for Layers 1-2: deploy Python, or PORT to Node. 0 Python is absent from the Dockerfile and has been offline for the project's life; the live path is 100% Node. Recommend PORTING bayesian.py (320 ln) to Node over resurrecting a service every live path has learned to route around.
2 Wire Layer 2 (Bayesian) first. 1 Its inputs (prior / recent / context) are ALL already available from featureCache.getStatRows — no new I/O. Reachable soonest; delivers a probability the grade can actually be a function of.
3 Reconnect Layer 3 forward — grade FROM the probability using the existing table. 2 Deletes the backwards confidence lookup and makes confidence a real probability. Kills the tautology in one move.
4 Wire Layer 1 (Similarity). 2, 3 Needs a historical instance store with context — S73-77 park/weather/platoon now PRODUCE that, so it is newly feasible. Set the min-instance threshold explicitly (the spec's 15 is unreconciled; code says 3).
5 Per-sport market-efficiency scaling. 3, 4 Needs per-sport calibration accrual — the per-sport ledger record already exists to supply it.
6 Promote park×weather / challengers — only on ledger evidence. 5 Currently NOT PROVEN (proj-v1.1 lost to champion). Gated, not scheduled.
7 Make "a sport" a real module (collapse the ~10-file core edit into a registry). 3 Prerequisite for NFL/CFB shipping as modules rather than core surgery.
8 NFL / CFB modules. 7 Each needs its input stack + a settled-result feed before launch.

MLB as the REFERENCE MODULE

MLB is the right reference and is already the deepest: free unlimited statsapi (game logs, season stats, probables, box scores), statcast row per batter, park base, weather modifier, platoon split, pitcher arsenal, a settled-result feed that works, and 526 settled rows of its own record. A new sport module should be required to declare the same contract: market list → normalizer mapping → stat whitelist (all three) → feature extractor emitting a projection → log-field map for features → settlement field map + a settled feed → live box map → archetype set → thresholds/efficiency score → its own ledger record.

Honest per-sport labeling — MECHANISM EXISTS, RULE MUST BE ENFORCED

The n≥20 per-sport gate and independent sports.{sport} records already exist. The rule for every future sport: launch with the engine fully built but the record rendered from sports.{sport} and labeled ACCURING until that sport's own games clear its own gate — never overall, which would silently borrow MLB/WNBA credibility. The pooled overall is the specific hazard to avoid on any new-sport surface.


TAGS

VERIFIED: 0.1, 0.2, all of Phase 1, the sport-boundary break-test, per-sport records, park×weather challenger status, MLB input inventory, Python-not-deployed. CANNOT DETERMINE: the spec's "min 15 instances" (founding spec text absent from the repo; code says 3). BLOCKED: none.