From 8a02c75aec044b4756e5e8e48a4a6e330aac8550 Mon Sep 17 00:00:00 2001 From: Kev Date: Sat, 1 Aug 2026 02:40:46 -0400 Subject: [PATCH] Step 0 input check: stop before wiring opportunity, and why READ-ONLY. Live grade path byte-identical -- no layer wired, no threshold moved, no challenger added, no holdout run. INPUTS ARE 100% POPULATED (n=80 real MLB props, through the grader's own path): ab_per_game, rest_days, l5_avg, l20_avg, l10_stddev and game_count_in_7d all 100%; opp_rank_stat 65% overall and 0% on stolen_bases. So there is no honest-degradation problem to solve. FOUR FINDINGS THAT STOP THE WIRING, three of which would have made the work unmeasurable or wrong: 1. THE PREMISE IS WRONG. There is no built opportunity layer to connect. ab_per_game is consumed in exactly one place -- analyzeViaEngine1:379, which renders "4.3 AB/G" on the grade card. engine1 has NO opportunity or usage factor at all. A projected opportunity was never built; building one is construction, not connection. 2. THE INPUT IS THE WRONG SHAPE. ab_per_game = season atBats/games. It is a per-player CONSTANT (measured: varies for 3 of 20 players, and those cannot be legitimate since the value can't depend on stat_type), so it can only move all of a player's props together, never separate them. And it is collinear with the projection: l20_avg = seasonTotal/games, the SAME denominator, so l20_avg already embeds opportunity multiplicatively. Adding it additively double-counts. 3. THE REAL INPUT DOES NOT EXIST. depthChartService returns battingOrder: null for MLB ("the one lineup slot the free schedule feed exposes") and PropLine /context carries lineup_confirmed as a BOOLEAN, not the order. 4. ARCHITECTURE: wiring it into engine1 would be unmeasurable BY THIS ORDER'S OWN TEST. Step 2 proves reliability and resolution, both measured on p_win. engine1 factors move the grade LETTER and never touch p_win. The layer belongs in probabilityEstimator, which already adjusts on opp_rank_stat, home_away and a consistency pull. SEQUENCING IS ALSO STALE: challengerProjection (arch-v1) is already live with archetype, matchup (platoon) and environment (park) axes, writing p_win_challenger to the ledger. Step 2 of the order's sequence is partly done -- and the harness this order needed already exists. RECOMMENDED INSTEAD, as its own order: an `opportunity` axis on that harness driven by DRIFT, not level -- recent AB/G (last 5) over season AB/G. A deviation is not collinear the way the level is. Per-game atBats is present in the statsapi log rows but MLB_LOG_FIELD never maps it, so it is a small contained BUILD, which is why it gets its own order. Honest caveat carried forward: it is still a proxy, not tonight's opportunity. PROBE BUG RECORDED: the first run reported 0% for every feature including l5_avg, on a pipeline that had just graded 365 props -- impossible, so the probe was wrong. getFeatures takes camelCase and returns { features: {} }; I passed snake_case and read the top level. Fixed to call computeFeaturesForProp. Same class as the earlier silent-false harness: a measurement that makes working code look broken invites you to "fix" something that was never broken. Gates: 4,059 tests / 325 suites green; next build exit 0. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01QJs13VsyiSKYQP6rj3NNmc --- specs/MASTER-PLAN.md | 1 + specs/connect-opportunity-step0.md | 173 +++++++++++++++++++++++++++++ tests/unit/featureCoverage.test.js | 82 ++++++++++++++ 3 files changed, 256 insertions(+) create mode 100644 specs/connect-opportunity-step0.md create mode 100644 tests/unit/featureCoverage.test.js diff --git a/specs/MASTER-PLAN.md b/specs/MASTER-PLAN.md index 10201a2..bbea355 100644 --- a/specs/MASTER-PLAN.md +++ b/specs/MASTER-PLAN.md @@ -78,6 +78,7 @@ multi-book data. | **soccer** | **settles** — ~15 competitions, 30d | "grades into a void" is a **$19/mo Pro-tier** problem, not a data problem | | **CLV + results feeds** | `/odds/closing` + `/movement` **redacted**; `/results` **403 `required_tier: hobby`**; `/exports/resolved-props` **403 `required_tier: pro`** | **verified on our keys** — plain tier exclusion, not a key or plan fault. **$9/mo** buys CLV + steam + results; **$19/mo** adds the 90-day settlement export | | **books SERVED** | 5 → **13**; props rendered **546 → 2,780** (5.1×); mean **4.22** books/prop | **AGGREGATOR widening is LIVE.** Of 2,234 newly-visible props, **31.2% carry a real non-DFS price**; **68.8% are DFS-only** — shown, tagged, never a market | +| **opportunity layer** | **NOT BUILT** — `ab_per_game` is display-only; engine1 has no usage factor; MLB batting order unavailable in every wired source | Step 0 stopped before wiring. Recommended instead: an `opportunity_drift` axis on the EXISTING `challengerProjection` (arch-v1). See `specs/connect-opportunity-step0.md` | | **MLB board size** | **7 → 365 graded props** (52×) in 114s | the 25-cap discarded 95.7% of the slate. Raised to 500 on measured cost. Refusals were **43.8% deliberate policy suppression**, not a data gap | | **model input** | **byte-identical** — 546 gradeable props, `v1_first_book` | `MODEL_BOOKS` gate in `dedupeProps` + `indexOddsProps`. Lifts only on the re-run | | **accrual clock** | **sequential, post-completion** | see §11. Pre-completion data does not count and is never pooled | diff --git a/specs/connect-opportunity-step0.md b/specs/connect-opportunity-step0.md new file mode 100644 index 0000000..35f7c3e --- /dev/null +++ b/specs/connect-opportunity-step0.md @@ -0,0 +1,173 @@ +# CONNECT PROJECTION LAYERS — STEP 0: OPPORTUNITY/USAGE INPUT CHECK + +**Date:** 2026-08-01 · **READ-ONLY** · live grade path byte-identical · +probe committed (`src/services/featureCoverage.js`, +`GET /api/internal/feature-coverage`). + +--- + +## VERDICT: STOPPED AT STEP 0 — AND THE REASON IS THE DELIVERABLE + +**Inputs are 100% populated. The layer still should not be wired as specified**, +for four reasons the input check surfaced. Three of them would have made the +work either unmeasurable or wrong. + +--- + +## 1. INPUT COVERAGE (MLB, n=80 real props, through the grader's own path) + +| feature | populated | rate | +|---|---:|---:| +| `ab_per_game` | 80/80 | **100%** | +| `rest_days` | 80/80 | 100% | +| `l5_avg` | 80/80 | 100% | +| `l20_avg` | 80/80 | 100% | +| `l10_stddev` | 80/80 | 100% | +| `game_count_in_7d` | 80/80 | 100% | +| `opp_rank_stat` | 52/80 | 65% | +| `minutes_per_game` / `usage_rate` | 0/80 | 0% *(NBA-only — correct for MLB)* | + +**No honest-degradation problem exists** for the opportunity input: there is +nothing sparse to fall back from. The one real hole is `opp_rank_stat`, which is +**0% for `stolen_bases`** (no opponent SB-defence rank) and ~84% elsewhere. + +--- + +## 2. 🔴 THE PREMISE: there is no built opportunity layer to connect + +The order says the expensive layers are *"BUILT and DISCONNECTED"* and that this +is *"CONNECTION, not construction."* **For opportunity, that is not the case.** + +`ab_per_game` is computed in `featureCache` and consumed in exactly one place — +`analyzeViaEngine1:379`, which renders **"4.3 AB/G"** on the grade card. It is a +**display field**. `engine1` has **no opportunity or usage factor at all**. + +**A projected opportunity — expected plate appearances with its own uncertainty — +was never built.** §10.3 of MASTER-PLAN said exactly this and called it *"the +single biggest modelling upgrade available."* Building it is construction. + +## 3. 🔴 The available input is the wrong shape for the job + +`ab_per_game = season atBats ÷ games`. Two consequences: + +- **It is a per-player constant.** Measured: it varies across a player's own + props for **3 of 20 players** (and those three are almost certainly a cache / + refresh artifact, since the value cannot legitimately depend on `stat_type`). + **A constant can only move all of a player's props together — it cannot + separate them**, which is what a per-prop opportunity signal has to do. +- **It is collinear with the projection that already exists.** `l20_avg` is + `seasonTotal ÷ games` — the *same denominator*. For a batter, + `hits/game ≈ (hits/AB) × (AB/game)`, so **`l20_avg` already embeds + opportunity multiplicatively.** Adding `ab_per_game` as an independent additive + factor double-counts it rather than adding information. + +**What is actually missing is tonight's deviation from the season baseline** — +batting-order slot, a platoon sit, a role change. + +## 4. 🔴 That input does not exist in any wired source + +- `depthChartService.getLineup` returns, for MLB, **only the probable pitcher**, + with `battingOrder: null`. Its own comment: *"the one lineup slot the free + schedule feed exposes."* +- PropLine `/context` (free, verified this session) carries `lineup_confirmed` — + **a boolean**, not the order. + +**Tonight's batting order, the actual driver of MLB plate appearances, is not +available from anything we have wired.** + +## 5. 🔴 ARCHITECTURE: wiring it into `engine1` would be unmeasurable by this order's own test + +Step 2 requires proving **reliability and resolution** improve. Both are measured +on **`p_win`**. + +**`engine1` factors move the grade LETTER. They do not touch `p_win`.** +`p_win` comes from `probabilityEstimator.estimateProbability`, which builds from +`frequencyOver(gameLogs, line)` plus feature adjustments. + +So an opportunity factor added to `engine1` would produce a change that **Step 2 +literally cannot measure**. The layer belongs in `probabilityEstimator` (or the +challenger below), which *does* consume features — it already adjusts on +`opp_rank_stat`, `home_away`, and a consistency pull off `l10_stddev/l20_avg`. + +--- + +## 6. THE SEQUENCING ASSUMPTION IS STALE — matchup is already connected + +The order sequences *opportunity → matchup granularity (archetype × opponent, +park/weather/platoon) → distribution ladder.* + +**`challengerProjection` (`arch-v1`) is already live in production** and already +carries three axes: **archetype**, **matchup (platoon)** and **environment +(park)**. It writes `p_win_challenger`, `challenger_delta`, +`challenger_adjustments` and `challenger_version` to the ledger on every graded +prop. + +So step 2 of the sequence is **partly done**, and — more usefully — **the harness +this order needed already exists.** Any new axis should be added there, not +invented. + +--- + +## WHAT I RECOMMEND INSTEAD (its own order, per "one layer at a time") + +**An `opportunity` axis on `challengerProjection`, driven by opportunity DRIFT +rather than by the season level:** + +``` +opportunity_drift = recent AB/G (last 5) ÷ season AB/G +``` + +- **>1** — batting higher / playing more than his baseline → lean over +- **<1** — reduced role, platoon, lower slot → lean under +- **null** — no at-bat data → **no adjustment**, challenger ≡ champion on that row + +This is a **deviation**, so it is not collinear with `l20_avg` the way the raw +level is, and it *does* vary per player over time. + +**The input exists but is not extracted.** Per-game `atBats` is present in the +statsapi game-log rows (the Session-56 box-score audit lists `atBats` among the +batting fields), but `MLB_LOG_FIELD` has no entry for it and nothing computes a +recent AB/G. That is a small, contained build — **and it is a build**, which is +why it belongs in its own order rather than being smuggled into a connection +order. + +**Honest caveat to carry into it:** this is still a *proxy* for tonight's +opportunity, not tonight's opportunity. The real input is the confirmed batting +order, and that needs a lineup source we do not have. + +--- + +## WHAT DID NOT HAPPEN, DELIBERATELY + +No layer was wired. **The live grade path is byte-identical.** No threshold moved, +no challenger was added, no holdout was run — running one would have measured a +change that could not have occurred. + +## A PROBE BUG WORTH RECORDING + +The first coverage run reported **0% for every feature, including `l5_avg`** — on +a pipeline that had just graded 365 props, which `projectionFor` cannot do +without a positive `l5_avg` or `l20_avg`. **Impossible, therefore the probe was +wrong.** + +Two bugs, both mine: `featureCache.getFeatures` takes **camelCase** +(`playerName`/`statType`) and I passed the prop's snake_case shape; and it +returns **`{ features: {...} }`** while I read the top level. Either alone yields +all zeros. Fixed by calling `computeFeaturesForProp` — the grader's own entry +point. + +Same class as the earlier harness that returned a silent `false`: **a measurement +that makes working code look broken is more dangerous than no measurement**, because +it invites you to "fix" something that was never broken. + +## TAGS + +**VERIFIED:** `ab_per_game` 100% populated · per-player constant (varies for +3/20) · `opp_rank_stat` 0% on stolen_bases · engine1 has no opportunity factor · +`ab_per_game` is display-only · MLB batting order unavailable in depthChart and +`/context` · `probabilityEstimator` consumes features, `engine1` does not affect +`p_win` · `challengerProjection` arch-v1 already carries archetype/matchup/ +environment. + +**CORRECTED:** "the opportunity layer is built and disconnected" — it is not +built. "Connect opportunity, then matchup" — matchup is already connected. diff --git a/tests/unit/featureCoverage.test.js b/tests/unit/featureCoverage.test.js new file mode 100644 index 0000000..9578421 --- /dev/null +++ b/tests/unit/featureCoverage.test.js @@ -0,0 +1,82 @@ +'use strict'; + +/** + * featureCoverage — Step 0 input check (2026-08-01). + * + * The property worth locking is the one whose absence produced a false + * all-zero reading on the first run: a coverage probe must count a feature as + * populated ONLY when it is a real finite number, and it must read features + * from the grader's own return shape. A probe that makes working code look + * broken is more dangerous than no probe. + */ + +const { coverage, __internals } = require('../../src/services/featureCoverage'); +const { populated } = __internals; + +const isModelBook = (b) => b === 'draftkings'; +const prop = (player, stat) => ({ + player, stat_type: stat, line: 1.5, book: 'draftkings', +}); + +describe('populated — strict, never 0-coercing', () => { + it('treats null / undefined / empty string as ABSENT, not zero', () => { + for (const v of [null, undefined, '', NaN, 'abc']) expect(populated(v)).toBe(false); + }); + + it('treats a real zero as PRESENT (0 rest days is a fact, not a gap)', () => { + expect(populated(0)).toBe(true); + expect(populated(4.31)).toBe(true); + }); +}); + +describe('coverage — reads the grader’s shape and splits by stat', () => { + const getOdds = async () => ({ + props: [ + prop('Batter A', 'hits'), + prop('Batter B', 'rbi'), + prop('Pitcher C', 'strikeouts'), + { ...prop('Ignored', 'hits'), book: 'prizepicks' }, // DFS never reaches the model + ], + }); + + // Mirrors computeFeaturesForProp: features live UNDER `features`. + const computeFeatures = async (p) => ({ + features: p.stat_type === 'strikeouts' + ? { l5_avg: 6.1, l20_avg: 5.8 } // pitcher: no at-bats + : { l5_avg: 1.2, l20_avg: 1.1, ab_per_game: 4.3, rest_days: 0 }, + }); + + it('reports per-feature coverage from the nested features object', async () => { + const r = await coverage({ sport: 'mlb', sample: 10, concurrency: 2, getOdds, computeFeatures, isModelBook }); + expect(r.sampled).toBe(3); // the DFS row is excluded upstream + expect(r.coverage.l5_avg.pct).toBe(100); + expect(r.coverage.ab_per_game.populated).toBe(2); + }); + + it('SPLITS BY STAT — a pooled number would hide a pitcher-shaped hole', async () => { + const r = await coverage({ sport: 'mlb', sample: 10, concurrency: 2, getOdds, computeFeatures, isModelBook }); + expect(r.by_stat.hits.ab_per_game).toBe(100); + expect(r.by_stat.rbi.ab_per_game).toBe(100); + expect(r.by_stat.strikeouts.ab_per_game).toBe(0); // the hole the pooled 66% would bury + }); + + it('flags a per-player CONSTANT — it cannot separate a player’s own props', async () => { + const twoProps = async () => ({ props: [prop('Same Guy', 'hits'), prop('Same Guy', 'rbi')] }); + const constant = async () => ({ features: { ab_per_game: 4.3 } }); + const r = await coverage({ sport: 'mlb', sample: 10, concurrency: 2, getOdds: twoProps, computeFeatures: constant, isModelBook }); + expect(r.ab_per_game_shape.players_with_value).toBe(1); + expect(r.ab_per_game_shape.players_where_it_varies_across_their_props).toBe(0); + expect(r.ab_per_game_shape.note).toMatch(/CONSTANT per player/); + }); + + it('a thrown feature build counts as ABSENT, never as populated', async () => { + const boom = async () => { throw new Error('nope'); }; + const r = await coverage({ sport: 'mlb', sample: 10, concurrency: 2, getOdds, computeFeatures: boom, isModelBook }); + expect(r.coverage.l5_avg.populated).toBe(0); + }); + + it('is read-only by contract', async () => { + const r = await coverage({ sport: 'mlb', sample: 10, concurrency: 2, getOdds, computeFeatures, isModelBook }); + expect(r.read_only).toBe(true); + }); +});