Files
vyndr/tests/unit/featureCoverage.test.js
builtbykev 8a02c75aec Step 0 input check: stop before wiring opportunity, and why
READ-ONLY. Live grade path byte-identical -- no layer wired, no threshold
moved, no challenger added, no holdout run.

INPUTS ARE 100% POPULATED (n=80 real MLB props, through the grader's own
path): ab_per_game, rest_days, l5_avg, l20_avg, l10_stddev and
game_count_in_7d all 100%; opp_rank_stat 65% overall and 0% on
stolen_bases. So there is no honest-degradation problem to solve.

FOUR FINDINGS THAT STOP THE WIRING, three of which would have made the
work unmeasurable or wrong:

1. THE PREMISE IS WRONG. There is no built opportunity layer to connect.
   ab_per_game is consumed in exactly one place -- analyzeViaEngine1:379,
   which renders "4.3 AB/G" on the grade card. engine1 has NO opportunity
   or usage factor at all. A projected opportunity was never built;
   building one is construction, not connection.

2. THE INPUT IS THE WRONG SHAPE. ab_per_game = season atBats/games. It is
   a per-player CONSTANT (measured: varies for 3 of 20 players, and those
   cannot be legitimate since the value can't depend on stat_type), so it
   can only move all of a player's props together, never separate them.
   And it is collinear with the projection: l20_avg = seasonTotal/games,
   the SAME denominator, so l20_avg already embeds opportunity
   multiplicatively. Adding it additively double-counts.

3. THE REAL INPUT DOES NOT EXIST. depthChartService returns battingOrder:
   null for MLB ("the one lineup slot the free schedule feed exposes") and
   PropLine /context carries lineup_confirmed as a BOOLEAN, not the order.

4. ARCHITECTURE: wiring it into engine1 would be unmeasurable BY THIS
   ORDER'S OWN TEST. Step 2 proves reliability and resolution, both
   measured on p_win. engine1 factors move the grade LETTER and never
   touch p_win. The layer belongs in probabilityEstimator, which already
   adjusts on opp_rank_stat, home_away and a consistency pull.

SEQUENCING IS ALSO STALE: challengerProjection (arch-v1) is already live
with archetype, matchup (platoon) and environment (park) axes, writing
p_win_challenger to the ledger. Step 2 of the order's sequence is partly
done -- and the harness this order needed already exists.

RECOMMENDED INSTEAD, as its own order: an `opportunity` axis on that
harness driven by DRIFT, not level -- recent AB/G (last 5) over season
AB/G. A deviation is not collinear the way the level is. Per-game atBats
is present in the statsapi log rows but MLB_LOG_FIELD never maps it, so it
is a small contained BUILD, which is why it gets its own order. Honest
caveat carried forward: it is still a proxy, not tonight's opportunity.

PROBE BUG RECORDED: the first run reported 0% for every feature including
l5_avg, on a pipeline that had just graded 365 props -- impossible, so the
probe was wrong. getFeatures takes camelCase and returns { features: {} };
I passed snake_case and read the top level. Fixed to call
computeFeaturesForProp. Same class as the earlier silent-false harness: a
measurement that makes working code look broken invites you to "fix"
something that was never broken.

Gates: 4,059 tests / 325 suites green; next build exit 0.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QJs13VsyiSKYQP6rj3NNmc
2026-08-01 02:40:46 -04:00

83 lines
3.6 KiB
JavaScript
Raw Permalink Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
'use strict';
/**
* featureCoverage — Step 0 input check (2026-08-01).
*
* The property worth locking is the one whose absence produced a false
* all-zero reading on the first run: a coverage probe must count a feature as
* populated ONLY when it is a real finite number, and it must read features
* from the grader's own return shape. A probe that makes working code look
* broken is more dangerous than no probe.
*/
const { coverage, __internals } = require('../../src/services/featureCoverage');
const { populated } = __internals;
const isModelBook = (b) => b === 'draftkings';
const prop = (player, stat) => ({
player, stat_type: stat, line: 1.5, book: 'draftkings',
});
describe('populated — strict, never 0-coercing', () => {
it('treats null / undefined / empty string as ABSENT, not zero', () => {
for (const v of [null, undefined, '', NaN, 'abc']) expect(populated(v)).toBe(false);
});
it('treats a real zero as PRESENT (0 rest days is a fact, not a gap)', () => {
expect(populated(0)).toBe(true);
expect(populated(4.31)).toBe(true);
});
});
describe('coverage — reads the graders shape and splits by stat', () => {
const getOdds = async () => ({
props: [
prop('Batter A', 'hits'),
prop('Batter B', 'rbi'),
prop('Pitcher C', 'strikeouts'),
{ ...prop('Ignored', 'hits'), book: 'prizepicks' }, // DFS never reaches the model
],
});
// Mirrors computeFeaturesForProp: features live UNDER `features`.
const computeFeatures = async (p) => ({
features: p.stat_type === 'strikeouts'
? { l5_avg: 6.1, l20_avg: 5.8 } // pitcher: no at-bats
: { l5_avg: 1.2, l20_avg: 1.1, ab_per_game: 4.3, rest_days: 0 },
});
it('reports per-feature coverage from the nested features object', async () => {
const r = await coverage({ sport: 'mlb', sample: 10, concurrency: 2, getOdds, computeFeatures, isModelBook });
expect(r.sampled).toBe(3); // the DFS row is excluded upstream
expect(r.coverage.l5_avg.pct).toBe(100);
expect(r.coverage.ab_per_game.populated).toBe(2);
});
it('SPLITS BY STAT — a pooled number would hide a pitcher-shaped hole', async () => {
const r = await coverage({ sport: 'mlb', sample: 10, concurrency: 2, getOdds, computeFeatures, isModelBook });
expect(r.by_stat.hits.ab_per_game).toBe(100);
expect(r.by_stat.rbi.ab_per_game).toBe(100);
expect(r.by_stat.strikeouts.ab_per_game).toBe(0); // the hole the pooled 66% would bury
});
it('flags a per-player CONSTANT — it cannot separate a players own props', async () => {
const twoProps = async () => ({ props: [prop('Same Guy', 'hits'), prop('Same Guy', 'rbi')] });
const constant = async () => ({ features: { ab_per_game: 4.3 } });
const r = await coverage({ sport: 'mlb', sample: 10, concurrency: 2, getOdds: twoProps, computeFeatures: constant, isModelBook });
expect(r.ab_per_game_shape.players_with_value).toBe(1);
expect(r.ab_per_game_shape.players_where_it_varies_across_their_props).toBe(0);
expect(r.ab_per_game_shape.note).toMatch(/CONSTANT per player/);
});
it('a thrown feature build counts as ABSENT, never as populated', async () => {
const boom = async () => { throw new Error('nope'); };
const r = await coverage({ sport: 'mlb', sample: 10, concurrency: 2, getOdds, computeFeatures: boom, isModelBook });
expect(r.coverage.l5_avg.populated).toBe(0);
});
it('is read-only by contract', async () => {
const r = await coverage({ sport: 'mlb', sample: 10, concurrency: 2, getOdds, computeFeatures, isModelBook });
expect(r.read_only).toBe(true);
});
});