Revive the dead probability layer + restore grade range ON MERIT
Folds re-sequenced steps 1+2 into one change (Kev's call): same bug
family — features wired to sources that return null.
THE PROBABILITY LAYER WAS DEAD IN PRODUCTION. p_win/ev_pct/kelly/
model_odds/value were absent on 0/8 live grades because
gameLogService.getGameLogs returns null for MLB by construction and
depends on the offline Python service for NBA/WNBA, so meta.gameLogs was
[] for every sport. This was the S46 bug in a second location — that fix
gave featureCache an MLB branch (why grades still worked) but never the
estimator. featureCache.getStatRows now supplies normalized rows
([{date,[statType]:v}], most-recent-first) for every sport, feeding the
estimator AND consistency AND game_count_in_7d from one fetch.
VERIFIED on real props: p_win 25/25 WNBA, 8/8 MLB (was 0).
GRADE RANGE, ON MERIT — never by rescaling (permanent founder ruling:
minting A's without new information is a relabelled B sold as an A and
corrupts an append-only ledger).
- refreshTeamStats wired into runSnapshot — it had ZERO production
callers, so opp_rank_stat was permanently null and a +/-1.0 factor
could never fire. Test-env no-op (opsNotify precedent).
- L20 made SYMMETRIC: both branches were delta +1.0, so the season
baseline could only ever ADD. No negative path was a structural reason
D was unreachable. New l20_contradicts_* carries -1.0.
- game_count_in_7d derived from real logged dates (heavy_workload_7d).
- NOT wired, deliberately, with reasons inline: teamId (no team_id
column; getFeatures reads it top-level; factor also needs a starter-id
list) and season_type (ESPN 2 = REGULAR season; threading it raw would
fire veteran_in_playoffs in July). Dead code dressed as a fix is the
thing we are removing, not adding.
CALIBRATION GUARD (found by verifying, not assuming): consistency CV is
NBA-tuned; for a Poisson-ish stat cv ~ 1/sqrt(mean), so any stat with
mean < 4 auto-classifies boom_bust. First verification run showed 8/8 MLB
props boom_bust — a blanket -1.0 that dropped the board to all-C. Floored
at CONSISTENCY_MIN_MEAN=4 -> 'unknown' below. Absent beats wrong. MLB
low-count stats therefore still get no consistency factor: honest, not
fixed. Scale-free index-of-dispersion classifier is the open follow-up.
CONFIDENCE IS NOT A PROBABILITY: payloads carry confidence_basis:
'grade_band'. Corrected mlb-grade-degradation.md — its "25/25
grade<->confidence agreement" is a TAUTOLOGY (confidence is derived FROM
the letter, so it would report 25/25 even if every grade were wrong), not
a validation. Removed dead mlbGrader.js (referenced only by its own test)
and the stale computeFeatures comment claiming a penalty that never ran.
VERIFICATION (scripts/verify-grade-range.js, real props/logs/engine):
WNBA 25 props B 68%->32%, C 32%->64%, D 0->1 (4%); 11-step spread went
from 2 steps to 5 (C/C+/B-/D). The D is earned: Angel Reese assists o2.5,
p_win 0.365. Nothing flooded — grades got HARDER. A did not emit locally
because opp_rank_stat needs the Redis cache only prod populates (local
ceiling +3.0 vs the +4.5 A needs); reachability is proven arithmetically
and locked in tests. Prod A-emission is the outstanding fingerprint.
MARKETING HOLD: "A-RATED" (AccuracyBadge, TopSignals) is unsupported
until that fingerprint. Confirmed honest fallbacks render today —
/api/ledger/accuracy has B and C buckets only, so the badge shows
"MODEL · 63% HIT" and TopSignals self-hides. Nothing fabricated ships.
Suite 276/3286 green, web build exit 0.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SmNjJAwEnqHPtXbvSZR8kA
This commit is contained in:
@@ -41,6 +41,32 @@ function classify(cv) {
|
||||
return { consistency: 'boom_bust', score: 0.1 };
|
||||
}
|
||||
|
||||
/**
|
||||
* Session 63 — the CV thresholds above are NBA-calibrated (points ~20/game,
|
||||
* cv ~0.2-0.4). They are MEANINGLESS for a low-count stat.
|
||||
*
|
||||
* For a Poisson-ish counting stat, cv ≈ 1/sqrt(mean). So mean < 4 forces
|
||||
* cv > 0.5 — i.e. EVERY such stat classifies 'boom_bust' no matter how the
|
||||
* player actually behaves. Verified against real logs: Alonso hits
|
||||
* [0,0,0,1,2,1,0,1,1,0] → mean 0.60, cv 1.17 → boom_bust; Henderson
|
||||
* [1,0,0,3,1,1,0,0,1,0] → mean 0.70, cv 1.36 → boom_bust.
|
||||
*
|
||||
* When the estimator path was revived, this would have stamped a blanket
|
||||
* -1.0 on nearly every MLB prop — a systematic downgrade masquerading as a
|
||||
* signal. Below the floor we return 'unknown' so engine1 adds NO factor:
|
||||
* absent beats wrong.
|
||||
*
|
||||
* The RIGHT long-term fix is an index-of-dispersion (variance/mean vs the
|
||||
* Poisson baseline) classifier, which is scale-free. That is a modelling
|
||||
* change with its own validation and is tracked separately — this floor is
|
||||
* the honest stopgap, not the answer.
|
||||
*/
|
||||
const MIN_MEAN_FOR_CV = Number(process.env.CONSISTENCY_MIN_MEAN || 4);
|
||||
|
||||
function cvIsMeaningful(mean) {
|
||||
return Number.isFinite(mean) && Math.abs(mean) >= MIN_MEAN_FOR_CV;
|
||||
}
|
||||
|
||||
function statsFor(values) {
|
||||
const clean = values.filter((v) => Number.isFinite(v));
|
||||
if (clean.length < 2) return null;
|
||||
@@ -60,7 +86,13 @@ async function getConsistency(input = {}) {
|
||||
const values = logs.map((row) => statFromGameLog(row, statType)).filter((v) => v != null);
|
||||
const s = statsFor(values);
|
||||
if (!s) return { consistency: 'unknown', score: null, games: values.length };
|
||||
// Session 63 — refuse to classify when CV cannot discriminate at this scale.
|
||||
if (!cvIsMeaningful(s.mean)) {
|
||||
return { ...s, consistency: 'unknown', score: null, reason: 'low_mean_cv_unreliable' };
|
||||
}
|
||||
return { ...s, ...classify(s.cv) };
|
||||
}
|
||||
|
||||
module.exports = { getConsistency, classify, statsFor, statFromGameLog };
|
||||
module.exports = {
|
||||
getConsistency, classify, statsFor, statFromGameLog, cvIsMeaningful, MIN_MEAN_FOR_CV,
|
||||
};
|
||||
|
||||
Reference in New Issue
Block a user