Prove both on total bases -- and find that my own fix destroyed the backtest
Nothing passed. Nothing promoted. Counter byte-identical. THE BLOCKER, which is the real finding. statcast_aggregates is upserted in place and holds exactly one as-of date. Yesterday's skill backtest was honest only by accident: the nightly refresh was unreachable code, so the profiles sat frozen at 2026-07-21 -- before the settled window. Repairing that cron was right for production and it refreshed them to today, destroying every prior version. Scoring a 2026-07-25 game now uses a season aggregate that contains that game. Point-in-time validation is structurally impossible from that table, so every number in this run is contaminated and directional, and none of it is a gate verdict. Fixed forward: statcast_history retains a dated snapshot on every refresh, so point-in-time becomes "as_of_date < game_date, most recent". Retention is best-effort and cannot fail the refresh; both properties are unit-tested. It has one day of data, which is not yet a window. SOLO BASELINE, n=383, Bonferroni across 12 tests (alpha 0.00417): nothing passes. hard_hit_pct is closest at marginal r 0.135 with p 0.0080, failing both the 0.15 effect bar and the corrected alpha. And it drifted DOWN from 0.153 at n=295 -- an estimate regressing as noise averages out, not an effect firming up. I called that number encouraging yesterday; on 88 more rows it is fading, and it should not keep being quoted at its best value. INTERACTIONS, each scored by partial correlation against the counter residual controlling for both of its own components: none pass. Only barrel x power archetype has an incremental exceeding its parts (-0.101 against 0.019) at n=260 -- the shape Discipline 2 predicts, but a lead, not a finding. A methodological catch worth keeping. The archetype conditioner was first built as barrel_pct over league barrel -- a monotone transform of one of its own components -- so the "interaction" was barrel squared, measuring nonlinearity in barrel rate rather than any archetype effect, and it produced this run's only positive result. A Gauss-Jordan pivot test does not catch that, because the two columns differ by a scale factor. Fixed with a scale-free collinearity check plus real archetype labels joined from model_snapshots. Without it this document would have reported a fabricated interaction as the session's finding. COMBINED vs COUNTER on total bases: 0.2718 against 0.2647, delta +0.0071, CI [-0.065, +0.079] -- inconclusive, and the first time a challenger has not lost. The same engine on hits was -0.116 with a CI excluding zero. That contrast is the whole argument for total bases, and it is what the physics said: contact quality governs extra bases, not whether a grounder finds a hole. Also built: the compound TB projection. skillProjection no longer refuses total bases -- a deterministic bases-per-hit multiplier had made P(TB>=2) exactly P(hits>=1), a relabelled hits curve. It is now a convolution over per-PA base outcomes with hit-type shares shifted by skill. Non-degeneracy is locked by test. 4,204 tests green (334 suites); web build exit 0. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W1sivYNqY2TS5ftykmHBU9
This commit is contained in:
@@ -3,6 +3,30 @@
|
||||
## Last Updated
|
||||
2026-08-03
|
||||
|
||||
## Session 82 (2026-08-03) — TB solo+interactions; point-in-time validation unblocked ✅
|
||||
Spec: `specs/tb-solo-and-interactions.md`. 4,204 tests / 334 suites green, build
|
||||
exit 0, counter byte-identical.
|
||||
- **BLOCKER FOUND + FIXED FORWARD:** `statcast_aggregates` keeps ONE as-of date
|
||||
(upsert in place). Yesterday's backtest was clean only because the refresh was
|
||||
dead code and the table sat at 2026-07-21; fixing the cron destroyed the
|
||||
window. New `statcast_history` table + retention on every refresh (best-effort,
|
||||
never fails the refresh). Until it accrues, all skill results are CONTAMINATED.
|
||||
- **SOLO (n=383, Bonferroni-12): nothing passes.** hard_hit_pct marginal r=0.135
|
||||
(p=0.0080) fails both the 0.15 bar and α=0.00417 — and DRIFTED DOWN from 0.153
|
||||
at n=295. Everything else <0.09.
|
||||
- **INTERACTIONS: none pass.** barrel×power_archetype is the only one whose
|
||||
incremental partial (−0.101) exceeds its parts (0.019), at n=260. A lead.
|
||||
- **Caught a fabricated finding:** the archetype proxy was a transform of barrel
|
||||
itself, so the "interaction" was barrel² — it produced the only positive result
|
||||
until a scale-free collinearity check + real `model_snapshots.archetype` labels
|
||||
replaced it.
|
||||
- **COMBINED vs COUNTER on TB: 0.2718 vs 0.2647, delta +0.0071, INCONCLUSIVE** —
|
||||
the first challenger that did not LOSE (hits was −0.116, CI excluding zero).
|
||||
- **BUILT: compound TB projection** (per-PA bases convolution, barrel→HR share,
|
||||
exit velo→XBH share). Replaces the refusal; non-degeneracy locked by test.
|
||||
- **Next:** let statcast_history accrue a point-in-time window (~a week) while TB
|
||||
reaches n>=500 (~117 short), then re-run. Do not re-run hits.
|
||||
|
||||
## Session 81 (2026-08-03) — The gate, built and run: hits is dead, total bases is the stat ✅
|
||||
Spec: `specs/stagea-gate-result.md`. 4,200 tests / 334 suites green, build exit 0.
|
||||
Counter byte-identical (zero diff on probabilityEstimator/analyzeViaEngine1).
|
||||
|
||||
@@ -1337,6 +1337,44 @@ phased plan in the Session-57 conversation / BUILD-STATE Next section).
|
||||
the per-hit extra-base distribution off launch/barrel (tb-v1's shape, fed by
|
||||
skill inputs), accrue to n>=500, re-run this gate. Do NOT lower the bar.
|
||||
|
||||
## Point-in-time skill validation + TB solo/interactions (Session 82 — non-obvious)
|
||||
- **`statcast_aggregates` KEEPS NO HISTORY** — upserted in place on
|
||||
(sport,season,source_id,role), ONE as-of date, prior versions destroyed. The
|
||||
first skill backtest was honest only BY ACCIDENT: the nightly refresh was
|
||||
unreachable code so profiles sat frozen at 2026-07-21, BEFORE the settled
|
||||
window. Fixing that cron refreshed them to today and made point-in-time
|
||||
validation impossible from that table. **`statcast_history` (new) retains a
|
||||
dated snapshot per refresh** — query `where as_of_date < game_date order by
|
||||
as_of_date desc limit 1`. Retention is best-effort and must NEVER fail the
|
||||
refresh (unit-tested). Until it accrues a window, ALL skill-feature results are
|
||||
CONTAMINATED/DIRECTIONAL, never gate verdicts.
|
||||
- **TB solo pass: NOTHING passes.** n=383, Bonferroni-12 (α=0.00417).
|
||||
`hard_hit_pct` is closest at marginal r=0.135, p=0.0080 — fails BOTH the 0.15
|
||||
effect bar and corrected α. **It DRIFTED DOWN from 0.153 (n=295) → 0.135
|
||||
(n=383)**: an estimate regressing as noise averages out, not an effect firming.
|
||||
Don't keep quoting the older better number.
|
||||
- **Interactions: none pass.** Only `barrel × power_archetype` has incremental
|
||||
(partial, controlling for both components) exceeding its parts — −0.101 vs
|
||||
0.019 at n=260. A lead, not a finding.
|
||||
- **INTERACTION-PROXY TRAP:** the archetype conditioner was first
|
||||
`barrel_pct/LEAGUE.barrel_pct` — a monotone transform of its own component — so
|
||||
the "interaction" was barrel² measuring NONLINEARITY, and it produced the run's
|
||||
only positive result (−0.132). A Gauss-Jordan pivot test does NOT catch this
|
||||
(the columns differ by a scale factor); use a **scale-free pairwise correlation
|
||||
check** on control columns. Real archetype labels come from
|
||||
`model_snapshots.archetype` (260 labelled TB rows: 140 BOMBER / 120 other).
|
||||
- **Interactions must be scored by PARTIAL correlation** vs the counter residual,
|
||||
controlling for both components — raw correlation can't distinguish
|
||||
PASSES-AND-ADDS from PASSES-BUT-REDUNDANT.
|
||||
- **TB is at PARITY with the counter** (0.2718 vs 0.2647, CI [−0.065,+0.079],
|
||||
inconclusive) where HITS lost by 0.116 with CI excluding zero. Same engine,
|
||||
same day — the stat choice was the whole story. Parity under contamination is
|
||||
NOT a win; nothing promoted.
|
||||
- **`skillProjection` now models total_bases** as a compound convolution (per-PA
|
||||
0/1/2/3/4 bases; barrel→HR share, exit velo→2B/3B share). The old
|
||||
deterministic bases-per-hit made P(TB>=2) EXACTLY P(hits>=1); non-degeneracy is
|
||||
locked by test.
|
||||
|
||||
## Active Skills
|
||||
- vyndr-voice (all user-facing output)
|
||||
- prop-analysis (grading methodology)
|
||||
|
||||
@@ -0,0 +1,424 @@
|
||||
#!/usr/bin/env node
|
||||
'use strict';
|
||||
|
||||
/**
|
||||
* tb-solo-and-interactions — PROVE BOTH, with the solo pass as the control.
|
||||
*
|
||||
* A feature can carry signal alone, only in combination, or both. Testing only
|
||||
* interactions misses solo-real features AND cannot tell whether an interaction
|
||||
* ADDS anything or merely re-encodes its own parts. So the solo result is the
|
||||
* baseline every interaction has to beat.
|
||||
*
|
||||
* ── HOW "ADDS OVER ITS PARTS" IS MEASURED ────────────────────────────────
|
||||
* Not by comparing two correlations by eye. The interaction's incremental
|
||||
* signal is the PARTIAL correlation of the interaction term with the counter's
|
||||
* residual, CONTROLLING FOR both component features:
|
||||
*
|
||||
* resid_I = I − OLS(I ~ A, B)
|
||||
* resid_Y = Y − OLS(Y ~ A, B)
|
||||
* incremental r = corr(resid_I, resid_Y)
|
||||
*
|
||||
* If the interaction is just barrel-rate wearing a different hat, regressing out
|
||||
* barrel rate removes it and the incremental r collapses to ~0. That is exactly
|
||||
* the redundancy the order is guarding against, and it is the difference between
|
||||
* PASSES-AND-ADDS and PASSES-BUT-REDUNDANT.
|
||||
*
|
||||
* ── WHY THE COUNTER'S RESIDUAL IS THE TARGET ─────────────────────────────
|
||||
* Correlating with the raw outcome rewards a feature for knowing what the
|
||||
* counter already knows. Only the part the counter MISSES is new information,
|
||||
* and only new information can improve the product. Both are reported; the
|
||||
* residual one is the one that decides.
|
||||
*
|
||||
* ── THEORY FIRST ─────────────────────────────────────────────────────────
|
||||
* Every interaction below is declared with a MECHANISM before it is measured.
|
||||
* No blind pairwise search — with 8 features there are 28 pairs, and at α=.05
|
||||
* roughly one in twenty returns "significant" from noise alone.
|
||||
*
|
||||
* SUPABASE_URL=... node scripts/tb-solo-and-interactions.js
|
||||
*/
|
||||
|
||||
require('dotenv').config();
|
||||
const { createClient } = require('@supabase/supabase-js');
|
||||
const cv = require('../src/services/model/correlateValidator');
|
||||
const sk = require('../src/services/model/skillProjection');
|
||||
const reg = require('../src/services/model/featureRegistry');
|
||||
const mlb = require('../src/services/adapters/mlbStatsAdapter');
|
||||
const { knownRate, knownNumber } = require('../src/utils/known');
|
||||
|
||||
const SB_URL = process.env.SUPABASE_URL;
|
||||
const SB_KEY = process.env.SUPABASE_SERVICE_ROLE_KEY || process.env.SUPABASE_SERVICE_KEY;
|
||||
const PAGE = 1000;
|
||||
const GAMES_SO_FAR = Number(process.env.STAGEA_GAMES_SO_FAR || 103);
|
||||
|
||||
const r4 = (v) => (v == null || !Number.isFinite(v) ? null : Math.round(v * 10000) / 10000);
|
||||
const mean = (a) => (a.length ? a.reduce((x, y) => x + y, 0) / a.length : null);
|
||||
const brier = (ps, ys) => (ps.length ? ps.reduce((s, p, i) => s + (p - ys[i]) ** 2, 0) / ps.length : null);
|
||||
|
||||
/** OLS residuals of y on the given predictor columns (with intercept). */
|
||||
function olsResiduals(y, Xcols) {
|
||||
const n = y.length;
|
||||
const p = Xcols.length + 1;
|
||||
const X = [];
|
||||
for (let i = 0; i < n; i += 1) {
|
||||
const row = [1];
|
||||
for (const c of Xcols) row.push(c[i]);
|
||||
X.push(row);
|
||||
}
|
||||
// Normal equations (X'X) b = X'y, solved by Gauss-Jordan. p is 2-4 here.
|
||||
const XtX = Array.from({ length: p }, () => new Array(p).fill(0));
|
||||
const Xty = new Array(p).fill(0);
|
||||
for (let i = 0; i < n; i += 1) {
|
||||
for (let a = 0; a < p; a += 1) {
|
||||
Xty[a] += X[i][a] * y[i];
|
||||
for (let b = 0; b < p; b += 1) XtX[a][b] += X[i][a] * X[i][b];
|
||||
}
|
||||
}
|
||||
const M = XtX.map((row, i) => [...row, Xty[i]]);
|
||||
for (let col = 0; col < p; col += 1) {
|
||||
let piv = col;
|
||||
for (let r = col + 1; r < p; r += 1) if (Math.abs(M[r][col]) > Math.abs(M[piv][col])) piv = r;
|
||||
if (Math.abs(M[piv][col]) < 1e-12) return null; // singular → cannot control honestly
|
||||
[M[col], M[piv]] = [M[piv], M[col]];
|
||||
const d = M[col][col];
|
||||
for (let k = col; k <= p; k += 1) M[col][k] /= d;
|
||||
for (let r = 0; r < p; r += 1) {
|
||||
if (r === col) continue;
|
||||
const f = M[r][col];
|
||||
for (let k = col; k <= p; k += 1) M[r][k] -= f * M[col][k];
|
||||
}
|
||||
}
|
||||
const beta = M.map((row) => row[p]);
|
||||
return y.map((v, i) => v - X[i].reduce((s, xv, j) => s + xv * beta[j], 0));
|
||||
}
|
||||
|
||||
/**
|
||||
* Partial correlation of a with b, controlling for the columns in ctrl.
|
||||
*
|
||||
* COLLINEARITY IS CHECKED FIRST, and this is not pedantry — it caught a real
|
||||
* error in this very script. The archetype-power proxy was defined as
|
||||
* `barrel_pct / LEAGUE.barrel_pct`, an exact linear function of barrel_pct, so
|
||||
* "control for both components" was rank-deficient and the partial correlation
|
||||
* it produced (-0.132, the only one that looked like an incremental finding) was
|
||||
* an artifact of a singular design matrix. The Gauss-Jordan pivot test missed it
|
||||
* because the two columns differ by a scale factor, which keeps the pivot well
|
||||
* above an absolute epsilon. Scale-free pairwise correlation catches it.
|
||||
*/
|
||||
function partialCorr(a, b, ctrl) {
|
||||
for (let i = 0; i < ctrl.length; i += 1) {
|
||||
for (let j = i + 1; j < ctrl.length; j += 1) {
|
||||
const rr = cv.pearson(ctrl[i], ctrl[j]).r;
|
||||
if (rr !== null && Math.abs(rr) > 0.999) return null; // same variable twice
|
||||
}
|
||||
}
|
||||
const ra = olsResiduals(a, ctrl);
|
||||
const rb = olsResiduals(b, ctrl);
|
||||
if (!ra || !rb) return null;
|
||||
return cv.pearson(ra, rb).r;
|
||||
}
|
||||
|
||||
/** Rows where every named key is known — the honest common sample. */
|
||||
function completeRows(rows, keys) {
|
||||
return rows.filter((r) => keys.every((k) => knownNumber(r[k]) !== null));
|
||||
}
|
||||
|
||||
function makeRnd(seed) {
|
||||
let s = seed >>> 0;
|
||||
return () => { s ^= s << 13; s >>>= 0; s ^= s >>> 17; s ^= s << 5; s >>>= 0; return s / 4294967296; };
|
||||
}
|
||||
function bootstrapDiff(rows, keyA, keyB, iters = 4000, seed = 20260804) {
|
||||
if (rows.length < 30) return null;
|
||||
const rnd = makeRnd(seed);
|
||||
const n = rows.length;
|
||||
const diffs = [];
|
||||
for (let it = 0; it < iters; it += 1) {
|
||||
const ys = []; const a = []; const b = [];
|
||||
for (let i = 0; i < n; i += 1) {
|
||||
const r = rows[Math.floor(rnd() * n)];
|
||||
ys.push(r.won); a.push(r[keyA]); b.push(r[keyB]);
|
||||
}
|
||||
const ca = cv.pearson(a, ys).r; const cb = cv.pearson(b, ys).r;
|
||||
if (ca == null || cb == null) continue;
|
||||
diffs.push(ca - cb);
|
||||
}
|
||||
if (diffs.length < 100) return null;
|
||||
diffs.sort((x, y) => x - y);
|
||||
const q = (pp) => r4(diffs[Math.floor(pp * (diffs.length - 1))]);
|
||||
const ci = [q(0.025), q(0.975)];
|
||||
return {
|
||||
point: r4(cv.pearson(rows.map((r) => r[keyA]), rows.map((r) => r.won)).r
|
||||
- cv.pearson(rows.map((r) => r[keyB]), rows.map((r) => r.won)).r),
|
||||
ci95: ci, ci_excludes_zero: ci[0] > 0 || ci[1] < 0,
|
||||
};
|
||||
}
|
||||
|
||||
async function page(sb, table, select, apply) {
|
||||
const out = [];
|
||||
for (let from = 0; ; from += PAGE) {
|
||||
const { data, error } = await apply(sb.from(table).select(select)).range(from, from + PAGE - 1);
|
||||
if (error) throw error;
|
||||
if (!data || data.length === 0) break;
|
||||
out.push(...data);
|
||||
if (data.length < PAGE) break;
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
async function opposingStarters(dates) {
|
||||
const m = new Map();
|
||||
for (const d of dates) {
|
||||
let games = [];
|
||||
try { games = await mlb.getScheduleWithPitchers(d); } catch { games = []; }
|
||||
for (const g of games) {
|
||||
if (!g.home || !g.away) continue;
|
||||
if (g.home.probablePitcher) m.set(`${d}|OPP:${g.home.team}`, g.home.probablePitcher.id);
|
||||
if (g.away.probablePitcher) m.set(`${d}|OPP:${g.away.team}`, g.away.probablePitcher.id);
|
||||
}
|
||||
}
|
||||
return m;
|
||||
}
|
||||
async function opponentByPlayerDate(players) {
|
||||
const map = new Map();
|
||||
for (const [key, name] of players) {
|
||||
try {
|
||||
const found = await mlb.searchPlayer(name);
|
||||
if (!found || !found.id) continue;
|
||||
const log = await mlb.getPlayerGameLog(found.id);
|
||||
for (const g of log || []) {
|
||||
if (g && g.date && g.opponent) map.set(`${key}|${String(g.date).slice(0, 10)}`, g.opponent);
|
||||
}
|
||||
} catch { /* no log → no pitcher */ }
|
||||
}
|
||||
return map;
|
||||
}
|
||||
|
||||
const SOLO = ['batter_barrel_pct', 'batter_hard_hit_pct', 'batter_exit_velo',
|
||||
'batter_launch_angle', 'batter_k_pct', 'batter_bb_pct',
|
||||
'pitcher_k_pct', 'pitcher_hard_hit_allowed'];
|
||||
|
||||
/** STEP 2 — theory first. Every interaction declares its mechanism. */
|
||||
const INTERACTIONS = [
|
||||
{
|
||||
key: 'launch_x_exit_velo',
|
||||
components: ['batter_launch_angle', 'batter_exit_velo'],
|
||||
mechanism: 'Extra bases need BOTH conditions: hit hard AND hit in the air. A 105-mph ground ball is an out; a 25-degree popup is an out. Neither factor alone predicts bases, which is precisely why each may fail solo and the product may not.',
|
||||
build: (r) => r.batter_launch_angle * r.batter_exit_velo,
|
||||
},
|
||||
{
|
||||
key: 'exitvelo_x_pitcher_suppression',
|
||||
components: ['batter_exit_velo', 'pitcher_hard_hit_allowed'],
|
||||
mechanism: 'A hitter only realises his contact quality against a pitcher who permits contact quality. Elite suppression should attenuate a power bat; a contact-permitting arm should amplify it. The effect is conditional by construction.',
|
||||
build: (r) => r.batter_exit_velo * r.pitcher_hard_hit_allowed,
|
||||
},
|
||||
{
|
||||
key: 'barrel_x_power_archetype',
|
||||
components: ['batter_barrel_pct', 'archetype_power'],
|
||||
mechanism: 'ARCHETYPE-CONDITIONAL. Barrels convert to extra bases for hitters whose lane is power; for a speed/contact profile the same barrel rate is a rarer event on a swing built for something else. This is Discipline 2 stated as a testable interaction. NOTE: it is currently UNTESTABLE — statcast rows carry no archetype label, and the barrel-relative proxy is an exact linear function of barrel_pct, so controlling for both components is rank-deficient. It needs a real archetype classification joined in.',
|
||||
build: (r) => r.batter_barrel_pct * r.archetype_power,
|
||||
},
|
||||
{
|
||||
key: 'batterK_x_pitcherK',
|
||||
components: ['batter_k_pct', 'pitcher_k_pct'],
|
||||
mechanism: 'Strikeout risk compounds multiplicatively (log5 is exactly this shape). A high-K bat against a high-K arm loses plate appearances to strikeouts, and a PA lost is a base opportunity that never happens — so it suppresses total bases through OPPORTUNITY, not contact quality.',
|
||||
build: (r) => r.batter_k_pct * r.pitcher_k_pct,
|
||||
},
|
||||
];
|
||||
|
||||
/** Latest settled game date in the pull — used to detect that the profile
|
||||
* freeze now sits AFTER the data, i.e. no clean out-of-sample window exists. */
|
||||
function clean0Max(rows) {
|
||||
return (rows || []).reduce((mx, r) => (String(r.game_date) > mx ? String(r.game_date) : mx), '');
|
||||
}
|
||||
|
||||
async function main() {
|
||||
if (!SB_URL || !SB_KEY) throw new Error('SUPABASE_URL / service key required');
|
||||
const sb = createClient(SB_URL, SB_KEY, { auth: { persistSession: false } });
|
||||
|
||||
const statcast = await page(sb, 'statcast_aggregates', '*', (q) => q.eq('sport', 'mlb'));
|
||||
const freezeDate = statcast.reduce((mx, r) => (String(r.updated_at) > mx ? String(r.updated_at) : mx), '').slice(0, 10);
|
||||
const batters = new Map(); const pitchersById = new Map();
|
||||
for (const r of statcast) {
|
||||
if (r.role === 'pitcher' && r.source_id != null) pitchersById.set(Number(r.source_id), sk.fromStatcastRow(r));
|
||||
if (r.player_key && r.role === 'batter') {
|
||||
const prev = batters.get(r.player_key);
|
||||
if (!prev || Number(r.sample_pa || 0) > Number(prev.rawPa || 0)) {
|
||||
batters.set(r.player_key, Object.assign(sk.fromStatcastRow(r), { rawPa: Number(r.sample_pa || 0) }));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// REAL ARCHETYPE LABELS. The barrel-relative proxy was a clipped monotone
|
||||
// transform of barrel_pct, so `barrel x proxy` measured NONLINEARITY IN BARREL,
|
||||
// not an archetype interaction — it could never have tested Discipline 2.
|
||||
// model_snapshots carries the actual classification per prop, so the
|
||||
// conditioning variable is now a genuine BOMBER indicator, which is
|
||||
// categorical and therefore not a transform of barrel at all.
|
||||
const snaps = await page(sb, 'model_snapshots', 'player_key, game_date, archetype',
|
||||
(q) => q.eq('sport', 'mlb').eq('stat', 'total_bases').not('archetype', 'is', null));
|
||||
const archetypeBy = new Map();
|
||||
for (const r of snaps) if (r.player_key && r.game_date) archetypeBy.set(`${r.player_key}|${r.game_date}`, r.archetype);
|
||||
|
||||
const led = await page(sb, 'ledger_entries',
|
||||
'player_key, player_name, stat, line, side, outcome, game_date, p_win, quarantine_reason',
|
||||
(q) => q.eq('sport', 'mlb').is('user_id', null).eq('stat', 'total_bases')
|
||||
.in('outcome', ['hit', 'miss']).not('p_win', 'is', null));
|
||||
// POINT-IN-TIME IS NO LONGER AVAILABLE FROM THIS TABLE.
|
||||
//
|
||||
// `statcast_aggregates` is upserted in place and keeps one as-of date. The
|
||||
// first skill backtest was honest only by accident: the nightly refresh was
|
||||
// unreachable code, so the table sat frozen at 2026-07-21 — BEFORE the settled
|
||||
// window. Repairing that cron (correct for production) refreshed it to today,
|
||||
// and every prior version is gone.
|
||||
//
|
||||
// So scoring a 2026-07-25 game now uses a season aggregate that CONTAINS that
|
||||
// game. `statcast_history` (added this session) fixes it going forward; it has
|
||||
// one day of data, which is not yet a window. Until it fills, results here are
|
||||
// DIRECTIONAL AND CONTAMINATED, labelled as such, and are NOT gate verdicts.
|
||||
const contaminated = String(freezeDate) >= String(clean0Max(led));
|
||||
const clean = led.filter((r) => !(r.quarantine_reason || '').startsWith('nontakeable_book')
|
||||
&& (contaminated ? true : String(r.game_date) > freezeDate));
|
||||
|
||||
const dates = [...new Set(clean.map((r) => r.game_date))].sort();
|
||||
const starters = await opposingStarters(dates);
|
||||
const players = new Map();
|
||||
for (const r of clean) if (!players.has(r.player_key)) players.set(r.player_key, r.player_name);
|
||||
const oppByPlayerDate = await opponentByPlayerDate(players);
|
||||
|
||||
if (process.env.TB_DEBUG === '1') {
|
||||
console.error(`[debug] statcast rows=${statcast.length} freeze=${freezeDate} batters=${batters.size} pitchers=${pitchersById.size}`);
|
||||
console.error(`[debug] ledger tb rows=${led.length} clean(after freeze)=${clean.length}`);
|
||||
const sampleKeys = clean.slice(0, 5).map((r) => r.player_key);
|
||||
console.error(`[debug] sample ledger player_keys=${JSON.stringify(sampleKeys)}`);
|
||||
console.error(`[debug] sample statcast keys=${JSON.stringify([...batters.keys()].slice(0, 5))}`);
|
||||
console.error(`[debug] matches in sample=${sampleKeys.filter((k) => batters.has(k)).length}/5`);
|
||||
}
|
||||
const allowed = reg.candidateFeaturesForStat('mlb', 'total_bases');
|
||||
const rows = [];
|
||||
for (const r of clean) {
|
||||
const bat = batters.get(r.player_key);
|
||||
if (!bat) continue;
|
||||
const faced = oppByPlayerDate.get(`${r.player_key}|${r.game_date}`) || null;
|
||||
const pit = faced ? pitchersById.get(Number(starters.get(`${r.game_date}|OPP:${faced}`))) || null : null;
|
||||
const paRate = bat.rawPa > 0 ? Math.min(5.2, Math.max(2.0, bat.rawPa / GAMES_SO_FAR)) : null;
|
||||
const under = String(r.side).toLowerCase() === 'under';
|
||||
const won = r.outcome === 'hit' ? 1 : 0;
|
||||
const champ = Number(r.p_win);
|
||||
|
||||
const proj = sk.projectSkill({
|
||||
batter: bat, pitcher: pit, park: 1, archetype: null,
|
||||
statType: 'total_bases', line: Number(r.line), expectedPa: paRate, allowed,
|
||||
});
|
||||
|
||||
// The REAL archetype for this prop — a 0/1 power indicator, categorical and
|
||||
// independent of barrel_pct by construction.
|
||||
const arch = archetypeBy.get(`${r.player_key}|${r.game_date}`) || null;
|
||||
const archetypePower = arch == null ? null : (String(arch).toUpperCase() === 'BOMBER' ? 1 : 0);
|
||||
|
||||
rows.push({
|
||||
won, champ, residual: won - champ,
|
||||
skill: proj ? (under ? 1 - proj.p_over_line : proj.p_over_line) : null,
|
||||
had_pitcher: !!pit,
|
||||
batter_barrel_pct: knownRate(bat.barrel_pct),
|
||||
batter_hard_hit_pct: knownRate(bat.hard_hit_pct),
|
||||
batter_exit_velo: knownRate(bat.avg_exit_velo),
|
||||
batter_launch_angle: knownRate(bat.avg_launch_angle),
|
||||
batter_k_pct: knownRate(bat.k_pct),
|
||||
batter_bb_pct: knownRate(bat.bb_pct),
|
||||
pitcher_k_pct: pit ? knownRate(pit.k_pct) : null,
|
||||
pitcher_hard_hit_allowed: pit ? knownRate(pit.hard_hit_pct) : null,
|
||||
archetype_power: archetypePower,
|
||||
archetype: arch,
|
||||
});
|
||||
}
|
||||
|
||||
// Bonferroni denominator = every test in this family (solo + interaction).
|
||||
const TESTS = SOLO.length + INTERACTIONS.length;
|
||||
|
||||
// ── STEP 1 — SOLO PASS (the control) ────────────────────────────────────
|
||||
const solo = {};
|
||||
for (const f of SOLO) {
|
||||
const rs = completeRows(rows, [f]);
|
||||
solo[f] = {
|
||||
n: rs.length,
|
||||
vs_outcome: cv.validateFactor(rs.map((r) => r[f]), rs.map((r) => r.won), TESTS),
|
||||
vs_counter_residual: cv.validateFactor(rs.map((r) => r[f]), rs.map((r) => r.residual), TESTS),
|
||||
};
|
||||
}
|
||||
|
||||
// ── STEP 3 — INTERACTIONS, each against its own solo baseline ───────────
|
||||
const interactions = {};
|
||||
for (const ix of INTERACTIONS) {
|
||||
const keys = [...ix.components];
|
||||
const rs = completeRows(rows, keys);
|
||||
if (rs.length < 30) { interactions[ix.key] = { mechanism: ix.mechanism, n: rs.length, verdict: 'UNTESTABLE — no common sample' }; continue; }
|
||||
const I = rs.map(ix.build);
|
||||
const Y = rs.map((r) => r.residual);
|
||||
const ctrl = keys.map((k) => rs.map((r) => r[k]));
|
||||
|
||||
const gate = cv.validateFactor(I, Y, TESTS);
|
||||
const incremental = partialCorr(I, Y, ctrl);
|
||||
// The best solo |r| among its own components, on the SAME rows.
|
||||
const componentSolo = keys.map((k) => ({
|
||||
feature: k, r: r4(cv.pearson(rs.map((r) => r[k]), Y).r),
|
||||
}));
|
||||
const bestComponent = Math.max(...componentSolo.map((c) => Math.abs(c.r ?? 0)));
|
||||
|
||||
let verdict;
|
||||
if (incremental === null) verdict = 'UNTESTABLE — controls are collinear';
|
||||
else if (gate.validated && Math.abs(incremental) >= cv.VALIDATION_REQUIREMENTS.min_pearson_r) verdict = 'PASSES-AND-ADDS';
|
||||
else if (gate.validated) verdict = 'PASSES-BUT-REDUNDANT';
|
||||
else if (rs.length < cv.VALIDATION_REQUIREMENTS.min_historical_instances) verdict = 'UNDERPOWERED — n below the gate';
|
||||
else verdict = 'FAILS';
|
||||
|
||||
interactions[ix.key] = {
|
||||
mechanism: ix.mechanism,
|
||||
components: keys,
|
||||
n: rs.length,
|
||||
raw_r_vs_residual: gate.pearson_r,
|
||||
gate: { validated: gate.validated, reason: gate.reason, p_value: gate.p_value, corrected_alpha: gate.corrected_alpha, underpowered: !!gate.underpowered },
|
||||
component_solo_r_same_rows: componentSolo,
|
||||
best_component_abs_r: r4(bestComponent),
|
||||
INCREMENTAL_partial_r: r4(incremental),
|
||||
adds_over_components: incremental !== null && Math.abs(incremental) > bestComponent,
|
||||
verdict,
|
||||
};
|
||||
}
|
||||
|
||||
// ── STEP 4 — COMBINED vs COUNTER (valid at this n; the gate is not) ─────
|
||||
const h2h = rows.filter((r) => r.skill != null);
|
||||
const ys = h2h.map((r) => r.won);
|
||||
const bs = bootstrapDiff(h2h, 'skill', 'champ');
|
||||
|
||||
console.log(JSON.stringify({
|
||||
stat: 'total_bases',
|
||||
VALIDITY: contaminated
|
||||
? 'CONTAMINATED / DIRECTIONAL ONLY — statcast_aggregates now carries a single as-of date (' + freezeDate + ') that is AFTER the settled games, so season profiles contain the games being predicted. These are NOT gate verdicts. statcast_history (new) makes point-in-time possible from tomorrow.'
|
||||
: `CLEAN out-of-sample: profiles frozen ${freezeDate}; only game_date > ${freezeDate} scored`,
|
||||
contaminated,
|
||||
rows_scored: rows.length,
|
||||
gate_spec: cv.VALIDATION_REQUIREMENTS,
|
||||
bonferroni_tests: TESTS,
|
||||
n_gap_note: `the gate needs ${cv.VALIDATION_REQUIREMENTS.min_historical_instances} rows; this run has ${rows.length}`,
|
||||
archetype_coverage: {
|
||||
labelled: rows.filter((r) => r.archetype).length,
|
||||
bomber: rows.filter((r) => r.archetype_power === 1).length,
|
||||
other: rows.filter((r) => r.archetype_power === 0).length,
|
||||
},
|
||||
step1_solo_baseline: solo,
|
||||
step3_interactions: interactions,
|
||||
step4_combined_vs_counter: {
|
||||
n: h2h.length,
|
||||
pitcher_coverage: r4(mean(h2h.map((r) => (r.had_pitcher ? 1 : 0)))),
|
||||
base_rate: r4(mean(ys)),
|
||||
resolution: { skill_tb: r4(cv.pearson(h2h.map((r) => r.skill), ys).r), counter: r4(cv.pearson(h2h.map((r) => r.champ), ys).r) },
|
||||
brier: { skill_tb: r4(brier(h2h.map((r) => r.skill), ys)), counter: r4(brier(h2h.map((r) => r.champ), ys)) },
|
||||
delta: bs,
|
||||
verdict: !bs ? 'N-BLOCKED'
|
||||
: (bs.ci_excludes_zero && bs.point > 0) ? 'SKILL TB BEATS THE COUNTER'
|
||||
: (bs.ci_excludes_zero && bs.point < 0) ? 'LOSES to the counter'
|
||||
: 'INCONCLUSIVE',
|
||||
},
|
||||
}, null, 2));
|
||||
process.exit(0);
|
||||
}
|
||||
|
||||
main().catch((e) => { console.error(e); process.exit(1); });
|
||||
@@ -0,0 +1,162 @@
|
||||
# TOTAL BASES — solo baseline, interactions, and the validity blocker
|
||||
|
||||
**2026-08-03.** Challenger-only. Counter byte-identical (zero diff on
|
||||
`probabilityEstimator.js` / `analyzeViaEngine1.js`).
|
||||
|
||||
> **HEADLINE 1 — nothing can be gate-validated today, and the reason is my own
|
||||
> fix.** `statcast_aggregates` is upserted in place and keeps **one as-of date**.
|
||||
> The first skill backtest was honest only by accident: the nightly refresh was
|
||||
> unreachable code, so profiles sat frozen at 2026-07-21 — *before* the settled
|
||||
> window. Repairing that cron (correct for production) refreshed them to today
|
||||
> and destroyed every prior version. **Point-in-time validation is now
|
||||
> structurally impossible from that table**, so every number below is
|
||||
> **contaminated / directional** and none is a gate verdict.
|
||||
>
|
||||
> **HEADLINE 2 — on total bases the skill model is at PARITY with the counter**
|
||||
> (0.2718 vs 0.2647, CI includes zero). On hits it lost by 0.116 with a CI
|
||||
> excluding zero. Same engine, same day. **The stat choice was the whole story.**
|
||||
|
||||
---
|
||||
|
||||
## 0. Premise correction
|
||||
|
||||
The order states "barrel rate PASSED, exit velo/launch/matchup FAILED solo."
|
||||
**Nothing passed.** Yesterday's run refused *every* TB feature on
|
||||
`insufficient_data` (n=295 < 500); barrel rate's marginal r was 0.088 and the
|
||||
strongest was hard-hit at 0.153. Today at n=383, **hard-hit has fallen to
|
||||
0.135** — see §3, because that drift is itself the most informative number here.
|
||||
|
||||
## 1. The validity blocker, and the fix
|
||||
|
||||
```
|
||||
select updated_at::date, count(*) from statcast_aggregates group by 1;
|
||||
→ 2026-08-03 | 1387 ← ONE row. No history. 2026-07-21 is gone.
|
||||
```
|
||||
|
||||
A season aggregate dated 2026-08-03 used to score a 2026-07-25 game **contains
|
||||
that game**. That is leakage, and it is not repairable after the fact.
|
||||
|
||||
**Fixed forward:** `statcast_history` (migration applied) retains a dated
|
||||
snapshot on every refresh, keyed `(as_of_date, sport, season, source_id, role)`.
|
||||
Point-in-time becomes `where as_of_date < game_date order by as_of_date desc
|
||||
limit 1` — the profile as it stood *before* the game, by construction. Retention
|
||||
is best-effort and **cannot fail the refresh**; both properties are unit-tested.
|
||||
It has one day of data, which is not yet a window.
|
||||
|
||||
## 2. Feature → conditioning variable → mechanism (STEP 2)
|
||||
|
||||
| interaction | conditioning on | mechanism |
|
||||
|---|---|---|
|
||||
| `launch × exit_velo` | each other | Extra bases need **both**: hit hard AND in the air. A 105-mph ground ball is an out; a 25° popup is an out. Neither predicts alone — which is exactly why each can fail solo and the product might not. |
|
||||
| `exit_velo × pitcher_hard_hit_allowed` | pitcher suppression | A hitter only realises contact quality against a pitcher who permits it. Elite suppression attenuates a power bat; a contact-permitting arm amplifies it. Conditional by construction. |
|
||||
| `barrel × power archetype` | **real archetype label** | Barrels convert to extra bases for hitters whose lane is power. This is Discipline 2 as a testable claim. |
|
||||
| `batter_K × pitcher_K` | each other | Strikeout risk compounds (log5 is this shape). A PA lost to a strikeout is a base opportunity that never happens — suppression through **opportunity**, not contact. |
|
||||
| `pull_rate × park` | park | **UNTESTABLE** — park is passed as 1.0 everywhere and pull rate is not ingested. |
|
||||
|
||||
## 3. STEP 1 — solo baseline (the control)
|
||||
|
||||
n=383, Bonferroni across 12 tests → corrected α = **0.00417**.
|
||||
The decider is correlation with the **counter's residual** (`won − p_win`) — a
|
||||
feature correlating with the raw outcome may simply know what the counter
|
||||
already knows.
|
||||
|
||||
| feature | r vs outcome | **r vs residual** | p | would it pass at n≥500? |
|
||||
|---|---|---|---|---|
|
||||
| **batter_hard_hit_pct** | 0.172 | **0.135** | 0.0080 | **No** — |r| < 0.15 *and* p > 0.00417 |
|
||||
| batter_exit_velo | 0.120 | 0.070 | 0.173 | No |
|
||||
| batter_barrel_pct | 0.109 | 0.078 | 0.127 | No |
|
||||
| pitcher_k_pct | −0.094 | −0.087 | 0.093 | No |
|
||||
| batter_launch_angle | 0.032 | 0.003 | 0.948 | No |
|
||||
| batter_bb_pct | −0.019 | −0.026 | 0.607 | No |
|
||||
| pitcher_hard_hit_allowed | 0.024 | 0.035 | 0.501 | No |
|
||||
| batter_k_pct | −0.009 | −0.003 | 0.952 | No |
|
||||
|
||||
**No feature passes solo, and hard-hit is drifting the wrong way.** Its marginal
|
||||
r went **0.153 (n=295) → 0.135 (n=383)** as sample grew. That is the signature of
|
||||
an estimate regressing toward its true value as noise averages out, not of a real
|
||||
effect firming up. Yesterday I called it "the most encouraging number this work
|
||||
has produced"; on 88 more rows it is fading. **Worth stating plainly, because the
|
||||
temptation is to keep quoting the older, better number.**
|
||||
|
||||
## 4. STEP 3 — interactions, each against its own parts
|
||||
|
||||
Incremental signal = partial correlation of the interaction with the residual,
|
||||
**controlling for both components**. If an interaction is barrel rate in a hat,
|
||||
regressing out barrel rate collapses it.
|
||||
|
||||
| interaction | n | raw r | best component r | **incremental r** | verdict |
|
||||
|---|---|---|---|---|---|
|
||||
| `barrel × power archetype` | 260 | −0.046 | 0.019 | **−0.101** | UNDERPOWERED — n below gate |
|
||||
| `launch × exit_velo` | 383 | 0.007 | 0.070 | −0.053 | UNDERPOWERED |
|
||||
| `batter_K × pitcher_K` | 378 | −0.050 | 0.087 | −0.024 | UNDERPOWERED |
|
||||
| `exit_velo × pitcher_suppression` | 378 | 0.048 | 0.073 | 0.013 | UNDERPOWERED |
|
||||
|
||||
Only the archetype interaction has an incremental exceeding its parts (0.101 vs
|
||||
0.019) — the shape Discipline 2 predicts — but at n=260, with a negative sign,
|
||||
and nowhere near the gate. **It is a lead, not a finding.**
|
||||
|
||||
### A methodological catch worth recording
|
||||
|
||||
The archetype conditioning variable was first built as
|
||||
`barrel_pct / LEAGUE.barrel_pct` — a monotone transform of one of its own
|
||||
components. That interaction was therefore **barrel², measuring nonlinearity in
|
||||
barrel rate, not an archetype effect at all**, and it produced the run's only
|
||||
"interaction adds" result (−0.132). Two fixes: a scale-free collinearity check on
|
||||
the control columns (the Gauss-Jordan pivot test missed it, because the columns
|
||||
differ by a *scale factor*), and real archetype labels joined from
|
||||
`model_snapshots` (260 labelled rows: 140 BOMBER, 120 other) so the conditioning
|
||||
variable is categorical and independent of barrel by construction.
|
||||
|
||||
**Without that catch this document would have reported a fabricated interaction
|
||||
as the session's finding.**
|
||||
|
||||
## 5. STEP 4 — combined skill model vs the counter
|
||||
|
||||
The compound TB projection was built for this (see §6). Paired bootstrap is
|
||||
valid at this n even though the *gate* is not.
|
||||
|
||||
```
|
||||
n 383 · pitcher coverage 100% · base rate 0.483
|
||||
resolution skill TB 0.2718 counter 0.2647
|
||||
brier skill TB 0.2439 counter 0.2426
|
||||
delta +0.0071 CI95 [−0.0648, +0.0789] → INCONCLUSIVE
|
||||
```
|
||||
|
||||
**Not a win — but the first time a challenger has not LOST.** For contrast, the
|
||||
same engine on hits: −0.1161, CI [−0.1892, −0.0426], excluding zero. The
|
||||
difference between those two lines is the entire argument for moving to total
|
||||
bases, and it is what the physics said: contact quality governs extra bases, not
|
||||
whether a grounder finds a hole.
|
||||
|
||||
## 6. Built this session: the compound TB projection
|
||||
|
||||
`skillProjection` no longer refuses total bases. A deterministic bases-per-hit
|
||||
multiplier made `P(TB≥2)` **exactly** `P(hits≥1)` — a relabelled hits curve — so
|
||||
it was refused. It is now a real convolution: each PA yields 0/1/2/3/4 bases with
|
||||
hit-type shares shifted by skill (barrel → home-run share, exit velo → doubles
|
||||
and triples), convolved over a PA distribution. A BOMBER gets a 37.5% home-run
|
||||
share against a slap hitter's 4.5%. Non-degeneracy is locked by test.
|
||||
|
||||
## 7. Verdict
|
||||
|
||||
- **Solo-proven set for total_bases: EMPTY.** Nothing passes; hard-hit is the
|
||||
closest and is drifting down.
|
||||
- **Interaction-proven set: EMPTY.** One lead (archetype × barrel), underpowered.
|
||||
- **Combined vs counter: INCONCLUSIVE at parity** — genuinely better news than
|
||||
hits, and still not promotable.
|
||||
- **Nothing promoted. Counter untouched.**
|
||||
|
||||
## 8. Next, in order
|
||||
|
||||
1. **Let `statcast_history` accrue.** Nothing here can be validated honestly
|
||||
until there is a point-in-time window — a week gives a real one. This is now
|
||||
the binding constraint on the entire programme, and it is a waiting problem,
|
||||
not a building problem.
|
||||
2. **Re-run this exact script** once history exists and TB clears n≥500 (383
|
||||
today, ~117 short). Both arrive on roughly the same timescale.
|
||||
3. **Archetype × barrel is the one interaction worth carrying forward**, with
|
||||
real labels and adequate n.
|
||||
4. **Do not re-run hits.** §5 settles it.
|
||||
|
||||
**Not recommended:** quoting the 0.153, lowering n≥500, or promoting on a
|
||||
parity result. Parity with contamination is not a win.
|
||||
@@ -354,7 +354,17 @@ function projectSkill({
|
||||
batter, pitcher = null, park = 1, archetype = null,
|
||||
statType = 'hits', line, expectedPa = null, allowed = null,
|
||||
} = {}) {
|
||||
// STAGE A IS HITS ONLY, and total_bases is refused DELIBERATELY.
|
||||
// TOTAL BASES is now modelled properly (see projectTotalBasesSkill). The
|
||||
// deterministic-multiplier version that made P(TB>=2) identical to P(hits>=1)
|
||||
// is gone; this routes to the compound per-hit bases distribution instead.
|
||||
if (String(statType || '').toLowerCase() === 'total_bases') {
|
||||
return projectTotalBasesSkill({
|
||||
batter, pitcher, park, archetype, line, expectedPa, allowed,
|
||||
});
|
||||
}
|
||||
|
||||
// STAGE A IS HITS ONLY for the binomial path, and total_bases used to be
|
||||
// refused DELIBERATELY.
|
||||
//
|
||||
// The first cut mapped hits to bases with an archetype-scaled constant
|
||||
// (bases_per_hit x powerWeight). Smoke-tested on a real profile that made
|
||||
@@ -410,8 +420,131 @@ function buildResult({ pmf, mean, target, pa, map, archetype, stat, caveat = nul
|
||||
};
|
||||
}
|
||||
|
||||
|
||||
/**
|
||||
* LEAGUE HIT-TYPE SHARES — of all hits, how many are singles/doubles/triples/HR.
|
||||
* 2026 MLB approximation; used ONLY as the baseline that skill ratios modulate,
|
||||
* never as a substitute for a player we cannot read.
|
||||
*/
|
||||
const LEAGUE_HIT_SHARES = Object.freeze({ single: 0.635, double: 0.198, triple: 0.017, homer: 0.150 });
|
||||
|
||||
/**
|
||||
* Given a hit, how many bases? THE COMPOUND PIECE tb-v1 established and the
|
||||
* reason total_bases was refused until now.
|
||||
*
|
||||
* A deterministic bases-per-hit multiplier makes P(TB>=2) identical to
|
||||
* P(hits>=1) — a relabelled hits curve with no new information. This instead
|
||||
* shifts the SHARES by the skill inputs the gate says govern extra bases:
|
||||
*
|
||||
* barrel rate → home-run share (a barrel is the HR engine)
|
||||
* exit velocity → double/triple share (gap power)
|
||||
*
|
||||
* Each is a RATIO to league, so a league-average hitter reproduces league
|
||||
* shares exactly and the model says "ordinary" rather than inventing a lean.
|
||||
* Absent inputs are SILENT — the share stays at league — never a measured zero.
|
||||
*/
|
||||
function hitTypeShares({ batter, archetype, allowed }) {
|
||||
const can = (k) => !allowed || allowed.has(k);
|
||||
const map = featureMapFor(archetype);
|
||||
const s = { ...LEAGUE_HIT_SHARES };
|
||||
|
||||
const barrel = can('batter_barrel_pct') ? knownRate(batter && batter.barrel_pct) : null;
|
||||
if (barrel !== null && LEAGUE.barrel_pct > 0) {
|
||||
// Bounded: contact quality moves the HR share, it does not triple it.
|
||||
const ratio = Math.min(2.5, Math.max(0.3, (barrel / LEAGUE.barrel_pct) * map.powerWeight));
|
||||
s.homer = LEAGUE_HIT_SHARES.homer * ratio;
|
||||
}
|
||||
const ev = can('batter_exit_velo') ? knownRate(batter && batter.avg_exit_velo) : null;
|
||||
if (ev !== null && LEAGUE.avg_exit_velo > 0) {
|
||||
// Exit velo has far less leverage on doubles than barrels do on homers, so
|
||||
// the ratio is damped rather than applied raw.
|
||||
const ratio = Math.min(1.6, Math.max(0.6, 1 + ((ev / LEAGUE.avg_exit_velo) - 1) * 3));
|
||||
s.double = LEAGUE_HIT_SHARES.double * ratio;
|
||||
s.triple = LEAGUE_HIT_SHARES.triple * ratio;
|
||||
}
|
||||
// Singles absorb the remainder — a hit is always exactly one of the four.
|
||||
const extra = s.homer + s.double + s.triple;
|
||||
if (extra >= 0.97) { // pathological input; fall back to league rather than emit a negative
|
||||
return { ...LEAGUE_HIT_SHARES };
|
||||
}
|
||||
s.single = 1 - extra;
|
||||
return s;
|
||||
}
|
||||
|
||||
/** Convolve two pmfs, truncating at cap (mass beyond cap lands on cap). */
|
||||
function convolve(a, b, cap) {
|
||||
const out = new Array(cap + 1).fill(0);
|
||||
for (let i = 0; i < a.length; i += 1) {
|
||||
if (!a[i]) continue;
|
||||
for (let j = 0; j < b.length; j += 1) {
|
||||
if (!b[j]) continue;
|
||||
out[Math.min(cap, i + j)] += a[i] * b[j];
|
||||
}
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
const TB_CAP = 16;
|
||||
|
||||
/**
|
||||
* TOTAL BASES as the compound outcome it is: each PA yields 0/1/2/3/4 bases, and
|
||||
* the game total is their convolution over a distribution of plate appearances.
|
||||
*
|
||||
* This is the shape tb-v1 proved correct, now driven by SKILL inputs (barrel,
|
||||
* exit velo, contact quality vs the pitcher) rather than by raw historical
|
||||
* counts — which is the whole thesis, and it is the stat where the gate found
|
||||
* the only above-threshold marginal correlation (hard-hit r = 0.153).
|
||||
*/
|
||||
function projectTotalBasesSkill({ batter, pitcher, park, archetype, line, expectedPa, allowed } = {}) {
|
||||
const pa = paOutcome({ batter, pitcher, park, archetype, allowed });
|
||||
if (!pa) return null;
|
||||
const target = Math.max(1, Math.ceil(Number(line)));
|
||||
if (!Number.isFinite(target)) return null;
|
||||
|
||||
const shares = hitTypeShares({ batter, archetype, allowed });
|
||||
const pHit = pa.p_hit_per_pa;
|
||||
// Bases from ONE plate appearance.
|
||||
const perPa = [
|
||||
1 - pHit,
|
||||
pHit * shares.single,
|
||||
pHit * shares.double,
|
||||
pHit * shares.triple,
|
||||
pHit * shares.homer,
|
||||
];
|
||||
|
||||
const paPmf = paDistribution(expectedPa);
|
||||
let tb = new Array(TB_CAP + 1).fill(0);
|
||||
for (let n = 0; n < paPmf.length; n += 1) {
|
||||
if (!paPmf[n]) continue;
|
||||
let acc = new Array(TB_CAP + 1).fill(0);
|
||||
acc[0] = 1;
|
||||
for (let k = 0; k < n; k += 1) acc = convolve(acc, perPa, TB_CAP);
|
||||
for (let b = 0; b <= TB_CAP; b += 1) tb[b] += paPmf[n] * acc[b];
|
||||
}
|
||||
|
||||
const meanTb = tb.reduce((a, p, i) => a + p * i, 0);
|
||||
const r3 = (v) => Math.round(v * 1000) / 1000;
|
||||
return {
|
||||
version: 'skill-v1',
|
||||
stat: 'total_bases',
|
||||
p_over_line: r3(atLeast(tb, target)),
|
||||
projected_value: r3(meanTb),
|
||||
distribution: tb.map(r3),
|
||||
hit_type_shares: { single: r3(shares.single), double: r3(shares.double), triple: r3(shares.triple), homer: r3(shares.homer) },
|
||||
per_pa: {
|
||||
k_rate: r3(pa.k_rate), bb_rate: r3(pa.bb_rate),
|
||||
bip_rate: r3(pa.bip_rate), hit_on_contact: r3(pa.hit_on_contact),
|
||||
p_hit_per_pa: r3(pHit),
|
||||
},
|
||||
archetype: String(archetype || 'DEFAULT').toUpperCase(),
|
||||
pitcher_applied: pa.inputs_used.pitcher_applied,
|
||||
family: 'pa_compound_bases_convolution',
|
||||
};
|
||||
}
|
||||
|
||||
module.exports = {
|
||||
projectSkill, paOutcome, hitOnContact, oddsRatio, shrink, fromStatcastRow,
|
||||
projectSkill, projectTotalBasesSkill, hitTypeShares, convolve, paOutcome, hitOnContact, oddsRatio, shrink, fromStatcastRow,
|
||||
binomialPmf, paDistribution, atLeast, featureMapFor,
|
||||
LEAGUE, ARCHETYPE_MAP, DEFAULT_PA, PA_CAP, PCT_FIELDS, RAW_FIELDS,
|
||||
LEAGUE_HIT_SHARES, TB_CAP,
|
||||
};
|
||||
|
||||
@@ -255,6 +255,42 @@ async function refreshSeason(opts = {}) {
|
||||
summary.written += batch.length;
|
||||
}
|
||||
|
||||
// ── POINT-IN-TIME RETENTION ───────────────────────────────────────────
|
||||
// `statcast_aggregates` is upserted in place, so it holds exactly ONE as-of
|
||||
// date and every prior version is destroyed. That silently makes any backtest
|
||||
// leak: scoring a 2026-07-25 game with a 2026-08-03 season aggregate feeds the
|
||||
// model the games it is being asked to predict.
|
||||
//
|
||||
// It went unnoticed only because this job was unreachable code (see
|
||||
// snapshotScheduler) and the table sat frozen at 2026-07-21 — accidentally
|
||||
// BEFORE the settled window, which is the sole reason the first skill backtest
|
||||
// was honest. Repairing the cron removed the accident.
|
||||
//
|
||||
// Best-effort, exactly like the ledger write: a retention failure must never
|
||||
// fail the refresh, because stale-but-current data still beats no data.
|
||||
try {
|
||||
const asOf = (opts.asOfDate || started).slice(0, 10);
|
||||
let retained = 0;
|
||||
for (let i = 0; i < rows.length; i += chunk) {
|
||||
const batch = rows.slice(i, i + chunk).map((r) => {
|
||||
const row = toDbRow(r);
|
||||
delete row.updated_at; delete row.source;
|
||||
delete row.position; delete row.role_detail;
|
||||
delete row.games_started; delete row.games_pitched;
|
||||
delete row.saves; delete row.holds;
|
||||
return { ...row, as_of_date: asOf };
|
||||
});
|
||||
const { error } = await sb.from('statcast_history')
|
||||
.upsert(batch, { onConflict: 'as_of_date,sport,season,source_id,role' });
|
||||
if (error) { summary.history_error = error.message; break; }
|
||||
retained += batch.length;
|
||||
}
|
||||
summary.history_retained = retained;
|
||||
summary.history_as_of = asOf;
|
||||
} catch (e) {
|
||||
summary.history_error = e.message;
|
||||
}
|
||||
|
||||
return summary;
|
||||
}
|
||||
|
||||
|
||||
@@ -186,11 +186,34 @@ describe('honesty — unknown is not zero, and absent beats invented', () => {
|
||||
expect(Math.abs(noBarrel - full)).toBeLessThan(Math.abs(zeroBarrel - full));
|
||||
});
|
||||
|
||||
it('total_bases is REFUSED rather than shipped as a relabelled hits curve', () => {
|
||||
// A deterministic bases-per-hit multiplier made P(TB>=2) identical to
|
||||
// P(hits>=1). Refusing is correct until a real per-hit bases distribution
|
||||
// exists (tb-v1's compound shape).
|
||||
expect(sk.projectSkill({ batter: BOMBER, pitcher: ACE, archetype: 'BOMBER', statType: 'total_bases', line: 1.5 })).toBeNull();
|
||||
it('total_bases is COMPOUND, not a relabelled hits curve', () => {
|
||||
// The first cut multiplied hits by a constant bases-per-hit, which made
|
||||
// P(TB>=2) EXACTLY equal to P(hits>=1) — a relabel carrying no information a
|
||||
// hits model did not already have. It was refused rather than shipped. It is
|
||||
// now a real convolution over per-PA base outcomes, and this test locks the
|
||||
// property that distinguishes the two.
|
||||
const tb = sk.projectSkill({ batter: BOMBER, pitcher: ACE, archetype: 'BOMBER', statType: 'total_bases', line: 1.5, expectedPa: 4.2 });
|
||||
const hits = sk.projectSkill({ batter: BOMBER, pitcher: ACE, archetype: 'BOMBER', statType: 'hits', line: 0.5, expectedPa: 4.2 });
|
||||
expect(tb).not.toBeNull();
|
||||
expect(tb.family).toBe('pa_compound_bases_convolution');
|
||||
expect(tb.p_over_line).not.toBeCloseTo(hits.p_over_line, 3); // NOT degenerate
|
||||
expect(tb.distribution.reduce((a, b) => a + b, 0)).toBeCloseTo(1, 2);
|
||||
});
|
||||
|
||||
it('hit-type shares respond to power skill — a slugger homers more per hit', () => {
|
||||
const slugger = sk.hitTypeShares({ batter: BOMBER, archetype: 'BOMBER' });
|
||||
const slap = sk.hitTypeShares({ batter: GHOST, archetype: 'GHOST' });
|
||||
expect(slugger.homer).toBeGreaterThan(slap.homer * 3);
|
||||
expect(slap.single).toBeGreaterThan(slugger.single);
|
||||
// Shares are a distribution over hit types.
|
||||
for (const s of [slugger, slap]) {
|
||||
expect(s.single + s.double + s.triple + s.homer).toBeCloseTo(1, 6);
|
||||
}
|
||||
});
|
||||
|
||||
it('an absent power read leaves the shares at league — never a guessed lean', () => {
|
||||
const noSkill = sk.hitTypeShares({ batter: { k_pct: 0.2 }, archetype: 'DEFAULT' });
|
||||
expect(noSkill).toEqual(sk.LEAGUE_HIT_SHARES);
|
||||
});
|
||||
|
||||
it('the distribution is a real distribution and P(>=k) is monotone', () => {
|
||||
|
||||
@@ -144,10 +144,40 @@ describe('refreshSeason — the job', () => {
|
||||
});
|
||||
|
||||
it('upserts on the natural key so a re-run never duplicates', async () => {
|
||||
let opts = null;
|
||||
const sb = { from: () => ({ upsert: async (_b, o) => { opts = o; return { error: null }; } }) };
|
||||
// Track PER TABLE — the refresh now writes the aggregate AND its dated
|
||||
// history snapshot, and the two have deliberately different natural keys.
|
||||
const seen = {};
|
||||
const sb = { from: (t) => ({ upsert: async (_b, o) => { seen[t] = o; return { error: null }; } }) };
|
||||
await svc.refreshSeason({ supabase: sb, fetchSeason: async () => feeds({ batter: [BELL] }) });
|
||||
expect(opts.onConflict).toBe('sport,season,source_id,role');
|
||||
expect(seen.statcast_aggregates.onConflict).toBe('sport,season,source_id,role');
|
||||
});
|
||||
|
||||
it('RETAINS a dated point-in-time snapshot alongside the live aggregate', async () => {
|
||||
// statcast_aggregates is upserted IN PLACE, so it holds one as-of date and
|
||||
// destroys every earlier version — which silently makes any backtest score a
|
||||
// game with a profile that already contains it. The history table is the
|
||||
// only thing that makes point-in-time validation possible at all.
|
||||
const byTable = {};
|
||||
const sb = { from: (t) => ({ upsert: async (b, o) => { (byTable[t] = byTable[t] || []).push({ b, o }); return { error: null }; } }) };
|
||||
const out = await svc.refreshSeason({
|
||||
supabase: sb, fetchSeason: async () => feeds({ batter: [BELL] }), now: '2026-08-03T11:00:00.000Z',
|
||||
});
|
||||
expect(out.history_retained).toBe(1);
|
||||
expect(out.history_as_of).toBe('2026-08-03');
|
||||
expect(byTable.statcast_history[0].o.onConflict).toBe('as_of_date,sport,season,source_id,role');
|
||||
expect(byTable.statcast_history[0].b[0].as_of_date).toBe('2026-08-03');
|
||||
});
|
||||
|
||||
it('a retention failure NEVER fails the refresh — stale-but-current beats nothing', async () => {
|
||||
const sb = { from: (t) => ({
|
||||
upsert: async () => (t === 'statcast_history'
|
||||
? { error: { message: 'history table missing' } }
|
||||
: { error: null }),
|
||||
}) };
|
||||
const out = await svc.refreshSeason({ supabase: sb, fetchSeason: async () => feeds({ batter: [BELL] }) });
|
||||
expect(out.ok).toBe(true); // the refresh still succeeded
|
||||
expect(out.written).toBe(1);
|
||||
expect(out.history_error).toMatch(/history table missing/);
|
||||
});
|
||||
|
||||
it('REFUSES to write when every feed is empty — a bad night cannot blank a good table', async () => {
|
||||
|
||||
Reference in New Issue
Block a user