Prove both on total bases -- and find that my own fix destroyed the backtest
Nothing passed. Nothing promoted. Counter byte-identical. THE BLOCKER, which is the real finding. statcast_aggregates is upserted in place and holds exactly one as-of date. Yesterday's skill backtest was honest only by accident: the nightly refresh was unreachable code, so the profiles sat frozen at 2026-07-21 -- before the settled window. Repairing that cron was right for production and it refreshed them to today, destroying every prior version. Scoring a 2026-07-25 game now uses a season aggregate that contains that game. Point-in-time validation is structurally impossible from that table, so every number in this run is contaminated and directional, and none of it is a gate verdict. Fixed forward: statcast_history retains a dated snapshot on every refresh, so point-in-time becomes "as_of_date < game_date, most recent". Retention is best-effort and cannot fail the refresh; both properties are unit-tested. It has one day of data, which is not yet a window. SOLO BASELINE, n=383, Bonferroni across 12 tests (alpha 0.00417): nothing passes. hard_hit_pct is closest at marginal r 0.135 with p 0.0080, failing both the 0.15 effect bar and the corrected alpha. And it drifted DOWN from 0.153 at n=295 -- an estimate regressing as noise averages out, not an effect firming up. I called that number encouraging yesterday; on 88 more rows it is fading, and it should not keep being quoted at its best value. INTERACTIONS, each scored by partial correlation against the counter residual controlling for both of its own components: none pass. Only barrel x power archetype has an incremental exceeding its parts (-0.101 against 0.019) at n=260 -- the shape Discipline 2 predicts, but a lead, not a finding. A methodological catch worth keeping. The archetype conditioner was first built as barrel_pct over league barrel -- a monotone transform of one of its own components -- so the "interaction" was barrel squared, measuring nonlinearity in barrel rate rather than any archetype effect, and it produced this run's only positive result. A Gauss-Jordan pivot test does not catch that, because the two columns differ by a scale factor. Fixed with a scale-free collinearity check plus real archetype labels joined from model_snapshots. Without it this document would have reported a fabricated interaction as the session's finding. COMBINED vs COUNTER on total bases: 0.2718 against 0.2647, delta +0.0071, CI [-0.065, +0.079] -- inconclusive, and the first time a challenger has not lost. The same engine on hits was -0.116 with a CI excluding zero. That contrast is the whole argument for total bases, and it is what the physics said: contact quality governs extra bases, not whether a grounder finds a hole. Also built: the compound TB projection. skillProjection no longer refuses total bases -- a deterministic bases-per-hit multiplier had made P(TB>=2) exactly P(hits>=1), a relabelled hits curve. It is now a convolution over per-PA base outcomes with hit-type shares shifted by skill. Non-degeneracy is locked by test. 4,204 tests green (334 suites); web build exit 0. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W1sivYNqY2TS5ftykmHBU9
This commit is contained in:
@@ -354,7 +354,17 @@ function projectSkill({
|
||||
batter, pitcher = null, park = 1, archetype = null,
|
||||
statType = 'hits', line, expectedPa = null, allowed = null,
|
||||
} = {}) {
|
||||
// STAGE A IS HITS ONLY, and total_bases is refused DELIBERATELY.
|
||||
// TOTAL BASES is now modelled properly (see projectTotalBasesSkill). The
|
||||
// deterministic-multiplier version that made P(TB>=2) identical to P(hits>=1)
|
||||
// is gone; this routes to the compound per-hit bases distribution instead.
|
||||
if (String(statType || '').toLowerCase() === 'total_bases') {
|
||||
return projectTotalBasesSkill({
|
||||
batter, pitcher, park, archetype, line, expectedPa, allowed,
|
||||
});
|
||||
}
|
||||
|
||||
// STAGE A IS HITS ONLY for the binomial path, and total_bases used to be
|
||||
// refused DELIBERATELY.
|
||||
//
|
||||
// The first cut mapped hits to bases with an archetype-scaled constant
|
||||
// (bases_per_hit x powerWeight). Smoke-tested on a real profile that made
|
||||
@@ -410,8 +420,131 @@ function buildResult({ pmf, mean, target, pa, map, archetype, stat, caveat = nul
|
||||
};
|
||||
}
|
||||
|
||||
|
||||
/**
|
||||
* LEAGUE HIT-TYPE SHARES — of all hits, how many are singles/doubles/triples/HR.
|
||||
* 2026 MLB approximation; used ONLY as the baseline that skill ratios modulate,
|
||||
* never as a substitute for a player we cannot read.
|
||||
*/
|
||||
const LEAGUE_HIT_SHARES = Object.freeze({ single: 0.635, double: 0.198, triple: 0.017, homer: 0.150 });
|
||||
|
||||
/**
|
||||
* Given a hit, how many bases? THE COMPOUND PIECE tb-v1 established and the
|
||||
* reason total_bases was refused until now.
|
||||
*
|
||||
* A deterministic bases-per-hit multiplier makes P(TB>=2) identical to
|
||||
* P(hits>=1) — a relabelled hits curve with no new information. This instead
|
||||
* shifts the SHARES by the skill inputs the gate says govern extra bases:
|
||||
*
|
||||
* barrel rate → home-run share (a barrel is the HR engine)
|
||||
* exit velocity → double/triple share (gap power)
|
||||
*
|
||||
* Each is a RATIO to league, so a league-average hitter reproduces league
|
||||
* shares exactly and the model says "ordinary" rather than inventing a lean.
|
||||
* Absent inputs are SILENT — the share stays at league — never a measured zero.
|
||||
*/
|
||||
function hitTypeShares({ batter, archetype, allowed }) {
|
||||
const can = (k) => !allowed || allowed.has(k);
|
||||
const map = featureMapFor(archetype);
|
||||
const s = { ...LEAGUE_HIT_SHARES };
|
||||
|
||||
const barrel = can('batter_barrel_pct') ? knownRate(batter && batter.barrel_pct) : null;
|
||||
if (barrel !== null && LEAGUE.barrel_pct > 0) {
|
||||
// Bounded: contact quality moves the HR share, it does not triple it.
|
||||
const ratio = Math.min(2.5, Math.max(0.3, (barrel / LEAGUE.barrel_pct) * map.powerWeight));
|
||||
s.homer = LEAGUE_HIT_SHARES.homer * ratio;
|
||||
}
|
||||
const ev = can('batter_exit_velo') ? knownRate(batter && batter.avg_exit_velo) : null;
|
||||
if (ev !== null && LEAGUE.avg_exit_velo > 0) {
|
||||
// Exit velo has far less leverage on doubles than barrels do on homers, so
|
||||
// the ratio is damped rather than applied raw.
|
||||
const ratio = Math.min(1.6, Math.max(0.6, 1 + ((ev / LEAGUE.avg_exit_velo) - 1) * 3));
|
||||
s.double = LEAGUE_HIT_SHARES.double * ratio;
|
||||
s.triple = LEAGUE_HIT_SHARES.triple * ratio;
|
||||
}
|
||||
// Singles absorb the remainder — a hit is always exactly one of the four.
|
||||
const extra = s.homer + s.double + s.triple;
|
||||
if (extra >= 0.97) { // pathological input; fall back to league rather than emit a negative
|
||||
return { ...LEAGUE_HIT_SHARES };
|
||||
}
|
||||
s.single = 1 - extra;
|
||||
return s;
|
||||
}
|
||||
|
||||
/** Convolve two pmfs, truncating at cap (mass beyond cap lands on cap). */
|
||||
function convolve(a, b, cap) {
|
||||
const out = new Array(cap + 1).fill(0);
|
||||
for (let i = 0; i < a.length; i += 1) {
|
||||
if (!a[i]) continue;
|
||||
for (let j = 0; j < b.length; j += 1) {
|
||||
if (!b[j]) continue;
|
||||
out[Math.min(cap, i + j)] += a[i] * b[j];
|
||||
}
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
const TB_CAP = 16;
|
||||
|
||||
/**
|
||||
* TOTAL BASES as the compound outcome it is: each PA yields 0/1/2/3/4 bases, and
|
||||
* the game total is their convolution over a distribution of plate appearances.
|
||||
*
|
||||
* This is the shape tb-v1 proved correct, now driven by SKILL inputs (barrel,
|
||||
* exit velo, contact quality vs the pitcher) rather than by raw historical
|
||||
* counts — which is the whole thesis, and it is the stat where the gate found
|
||||
* the only above-threshold marginal correlation (hard-hit r = 0.153).
|
||||
*/
|
||||
function projectTotalBasesSkill({ batter, pitcher, park, archetype, line, expectedPa, allowed } = {}) {
|
||||
const pa = paOutcome({ batter, pitcher, park, archetype, allowed });
|
||||
if (!pa) return null;
|
||||
const target = Math.max(1, Math.ceil(Number(line)));
|
||||
if (!Number.isFinite(target)) return null;
|
||||
|
||||
const shares = hitTypeShares({ batter, archetype, allowed });
|
||||
const pHit = pa.p_hit_per_pa;
|
||||
// Bases from ONE plate appearance.
|
||||
const perPa = [
|
||||
1 - pHit,
|
||||
pHit * shares.single,
|
||||
pHit * shares.double,
|
||||
pHit * shares.triple,
|
||||
pHit * shares.homer,
|
||||
];
|
||||
|
||||
const paPmf = paDistribution(expectedPa);
|
||||
let tb = new Array(TB_CAP + 1).fill(0);
|
||||
for (let n = 0; n < paPmf.length; n += 1) {
|
||||
if (!paPmf[n]) continue;
|
||||
let acc = new Array(TB_CAP + 1).fill(0);
|
||||
acc[0] = 1;
|
||||
for (let k = 0; k < n; k += 1) acc = convolve(acc, perPa, TB_CAP);
|
||||
for (let b = 0; b <= TB_CAP; b += 1) tb[b] += paPmf[n] * acc[b];
|
||||
}
|
||||
|
||||
const meanTb = tb.reduce((a, p, i) => a + p * i, 0);
|
||||
const r3 = (v) => Math.round(v * 1000) / 1000;
|
||||
return {
|
||||
version: 'skill-v1',
|
||||
stat: 'total_bases',
|
||||
p_over_line: r3(atLeast(tb, target)),
|
||||
projected_value: r3(meanTb),
|
||||
distribution: tb.map(r3),
|
||||
hit_type_shares: { single: r3(shares.single), double: r3(shares.double), triple: r3(shares.triple), homer: r3(shares.homer) },
|
||||
per_pa: {
|
||||
k_rate: r3(pa.k_rate), bb_rate: r3(pa.bb_rate),
|
||||
bip_rate: r3(pa.bip_rate), hit_on_contact: r3(pa.hit_on_contact),
|
||||
p_hit_per_pa: r3(pHit),
|
||||
},
|
||||
archetype: String(archetype || 'DEFAULT').toUpperCase(),
|
||||
pitcher_applied: pa.inputs_used.pitcher_applied,
|
||||
family: 'pa_compound_bases_convolution',
|
||||
};
|
||||
}
|
||||
|
||||
module.exports = {
|
||||
projectSkill, paOutcome, hitOnContact, oddsRatio, shrink, fromStatcastRow,
|
||||
projectSkill, projectTotalBasesSkill, hitTypeShares, convolve, paOutcome, hitOnContact, oddsRatio, shrink, fromStatcastRow,
|
||||
binomialPmf, paDistribution, atLeast, featureMapFor,
|
||||
LEAGUE, ARCHETYPE_MAP, DEFAULT_PA, PA_CAP, PCT_FIELDS, RAW_FIELDS,
|
||||
LEAGUE_HIT_SHARES, TB_CAP,
|
||||
};
|
||||
|
||||
@@ -255,6 +255,42 @@ async function refreshSeason(opts = {}) {
|
||||
summary.written += batch.length;
|
||||
}
|
||||
|
||||
// ── POINT-IN-TIME RETENTION ───────────────────────────────────────────
|
||||
// `statcast_aggregates` is upserted in place, so it holds exactly ONE as-of
|
||||
// date and every prior version is destroyed. That silently makes any backtest
|
||||
// leak: scoring a 2026-07-25 game with a 2026-08-03 season aggregate feeds the
|
||||
// model the games it is being asked to predict.
|
||||
//
|
||||
// It went unnoticed only because this job was unreachable code (see
|
||||
// snapshotScheduler) and the table sat frozen at 2026-07-21 — accidentally
|
||||
// BEFORE the settled window, which is the sole reason the first skill backtest
|
||||
// was honest. Repairing the cron removed the accident.
|
||||
//
|
||||
// Best-effort, exactly like the ledger write: a retention failure must never
|
||||
// fail the refresh, because stale-but-current data still beats no data.
|
||||
try {
|
||||
const asOf = (opts.asOfDate || started).slice(0, 10);
|
||||
let retained = 0;
|
||||
for (let i = 0; i < rows.length; i += chunk) {
|
||||
const batch = rows.slice(i, i + chunk).map((r) => {
|
||||
const row = toDbRow(r);
|
||||
delete row.updated_at; delete row.source;
|
||||
delete row.position; delete row.role_detail;
|
||||
delete row.games_started; delete row.games_pitched;
|
||||
delete row.saves; delete row.holds;
|
||||
return { ...row, as_of_date: asOf };
|
||||
});
|
||||
const { error } = await sb.from('statcast_history')
|
||||
.upsert(batch, { onConflict: 'as_of_date,sport,season,source_id,role' });
|
||||
if (error) { summary.history_error = error.message; break; }
|
||||
retained += batch.length;
|
||||
}
|
||||
summary.history_retained = retained;
|
||||
summary.history_as_of = asOf;
|
||||
} catch (e) {
|
||||
summary.history_error = e.message;
|
||||
}
|
||||
|
||||
return summary;
|
||||
}
|
||||
|
||||
|
||||
Reference in New Issue
Block a user