builtbykev/vyndr
1
0
Fork 0
Code Issues Pull Requests Actions 5 Packages Projects Releases Wiki Activity

runs + RBI: mostly-base-rate confirmed, and one level deeper than expected

Browse Source 
Nothing proved. For RBI even the ARCHETYPE split is theatre, so the honest
grade is the POOLED base rate.

PREMISE NOTE: the order's closing line says the batter board is
per-archetype-graded after this. Nothing has been rescaled for hits or
total_bases either -- no archetype slot has ever reached sample and
gradeBands remains built, gated and unwired. This is the fourth stat
measured, not the completion of three.

AUDIT: RBI 935 clean / 43 games; RUNS 617 clean / 33 games. Zero
quarantined. No archetype slot reaches 500 -- and the signature
archetypes the order names are the two SMALLEST slots on the board,
RBI->DRIVER at n=24 and runs->CATALYST at n=9. RUNS is refused
structurally before any factor is tested: 33 game clusters against a 40
floor.

INPUTS RECONSTRUCTED rather than declared missing. lineup_context only
covers 08-04 onward while settled rows start 07-31, so 187/617 runs rows
joined. But the play-by-play cache runs from 05-01 and the batting order
IS the order batters first appear -- slot, power-behind and reach-base
all rebuilt point-in-time, coverage 187 -> 574.

RBI, all THEATER: risp_opportunity +0.0047, extra_base_skill +0.0010,
risp x extra_base +0.0056. RUNS, all refused on clusters and all pointing
the wrong way: +0.0043 / +0.0008 / +0.0054.

THE COMPOUND IS THE WORST VERSION IN BOTH STATS. The causally-correct
compound was the most promising factor on the sheet and is the most
harmful in each. Two multipliers that individually carry nothing do not
cancel -- they compound each other's noise. Distinct from the
collapsed-sequence lesson: there the product of two REAL effects was too
small to use; here the product of two NULL effects is worse than either.

THE ARCHETYPE DOES NOT RESCUE IT, and this is where the session nearly
went wrong. The base rates look strongly differentiated (RBI DRIVER 0.609
vs BOMBER 0.413; runs GHOST 0.716 vs BOMBER 0.460). Gated directly
against the pooled base rate: RBI +0.0010 CI [-0.0034,+0.0050] THEATER;
runs -0.0028 CI [-0.0147,+0.0108] candidate at k=33. DRIVER's 0.609 is
n=23 -- small-slot noise wearing a decimal point. Read off the table
instead of gated, this would have shipped as "archetype differentiation
is real and large". It is not.

THE CROSS-STAT PATTERN THAT IS REAL -- the counter over-predicts every
batter counting stat measured:

  total_bases  p_win 0.5698 vs actual 0.5074  bias +0.0624
  rbi          p_win 0.4860 vs actual 0.4313  bias +0.0547
  runs         p_win 0.5949 vs actual 0.5749  bias +0.0200

Across four stats and three sessions, calibration is the systematic
defect and factor scarcity is not. TB's held-out isotonic fix (-0.0039)
still outperforms every factor tried on any stat, all null or theatre.

NO RESCALE. Nothing proved, nothing certified calibrated, no slot at
sample, and for RBI the archetype split is itself theatre -- so the
honest band is the pooled base rate, which gradeBands returns by
construction.

Counter and frozen clusters byte-identical.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W1sivYNqY2TS5ftykmHBU9
This commit is contained in:
Kev
2026-08-06 14:42:33 -04:00
parent 6a327d9114
commit 23d1b13176
3 changed files with 445 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files
scripts/prove-runs-rbi.js
+301
View File
@@ -0,0 +1,301 @@
#!/usr/bin/env node
'use strict';
/**
* prove-runs-rbi — THE CONTEXT-HEAVY STATS, WHERE INFLATION IS EASIEST.
*
* A large share of both stats is genuinely outside the hitter's control: a run
* needs someone behind you, an RBI needs someone in front of you. The job is to
* prove the HITTER-CONTROLLABLE part above the archetype's own base rate and
* grade the rest honestly as base-rate — which is the CORRECT answer for a
* context stat, not a failure to find something.
*
* ── THE NULL IS THE ARCHETYPE'S BASE RATE ────────────────────────────────
* Deliberately, and per the order: these base rates are spread and
* context-inflated, so beating "hitters like him" is the only meaningful bar. A
* per-player leave-one-out rate is not available here — 935 RBI rows over 344
* players is ~2.7 rows each, and estimating a personal rate from two rows would
* be inventing one. Leave-one-out is applied at the ARCHETYPE level so a row
* never contributes to its own baseline.
*
* ── INPUTS RECONSTRUCTED RATHER THAN DECLARED MISSING ────────────────────
* `lineup_context` only starts 2026-08-04 (ingest began last week) while settled
* rows run from 07-31, so only 187 of 617 runs rows join to a batting order.
* That would be input-blocked — except the play-by-play cache covers 05-01
* onward, and the batting order IS the order batters first appear. Reach-base
* skill and lineup power behind are derived from the same cache, point-in-time.
*
* SUPABASE_URL=... node scripts/prove-runs-rbi.js
*/
require('dotenv').config();
const fs = require('fs');
const path = require('path');
const { createClient } = require('@supabase/supabase-js');
const fg = require('../src/services/model/factorGate');
const tl = require('../src/services/model/testLedger');
const sk = require('../src/services/model/skillProjection');
const { knownNumber, knownRate } = require('../src/utils/known');
const { nameKey } = require('../src/utils/playerName');
const SB_URL = process.env.SUPABASE_URL;
const SB_KEY = process.env.SUPABASE_SERVICE_ROLE_KEY || process.env.SUPABASE_SERVICE_KEY;
const CACHE = process.env.SEQ_OUT || path.join(process.cwd(), '.seq-cache', 'sequences.json');
const PAGE = 1000;
const ARCHS = (process.env.RR_ARCHETYPES || 'ALL,BOMBER,GHOST,BRUSH,DRIVER').split(',');
const HIT = new Set(['single', 'double', 'triple', 'home_run']);
const ONBASE = new Set(['single', 'double', 'triple', 'home_run', 'walk', 'hit_by_pitch', 'intent_walk']);
const PA_EVENT = new Set([...HIT, 'field_out', 'strikeout', 'grounded_into_double_play', 'force_out',
'field_error', 'fielders_choice', 'fielders_choice_out', 'double_play', 'sac_fly', 'pop_out',
'line_out', 'fly_out', 'strikeout_double_play', 'walk', 'hit_by_pitch', 'intent_walk']);
const mean = (xs) => (xs.length ? xs.reduce((a, b) => a + b, 0) / xs.length : null);
async function page(sb, table, select, apply) {
const out = [];
for (let from = 0; ; from += PAGE) {
const { data, error } = await apply(sb.from(table).select(select)).range(from, from + PAGE - 1);
if (error) throw error;
if (!data || data.length === 0) break;
out.push(...data);
if (data.length < PAGE) break;
}
return out;
}
/**
* Reconstruct, point-in-time, from play-by-play:
* order[date|nameKey] the hitter's batting slot that game
* behind[date|nameKey] mean barrel-ish power of the three slots after him
* onbase[nameKey] his reach-base rate over PRIOR games only
*/
function reconstruct(barrelByKey) {
const { games } = JSON.parse(fs.readFileSync(CACHE, 'utf8'));
games.sort((a, b) => String(a.date).localeCompare(String(b.date)) || a.gamePk - b.gamePk);
const order = new Map();
const behind = new Map();
const onbaseNow = new Map(); // running totals, folded in AFTER each game
const onbasePrior = new Map(); // snapshot used for that game's rows
for (const g of games) {
for (const half of ['top', 'bottom']) {
const pas = g.pas.filter((p) => p.half === half && PA_EVENT.has(p.event));
if (!pas.length) continue;
// The batting order IS the order batters first appear.
const seen = [];
const seenSet = new Set();
for (const p of pas) {
if (!seenSet.has(p.batter)) { seenSet.add(p.batter); seen.push(p); }
if (seen.length >= 9) break;
}
const slots = seen.map((p) => ({ id: p.batter, key: nameKey(p.batter_name || '') }));
for (let i = 0; i < slots.length; i += 1) {
const k = `${g.date}|${slots[i].key}`;
order.set(k, i + 1);
// Power BEHIND him — the hitters who would drive him in.
const nxt = [1, 2, 3].map((d) => slots[(i + d) % slots.length])
.map((s) => (s ? knownRate(barrelByKey.get(s.key)) : null))
.filter((v) => v !== null);
if (nxt.length) behind.set(k, mean(nxt));
const prior = onbaseNow.get(slots[i].key);
if (prior && prior.pa >= 60) onbasePrior.set(k, prior.ob / prior.pa);
}
for (const p of pas) {
const key = nameKey(p.batter_name || '');
const cur = onbaseNow.get(key) || { pa: 0, ob: 0 };
cur.pa += 1; cur.ob += ONBASE.has(p.event) ? 1 : 0;
onbaseNow.set(key, cur);
}
}
}
return { order, behind, onbase: onbasePrior };
}
/** RBI and RUNS have different causal stories, so different factors. */
const FACTORS = {
rbi: [
{
key: 'risp_opportunity',
needs: ['risp_share'],
entity: (r) => r.player_key,
mechanism: 'HOW OFTEN HE BATS WITH RUNNERS IN SCORING POSITION. Half of an RBI is opportunity, and this is the ingested measure of it.',
apply: (r) => 1 + Math.max(-0.20, Math.min(0.20, (r.risp_share - 0.22) * 1.6)),
},
{
key: 'extra_base_skill',
needs: ['barrel_pct'],
entity: (r) => r.player_key,
mechanism: 'The other half — having batted with runners on, can he drive them in.',
apply: (r) => 1 + Math.max(-0.20, Math.min(0.20, (r.barrel_pct - 0.078) * 1.8)),
},
{
key: 'risp_x_extra_base',
needs: ['risp_share', 'barrel_pct'],
entity: (r) => r.player_key,
mechanism: 'THE CAUSALLY-CORRECT COMPOUND: opportunity AND the power to convert it. Neither half alone is an RBI.',
apply: (r) => (1 + Math.max(-0.20, Math.min(0.20, (r.risp_share - 0.22) * 1.6)))
* (1 + Math.max(-0.20, Math.min(0.20, (r.barrel_pct - 0.078) * 1.8))),
},
],
runs: [
{
key: 'reach_base',
needs: ['onbase'],
entity: (r) => r.player_key,
mechanism: 'You cannot score without first reaching base. The most hitter-controllable component of a run.',
apply: (r) => 1 + Math.max(-0.25, Math.min(0.25, (r.onbase - 0.318) * 2.2)),
},
{
key: 'lineup_power_behind',
needs: ['power_behind'],
entity: (r) => `${r.game_id}|${r.batting_order}`,
mechanism: 'Who bats after him — the hitters who would drive him in. Pure context, and the part he does not control.',
apply: (r) => 1 + Math.max(-0.20, Math.min(0.20, (r.power_behind - 0.078) * 1.8)),
},
{
key: 'reach_x_power_behind',
needs: ['onbase', 'power_behind'],
entity: (r) => r.player_key,
mechanism: 'THE CAUSALLY-CORRECT COMPOUND: reach base AND have someone behind you who can drive you in.',
apply: (r) => (1 + Math.max(-0.25, Math.min(0.25, (r.onbase - 0.318) * 2.2)))
* (1 + Math.max(-0.20, Math.min(0.20, (r.power_behind - 0.078) * 1.8))),
},
],
};
async function main() {
const sb = createClient(SB_URL, SB_KEY, { auth: { persistSession: false } });
const statcast = await page(sb, 'statcast_aggregates', '*', (q) => q.eq('sport', 'mlb').eq('role', 'batter'));
const batByKey = new Map();
const barrelByKey = new Map();
for (const r of statcast) {
if (!r.player_key) continue;
const prof = sk.fromStatcastRow(r);
batByKey.set(r.player_key, prof);
if (prof.barrel_pct != null) barrelByKey.set(r.player_key, prof.barrel_pct);
}
const oppRows = await page(sb, 'hitter_opportunity', '*', (q) => q.eq('sport', 'mlb'));
const oppByKey = new Map();
for (const r of oppRows) {
const prev = oppByKey.get(r.player_key);
if (!prev || String(r.as_of_date) > String(prev.as_of_date)) oppByKey.set(r.player_key, r);
}
const recon = reconstruct(barrelByKey);
const out = { generated_note: 'null is the ARCHETYPE base rate, leave-one-out' };
for (const stat of ['rbi', 'runs']) {
const snaps = await page(sb, 'model_snapshots', 'player_key, game_date, archetype',
(q) => q.eq('sport', 'mlb').eq('stat', stat).not('archetype', 'is', null));
const archOf = new Map();
for (const s of snaps) archOf.set(`${s.player_key}|${s.game_date}`, s.archetype);
const led = await page(sb, 'ledger_entries',
'id, game_id, player_key, player_name, line, side, outcome, game_date, p_win, quarantine_reason',
(q) => q.eq('sport', 'mlb').is('user_id', null).eq('stat', stat).in('outcome', ['hit', 'miss']));
const clean = led.filter((r) => !(r.quarantine_reason || '').startsWith('nontakeable_book')
&& knownNumber(r.line) === 0.5);
// Archetype-level leave-one-out base rate — the context-inflated null.
const byArch = new Map();
for (const r of clean) {
const a = String(archOf.get(`${r.player_key}|${r.game_date}`) || 'UNLABELLED').toUpperCase();
const cur = byArch.get(a) || { n: 0, w: 0 };
cur.n += 1; cur.w += r.outcome === 'hit' ? 1 : 0;
byArch.set(a, cur);
}
const rows = [];
const loss = { no_archetype_base: 0, kept: 0 };
for (const r of clean) {
const a = String(archOf.get(`${r.player_key}|${r.game_date}`) || 'UNLABELLED').toUpperCase();
const ab = byArch.get(a);
if (!ab || ab.n < 4) { loss.no_archetype_base += 1; continue; }
const baseline = (ab.w - (r.outcome === 'hit' ? 1 : 0)) / (ab.n - 1);
const bat = batByKey.get(r.player_key);
const opp = oppByKey.get(r.player_key);
const okey = `${r.game_date}|${r.player_key}`;
rows.push({
archetype: a,
player_key: r.player_key,
game_id: r.game_id,
cluster: r.game_id,
baseline,
won: r.outcome === 'hit' ? 1 : 0,
risp_share: opp ? knownNumber(opp.risp_share) : null,
barrel_pct: bat ? knownRate(bat.barrel_pct) : null,
onbase: recon.onbase.has(okey) ? recon.onbase.get(okey) : null,
power_behind: recon.behind.has(okey) ? recon.behind.get(okey) : null,
batting_order: recon.order.get(okey) ?? null,
});
loss.kept += 1;
}
const mc = await tl.recordAndCount(tl.supabaseStore(sb), FACTORS[stat].flatMap((f) =>
ARCHS.map((a) => ({
sport: 'mlb', stat, archetype: a === 'ALL' ? null : a,
interaction: `factor:${f.key}`, target: 'outcome',
}))));
const audit = [];
const results = [];
for (const arch of ARCHS) {
const slot = arch === 'ALL' ? rows : rows.filter((r) => r.archetype === arch);
for (const f of FACTORS[stat]) {
const usable = slot.filter((r) => f.needs.every((k) => knownNumber(r[k]) !== null));
const ents = new Set(usable.map((r) => String(f.entity(r))));
const games = new Set(usable.map((r) => String(r.cluster)));
if (arch === 'ALL') {
audit.push({ factor: f.key, archetype: arch, rows: usable.length, games: games.size, entities: ents.size });
}
const useEntity = ents.size < games.size;
const paired = usable.map((r) => {
const m = f.apply(r);
return {
baseline: r.baseline,
conditioned: m === null ? null : Math.min(0.99, Math.max(0.01, r.baseline * m)),
won: r.won,
cluster: useEntity ? `e:${f.entity(r)}` : r.cluster,
};
});
const v = fg.adjudicate(paired, { factor: f.key, archetype: arch, stat, cumulativeTests: mc.cumulative_tests });
results.push({
archetype: arch, factor: f.key, n: v.movement.n,
clusters: v.improvement ? v.improvement.effective_n : null,
clustered_on: useEntity ? 'treatment_entity' : 'game',
distinct_games: games.size,
mean_abs_shift: v.movement.mean_abs_shift,
brier_delta: v.improvement ? v.improvement.brier_delta : null,
ci: v.improvement ? v.improvement.ci : null,
verdict: v.verdict,
});
}
}
out[stat] = {
clean_rows_line_0_5: clean.length,
rows_built: rows.length,
row_loss: loss,
distinct_games: new Set(rows.map((r) => r.cluster)).size,
archetype_base_rates: Object.fromEntries([...byArch.entries()]
.sort((a, b) => b[1].n - a[1].n)
.map(([a, v]) => [a, { n: v.n, base_rate: Math.round((v.w / v.n) * 10000) / 10000 }])),
cumulative_tests: mc.cumulative_tests,
input_audit: audit,
results,
proven: results.filter((r) => r.verdict === 'PROVES'),
};
}
console.log(JSON.stringify(out, null, 2));
process.exit(0);
}
main().catch((e) => { console.error(e); process.exit(1); });
specs/runs-rbi-per-archetype.md
+143
View File
@@ -0,0 +1,143 @@
# runs + RBI — mostly-base-rate confirmed, and stronger than expected
**Nothing proved. For RBI even the ARCHETYPE split is theatre, so the honest
grade is the POOLED base rate.** The order anticipated that mostly-base-rate
would be the correct answer for context-heavy stats. It is, and one level deeper
than predicted.
## Premise note
The order's closing line says the batter board is "per-archetype-graded" after
this. It is not: **nothing has been rescaled for hits or total_bases either** —
no archetype slot has ever reached sample, and `gradeBands` remains built, gated
and unwired. This is the fourth stat measured, not the completion of three.
---
## STEP 1 — Full-history audit
| | clean | with p_win | players | dates | **games** | lines |
|---|---|---|---|---|---|---|
| RBI | 935 | 931 | 344 | 5 | **43** | 0.5 on 885 |
| RUNS | 617 | 614 | 334 | 5 | **33** | 0.5 on all |
Zero quarantined in either. **No archetype slot reaches 500.**
| archetype | RBI n | RUNS n |
|---|---|---|
| UNLABELLED | 546 | 341 |
| BOMBER | 207 | 148 |
| GHOST | 92 | 67 |
| BRUSH | 43 | 26 |
| **DRIVER** | **24** | 20 |
| **CATALYST** | 13 | **9** |
**The signature archetypes the order names are the two smallest slots on the
board** — RBI→DRIVER at n=24, runs→CATALYST at n=9. The hypothesis is reasonable
and we are three orders of magnitude from being able to test it.
**RUNS is refused structurally before any factor is tested: 33 game clusters
against a 40 floor.** Reported as such rather than dressed up as a factor result.
## Inputs reconstructed rather than declared missing
`lineup_context` only covers 2026-08-04/05/06 (ingest began last week) while
settled rows start 07-31, so just 187 of 617 runs rows join to a batting order.
That reads as input-blocked — but the play-by-play cache runs from 05-01, and
**the batting order IS the order batters first appear.** Batting slot, power
behind (mean barrel of the next three slots) and reach-base rate were all
reconstructed point-in-time from it: coverage went 187 → 574.
---
## STEP 2/3 — The gate (153 / 168 cumulative tests)
### RBI — all THEATER
| factor | n | clusters | shift | Brier Δ | CI | verdict |
|---|---|---|---|---|---|---|
| `risp_opportunity` | 803 | 43 | 0.0585 | **+0.0047** | [−0.0023, +0.0122] | **THEATER** |
| `extra_base_skill` | 881 | 43 | 0.0271 | **+0.0010** | [−0.0023, +0.0042] | **THEATER** |
| `risp × extra_base` | 803 | 43 | 0.0624 | **+0.0056** | [−0.0042, +0.0135] | **THEATER** |
### RUNS — all refused on cluster count, all pointing the wrong way
| factor | n | clusters | shift | Brier Δ | verdict |
|---|---|---|---|---|---|
| `reach_base` | 525 | 32 | 0.0383 | +0.0043 | PENDING — k<40 |
| `lineup_power_behind` | 574 | 32 | 0.0232 | +0.0008 | PENDING — k<40 |
| `reach × power_behind` | 525 | 32 | 0.0482 | +0.0054 | PENDING — k<40 |
### The compound is the WORST version, in both stats
RBI: 0.0047 / 0.0010 → **0.0056** compounded. RUNS: 0.0043 / 0.0008 →
**0.0054** compounded.
The causally-correct compound was the most promising factor on the sheet and is
the most harmful in both. Two multipliers that individually carry nothing do not
cancel — they compound each other's noise. Related to but distinct from the
collapsed-sequence lesson: there the product of two REAL effects was too small to
use; here the product of two NULL effects is actively worse than either.
---
## The archetype itself does not rescue it
The archetype base rates look strongly differentiated, and that appearance is
most of the trap:
| | RBI | RUNS |
|---|---|---|
| DRIVER | **0.609** (n=23) | 0.500 (n=20) |
| GHOST | 0.467 | **0.716** (n=67) |
| BOMBER | 0.413 | 0.460 |
| pooled | 0.4305 | 0.5721 |
Tested directly — is the archetype's leave-one-out base rate better than the
pooled one?
| | shift | Brier Δ | CI | verdict |
|---|---|---|---|---|
| **RBI** | 0.0231 | +0.0010 | [−0.0034, +0.0050] | **THEATER** |
| **RUNS** | 0.0642 | −0.0028 | [−0.0147, +0.0108] | CANDIDATE — k=33 |
**For RBI, knowing the archetype makes the forecast worse.** DRIVER's 0.609 is
n=23 — the spread is small-slot noise wearing a decimal point. Runs is at least
directionally favourable, and unproven.
Had this been read off the base-rate table instead of gated, the session would
have shipped "archetype differentiation is real and large" as a finding. It is
not one.
---
## The cross-stat pattern that IS real
| stat | mean p_win | actual | counter bias |
|---|---|---|---|
| total_bases | 0.5698 | 0.5074 | **+0.0624** |
| RBI | 0.4860 | 0.4313 | **+0.0547** |
| runs | 0.5949 | 0.5749 | **+0.0200** |
**The counter over-predicts every batter counting stat measured.** Across four
stats and three sessions, calibration is the systematic defect and factor
scarcity is not — TB's held-out isotonic fix (−0.0039 Brier) still outperforms
every factor tried on any stat, all of which have been null or theatre.
## STEP 4 — No rescale
Two-bar rule: nothing proved, nothing certified calibrated, no slot at sample —
and for RBI the archetype split is itself theatre, so the honest band is the
POOLED base rate rather than a per-archetype one. `gradeBands` returns exactly
that by construction.
## Next, by value
1. **Calibrate the counter across all four stats.** One systematic bias,
measured four times, larger than anything else on the board.
2. **Stop adding factors to context stats.** Six tested across runs/RBI, six
null-or-worse, and the compounds worst of all.
3. Runs needs game-date accrual to clear the cluster floor before it can be
gated at all.
Counter and frozen clusters byte-identical.
web/public/sw.js
+1 -1
View File
File diff suppressed because one or more lines are too long