Build the conditioning registry, and a probe so "proven" stops drifting
The order opens with "two proven clusters live". They are not proven -- the
proven set is empty -- and this is the fourth consecutive order to start from
a stronger claim than the measurements support. Correcting that in prose four
times has not worked, so this session adds scripts/proven-status.js, which
recomputes the answer from the ledger: hits LOSES (-0.096, CI excluding zero),
total_bases INCONCLUSIVE (+0.004), strikeouts INCONCLUSIVE (+0.259 at n=57).
It deliberately reports sample readiness separately from recorded verdicts, so
"n>=500" can never again be read as "passed".
A counting error worth recording. The first read of the top-volume archetype
said BOMBER x hits was 641 rows -- gate-ready. It is 287. model_snapshots
holds one row per prop PER SNAPSHOT CYCLE, so joining it to ledger_entries
counts each ledger row once per cycle it appeared in. Deduping on the ledger
row id gives the true figure, and my own status script had the same bug until
it was fixed. That is the difference between running the gate and being short
by 213.
So no archetype x stat combination reaches the gate. BOMBER x hits at 287 is
the closest; pitcher archetypes are untestable at 58 settled strikeout rows
across all of them, so the pitcher half of this order could not be run.
The registry is built: recordConditioning keys archetype x underlying-skill x
interaction x status with measured lift, and the skill tag is MANDATORY and
enforced -- untagged entries are refused, and PROVEN without sufficient
evidence is refused. validatedSkills() returns the coherent profile as it
stands, which is {} for every archetype, by design.
BOMBER x hits conditioning was tested across the order's categories and every
result is underpowered: arsenal (barrel x breaking share) incremental +0.043,
batted-ball (launch x pitcher GB) +0.001, contact quality -0.020 and -0.015,
K x K -0.063. Within BOMBER the counter still leads on hits, 0.218 to 0.160,
consistent with the closed pooled negative.
One bug fixed mid-run: fromStatcastRow maps percentage and raw fields only and
does not carry pitch_mix, so the arsenal category first reported n=0 for every
row -- it was measuring nothing rather than failing. Without catching it,
"arsenal doesn't matter" would have been recorded from a column that was never
populated.
On defense: I looked for a derivable proxy before calling it unsourceable, and
there isn't one. We ingest no fielding data at all, and opposing pitchers'
hits-allowed conflates pitching with defense, so it would validate the wrong
skill. It needs Savant's fielding endpoint -- free, same host as the five
feeds already ingested -- and it is not sourced here, because sourcing it to
test at n=282 would answer nothing.
Nothing proved, so nothing was recalibrated and nothing shipped.
4,221 tests green (335 suites); web build exit 0.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W1sivYNqY2TS5ftykmHBU9
This commit is contained in:
@@ -3,6 +3,31 @@
|
|||||||
## Last Updated
|
## Last Updated
|
||||||
2026-08-03
|
2026-08-03
|
||||||
|
|
||||||
|
## Session 86 (2026-08-03) — Conditioning registry + a probe so "proven" stops drifting ✅
|
||||||
|
Spec: `specs/conditioning-registry.md`. 4,221 tests / 335 suites green, build exit
|
||||||
|
0. Counter + batter model + pitcher engine byte-identical.
|
||||||
|
- **`scripts/proven-status.js`** recomputes the proven set from the ledger.
|
||||||
|
PROVEN_SET = **EMPTY**. Built because four consecutive orders opened by calling
|
||||||
|
null results proven; prose decays, a recomputed number does not.
|
||||||
|
- **COUNTING BUG CAUGHT:** joining model_snapshots to ledger_entries fans out
|
||||||
|
(one snapshot row per cycle) — BOMBER x hits read 641, true distinct 287.
|
||||||
|
Fixed in both the analysis and the status probe.
|
||||||
|
- **NO archetype x stat reaches the gate.** BOMBER x hits 287 (short 213) is
|
||||||
|
closest; pitcher archetypes untestable (58 settled Ks total).
|
||||||
|
- **Structured registry built:** `recordConditioning` keys archetype x SKILL x
|
||||||
|
interaction x status + lift, with the skill tag ENFORCED (untagged refused,
|
||||||
|
PROVEN-without-evidence refused). `validatedSkills()` = {} everywhere, by design.
|
||||||
|
- **BOMBER x hits conditioning tested, all UNDERPOWERED:** arsenal (barrel x
|
||||||
|
breaking share) incr +0.043, batted-ball (launch x pitcher GB) +0.001, contact
|
||||||
|
quality −0.020/−0.015, K x K −0.063. Within BOMBER the counter still leads
|
||||||
|
(0.218 vs 0.160).
|
||||||
|
- **Bug fixed mid-run:** `fromStatcastRow` doesn't carry pitch_mix, so the arsenal
|
||||||
|
category read n=0 — it was measuring nothing, not failing.
|
||||||
|
- **DEFENSE: genuinely not derivable** from ingested data (no OAA/DRS; pitching
|
||||||
|
proxies conflate skills). Needs Savant's free fielding feed — not sourced,
|
||||||
|
because sourcing it to test at n=282 answers nothing.
|
||||||
|
- **Nothing proved → nothing recalibrated, nothing shipped.**
|
||||||
|
|
||||||
## Session 85 (2026-08-03) — Rung 1 derived free; the cap fix fingerprinted ✅
|
## Session 85 (2026-08-03) — Rung 1 derived free; the cap fix fingerprinted ✅
|
||||||
Spec: `specs/lineup-k-rate-rung1.md`. 4,221 tests / 335 suites green, build exit
|
Spec: `specs/lineup-k-rate-rung1.md`. 4,221 tests / 335 suites green, build exit
|
||||||
0. Counter + batter cluster + pitcher engine byte-identical.
|
0. Counter + batter cluster + pitcher engine byte-identical.
|
||||||
|
|||||||
@@ -1466,6 +1466,36 @@ phased plan in the Session-57 conversation / BUILD-STATE Next section).
|
|||||||
Four exceed the |r|>=0.15 effect bar (arm_angle −0.250, whiff +0.213, k_pct
|
Four exceed the |r|>=0.15 effect bar (arm_angle −0.250, whiff +0.213, k_pct
|
||||||
+0.206, chase +0.195) but exceeding one of three thresholds is not passing.
|
+0.206, chase +0.195) but exceeding one of three thresholds is not passing.
|
||||||
|
|
||||||
|
## Conditioning registry + the proven-status probe (Session 86 — non-obvious)
|
||||||
|
- **RUN `node scripts/proven-status.js` BEFORE planning on a "proven" claim.**
|
||||||
|
Four consecutive orders opened by calling null results proven. The script
|
||||||
|
recomputes from the ledger: PROVEN_SET is **EMPTY** (hits LOSES −0.096 CI
|
||||||
|
excluding zero; total_bases +0.004 inconclusive; strikeouts +0.259 inconclusive
|
||||||
|
at n=57). It deliberately reports SAMPLE READINESS separately from RECORDED
|
||||||
|
VERDICTS so "n>=500" is never mistaken for "passed".
|
||||||
|
- **JOINING `model_snapshots` TO `ledger_entries` FANS OUT.** model_snapshots
|
||||||
|
holds one row per prop PER SNAPSHOT CYCLE, so a naive join counts each ledger
|
||||||
|
row once per cycle: BOMBER x hits read as **641** when the true distinct figure
|
||||||
|
is **287**. Always dedupe on `ledger_entries.id`. This is the difference
|
||||||
|
between "gate-ready" and "short by 213".
|
||||||
|
- **NO archetype x stat reaches n>=500.** Best: BOMBER x hits 287, BOMBER x TB
|
||||||
|
142, BOMBER x rbi 128, GHOST x hits 124. Pitcher archetypes are untestable
|
||||||
|
(58 settled Ks across ALL archetypes).
|
||||||
|
- **`featureRegistry.recordConditioning`** keys archetype x SKILL x interaction x
|
||||||
|
status + lift. The skill tag is MANDATORY and enforced (untagged → refused;
|
||||||
|
PROVEN without sufficient evidence → refused). `validatedSkills()` returns the
|
||||||
|
coherent profile — currently `{}` for every archetype, by design.
|
||||||
|
- **`fromStatcastRow` does NOT carry `pitch_mix`** (it maps PCT_FIELDS/RAW_FIELDS
|
||||||
|
only). Attach it explicitly or arsenal features silently read n=0 — which
|
||||||
|
would have recorded "arsenal doesn't matter" from a column that was never
|
||||||
|
populated. pitch_mix shape is `[{type, usage_pct, velo, whiff_pct, ...}]`.
|
||||||
|
- **DEFENSE IS GENUINELY NOT DERIVABLE from what we ingest.** No OAA/DRS/range
|
||||||
|
anywhere; opposing pitchers' hits-allowed conflates pitching WITH defense so it
|
||||||
|
would validate the wrong skill. It needs Savant's fielding endpoint (free, same
|
||||||
|
host as the five feeds already ingested). Don't proxy it.
|
||||||
|
- Within BOMBER, the counter still leads on hits (0.218 vs 0.160) — consistent
|
||||||
|
with the closed pooled hits negative.
|
||||||
|
|
||||||
## Active Skills
|
## Active Skills
|
||||||
- vyndr-voice (all user-facing output)
|
- vyndr-voice (all user-facing output)
|
||||||
- prop-analysis (grading methodology)
|
- prop-analysis (grading methodology)
|
||||||
|
|||||||
@@ -50,6 +50,10 @@ const SB_KEY = process.env.SUPABASE_SERVICE_ROLE_KEY || process.env.SUPABASE_SER
|
|||||||
const PAGE = 1000;
|
const PAGE = 1000;
|
||||||
const GAMES_SO_FAR = Number(process.env.STAGEA_GAMES_SO_FAR || 103);
|
const GAMES_SO_FAR = Number(process.env.STAGEA_GAMES_SO_FAR || 103);
|
||||||
const STAT = process.env.CLUSTER_STAT || 'total_bases';
|
const STAT = process.env.CLUSTER_STAT || 'total_bases';
|
||||||
|
/** Restrict every test to ONE archetype — the pooled result can hide an
|
||||||
|
* archetype-conditional effect entirely (the pitcher strata showed opposite
|
||||||
|
* signs cancelling to near-zero when pooled). */
|
||||||
|
const ARCH = process.env.CLUSTER_ARCHETYPE || null;
|
||||||
|
|
||||||
const r4 = (v) => (v == null || !Number.isFinite(v) ? null : Math.round(v * 10000) / 10000);
|
const r4 = (v) => (v == null || !Number.isFinite(v) ? null : Math.round(v * 10000) / 10000);
|
||||||
const mean = (a) => (a.length ? a.reduce((x, y) => x + y, 0) / a.length : null);
|
const mean = (a) => (a.length ? a.reduce((x, y) => x + y, 0) / a.length : null);
|
||||||
@@ -194,7 +198,8 @@ async function opponentByPlayerDate(players) {
|
|||||||
|
|
||||||
const SOLO = ['batter_barrel_pct', 'batter_hard_hit_pct', 'batter_exit_velo',
|
const SOLO = ['batter_barrel_pct', 'batter_hard_hit_pct', 'batter_exit_velo',
|
||||||
'batter_launch_angle', 'batter_k_pct', 'batter_bb_pct',
|
'batter_launch_angle', 'batter_k_pct', 'batter_bb_pct',
|
||||||
'pitcher_k_pct', 'pitcher_hard_hit_allowed'];
|
'pitcher_k_pct', 'pitcher_hard_hit_allowed',
|
||||||
|
'pitcher_gb_pct', 'pitcher_fb_pct', 'pitcher_breaking_share'];
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* PER-STAT INTERACTION SETS — the total_bases conditioning map RE-WEIGHTED, not
|
* PER-STAT INTERACTION SETS — the total_bases conditioning map RE-WEIGHTED, not
|
||||||
@@ -204,8 +209,8 @@ const SOLO = ['batter_barrel_pct', 'batter_hard_hit_pct', 'batter_exit_velo',
|
|||||||
* and RBI through another (does anyone happen to be on base when it does).
|
* and RBI through another (does anyone happen to be on base when it does).
|
||||||
*/
|
*/
|
||||||
const STAT_INTERACTIONS = {
|
const STAT_INTERACTIONS = {
|
||||||
total_bases: ['launch_x_exit_velo', 'exitvelo_x_pitcher_suppression', 'barrel_x_power_archetype', 'batterK_x_pitcherK'],
|
total_bases: ['launch_x_exit_velo', 'exitvelo_x_pitcher_suppression', 'barrel_x_power_archetype', 'batterK_x_pitcherK', 'launch_x_pitcher_gb', 'barrel_x_breaking_share'],
|
||||||
hits: ['launch_x_exit_velo', 'exitvelo_x_pitcher_suppression', 'batterK_x_pitcherK'],
|
hits: ['launch_x_exit_velo', 'exitvelo_x_pitcher_suppression', 'batterK_x_pitcherK', 'launch_x_pitcher_gb', 'barrel_x_breaking_share'],
|
||||||
// HOME RUNS are the purest barrel stat: the ball must be hit hard AND at the
|
// HOME RUNS are the purest barrel stat: the ball must be hit hard AND at the
|
||||||
// right angle, and the pitcher must be the kind who allows that combination.
|
// right angle, and the pitcher must be the kind who allows that combination.
|
||||||
home_runs: ['launch_x_exit_velo', 'barrel_x_power_archetype', 'exitvelo_x_pitcher_suppression'],
|
home_runs: ['launch_x_exit_velo', 'barrel_x_power_archetype', 'exitvelo_x_pitcher_suppression'],
|
||||||
@@ -238,6 +243,18 @@ const INTERACTIONS = [
|
|||||||
mechanism: 'ARCHETYPE-CONDITIONAL. Barrels convert to extra bases for hitters whose lane is power; for a speed/contact profile the same barrel rate is a rarer event on a swing built for something else. This is Discipline 2 stated as a testable interaction. NOTE: it is currently UNTESTABLE — statcast rows carry no archetype label, and the barrel-relative proxy is an exact linear function of barrel_pct, so controlling for both components is rank-deficient. It needs a real archetype classification joined in.',
|
mechanism: 'ARCHETYPE-CONDITIONAL. Barrels convert to extra bases for hitters whose lane is power; for a speed/contact profile the same barrel rate is a rarer event on a swing built for something else. This is Discipline 2 stated as a testable interaction. NOTE: it is currently UNTESTABLE — statcast rows carry no archetype label, and the barrel-relative proxy is an exact linear function of barrel_pct, so controlling for both components is rank-deficient. It needs a real archetype classification joined in.',
|
||||||
build: (r) => r.batter_barrel_pct * r.archetype_power,
|
build: (r) => r.batter_barrel_pct * r.archetype_power,
|
||||||
},
|
},
|
||||||
|
{
|
||||||
|
key: 'launch_x_pitcher_gb',
|
||||||
|
components: ['batter_launch_angle', 'pitcher_gb_pct'],
|
||||||
|
mechanism: 'PITCHER BATTED-BALL TYPE. A ground-ball arm takes the air away, and a hitter whose value lives in the air needs the air. An air hitter against a sinkerballer and a ground-ball hitter against a fly-ball arm are both mismatches that neither factor states alone.',
|
||||||
|
build: (r) => r.batter_launch_angle * r.pitcher_gb_pct,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
key: 'barrel_x_breaking_share',
|
||||||
|
components: ['batter_barrel_pct', 'pitcher_breaking_share'],
|
||||||
|
mechanism: 'ARSENAL MATCHUP. Barrel rate is far more a fastball skill than a breaking-ball skill, so a power bat facing a breaking-heavy arm should convert less of it. The pitch mix is already ingested, so this costs nothing to test.',
|
||||||
|
build: (r) => r.batter_barrel_pct * r.pitcher_breaking_share,
|
||||||
|
},
|
||||||
{
|
{
|
||||||
key: 'batterK_x_pitcherK',
|
key: 'batterK_x_pitcherK',
|
||||||
components: ['batter_k_pct', 'pitcher_k_pct'],
|
components: ['batter_k_pct', 'pitcher_k_pct'],
|
||||||
@@ -246,6 +263,26 @@ const INTERACTIONS = [
|
|||||||
},
|
},
|
||||||
];
|
];
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Breaking-ball share of a pitcher's mix, from `pitch_mix` already ingested.
|
||||||
|
* Sliders/curves/sweepers/cutters vs fastballs — absent mix -> null, never 0.
|
||||||
|
*/
|
||||||
|
function breakingShare(mix) {
|
||||||
|
if (!mix || typeof mix !== 'object') return null;
|
||||||
|
const rows = Array.isArray(mix) ? mix : Object.values(mix);
|
||||||
|
let breaking = 0; let total = 0;
|
||||||
|
for (const p of rows) {
|
||||||
|
if (!p) continue;
|
||||||
|
const type = String(p.type || p.pitch_type || '').toUpperCase();
|
||||||
|
const usage = knownNumber(p.usage_pct ?? p.usage ?? p.pct);
|
||||||
|
if (!type || usage === null || usage < 0) continue;
|
||||||
|
total += usage;
|
||||||
|
if (['SL', 'CU', 'KC', 'ST', 'SV', 'FC', 'SC'].includes(type)) breaking += usage;
|
||||||
|
}
|
||||||
|
if (total <= 0) return null;
|
||||||
|
return breaking / total;
|
||||||
|
}
|
||||||
|
|
||||||
/** Latest settled game date in the pull — used to detect that the profile
|
/** Latest settled game date in the pull — used to detect that the profile
|
||||||
* freeze now sits AFTER the data, i.e. no clean out-of-sample window exists. */
|
* freeze now sits AFTER the data, i.e. no clean out-of-sample window exists. */
|
||||||
function clean0Max(rows) {
|
function clean0Max(rows) {
|
||||||
@@ -260,7 +297,12 @@ async function main() {
|
|||||||
const freezeDate = statcast.reduce((mx, r) => (String(r.updated_at) > mx ? String(r.updated_at) : mx), '').slice(0, 10);
|
const freezeDate = statcast.reduce((mx, r) => (String(r.updated_at) > mx ? String(r.updated_at) : mx), '').slice(0, 10);
|
||||||
const batters = new Map(); const pitchersById = new Map();
|
const batters = new Map(); const pitchersById = new Map();
|
||||||
for (const r of statcast) {
|
for (const r of statcast) {
|
||||||
if (r.role === 'pitcher' && r.source_id != null) pitchersById.set(Number(r.source_id), sk.fromStatcastRow(r));
|
// pitch_mix is NOT part of fromStatcastRow's output (it maps pct/raw fields
|
||||||
|
// only), so it must be attached explicitly — without it the arsenal category
|
||||||
|
// silently measures nothing and reports n=0.
|
||||||
|
if (r.role === 'pitcher' && r.source_id != null) {
|
||||||
|
pitchersById.set(Number(r.source_id), { ...sk.fromStatcastRow(r), pitch_mix: r.pitch_mix });
|
||||||
|
}
|
||||||
if (r.player_key && r.role === 'batter') {
|
if (r.player_key && r.role === 'batter') {
|
||||||
const prev = batters.get(r.player_key);
|
const prev = batters.get(r.player_key);
|
||||||
if (!prev || Number(r.sample_pa || 0) > Number(prev.rawPa || 0)) {
|
if (!prev || Number(r.sample_pa || 0) > Number(prev.rawPa || 0)) {
|
||||||
@@ -315,7 +357,9 @@ async function main() {
|
|||||||
console.error(`[debug] matches in sample=${sampleKeys.filter((k) => batters.has(k)).length}/5`);
|
console.error(`[debug] matches in sample=${sampleKeys.filter((k) => batters.has(k)).length}/5`);
|
||||||
}
|
}
|
||||||
const allowed = reg.candidateFeaturesForStat('mlb', STAT);
|
const allowed = reg.candidateFeaturesForStat('mlb', STAT);
|
||||||
const rows = [];
|
|
||||||
|
const rowsAll = [];
|
||||||
|
const rows = rowsAll;
|
||||||
for (const r of clean) {
|
for (const r of clean) {
|
||||||
const bat = batters.get(r.player_key);
|
const bat = batters.get(r.player_key);
|
||||||
if (!bat) continue;
|
if (!bat) continue;
|
||||||
@@ -350,9 +394,25 @@ async function main() {
|
|||||||
pitcher_hard_hit_allowed: pit ? knownRate(pit.hard_hit_pct) : null,
|
pitcher_hard_hit_allowed: pit ? knownRate(pit.hard_hit_pct) : null,
|
||||||
archetype_power: archetypePower,
|
archetype_power: archetypePower,
|
||||||
archetype: arch,
|
archetype: arch,
|
||||||
|
// ── CONDITIONING CATEGORIES (this order) ──────────────────────────
|
||||||
|
// PITCHER BATTED-BALL TYPE: a ground-ball arm suppresses air contact, so
|
||||||
|
// it should matter differently to a hitter whose value is in the air.
|
||||||
|
pitcher_gb_pct: pit ? knownRate(pit.gb_pct) : null,
|
||||||
|
pitcher_fb_pct: pit ? knownRate(pit.fb_pct) : null,
|
||||||
|
// ARSENAL: breaking-ball share, from the pitch mix already ingested. A
|
||||||
|
// power hitter's barrel rate is a fastball skill far more than a
|
||||||
|
// breaking-ball skill, so the mix should condition it.
|
||||||
|
pitcher_breaking_share: pit && pit.pitch_mix ? breakingShare(pit.pitch_mix) : null,
|
||||||
|
// DEFENSE: NOT DERIVABLE from what we ingest — see the report. Recorded as
|
||||||
|
// null rather than proxied by something that is really pitching quality.
|
||||||
|
team_defense: null,
|
||||||
});
|
});
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// ARCHETYPE RESTRICTION — applied AFTER building rows so coverage is visible.
|
||||||
|
const archRows = ARCH ? rowsAll.filter((r) => String(r.archetype || '').toUpperCase() === ARCH) : rowsAll;
|
||||||
|
rows.length = 0; rows.push(...archRows);
|
||||||
|
|
||||||
// Bonferroni denominator = every test in this family (solo + interaction).
|
// Bonferroni denominator = every test in this family (solo + interaction).
|
||||||
const TESTS = SOLO.length + INTERACTIONS.length;
|
const TESTS = SOLO.length + INTERACTIONS.length;
|
||||||
|
|
||||||
@@ -414,6 +474,7 @@ async function main() {
|
|||||||
|
|
||||||
console.log(JSON.stringify({
|
console.log(JSON.stringify({
|
||||||
stat: STAT,
|
stat: STAT,
|
||||||
|
archetype_restriction: ARCH || 'none (pooled)',
|
||||||
VALIDITY: contaminated
|
VALIDITY: contaminated
|
||||||
? 'CONTAMINATED / DIRECTIONAL ONLY — statcast_aggregates now carries a single as-of date (' + freezeDate + ') that is AFTER the settled games, so season profiles contain the games being predicted. These are NOT gate verdicts. statcast_history (new) makes point-in-time possible from tomorrow.'
|
? 'CONTAMINATED / DIRECTIONAL ONLY — statcast_aggregates now carries a single as-of date (' + freezeDate + ') that is AFTER the settled games, so season profiles contain the games being predicted. These are NOT gate verdicts. statcast_history (new) makes point-in-time possible from tomorrow.'
|
||||||
: `CLEAN out-of-sample: profiles frozen ${freezeDate}; only game_date > ${freezeDate} scored`,
|
: `CLEAN out-of-sample: profiles frozen ${freezeDate}; only game_date > ${freezeDate} scored`,
|
||||||
|
|||||||
@@ -0,0 +1,126 @@
|
|||||||
|
#!/usr/bin/env node
|
||||||
|
'use strict';
|
||||||
|
|
||||||
|
/**
|
||||||
|
* proven-status — WHAT IS ACTUALLY PROVEN, computed from the ledger.
|
||||||
|
*
|
||||||
|
* WHY THIS EXISTS. Four consecutive build orders have opened by describing
|
||||||
|
* results as proven that the measurements did not support: "barrel rate PASSED
|
||||||
|
* solo" (every total_bases feature was refused on sample), "total_bases has
|
||||||
|
* passed BAR 1" (inconclusive at parity, CI spanning zero), "whiff/stuff prove
|
||||||
|
* SOLO through the gate" (refused at n=57), "two proven clusters live" (the
|
||||||
|
* proven set is empty). Each time the correction had to be re-derived by hand
|
||||||
|
* from a spec written days earlier.
|
||||||
|
*
|
||||||
|
* Prose decays. A number recomputed from the ledger does not. So this prints the
|
||||||
|
* proven set on demand, from the same gate everything else is held to, and any
|
||||||
|
* session can run it in one command before planning on top of a claim.
|
||||||
|
*
|
||||||
|
* IT DELIBERATELY CANNOT SAY "PROVEN" ON ITS OWN. A stat is proven only if a
|
||||||
|
* recorded head-to-head beat the counter out-of-sample with a CI excluding zero,
|
||||||
|
* which is a measurement this script does not perform — it reports SAMPLE
|
||||||
|
* READINESS (can the gate even be run?) and the recorded verdicts, so the two
|
||||||
|
* are never confused again.
|
||||||
|
*
|
||||||
|
* SUPABASE_URL=... node scripts/proven-status.js
|
||||||
|
*/
|
||||||
|
|
||||||
|
require('dotenv').config();
|
||||||
|
const { createClient } = require('@supabase/supabase-js');
|
||||||
|
const cv = require('../src/services/model/correlateValidator');
|
||||||
|
|
||||||
|
const SB_URL = process.env.SUPABASE_URL;
|
||||||
|
const SB_KEY = process.env.SUPABASE_SERVICE_ROLE_KEY || process.env.SUPABASE_SERVICE_KEY;
|
||||||
|
const PAGE = 1000;
|
||||||
|
const MIN_N = cv.VALIDATION_REQUIREMENTS.min_historical_instances;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* RECORDED VERDICTS — every head-to-head this programme has actually run, with
|
||||||
|
* its spec. Add a row when a head-to-head is run; never edit one to be kinder.
|
||||||
|
*/
|
||||||
|
const RECORDED = [
|
||||||
|
{ stat: 'hits', n: 803, model: 0.0842, counter: 0.1803, delta: -0.0961, ci: [-0.1648, -0.0285],
|
||||||
|
verdict: 'LOSES', spec: 'specs/batter-cluster-prove.md' },
|
||||||
|
{ stat: 'total_bases', n: 383, model: 0.2685, counter: 0.2647, delta: 0.0038, ci: [-0.0675, 0.0753],
|
||||||
|
verdict: 'INCONCLUSIVE', spec: 'specs/tb-solo-and-interactions.md' },
|
||||||
|
{ stat: 'strikeouts', n: 57, model: 0.1953, counter: -0.0639, delta: 0.2592, ci: [-0.0167, 0.5645],
|
||||||
|
verdict: 'INCONCLUSIVE', spec: 'specs/lineup-k-rate-rung1.md' },
|
||||||
|
];
|
||||||
|
|
||||||
|
async function page(sb, table, select, apply) {
|
||||||
|
const out = [];
|
||||||
|
for (let from = 0; ; from += PAGE) {
|
||||||
|
const { data, error } = await apply(sb.from(table).select(select)).range(from, from + PAGE - 1);
|
||||||
|
if (error) throw error;
|
||||||
|
if (!data || data.length === 0) break;
|
||||||
|
out.push(...data);
|
||||||
|
if (data.length < PAGE) break;
|
||||||
|
}
|
||||||
|
return out;
|
||||||
|
}
|
||||||
|
|
||||||
|
async function main() {
|
||||||
|
if (!SB_URL || !SB_KEY) throw new Error('SUPABASE_URL / service key required');
|
||||||
|
const sb = createClient(SB_URL, SB_KEY, { auth: { persistSession: false } });
|
||||||
|
|
||||||
|
const led = await page(sb, 'ledger_entries', 'stat, outcome, quarantine_reason, p_win',
|
||||||
|
(q) => q.eq('sport', 'mlb').is('user_id', null));
|
||||||
|
const settled = {};
|
||||||
|
for (const r of led) {
|
||||||
|
if ((r.quarantine_reason || '').startsWith('nontakeable_book')) continue;
|
||||||
|
if (r.outcome !== 'hit' && r.outcome !== 'miss') continue;
|
||||||
|
if (r.p_win == null) continue;
|
||||||
|
settled[r.stat] = (settled[r.stat] || 0) + 1;
|
||||||
|
}
|
||||||
|
|
||||||
|
const snaps = await page(sb, 'model_snapshots', 'stat, archetype, player_key, line, side, game_date',
|
||||||
|
(q) => q.eq('sport', 'mlb').not('archetype', 'is', null));
|
||||||
|
const archOf = new Map();
|
||||||
|
for (const s of snaps) archOf.set(`${s.player_key}|${s.stat}|${s.line}|${String(s.side).toLowerCase()}|${s.game_date}`, s.archetype);
|
||||||
|
|
||||||
|
// COUNT DISTINCT LEDGER ROWS. `model_snapshots` holds one row per prop PER
|
||||||
|
// SNAPSHOT CYCLE, so a naive join fans out and inflates the count — it read
|
||||||
|
// BOMBER x hits as 641 when the true figure is 287, which is the difference
|
||||||
|
// between "gate-ready" and "not close". Dedupe on the ledger row's identity.
|
||||||
|
const led2 = await page(sb, 'ledger_entries', 'id, stat, outcome, quarantine_reason, player_key, line, side, game_date',
|
||||||
|
(q) => q.eq('sport', 'mlb').is('user_id', null).in('outcome', ['hit', 'miss']));
|
||||||
|
const byArch = {};
|
||||||
|
const seen = new Set();
|
||||||
|
for (const r of led2) {
|
||||||
|
if ((r.quarantine_reason || '').startsWith('nontakeable_book')) continue;
|
||||||
|
if (seen.has(r.id)) continue;
|
||||||
|
seen.add(r.id);
|
||||||
|
const a = archOf.get(`${r.player_key}|${r.stat}|${r.line}|${String(r.side).toLowerCase()}|${r.game_date}`);
|
||||||
|
if (!a) continue;
|
||||||
|
const k = `${a} x ${r.stat}`;
|
||||||
|
byArch[k] = (byArch[k] || 0) + 1;
|
||||||
|
}
|
||||||
|
|
||||||
|
const gateReady = Object.entries(settled).filter(([, n]) => n >= MIN_N).map(([s, n]) => ({ stat: s, n }));
|
||||||
|
const archReady = Object.entries(byArch).filter(([, n]) => n >= MIN_N)
|
||||||
|
.sort((a, b) => b[1] - a[1]).map(([k, n]) => ({ combo: k, n }));
|
||||||
|
|
||||||
|
const proven = RECORDED.filter((r) => r.verdict === 'BEATS');
|
||||||
|
|
||||||
|
console.log(JSON.stringify({
|
||||||
|
generated_at_note: 'computed from the ledger; prose in specs may lag this',
|
||||||
|
gate_spec: cv.VALIDATION_REQUIREMENTS,
|
||||||
|
|
||||||
|
PROVEN_SET: proven.length === 0 ? 'EMPTY — no stat has beaten the counter out-of-sample with a CI excluding zero' : proven,
|
||||||
|
|
||||||
|
recorded_head_to_heads: RECORDED,
|
||||||
|
|
||||||
|
sample_readiness: {
|
||||||
|
note: 'n >= 500 means the gate CAN be run — it does not mean anything passed it',
|
||||||
|
stats_at_or_above_gate: gateReady,
|
||||||
|
stats_below_gate: Object.entries(settled).filter(([, n]) => n < MIN_N)
|
||||||
|
.sort((a, b) => b[1] - a[1]).map(([s, n]) => ({ stat: s, n, short_by: MIN_N - n })),
|
||||||
|
archetype_x_stat_at_or_above_gate: archReady,
|
||||||
|
archetype_x_stat_closest_below: Object.entries(byArch).filter(([, n]) => n < MIN_N)
|
||||||
|
.sort((a, b) => b[1] - a[1]).slice(0, 6).map(([k, n]) => ({ combo: k, n, short_by: MIN_N - n })),
|
||||||
|
},
|
||||||
|
}, null, 2));
|
||||||
|
process.exit(0);
|
||||||
|
}
|
||||||
|
|
||||||
|
main().catch((e) => { console.error(e); process.exit(1); });
|
||||||
@@ -0,0 +1,147 @@
|
|||||||
|
# THE CONDITIONING REGISTRY — built, and what it currently holds
|
||||||
|
|
||||||
|
**2026-08-03.** Challenger-only. Counter, batter model and pitcher engine
|
||||||
|
byte-identical (verified by diff).
|
||||||
|
|
||||||
|
> **No archetype × stat combination reaches the gate. The best is BOMBER × hits
|
||||||
|
> at n=287, short by 213.** So the conditioning categories were tested and are
|
||||||
|
> all UNDERPOWERED — none proved, none died. Nothing was recalibrated and nothing
|
||||||
|
> shipped, because nothing earned it.
|
||||||
|
>
|
||||||
|
> **The durable deliverables are the registry itself and a status probe** that
|
||||||
|
> makes "what is proven" a query instead of a memory.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 0. The recurring premise problem, and a structural fix
|
||||||
|
|
||||||
|
This order opens with "two proven clusters live (batter contact stats + pitcher
|
||||||
|
strikeouts)". They are not proven. **The proven set is empty**, and this is the
|
||||||
|
fourth consecutive order to start from a stronger claim than the measurements
|
||||||
|
support:
|
||||||
|
|
||||||
|
| order said | measurement said |
|
||||||
|
|---|---|
|
||||||
|
| "barrel rate PASSED solo" | every total_bases feature refused on sample |
|
||||||
|
| "total_bases has passed BAR 1" | inconclusive at parity, CI spanning zero |
|
||||||
|
| "whiff/stuff prove SOLO through the gate" | all refused at n=57 |
|
||||||
|
| "two proven clusters live" | **proven set EMPTY** |
|
||||||
|
|
||||||
|
Correcting it in prose four times has not worked, so this session added
|
||||||
|
**`scripts/proven-status.js`** — it recomputes the answer from the ledger:
|
||||||
|
|
||||||
|
```
|
||||||
|
PROVEN_SET: EMPTY — no stat has beaten the counter out-of-sample
|
||||||
|
with a CI excluding zero
|
||||||
|
|
||||||
|
hits n=803 delta −0.0961 [−0.165, −0.029] LOSES
|
||||||
|
total_bases n=383 delta +0.0038 [−0.068, +0.075] INCONCLUSIVE
|
||||||
|
strikeouts n=57 delta +0.2592 [−0.017, +0.564] INCONCLUSIVE
|
||||||
|
|
||||||
|
stats at/above the gate: hits (806) — and hits is a closed negative
|
||||||
|
archetype × stat at/above the gate: NONE
|
||||||
|
closest: BOMBER×hits 287 (short 213) · BOMBER×TB 142 · BOMBER×rbi 128
|
||||||
|
```
|
||||||
|
|
||||||
|
**Run it before planning on top of a claim.** It deliberately cannot say
|
||||||
|
"proven" on its own — it reports sample readiness and *recorded* verdicts, so the
|
||||||
|
two can never be conflated again.
|
||||||
|
|
||||||
|
## 1. The structured registry (STEP 1) — built
|
||||||
|
|
||||||
|
`featureRegistry.recordConditioning()` keys **archetype × underlying-skill ×
|
||||||
|
conditioning-interaction × status**, with measured lift.
|
||||||
|
|
||||||
|
**The skill tag is mandatory and enforced.** An untagged entry is refused
|
||||||
|
(`untagged_or_unknown_skill`), and a `PROVEN` entry without sufficient evidence is
|
||||||
|
refused (`insufficient_evidence_for_proven`). Skills: POWER, CONTACT, SPEED,
|
||||||
|
WHIFF, COMMAND, OPPORTUNITY.
|
||||||
|
|
||||||
|
Why the tag matters: a proven interaction is not merely "this helps this stat" —
|
||||||
|
it is evidence that **one underlying skill is real and measurable for this
|
||||||
|
archetype**. `validatedSkills(sport, archetype)` returns the coherent profile as
|
||||||
|
it currently stands. **It returns `{}` for every archetype**, because nothing has
|
||||||
|
been proven, and seeding it with hopeful rows would defeat its purpose exactly as
|
||||||
|
seeding PROVEN features would.
|
||||||
|
|
||||||
|
## 2. Top-volume selection (STEP 2)
|
||||||
|
|
||||||
|
**Batter: BOMBER** — the highest-volume archetype by a distance (287 settled hits
|
||||||
|
rows; next is GHOST×hits at 124).
|
||||||
|
**Pitcher: none testable.** Strikeouts total 58 settled rows across *all*
|
||||||
|
archetypes, so no pitcher archetype has a sample. STEP 4 could not be run.
|
||||||
|
|
||||||
|
### A counting error caught, worth recording
|
||||||
|
|
||||||
|
The first read said BOMBER × hits was **641** — gate-ready. It is **287**. The
|
||||||
|
join to `model_snapshots` fans out, because that table holds one row per prop
|
||||||
|
**per snapshot cycle**, so each ledger row was counted once per cycle it appeared
|
||||||
|
in. Deduping on the ledger row's identity gives the true figure. **That is the
|
||||||
|
difference between "run the gate" and "not close", and my own status script had
|
||||||
|
the same bug until it was fixed.**
|
||||||
|
|
||||||
|
## 3. BOMBER × hits conditioning (STEP 3) — all underpowered
|
||||||
|
|
||||||
|
n=282, Bonferroni across 17 tests. Every result refused on sample.
|
||||||
|
|
||||||
|
| conditioning | category | n | raw r | best part | **incremental** |
|
||||||
|
|---|---|---|---|---|---|
|
||||||
|
| barrel × breaking share | **ARSENAL** | 282 | −0.063 | 0.105 | +0.043 |
|
||||||
|
| launch × pitcher GB% | **BATTED-BALL** | 282 | −0.054 | 0.052 | +0.001 |
|
||||||
|
| launch × exit velo | contact quality | 282 | −0.060 | 0.087 | −0.020 |
|
||||||
|
| exit velo × pitcher suppression | contact quality | 282 | +0.008 | 0.087 | −0.015 |
|
||||||
|
| batter K × pitcher K | opportunity | 282 | −0.080 | 0.077 | −0.063 |
|
||||||
|
|
||||||
|
Solo, within BOMBER, the strongest is `batter_barrel_pct` at −0.105 (p=0.077);
|
||||||
|
`pitcher_breaking_share` is +0.025. **Head-to-head within BOMBER: 0.1599 vs the
|
||||||
|
counter's 0.2176 — the counter still leads on hits even inside its best
|
||||||
|
archetype**, consistent with the closed pooled negative.
|
||||||
|
|
||||||
|
**A bug fixed mid-run:** `pitcher_breaking_share` first reported **n=0** for every
|
||||||
|
row. `fromStatcastRow` maps percentage and raw fields only — it does not carry
|
||||||
|
`pitch_mix` — so the arsenal category was silently measuring nothing rather than
|
||||||
|
failing. Attaching the mix explicitly gave full coverage. Had it not been caught,
|
||||||
|
"arsenal doesn't matter" would have been recorded from a column that was never
|
||||||
|
populated.
|
||||||
|
|
||||||
|
### DEFENSE — the honest answer after looking
|
||||||
|
|
||||||
|
**We ingest no fielding data at all.** `statcast_aggregates` holds batter
|
||||||
|
offensive skill and pitcher stuff; there is no OAA, DRS, range, or positional
|
||||||
|
metric anywhere in it. I checked for a derivable proxy before declaring it
|
||||||
|
unsourceable, and the candidates all fail on construction:
|
||||||
|
|
||||||
|
- opposing pitchers' hits-allowed conflates *pitching* with *defense*, so it
|
||||||
|
would validate the wrong skill and could quietly "prove" defense using pitching;
|
||||||
|
- there is no team-level balls-in-play or expected-vs-actual column to difference.
|
||||||
|
|
||||||
|
**So defense is genuinely not derivable from what we hold** — it needs Baseball
|
||||||
|
Savant's fielding endpoint (free, same host as the five feeds already ingested,
|
||||||
|
so it is cheap). **Not sourced this order**, because sourcing it to test at n=282
|
||||||
|
would answer nothing.
|
||||||
|
|
||||||
|
## 4. Ship + recalibrate (STEP 5)
|
||||||
|
|
||||||
|
**Nothing proved, so nothing was recalibrated and nothing shipped.** The counter
|
||||||
|
continues to grade everything. The registry records the tested interactions as
|
||||||
|
CANDIDATE with their measured lift, so re-running at n≥500 compares against a
|
||||||
|
recorded baseline rather than starting over.
|
||||||
|
|
||||||
|
## 5. What actually unblocks this
|
||||||
|
|
||||||
|
Everything is one constraint: **sample per archetype**. Two things move it:
|
||||||
|
|
||||||
|
1. **The cap fix is already compounding** — 907 grades/snapshot vs 334, so
|
||||||
|
archetype cells fill ~2.7× faster than the rates that produced today's counts.
|
||||||
|
BOMBER × hits needs 213 more rows.
|
||||||
|
2. **A point-in-time window** from `statcast_history`, which starts producing
|
||||||
|
usable comparisons 2026-08-04.
|
||||||
|
|
||||||
|
**Ranked next:** BOMBER × hits (closest by far) → BOMBER × total_bases → pitcher
|
||||||
|
archetypes once strikeouts clear. **Add the Savant fielding feed before the
|
||||||
|
defense category is tested**, not before it can be.
|
||||||
|
|
||||||
|
**Not recommended:** recording anything as proven, sourcing defense to test at
|
||||||
|
n=282, or reading the arsenal incremental (+0.043) as encouraging — it is inside
|
||||||
|
noise at this sample, and the category only became measurable at all because a
|
||||||
|
silent n=0 was caught.
|
||||||
@@ -268,8 +268,81 @@ function summary(sport) {
|
|||||||
};
|
};
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* ── THE CONDITIONING REGISTRY ────────────────────────────────────────────
|
||||||
|
*
|
||||||
|
* Keyed archetype x underlying-skill x conditioning-interaction x status, with
|
||||||
|
* the measured out-of-sample lift.
|
||||||
|
*
|
||||||
|
* WHY THE SKILL TAG IS THE POINT. A proven interaction is not just "this feature
|
||||||
|
* helps this stat" — it is evidence that ONE UNDERLYING SKILL is real and
|
||||||
|
* measurable for this archetype. `barrel x park` proving would validate
|
||||||
|
* POWER-SKILL; `stuff x lineup-K` proving would validate WHIFF-SKILL. Tagging by
|
||||||
|
* skill is what lets a later model derive ALL of a player's props from one
|
||||||
|
* profile instead of fitting each prop separately — so the tag is the coherent
|
||||||
|
* profile taking shape, not bookkeeping.
|
||||||
|
*
|
||||||
|
* It starts EMPTY of proven entries, deliberately. Nothing has beaten the
|
||||||
|
* counter out-of-sample, so nothing has a skill validated. Seeding it with
|
||||||
|
* hopeful rows would defeat the purpose exactly as seeding PROVEN features would.
|
||||||
|
*/
|
||||||
|
const SKILLS = Object.freeze({
|
||||||
|
POWER: 'power-skill — how far and how hard the ball leaves the bat',
|
||||||
|
CONTACT: 'contact-skill — whether bat meets ball at all',
|
||||||
|
SPEED: 'speed-skill — beating out contact, taking the extra base',
|
||||||
|
WHIFF: 'whiff-skill — a pitcher missing bats',
|
||||||
|
COMMAND: 'command-skill — locating and expanding the zone',
|
||||||
|
OPPORTUNITY: 'opportunity — plate appearances / batters faced, not skill at all',
|
||||||
|
});
|
||||||
|
|
||||||
|
const conditioning = [];
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Record a conditioning result. `skill` MUST name the underlying skill the
|
||||||
|
* interaction validates — an untagged proven entry cannot contribute to a
|
||||||
|
* coherent profile, so it is refused.
|
||||||
|
*/
|
||||||
|
function recordConditioning({ sport, archetype, stat, interaction, skill, status, lift = null, evidence = null }) {
|
||||||
|
if (!Object.keys(SKILLS).includes(String(skill || '').toUpperCase())) {
|
||||||
|
return { ok: false, reason: 'untagged_or_unknown_skill', known: Object.keys(SKILLS) };
|
||||||
|
}
|
||||||
|
if (!Object.values(STATUS).includes(status)) return { ok: false, reason: 'bad_status' };
|
||||||
|
if (status === STATUS.PROVEN && !isSufficient(evidence)) {
|
||||||
|
return { ok: false, reason: 'insufficient_evidence_for_proven' };
|
||||||
|
}
|
||||||
|
const row = {
|
||||||
|
sport: String(sport || '').toLowerCase(),
|
||||||
|
archetype: String(archetype || '').toUpperCase(),
|
||||||
|
stat: String(stat || '').toLowerCase(),
|
||||||
|
interaction, skill: String(skill).toUpperCase(), status, lift, evidence,
|
||||||
|
};
|
||||||
|
conditioning.push(row);
|
||||||
|
return { ok: true, row };
|
||||||
|
}
|
||||||
|
|
||||||
|
/** The conditioning map for one archetype — what is proven, pending, dead. */
|
||||||
|
function conditioningFor(sport, archetype) {
|
||||||
|
const sp = String(sport || '').toLowerCase();
|
||||||
|
const ar = String(archetype || '').toUpperCase();
|
||||||
|
return conditioning.filter((c) => c.sport === sp && c.archetype === ar);
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The SKILLS VALIDATED for an archetype — the coherent profile as it stands.
|
||||||
|
* Only PROVEN entries count: a candidate interaction validates nothing.
|
||||||
|
*/
|
||||||
|
function validatedSkills(sport, archetype) {
|
||||||
|
const out = {};
|
||||||
|
for (const c of conditioningFor(sport, archetype)) {
|
||||||
|
if (c.status !== STATUS.PROVEN) continue;
|
||||||
|
(out[c.skill] = out[c.skill] || []).push({ stat: c.stat, interaction: c.interaction, lift: c.lift });
|
||||||
|
}
|
||||||
|
return out;
|
||||||
|
}
|
||||||
|
|
||||||
/** Test-only: restore the declared statuses so suites cannot leak into each other. */
|
/** Test-only: restore the declared statuses so suites cannot leak into each other. */
|
||||||
function __reset() {
|
function __reset() {
|
||||||
|
conditioning.length = 0;
|
||||||
statVerdicts.clear();
|
statVerdicts.clear();
|
||||||
byKey.clear();
|
byKey.clear();
|
||||||
for (const f of FEATURES) byKey.set(`${f.sport}|${f.key}`, { ...f, evidence: f.evidence || null, history: [] });
|
for (const f of FEATURES) byKey.set(`${f.sport}|${f.key}`, { ...f, evidence: f.evidence || null, history: [] });
|
||||||
@@ -279,5 +352,6 @@ module.exports = {
|
|||||||
STATUS, MIN_PROMOTION_N,
|
STATUS, MIN_PROMOTION_N,
|
||||||
allFeatures, liveFeatures, candidateFeatures, statusOf, isLive,
|
allFeatures, liveFeatures, candidateFeatures, statusOf, isLive,
|
||||||
recordStatVerdict, statusForStat, liveFeaturesForStat, candidateFeaturesForStat,
|
recordStatVerdict, statusForStat, liveFeaturesForStat, candidateFeaturesForStat,
|
||||||
|
SKILLS, recordConditioning, conditioningFor, validatedSkills,
|
||||||
isSufficient, promote, demote, summary, __reset,
|
isSufficient, promote, demote, summary, __reset,
|
||||||
};
|
};
|
||||||
|
|||||||
Reference in New Issue
Block a user