vyndr

builtbykev/vyndr

Fork 0

Commit Graph

Author	SHA1	Message	Date
builtbykev	ece2b9f5f9	Ingest defence, and make Bonferroni cumulative across the programme Two things shipped that stand regardless of sample. DEFENCE. Statcast Outs Above Average is free on the host we already pull six feeds from, so there was nothing to decide. 514 fielders, aggregated to team level -- the unit a batter's prop actually needs, the defence behind the pitcher he faces -- and persisted as 31 team rows. Verified in production. Cubs +56 best, Mariners -29 worst. Unknown is not zero, and it bites unusually hard here: an OAA of 0 is a REAL reading meaning exactly average, so coercing absence to 0 would assert that every unmeasured fielder is league-average, which is the commonest defensive profile there is. team_defense also carries as_of_date in its primary key from the first row -- statcast_aggregates was built upsert-in-place and that silently made every backtest leak the games it predicted, so point-in-time is available here before it is needed rather than after a wrong answer. A bug worth recording as a class: BASE already ends in /leaderboard, so the new feed built a doubled path and 404'd. Because a failing feed degrades to an empty index by design -- correct, so one broken source cannot fail the whole pull -- it surfaced as "fielding_oaa: 0 rows", which reads exactly like "Statcast has no fielding data". Graceful degradation makes a wiring bug look like an honest absence. CUMULATIVE CORRECTION. Bonferroni had been applied per session throughout: a run testing eight features corrected by eight. Across a programme's lifetime that is wrong in the dangerous direction, because every order gets a fresh generous alpha and the false-positive rate compounds quietly. Correcting by 8 when sixty have been tried is how a noise result eventually gets recorded as PROVEN with a p-value to point at. The denominator is now distinct hypotheses ever tested, persisted, and it moved 19 -> 38 within this session alone, alpha 0.0026 -> 0.0013. Re-tests deliberately do not inflate it: re-asking the same question on more data is not a new shot on goal, and counting it would punish the discipline of waiting for sample. THE MEASUREMENT. The differential the theory predicted is present: defence correlates with the counter's residual at +0.130 for GHOST, the contact and speed archetype, and -0.018 for BOMBER, the power archetype. A GHOST's hits depend on whether anyone can range to the ball; a BOMBER's barrels clear the defence entirely. So a flat BOMBER result is the theory working rather than the test failing. It is not a result. GHOST is n=104 against a 500 bar, with p=0.188 against a corrected alpha of 0.0013 -- three orders of magnitude short. Both are recorded as CANDIDATE with their measured lift, tagged contact-skill, so the re-run at full sample compares against a recorded baseline. Nothing proved, so nothing was recalibrated and nothing shipped. 4,228 tests green (336 suites); web build exit 0. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W1sivYNqY2TS5ftykmHBU9	2026-08-03 22:20:30 -04:00
builtbykev	ac1361486e	Build the conditioning registry, and a probe so "proven" stops drifting The order opens with "two proven clusters live". They are not proven -- the proven set is empty -- and this is the fourth consecutive order to start from a stronger claim than the measurements support. Correcting that in prose four times has not worked, so this session adds scripts/proven-status.js, which recomputes the answer from the ledger: hits LOSES (-0.096, CI excluding zero), total_bases INCONCLUSIVE (+0.004), strikeouts INCONCLUSIVE (+0.259 at n=57). It deliberately reports sample readiness separately from recorded verdicts, so "n>=500" can never again be read as "passed". A counting error worth recording. The first read of the top-volume archetype said BOMBER x hits was 641 rows -- gate-ready. It is 287. model_snapshots holds one row per prop PER SNAPSHOT CYCLE, so joining it to ledger_entries counts each ledger row once per cycle it appeared in. Deduping on the ledger row id gives the true figure, and my own status script had the same bug until it was fixed. That is the difference between running the gate and being short by 213. So no archetype x stat combination reaches the gate. BOMBER x hits at 287 is the closest; pitcher archetypes are untestable at 58 settled strikeout rows across all of them, so the pitcher half of this order could not be run. The registry is built: recordConditioning keys archetype x underlying-skill x interaction x status with measured lift, and the skill tag is MANDATORY and enforced -- untagged entries are refused, and PROVEN without sufficient evidence is refused. validatedSkills() returns the coherent profile as it stands, which is {} for every archetype, by design. BOMBER x hits conditioning was tested across the order's categories and every result is underpowered: arsenal (barrel x breaking share) incremental +0.043, batted-ball (launch x pitcher GB) +0.001, contact quality -0.020 and -0.015, K x K -0.063. Within BOMBER the counter still leads on hits, 0.218 to 0.160, consistent with the closed pooled negative. One bug fixed mid-run: fromStatcastRow maps percentage and raw fields only and does not carry pitch_mix, so the arsenal category first reported n=0 for every row -- it was measuring nothing rather than failing. Without catching it, "arsenal doesn't matter" would have been recorded from a column that was never populated. On defense: I looked for a derivable proxy before calling it unsourceable, and there isn't one. We ingest no fielding data at all, and opposing pitchers' hits-allowed conflates pitching with defense, so it would validate the wrong skill. It needs Savant's fielding endpoint -- free, same host as the five feeds already ingested -- and it is not sourced here, because sourcing it to test at n=282 would answer nothing. Nothing proved, so nothing was recalibrated and nothing shipped. 4,221 tests green (335 suites); web build exit 0. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W1sivYNqY2TS5ftykmHBU9	2026-08-03 21:49:51 -04:00
builtbykev	c0621e7aa2	Measure the batter cluster: the proven set is empty, and hits is closed PREMISE CORRECTION FIRST, because it defines the bar. total_bases has not passed BAR 1. Its head-to-head is inconclusive at parity -- delta +0.004 to +0.007 with a CI spanning zero -- and it is contaminated, and no feature of its passed the gate. It was described last session as the first challenger that did not LOSE, which is not the same as proven. If it is installed as the frozen proven reference and every other stat is held to "the identical bar total_bases cleared", the bar becomes "be inconclusive at parity" and the whole cluster passes on a null result. The proven set is EMPTY. HITS IS NOW A FINAL ANSWER. At n=803 it clears the gate's sample requirement, so its features were properly TESTED rather than refused: every one fails on effect size (max marginal \|r\| 0.053 against a 0.15 bar), every interaction's incremental contribution collapses to about zero, and the model loses head-to-head by 0.096 with a CI excluding zero. That is a well-powered negative and hits should be closed rather than retried. The rest are n-blocked: total_bases 383, rbi 391, home_runs 228, runs 188, against a bar of 500. Two leads are worth carrying. home_runs barrel rate has a marginal r of -0.135, and the sign matters -- higher barrel rate goes with the counter OVER-predicting, which would be a correction rather than a new predictor. And runs batterK x pitcherK has the largest incremental in the cluster at +0.132, with a clean mechanism: strikeouts destroy plate appearances, and a PA that never happens cannot score. RBI deserves a caveat rather than a verdict. It is power times OPPORTUNITY, and we ingest no baserunner state at all, so half its mechanism is missing. A weak RBI result is evidence that we are modelling half the stat. total_bases was held frozen: git diff on skillProjection against the prior commit is empty. The counter is untouched. Also fixed and verified in production: the point-in-time retention shipped after yesterday's refresh had already run, so statcast_history was empty, and its first run then failed on a hand-enumerated schema that had already drifted from its source ("could not find the 'swing_pct' column"). The refresh itself still succeeded and wrote all 1,387 aggregate rows, which confirmed the best-effort guard in prod. The table now mirrors the source via LIKE and the writer passes rows through whole. Verified live: 1,387 rows retained at as_of 2026-08-03. A usable point-in-time window starts 2026-08-04. Stage B has nothing to calibrate. Everything now waits on a point-in-time window and on sample -- both waiting problems, not building problems. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W1sivYNqY2TS5ftykmHBU9	2026-08-03 16:58:57 -04:00

Author

SHA1

Message

Date

builtbykev

ece2b9f5f9

Ingest defence, and make Bonferroni cumulative across the programme

Two things shipped that stand regardless of sample.

DEFENCE. Statcast Outs Above Average is free on the host we already pull six
feeds from, so there was nothing to decide. 514 fielders, aggregated to team
level -- the unit a batter's prop actually needs, the defence behind the
pitcher he faces -- and persisted as 31 team rows. Verified in production.
Cubs +56 best, Mariners -29 worst.

Unknown is not zero, and it bites unusually hard here: an OAA of 0 is a REAL
reading meaning exactly average, so coercing absence to 0 would assert that
every unmeasured fielder is league-average, which is the commonest defensive
profile there is. team_defense also carries as_of_date in its primary key from
the first row -- statcast_aggregates was built upsert-in-place and that
silently made every backtest leak the games it predicted, so point-in-time is
available here before it is needed rather than after a wrong answer.

A bug worth recording as a class: BASE already ends in /leaderboard, so the
new feed built a doubled path and 404'd. Because a failing feed degrades to an
empty index by design -- correct, so one broken source cannot fail the whole
pull -- it surfaced as "fielding_oaa: 0 rows", which reads exactly like
"Statcast has no fielding data". Graceful degradation makes a wiring bug look
like an honest absence.

CUMULATIVE CORRECTION. Bonferroni had been applied per session throughout: a
run testing eight features corrected by eight. Across a programme's lifetime
that is wrong in the dangerous direction, because every order gets a fresh
generous alpha and the false-positive rate compounds quietly. Correcting by 8
when sixty have been tried is how a noise result eventually gets recorded as
PROVEN with a p-value to point at. The denominator is now distinct hypotheses
ever tested, persisted, and it moved 19 -> 38 within this session alone, alpha
0.0026 -> 0.0013. Re-tests deliberately do not inflate it: re-asking the same
question on more data is not a new shot on goal, and counting it would punish
the discipline of waiting for sample.

THE MEASUREMENT. The differential the theory predicted is present: defence
correlates with the counter's residual at +0.130 for GHOST, the contact and
speed archetype, and -0.018 for BOMBER, the power archetype. A GHOST's hits
depend on whether anyone can range to the ball; a BOMBER's barrels clear the
defence entirely. So a flat BOMBER result is the theory working rather than
the test failing.

It is not a result. GHOST is n=104 against a 500 bar, with p=0.188 against a
corrected alpha of 0.0013 -- three orders of magnitude short. Both are
recorded as CANDIDATE with their measured lift, tagged contact-skill, so the
re-run at full sample compares against a recorded baseline.

Nothing proved, so nothing was recalibrated and nothing shipped.

4,228 tests green (336 suites); web build exit 0.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W1sivYNqY2TS5ftykmHBU9

2026-08-03 22:20:30 -04:00

builtbykev

ac1361486e

Build the conditioning registry, and a probe so "proven" stops drifting

The order opens with "two proven clusters live". They are not proven -- the
proven set is empty -- and this is the fourth consecutive order to start from
a stronger claim than the measurements support. Correcting that in prose four
times has not worked, so this session adds scripts/proven-status.js, which
recomputes the answer from the ledger: hits LOSES (-0.096, CI excluding zero),
total_bases INCONCLUSIVE (+0.004), strikeouts INCONCLUSIVE (+0.259 at n=57).
It deliberately reports sample readiness separately from recorded verdicts, so
"n>=500" can never again be read as "passed".

A counting error worth recording. The first read of the top-volume archetype
said BOMBER x hits was 641 rows -- gate-ready. It is 287. model_snapshots
holds one row per prop PER SNAPSHOT CYCLE, so joining it to ledger_entries
counts each ledger row once per cycle it appeared in. Deduping on the ledger
row id gives the true figure, and my own status script had the same bug until
it was fixed. That is the difference between running the gate and being short
by 213.

So no archetype x stat combination reaches the gate. BOMBER x hits at 287 is
the closest; pitcher archetypes are untestable at 58 settled strikeout rows
across all of them, so the pitcher half of this order could not be run.

The registry is built: recordConditioning keys archetype x underlying-skill x
interaction x status with measured lift, and the skill tag is MANDATORY and
enforced -- untagged entries are refused, and PROVEN without sufficient
evidence is refused. validatedSkills() returns the coherent profile as it
stands, which is {} for every archetype, by design.

BOMBER x hits conditioning was tested across the order's categories and every
result is underpowered: arsenal (barrel x breaking share) incremental +0.043,
batted-ball (launch x pitcher GB) +0.001, contact quality -0.020 and -0.015,
K x K -0.063. Within BOMBER the counter still leads on hits, 0.218 to 0.160,
consistent with the closed pooled negative.

One bug fixed mid-run: fromStatcastRow maps percentage and raw fields only and
does not carry pitch_mix, so the arsenal category first reported n=0 for every
row -- it was measuring nothing rather than failing. Without catching it,
"arsenal doesn't matter" would have been recorded from a column that was never
populated.

On defense: I looked for a derivable proxy before calling it unsourceable, and
there isn't one. We ingest no fielding data at all, and opposing pitchers'
hits-allowed conflates pitching with defense, so it would validate the wrong
skill. It needs Savant's fielding endpoint -- free, same host as the five
feeds already ingested -- and it is not sourced here, because sourcing it to
test at n=282 would answer nothing.

Nothing proved, so nothing was recalibrated and nothing shipped.

4,221 tests green (335 suites); web build exit 0.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W1sivYNqY2TS5ftykmHBU9

2026-08-03 21:49:51 -04:00

builtbykev

c0621e7aa2

Measure the batter cluster: the proven set is empty, and hits is closed

PREMISE CORRECTION FIRST, because it defines the bar. total_bases has not
passed BAR 1. Its head-to-head is inconclusive at parity -- delta +0.004 to
+0.007 with a CI spanning zero -- and it is contaminated, and no feature of
its passed the gate. It was described last session as the first challenger
that did not LOSE, which is not the same as proven. If it is installed as the
frozen proven reference and every other stat is held to "the identical bar
total_bases cleared", the bar becomes "be inconclusive at parity" and the
whole cluster passes on a null result. The proven set is EMPTY.

HITS IS NOW A FINAL ANSWER. At n=803 it clears the gate's sample requirement,
so its features were properly TESTED rather than refused: every one fails on
effect size (max marginal |r| 0.053 against a 0.15 bar), every interaction's
incremental contribution collapses to about zero, and the model loses
head-to-head by 0.096 with a CI excluding zero. That is a well-powered
negative and hits should be closed rather than retried.

The rest are n-blocked: total_bases 383, rbi 391, home_runs 228, runs 188,
against a bar of 500. Two leads are worth carrying. home_runs barrel rate has
a marginal r of -0.135, and the sign matters -- higher barrel rate goes with
the counter OVER-predicting, which would be a correction rather than a new
predictor. And runs batterK x pitcherK has the largest incremental in the
cluster at +0.132, with a clean mechanism: strikeouts destroy plate
appearances, and a PA that never happens cannot score.

RBI deserves a caveat rather than a verdict. It is power times OPPORTUNITY,
and we ingest no baserunner state at all, so half its mechanism is missing. A
weak RBI result is evidence that we are modelling half the stat.

total_bases was held frozen: git diff on skillProjection against the prior
commit is empty. The counter is untouched.

Also fixed and verified in production: the point-in-time retention shipped
after yesterday's refresh had already run, so statcast_history was empty, and
its first run then failed on a hand-enumerated schema that had already drifted
from its source ("could not find the 'swing_pct' column"). The refresh itself
still succeeded and wrote all 1,387 aggregate rows, which confirmed the
best-effort guard in prod. The table now mirrors the source via LIKE and the
writer passes rows through whole. Verified live: 1,387 rows retained at as_of
2026-08-03. A usable point-in-time window starts 2026-08-04.

Stage B has nothing to calibrate. Everything now waits on a point-in-time
window and on sample -- both waiting problems, not building problems.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W1sivYNqY2TS5ftykmHBU9

2026-08-03 16:58:57 -04:00

3 Commits