Files
vyndr/specs/conditioning-registry.md
T
builtbykev ac1361486e Build the conditioning registry, and a probe so "proven" stops drifting
The order opens with "two proven clusters live". They are not proven -- the
proven set is empty -- and this is the fourth consecutive order to start from
a stronger claim than the measurements support. Correcting that in prose four
times has not worked, so this session adds scripts/proven-status.js, which
recomputes the answer from the ledger: hits LOSES (-0.096, CI excluding zero),
total_bases INCONCLUSIVE (+0.004), strikeouts INCONCLUSIVE (+0.259 at n=57).
It deliberately reports sample readiness separately from recorded verdicts, so
"n>=500" can never again be read as "passed".

A counting error worth recording. The first read of the top-volume archetype
said BOMBER x hits was 641 rows -- gate-ready. It is 287. model_snapshots
holds one row per prop PER SNAPSHOT CYCLE, so joining it to ledger_entries
counts each ledger row once per cycle it appeared in. Deduping on the ledger
row id gives the true figure, and my own status script had the same bug until
it was fixed. That is the difference between running the gate and being short
by 213.

So no archetype x stat combination reaches the gate. BOMBER x hits at 287 is
the closest; pitcher archetypes are untestable at 58 settled strikeout rows
across all of them, so the pitcher half of this order could not be run.

The registry is built: recordConditioning keys archetype x underlying-skill x
interaction x status with measured lift, and the skill tag is MANDATORY and
enforced -- untagged entries are refused, and PROVEN without sufficient
evidence is refused. validatedSkills() returns the coherent profile as it
stands, which is {} for every archetype, by design.

BOMBER x hits conditioning was tested across the order's categories and every
result is underpowered: arsenal (barrel x breaking share) incremental +0.043,
batted-ball (launch x pitcher GB) +0.001, contact quality -0.020 and -0.015,
K x K -0.063. Within BOMBER the counter still leads on hits, 0.218 to 0.160,
consistent with the closed pooled negative.

One bug fixed mid-run: fromStatcastRow maps percentage and raw fields only and
does not carry pitch_mix, so the arsenal category first reported n=0 for every
row -- it was measuring nothing rather than failing. Without catching it,
"arsenal doesn't matter" would have been recorded from a column that was never
populated.

On defense: I looked for a derivable proxy before calling it unsourceable, and
there isn't one. We ingest no fielding data at all, and opposing pitchers'
hits-allowed conflates pitching with defense, so it would validate the wrong
skill. It needs Savant's fielding endpoint -- free, same host as the five
feeds already ingested -- and it is not sourced here, because sourcing it to
test at n=282 would answer nothing.

Nothing proved, so nothing was recalibrated and nothing shipped.

4,221 tests green (335 suites); web build exit 0.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W1sivYNqY2TS5ftykmHBU9
2026-08-03 21:49:51 -04:00

7.0 KiB
Raw Blame History

THE CONDITIONING REGISTRY — built, and what it currently holds

2026-08-03. Challenger-only. Counter, batter model and pitcher engine byte-identical (verified by diff).

No archetype × stat combination reaches the gate. The best is BOMBER × hits at n=287, short by 213. So the conditioning categories were tested and are all UNDERPOWERED — none proved, none died. Nothing was recalibrated and nothing shipped, because nothing earned it.

The durable deliverables are the registry itself and a status probe that makes "what is proven" a query instead of a memory.


0. The recurring premise problem, and a structural fix

This order opens with "two proven clusters live (batter contact stats + pitcher strikeouts)". They are not proven. The proven set is empty, and this is the fourth consecutive order to start from a stronger claim than the measurements support:

order said measurement said
"barrel rate PASSED solo" every total_bases feature refused on sample
"total_bases has passed BAR 1" inconclusive at parity, CI spanning zero
"whiff/stuff prove SOLO through the gate" all refused at n=57
"two proven clusters live" proven set EMPTY

Correcting it in prose four times has not worked, so this session added scripts/proven-status.js — it recomputes the answer from the ledger:

PROVEN_SET: EMPTY — no stat has beaten the counter out-of-sample
            with a CI excluding zero

hits         n=803  delta 0.0961  [0.165, 0.029]  LOSES
total_bases  n=383  delta +0.0038  [0.068, +0.075]  INCONCLUSIVE
strikeouts   n=57   delta +0.2592  [0.017, +0.564]  INCONCLUSIVE

stats at/above the gate:              hits (806) — and hits is a closed negative
archetype × stat at/above the gate:   NONE
closest:  BOMBER×hits 287 (short 213) · BOMBER×TB 142 · BOMBER×rbi 128

Run it before planning on top of a claim. It deliberately cannot say "proven" on its own — it reports sample readiness and recorded verdicts, so the two can never be conflated again.

1. The structured registry (STEP 1) — built

featureRegistry.recordConditioning() keys archetype × underlying-skill × conditioning-interaction × status, with measured lift.

The skill tag is mandatory and enforced. An untagged entry is refused (untagged_or_unknown_skill), and a PROVEN entry without sufficient evidence is refused (insufficient_evidence_for_proven). Skills: POWER, CONTACT, SPEED, WHIFF, COMMAND, OPPORTUNITY.

Why the tag matters: a proven interaction is not merely "this helps this stat" — it is evidence that one underlying skill is real and measurable for this archetype. validatedSkills(sport, archetype) returns the coherent profile as it currently stands. It returns {} for every archetype, because nothing has been proven, and seeding it with hopeful rows would defeat its purpose exactly as seeding PROVEN features would.

2. Top-volume selection (STEP 2)

Batter: BOMBER — the highest-volume archetype by a distance (287 settled hits rows; next is GHOST×hits at 124). Pitcher: none testable. Strikeouts total 58 settled rows across all archetypes, so no pitcher archetype has a sample. STEP 4 could not be run.

A counting error caught, worth recording

The first read said BOMBER × hits was 641 — gate-ready. It is 287. The join to model_snapshots fans out, because that table holds one row per prop per snapshot cycle, so each ledger row was counted once per cycle it appeared in. Deduping on the ledger row's identity gives the true figure. That is the difference between "run the gate" and "not close", and my own status script had the same bug until it was fixed.

3. BOMBER × hits conditioning (STEP 3) — all underpowered

n=282, Bonferroni across 17 tests. Every result refused on sample.

conditioning category n raw r best part incremental
barrel × breaking share ARSENAL 282 0.063 0.105 +0.043
launch × pitcher GB% BATTED-BALL 282 0.054 0.052 +0.001
launch × exit velo contact quality 282 0.060 0.087 0.020
exit velo × pitcher suppression contact quality 282 +0.008 0.087 0.015
batter K × pitcher K opportunity 282 0.080 0.077 0.063

Solo, within BOMBER, the strongest is batter_barrel_pct at 0.105 (p=0.077); pitcher_breaking_share is +0.025. Head-to-head within BOMBER: 0.1599 vs the counter's 0.2176 — the counter still leads on hits even inside its best archetype, consistent with the closed pooled negative.

A bug fixed mid-run: pitcher_breaking_share first reported n=0 for every row. fromStatcastRow maps percentage and raw fields only — it does not carry pitch_mix — so the arsenal category was silently measuring nothing rather than failing. Attaching the mix explicitly gave full coverage. Had it not been caught, "arsenal doesn't matter" would have been recorded from a column that was never populated.

DEFENSE — the honest answer after looking

We ingest no fielding data at all. statcast_aggregates holds batter offensive skill and pitcher stuff; there is no OAA, DRS, range, or positional metric anywhere in it. I checked for a derivable proxy before declaring it unsourceable, and the candidates all fail on construction:

  • opposing pitchers' hits-allowed conflates pitching with defense, so it would validate the wrong skill and could quietly "prove" defense using pitching;
  • there is no team-level balls-in-play or expected-vs-actual column to difference.

So defense is genuinely not derivable from what we hold — it needs Baseball Savant's fielding endpoint (free, same host as the five feeds already ingested, so it is cheap). Not sourced this order, because sourcing it to test at n=282 would answer nothing.

4. Ship + recalibrate (STEP 5)

Nothing proved, so nothing was recalibrated and nothing shipped. The counter continues to grade everything. The registry records the tested interactions as CANDIDATE with their measured lift, so re-running at n≥500 compares against a recorded baseline rather than starting over.

5. What actually unblocks this

Everything is one constraint: sample per archetype. Two things move it:

  1. The cap fix is already compounding — 907 grades/snapshot vs 334, so archetype cells fill ~2.7× faster than the rates that produced today's counts. BOMBER × hits needs 213 more rows.
  2. A point-in-time window from statcast_history, which starts producing usable comparisons 2026-08-04.

Ranked next: BOMBER × hits (closest by far) → BOMBER × total_bases → pitcher archetypes once strikeouts clear. Add the Savant fielding feed before the defense category is tested, not before it can be.

Not recommended: recording anything as proven, sourcing defense to test at n=282, or reading the arsenal incremental (+0.043) as encouraging — it is inside noise at this sample, and the category only became measurable at all because a silent n=0 was caught.