Build the gate, run it, and find we were proving things on the wrong stat

PREMISE CORRECTION FIRST. statModel.js and correlateValidator.js do not exist
in this repository. The validation spec's only prior form is
src/services/python/blueprints/unconventional.py -- a Flask blueprint in the
Python service that is offline in production, scoring NBA factors against a
warehouse that was never populated -- and tests/unit/supplementSystems.test.js
requires only fs and path while defining its own validateFactor inline at line
368. Those tests assert a re-implementation of the thresholds, not an
implementation, which is exactly why they passed for months while nothing was
connected. The diagnosis behind the order is right -- every challenger was
measured without a gate -- but the cause is that there was no gate on the Node
side to import. So it is built, to the exact spec.

correlateValidator: n>=500, |r|>=0.15, p<0.05, Bonferroni across the sweep.
The p-value is exact rather than approximated (t-transform through a
regularized incomplete beta) and is verified in the suite against known
values, because scipy is not available here. Pairs with an unknown side are
dropped, never zero-filled -- a zero-fill inside a correlation does not add
noise, it invents a point at the origin.

THE RUN, hits, n=570, Bonferroni-8: every skill feature fails, and not
narrowly. The strongest marginal correlation against the counter's residual is
0.062 against a 0.15 bar. That is an effect-size failure at a sample that
would have found a real effect comfortably -- a clean, well-powered negative.
The head-to-head agrees: value engine 0.0499 against the counter's 0.166,
delta -0.116 with CI [-0.189, -0.043]. Not promoted.

THE RUN, total bases, n=295: cannot be tested, and that is the finding.
hard_hit_pct shows a marginal r of 0.153 -- above the threshold -- and exit
velo 0.124, refused solely because n is 205 short of 500. It is the most
encouraging number this work has produced, and it is what the physics
predicts: contact quality governs extra bases, not whether a grounder finds a
hole. We have been testing skill inputs on the one stat where they should not
matter much.

Two things the run forced. Feature verdicts are now PER STAT, because marking
these DEAD sport-wide on hits evidence would have killed, for total bases, the
features that look most alive there -- per-sport doctrine one level deeper.
And the gate now reports r and p even when underpowered, because "not enough
data yet" and "nothing here" demand opposite decisions and a bare refusal was
hiding the best signal on the board.

Next: build the compound TB projection (skillProjection still refuses total
bases by design, since a deterministic bases-per-hit made P(TB>=2) identical
to P(hits>=1)), accrue to n>=500, re-run this gate. Leave hits alone.

4,200 tests green (334 suites); web build exit 0; counter byte-identical.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W1sivYNqY2TS5ftykmHBU9
This commit is contained in:
Kev
2026-08-03 02:34:02 -04:00
parent 258d8a6655
commit c7cc8f5e52
8 changed files with 883 additions and 0 deletions
+36
View File
@@ -1301,6 +1301,42 @@ phased plan in the Session-57 conversation / BUILD-STATE Next section).
branch. Now its own `statcastTick`. The old test only checked the string
existed — the new one asserts it is not behind the snapshot-hours guard.
## The validation gate + the stat that was wrong (Session 81 — non-obvious)
- **`statModel.js` and `correlateValidator.js` NEVER EXISTED** in this repo. The
spec's only prior form was `src/services/python/blueprints/unconventional.py`
(Flask, in the OFFLINE python service, scoring NBA factors against an empty
warehouse), and `tests/unit/supplementSystems.test.js` INLINES its own
`validateFactor` (line 368; only `fs`/`path` are required). So those tests
passed for months with no implementation to connect — that is the real reason
every challenger was measured ungated.
- **`src/services/model/correlateValidator.js` is the gate now** — n>=500,
|r|>=0.15, p<0.05, Bonferroni. The p-value is EXACT (t-transform via a
regularized incomplete beta, Lentz CF) and unit-verified against known values;
scipy isn't available in Node so don't reach for an approximation. Pairs with
an unknown side are DROPPED — zero-filling a correlation invents a point at
the origin.
- **HITS IS A CLEAN NEGATIVE — stop modelling it.** Gate run at n=570,
Bonferroni-8: EVERY skill feature fails, max marginal |r| = 0.062 vs a 0.15
bar. Not a power problem — an effect-size problem. And the value engine loses
head-to-head (0.0499 vs 0.166, CI [0.189,0.043]). At the 0.5 hits line there
is very little for skill inputs to know.
- **TOTAL BASES IS WHERE THE SIGNAL IS, and it is n-blocked.** Same features:
`hard_hit_pct` marginal r = **0.153** (above threshold), `exit_velo` 0.124,
raw r 0.196/0.167 — refused ONLY because n=295 < 500. Needs ~205 more settled
rows. This is what the physics predicts: contact quality drives EXTRA BASES,
not whether a grounder finds a hole.
- **The gate reports r and p even when underpowered** (`underpowered: true`,
`rows_needed`). "Not enough data yet" and "nothing here" need OPPOSITE
decisions — collapsing them into a bare refusal hid the best signal on the board.
- **Feature verdicts are PER STAT** (`recordStatVerdict` / `statusForStat` /
`candidateFeaturesForStat`). Marking these DEAD sport-wide on hits evidence
would have killed the features most alive on TB. Per-sport doctrine one level
deeper: physics differ per stat.
- **Next is the compound TB projection** — `skillProjection` still REFUSES
total_bases (a deterministic bases-per-hit made P(TB>=2) == P(hits>=1)). Build
the per-hit extra-base distribution off launch/barrel (tb-v1's shape, fed by
skill inputs), accrue to n>=500, re-run this gate. Do NOT lower the bar.
## Active Skills
- vyndr-voice (all user-facing output)
- prop-analysis (grading methodology)