main
3 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 builtbykev
|
ff037e40c2 |
Re-adjudicate: nothing to demote, and close the hole that would have mattered
There is nothing to re-adjudicate. The proven set is empty and always has
been -- verified three ways: proven-status reports EMPTY, validatedSkills()
returns {} for every archetype, and zero conditioning entries have ever
reached PROVEN. The one PROVEN feature is recent_frequency_prior, which is the
incumbent counter itself, proven by the S78 ablation as ~100% of the
champion's resolution. It is the baseline every challenger is measured
against, not a conditioning interaction, and demoting it would leave the model
with nothing to grade from.
A correction to the premise: the cumulative gate did NOT catch a false
positive last session. It caught nothing, because there was nothing in the
proven set to catch. What it did was tighten alpha from 0.0026 to 0.0013
within one session, which demonstrated the mechanism working rather than a
demotion. So steps 3 and 4 -- demote, recalibrate -- are vacuous here, and
readjudicateAll says so plainly rather than glossing a no-op.
But the worry behind the order was well founded, and the audit found the real
exposure: promote() did not require the cumulative denominator. It checked n,
lift and CI, and nothing stopped a future session from testing eight
hypotheses, correcting by eight, and promoting on a p-value that would not
survive the programme's real denominator. That is precisely the hole that
makes a retroactive re-adjudication pass necessary later, so it is closed at
promotion time instead. isSufficient now refuses evidence carrying no
correction, evidence corrected against fewer tests than the cumulative count,
and any p-value that does not clear 0.05 over its own test count. The same
rule guards a PROVEN conditioning entry.
The second audit found two of four analysis scripts still correcting
per-session; pitcher-prove-k and tb-solo-and-interactions now use the
cumulative ledger, so the correction is native on every path.
reAblation.js is the standing second line: pure and injectable, so the
decision rule cannot drift from the gate's, and every verdict records both
p-values and both test counts so a demotion is re-derivable by anyone. A
feature promoted at alpha 0.05/20 can demote on the same p-value once the bar
is 0.05/60 -- correct, because the bar rose only after the programme had more
chances to get lucky. No fresh measurement is PENDING_RETEST and never a
demotion: absence of a re-test is not evidence, and demoting on it would
punish whichever stat happens to be off-season.
Net effect on the proven set is zero. No demotions, no recalibrations, and no
public ledger event -- announcing "recalibrated after re-adjudication" when
nothing changed would itself be a false signal of rigour.
4,238 tests green (337 suites); web build exit 0; counter byte-identical.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W1sivYNqY2TS5ftykmHBU9
|
||
|
builtbykev
|
9538e11198 |
Derive the lineup K-rate free, and fingerprint the cap fix
Two premise corrections first. Pitcher stuff features have NOT proven solo through the gate -- every one was refused on sample (n=57 against 500). Four exceed the effect-size bar (arm angle -0.250, whiff +0.213, k rate +0.206, chase +0.195), which is why they are worth pursuing, but clearing one of three thresholds is not passing. And the carrier was not blocked only on the lineup input: that input was built and measured last session at 94.7% coverage. What blocks it is n, and n was being throttled by the grading cap. RUNG 1 IS DERIVED AND COSTS NOTHING. Opposing-team K-rate comes from joining the opposing roster to the batter k_pct values already in statcast_aggregates -- no new feed. The improvement this session is that it is PA-WEIGHTED: an unweighted roster mean counts a 12-PA callup the same as an everyday starter, which is not the lineup a pitcher faces. That change alone reversed the term's sign. Unweighted, the lineup term HURT the model (0.1738 -> 0.1285). PA-weighted, it HELPS (0.1738 -> 0.1953). Same hypothesis, same data -- the derivation was the problem, not the signal, which is the entire argument for deriving the best honest version before sourcing anything. Head-to-head is now +0.2592 with a CI of [-0.0167, +0.5645], very nearly excluding zero, at n=57. Within archetype, the two strata come out with OPPOSITE signs -- FLAME incremental -0.152, non-FLAME +0.145 -- and the pooled value (+0.077) sits between them, which is the shape a conditional effect makes and is invisible when pooled. That is what stratifying was for. But n is 20 and 24, the standard error on a correlation there is about 0.22, and the direction contradicts the theory that predicted a stronger effect for finesse arms. It is recorded as a structure to re-test, not as a finding. Rungs 2 and 3 are NOT triggered. A rung fails only once it has been fairly tested, and Rung 1 is n-blocked rather than failed. Sourcing confirmed lineups now would be paying for precision on top of a proxy we have not yet measured. THE RESULT THAT DECIDES THE TIMELINE: yesterday's cap raise is fingerprinted in production at 907 grades per snapshot, up from 334, with strikeouts going 6 to 17. That puts n>=500 for pitcher Ks about a week out instead of three months. Operational note: the manual internal snapshot endpoint now 524s at the Cloudflare edge because grading the full board exceeds 100s -- the run still completes server-side (this very snapshot was written by a 524'd request) and the cron is in-process, so a 524 there is not a failure. Nothing proven, nothing calibrated, nothing shipped. The counter remains anti-predictive on strikeouts at -0.064 and the skill model leads it by 0.26. 4,221 tests green (335 suites); web build exit 0. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W1sivYNqY2TS5ftykmHBU9 |
||
|
builtbykev
|
843c8c6d4b |
Build the pitcher engine, and find the cap was eating the whole board
Strikeouts are NOT proven -- n=57 against a bar of 500. But the finding that matters is not a correlation. THE CAP. Measured on the live slate via the refusal diagnostic: 1,244 unique gradeable props exist, the 500 cap graded about 334, and because dedupeProps takes first-row-wins in FEED ORDER, what survives is decided by feed position rather than value. Pitchers are 2.6% of a batter-dominated feed, so we were grading SIX strikeout props a slate against 32 available -- putting n>=500 three months away for every pitcher stat. Pitcher props were never being refused (graded 5, refused 0, suppressed 0); it was truncation. Raised 500 -> 1500 on measured cost: 721ms per prop at concurrency 5 is about 179 seconds for the full board, against a cron that runs five times a day and a fire-and-forget caller that never holds an HTTP response. statsapi is free and unlimited. Concurrency stays at 5 -- one variable at a time. This unblocks every n-blocked stat in the programme, not just pitchers. THE ENGINE. pitcherEngine.js is its own engine, not the batter engine pointed at pitchers: the batter model asks whether contact becomes a hit and reads contact quality, the pitcher model asks whether the plate appearance ends without contact at all and reads stuff. Archetypes are FLAME (whiff-led), SCALPEL (chase-led), SINKER (pitches to contact) and DEFAULT, and a test asserts the weight keys are not the batter engine's. The projection is K% by log5 against THIS lineup, times batters faced, through a binomial. An unclassifiable arm gets the balanced map, never a guessed archetype. THE MEASUREMENT, at n=57 and contaminated. Four solo features clear the 0.15 effect bar and fail only on sample: arm angle at -0.250 -- the largest correlation measured anywhere in this programme -- then whiff +0.213, k rate +0.206, chase +0.195. The batter cluster's best was 0.135. Head to head, pitch-v1 resolves 0.1285 against the counter's -0.0639, delta +0.192 with a CI spanning zero. That negative is the interesting number. The counter is ANTI-PREDICTIVE on strikeouts: counting a pitcher's recent Ks is worse than useless, because his recent totals track which lineups he drew and how long he was left in rather than his skill. It is the one stat where the incumbent has no defensible edge. A bug caught on the way. resolveTeam wants an abbreviation and the game log supplies full team names, so the roster join silently resolved nothing and the first run reported 0% lineup coverage -- the theorized stuff x lineup carrier was never being tested, not failing. Fixed; coverage is now 94.7%. The carrier still shows no incremental signal over whiff alone, and adding the lineup term lowered head-to-head resolution, which is recorded rather than dropped. Calibration was not reached: nothing passed the first bar. The batter model and the counter are byte-identical, verified by diff. 4,221 tests green (335 suites); web build exit 0. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W1sivYNqY2TS5ftykmHBU9 |