Files
vyndr/specs/per-archetype-grade-bands.md
T
builtbykev 3081c92e00 Per-archetype grade bands: built, gated, and the rescale blocked twice
The premise does not hold. proven-status.js run fresh: PROVEN_SET is
EMPTY, no archetype x stat reaches the gate. pitcher_contact_profile has
a CI upper bound of exactly 0.0000 and platoon_severity is held on
4.5%-contaminated splits, so the proven set is one factor, pooled, not
three archetype-conditioned ones. The specific pattern the order names --
defense strong for GHOST/BRUSH, null for BOMBER -- is the one I measured
running the OTHER WAY yesterday, both noise-dominated.

But the second blocker is new and matters more, because it would stop the
rescale even if the factors had proved: the grade does not separate
within any archetype. Every archetype collapses to ONE band at the
corrected bar, because bands merge when their intervals overlap and
publishing two letters we cannot tell apart is a distinction we have not
measured.

Uncorrected, so the ranking is visible rather than hidden by the bar,
this INVERTS the order's design. The order gives contact types the
factor-rich treatment and power types honest base-rate, reasoning that
single-game hits are variance for a power profile. Measured:

  BOMBER n=466  corr(p_win,outcome) +0.207  quintiles 0.75 0.62 0.60 0.48 0.48
  GHOST  n=192  corr(p_win,outcome) -0.007  quintiles 0.47 0.63 0.74 0.58 0.45

BOMBER is the one archetype the model ranks, and it splits into a real
A 0.660 / B 0.481 at 95%. GHOST is flat, and non-monotone -- its most
confident reads hit 47% while its middle reads hit 74%. Shipping as
specified would have given the factor-rich treatment to the archetype the
model reads worst and left base-rate on the one it reads best. That is
mechanically sensible in hindsight: a power hitter's hit tracks whether
he can damage the arm, a contact hitter's depends on balls finding holes.

BOMBER's split does not survive the cumulative correction at 106 tests.
Exposing it by loosening the correction is the curve-to-make-A's the
order forbids, so it stays one band.

BUILT: gradeBands.js -- lift against the archetype's OWN base rate (the
same 62% is lift for a 45% profile and a deficit for a 68% one),
indistinguishable neighbours merged, thin bands PROVISIONAL not dropped,
Wilson intervals widened by the cumulative correction. The two-bar rule
is structural: proven-alone, calibrated-alone and neither all return
base_rate with the reason stated, so with nothing proven no
factor-informed band can be produced at all.

reasoning() is built and tested but NOT wired to the card -- there is no
per-archetype band being served, so attaching the copy now would ship
product language for a rescale that does not exist.

NOT BUILT: the specified power-type reason "the matchup edge is in
total_bases". total_bases is recorded INCONCLUSIVE (+0.0038, CI
[-0.068,+0.075]). Wiring it would assert an edge measured as
indistinguishable from zero -- the exact fabricated-reason failure this
module exists to prevent.

BOMBER x hits is 29 rows short of the gate and is the archetype the model
actually reads. That is the first slot to test, not GHOST.

Counter and frozen clusters byte-identical. No letter was moved.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W1sivYNqY2TS5ftykmHBU9
2026-08-06 01:03:10 -04:00

6.0 KiB
Raw Blame History

Per-archetype grade bands — built, gated, and not yet shippable

The rescale does not ship, for two independent reasons — and the second one is new and more interesting than the first.


1. The premise, checked against the ledger

scripts/proven-status.js (run fresh, not recalled):

PROVEN_SET: EMPTY — no stat has beaten the counter out-of-sample
archetype_x_stat_at_or_above_gate: []
closest: BOMBER x hits n=471 (short 29) · GHOST x hits n=196 (short 304)
the order states measured
three proven causally-correct factors one. pitcher_contact_profile CI upper bound is exactly 0.0000 (not proven); platoon_severity passes but is held on 4.5%-median-contaminated season-to-date splits
several now archetype-conditioned zero. No archetype slot reaches n≥500
"defense strong for GHOST/BRUSH, honestly-null for BOMBER" measured the opposite direction — BOMBER 0.0036, GHOST 0.0024, both noise-dominated (specs/per-archetype-re-audit.md)

Pooled proof does not qualify a slot. PROVEN_BY_ARCHETYPE is {} and the band builder reads it directly.


2. The new finding: the grade does not separate within archetype

This is the reason that matters, because it would block the rescale even if the factors had proved.

Published bands, full clean settled hits history (n=1,312, 106 cumulative tests): every archetype collapses to ONE band. Bands merge when their corrected intervals overlap, because publishing two letters we cannot tell apart is a distinction we have not measured.

Uncorrected, so the underlying ranking is visible rather than hidden by the bar:

archetype n base rate quintiles (hi→lo) corr(p_win, outcome) bands @95%
BOMBER 466 0.588 0.75 · 0.62 · 0.60 · 0.48 · 0.48 +0.207 2 — A 0.660 (n=279) vs B 0.481 (n=187), A shows lift
UNLABELLED 490 0.539 0.66 · 0.53 · 0.58 · 0.51 · 0.41 +0.155 2, no lift
GHOST 192 0.578 0.47 · 0.63 · 0.74 · 0.58 · 0.45 0.007 1
BRUSH 80 0.537 0.81 · 0.63 · 0.44 · 0.50 · 0.31 +0.350 1

This inverts the order's design

The order assigns contact types the factor-rich treatment and power types the honest base-rate treatment, on the reasoning that single-game hits are variance for a power profile.

Measured, it is the other way round. BOMBER is the one archetype where the model ranks (+0.207, monotone across quintiles, and it separates into a genuine A/B at 95%). GHOST is where it is flat — corr 0.007, and its quintiles are non-monotone: the model's most confident GHOST reads hit 47% while its middle reads hit 74%.

That ordering is mechanically sensible in hindsight. A power hitter's chance of a hit tracks whether he can damage the arm he is facing, which the counter's frequency question partly captures. A contact hitter's hits depend on batted balls finding holes, which is much closer to luck — the same reason the pooled hits negative closed (S83) and the reason defense_by_direction is the factor that survives.

Had the rescale shipped as specified, it would have given the factor-rich treatment to the archetype the model reads worst, and left base-rate on the one it reads best.

BOMBER still cannot publish two letters

BOMBER's A/B split is real at 95% but does not survive the cumulative correction at 106 tests — the intervals merge. So even the best archetype gets one honest band today. Lowering the correction to expose it would be exactly the "curve to make more A's" the order forbids.


3. What was built

src/services/model/gradeBands.js — per-archetype bands from realized outcomes:

  • Lift, not raw rate. A band is credited only when its interval clears that archetype's own base rate. Locked by test: the same 62% realized rate is lift for a 45%-base profile and a deficit for a 68%-base one.
  • The two-bar rule is structural. factor_informed requires proven AND calibrated for that archetype. Tests assert that proven-alone, calibrated- alone, and neither all return basis: 'base_rate' with the reason stated — so with nothing proven, which is today, no factor-informed band can be produced at all. Same shape as featureRegistry.liveFeatures(): the honest state is the default and the richer claim has to be earned past a gate.
  • Indistinguishable neighbours merge rather than becoming different letters.
  • Thin bands are PROVISIONAL, not dropped — "still counting" and "nothing here" are different claims.
  • Wilson intervals, widened by the cumulative correction. Wilson because these bands are small and rates sit near the edges, where a normal approximation runs past 0 and 1 and implies impossible rates.

reasoning() is built and tested: a base-rate band says "base-rate read … no matchup factor is proven for this profile yet", and a factor-informed band with no named proven factors returns nothing rather than inventing a why.

It is not wired to the card, deliberately. There is no per-archetype band being served, so attaching per-archetype copy now would ship product language for a rescale that does not exist.

One piece of the order I did not build

The specified power-type reasoning — "the matchup edge is in total_bases" — is not supported by any measurement. proven-status.js records total_bases as INCONCLUSIVE (delta +0.0038, CI [0.068, +0.075]). Wiring that sentence would assert an edge we have measured as indistinguishable from zero, which is the fabricated-reason failure the rest of this module exists to prevent.


4. What would unblock the rescale

  1. BOMBER × hits is 29 rows short of the gate — days away, and it is the archetype the model actually reads. That is the first slot to test, not GHOST.
  2. Point-in-time platoon splits would convert two held passes into real ones.
  3. Separation must survive the correction, not just 95%. More sample tightens the intervals; nothing else legitimately does.

Counter and frozen clusters untouched. No letter was moved.