builtbykev
91927a4a8a
number beside it PHASE 0 corrects the order's premise. A grade letter has been served all along -- engine1.gradeProp builds it from an additive factor index, computed INDEPENDENTLY of p_win. gradeBands is orphaned for a different reason than assumed: it defines what a letter MEANS from realized outcomes, and every band collapses to base-rate at current resolution. The measurement that changed this order, on 3,417 settled props: grade n realized mean p_win A 8 0.500 0.647 <- the TOP grade did WORST B 985 0.640 0.700 C 1,695 0.602 0.676 D 303 0.558 0.604 F 426 0.535 0.588 letter resolution 0.00116 (0.48% of variance) p_win resolution 0.00715 (2.98%) -> the letter carried 0.16x the information of the number beside it Concretely, from the hand-verify: Christian Encarnacion's 0.95 over graded C and his 0.05 under ALSO graded C -- same hitter, opposite forecasts, same letter. The gap was never that grades don't ship; it is that the weaker of two available signals shipped as the headline. PHASE 1 — model/servedGrade.js derives the letter from p_win with bands anchored on MEASURED realized rates (B+ 0.663 / B 0.646 / C+ 0.615 / C 0.589 / C- 0.548 / D 0.512 / F 0.447, base 0.6005). NO MANUFACTURED A, structurally: A+/A/A- are UNISSUABLE, not rare. The realized rate plateaus at 0.65-0.68 above p_win 0.70, so no band has earned a top letter; a test sweeps every p_win 0..1 and asserts none produces one. Even 0.99 tops out at B+ with its realized 0.663 attached. Raising that ceiling later is a deliberate, visible act. Bands that cannot separate SAY so -- C+/C/C- carry separates_from_base_rate false and copy naming it, which is the honest description of a forecast explaining 3% of variance. Every grade states its basis (forecast_only vs forecast_plus_matchup_factors, naming which factors fired) and calibrated:false. engine1.grade is preserved as engine_grade so nothing downstream breaks. PHASE 2 — refusals render real states: insufficient_data -> "not enough history to call this one"; juiced_no_edge -> "the book has priced the vig past any edge on this side". 1,870 refused snapshots carry exactly those two reasons and both now surface. PHASE 3 — hand-verified on 12 real served props. Freeman/Rice/Encarnacion 0.95 overs now B+ (was B, C, B); the 0.05 unders now F (was C). Refused doubles render NO READ with their reason. never-blank PASS, no-manufactured-A PASS. Serving change; nine frozen model modules unchanged including engine1; p_win never mutated; no calibrated number leaks (deployed set empty); no Bonferroni slot. STILL TRUE: the forecast explains ~3% of outcome variance. This order did not make the model better. It made the letter stop overstating it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W1sivYNqY2TS5ftykmHBU9
133 lines
5.0 KiB
Markdown
133 lines
5.0 KiB
Markdown
# The grade surface — it wasn't missing, it was serving the weaker signal
|
||
|
||
## PHASE 0 — what actually reaches a user
|
||
|
||
**Correction to the order's premise: a grade letter has been served all along.**
|
||
`engine1.gradeProp` produces it from an additive factor index, and it is
|
||
**not derived from `p_win` at all** — the two are computed independently and both
|
||
ride the payload.
|
||
|
||
`gradeBands` is orphaned for a different reason than assumed: it defines what a
|
||
letter *means* from realized outcomes, and every band it produces collapses to
|
||
base-rate at current resolution. It was never the missing link to the surface.
|
||
|
||
### The measurement that changed the order
|
||
|
||
3,417 settled props, pooled across the four batter stats:
|
||
|
||
| grade | n | realized | mean p_win |
|
||
|---|---|---|---|
|
||
| **A** | 8 | **0.5000** | 0.6466 |
|
||
| B | 985 | 0.6396 | 0.7002 |
|
||
| C | 1,695 | 0.6024 | 0.6755 |
|
||
| D | 303 | 0.5578 | 0.6042 |
|
||
| F | 426 | 0.5352 | 0.5875 |
|
||
|
||
```
|
||
grade-letter resolution 0.00116 (0.48% of variance)
|
||
p_win resolution 0.00715 (2.98%)
|
||
=> the letter carries 0.16x the information of the number beside it
|
||
```
|
||
|
||
**The top grade hit worse than the bottom grade.** And concretely, from the
|
||
hand-verify: **Christian Encarnación's 0.95 over graded `C`, and his 0.05 under
|
||
also graded `C`** — same hitter, opposite forecasts, same letter.
|
||
|
||
The gap was never that grades don't ship. It is that **the weaker of two
|
||
available signals was shipping as the headline.**
|
||
|
||
---
|
||
|
||
## PHASE 1 — the honest grade
|
||
|
||
`model/servedGrade.js` derives the letter from `p_win`, with bands anchored on
|
||
**measured realized rates**, not targets:
|
||
|
||
| letter | p_win ≥ | realized | separates from base rate? |
|
||
|---|---|---|---|
|
||
| B+ | 0.780 | 0.663 | yes |
|
||
| B | 0.700 | 0.646 | yes |
|
||
| C+ | 0.640 | 0.615 | **no** |
|
||
| C | 0.560 | 0.589 | **no** |
|
||
| C- | 0.480 | 0.548 | **no** |
|
||
| D | 0.350 | 0.512 | yes |
|
||
| F | 0.000 | 0.447 | yes |
|
||
|
||
Base rate 0.6005.
|
||
|
||
### No manufactured A — structurally
|
||
|
||
**`A+`, `A` and `A-` are UNISSUABLE.** Not rare — absent by construction. The
|
||
realized rate plateaus at 0.65–0.68 above p_win 0.70 (the 0.9+ bucket does no
|
||
better than the 0.8 bucket), so no band of this forecast has earned a top letter.
|
||
A test sweeps every p_win from 0 to 1 and asserts none produces one. Even a 0.99
|
||
forecast tops out at B+ with its realized 0.663 attached.
|
||
|
||
When resolution improves enough to earn an A, the ceiling gets raised
|
||
deliberately and visibly — not by a threshold quietly drifting.
|
||
|
||
### Bands that cannot separate SAY so
|
||
|
||
`C+ / C / C-` carry `separates_from_base_rate: false` and copy that names it —
|
||
*"a base-rate read; the model sees nothing that separates this."* That covers the
|
||
bulk of the board, and it is the honest description of a forecast explaining 3%
|
||
of variance.
|
||
|
||
### The basis is stated, never implied
|
||
|
||
Each grade carries `basis`: `forecast_plus_matchup_factors` (naming which of the
|
||
three proven factors fired) or `forecast_only`, plus `calibrated: false` —
|
||
calibration is withdrawn and nothing here rides on a number that doesn't exist.
|
||
|
||
`engine1.grade` is preserved as `engine_grade` so nothing downstream breaks and
|
||
the two stay comparable.
|
||
|
||
---
|
||
|
||
## PHASE 2 — the refusal surface
|
||
|
||
Refusals render a real state, never a blank or a fabricated number:
|
||
|
||
- `insufficient_data` → **NO READ** — *"not enough history to call this one"*
|
||
- `juiced_no_edge` → **NO READ** — *"the book has priced the vig past any edge on
|
||
this side"*
|
||
|
||
1,870 refused snapshots carry exactly these two reasons, and both now surface.
|
||
`projectionFor` reads the repaired full-window reference, so refusals are
|
||
computed on the repaired champion.
|
||
|
||
---
|
||
|
||
## PHASE 3 — hand-verified on real served props
|
||
|
||
| prop | p_win | OLD | NEW | separates | state |
|
||
|---|---|---|---|---|---|
|
||
| Freddie Freeman hits 0.5o | 0.95 | B | **B+** | true | graded |
|
||
| Christian Encarnación hits 0.5o | 0.95 | **C** | **B+** | true | graded |
|
||
| Ben Rice hits 0.5o | 0.95 | B | **B+** | true | graded |
|
||
| Christian Encarnación hits 0.5u | 0.05 | **C** | **F** | true | graded |
|
||
| Ben Rice hits 0.5u | 0.05 | C | **F** | true | graded |
|
||
| Eliezer Alfonso Jr doubles 0.5o | — | — | **NO READ** | — | refused |
|
||
| Eliezer Alfonso Jr doubles 0.5u | — | — | **NO READ** (vig) | — | refused |
|
||
| Paul Goldschmidt doubles 0.5o | — | — | **NO READ** | — | refused |
|
||
|
||
```
|
||
never-blank check: PASS — every prop renders a label and a meaning
|
||
no-manufactured-A check: PASS
|
||
```
|
||
|
||
The Encarnación rows are the clearest evidence: under the old letter his 0.95 and
|
||
his 0.05 were both `C`. Under the new one they are `B+` and `F`.
|
||
|
||
---
|
||
|
||
## Invariants
|
||
|
||
Grades ride on repaired-champion raw `p_win` plus factors where they fire. No
|
||
calibrated number leaks — the deployed set is empty and `calibrated: false` is
|
||
stated on every grade. `p_win` never mutated. Nine frozen model modules verified
|
||
unchanged, `engine1` included. No Bonferroni slot — no new factor.
|
||
|
||
**Still true and unchanged:** the forecast explains ~3% of outcome variance. This
|
||
order did not make the model better. It made the letter stop overstating it.
|