builtbykev
6c97f59546
PART A -- WNBA TRUTH CORRECTION (no behaviour change).
WNBA does not "abstain" and is not "anti-predictive". The -0.12 that
produced those words was NBA-template machinery run on WNBA data -- WNBA
has never had its own archetypes, variables, conditions or calibration,
which is precisely the "sport stubbed in on another sport's template"
CLAUDE.md forbids. That is an UNBUILT MODEL'S EXPECTED FAILURE, not a
verdict on the sport; reading it as a verdict would quietly retire a sport
we never actually attempted. Its own build is QUEUED, after MLB.
The guard CODE is unchanged -- FORECAST_RANKED_SPORTS = {'mlb'} and the
inheritance test are correct live safety either way. Only the meaning is
corrected, and generalised into the doctrine-as-a-gate: a sport ranks on
p_win ONLY once its OWN model is built and shown to predict (calibration
AND resolution on its own holdout). Others are held out as NOT-BUILT,
never as failed. Re-labelled across gradeRanking, snapshot route, tests,
MASTER-PLAN and the challenger report.
PART B -- THE FLIP, gated on a full-slate re-run.
The re-run found something better than a bigger sample. An induced
snapshot graded 7 props: gradeAndCacheSlate runs with DEFAULT_LIMIT = 25
and ~72% of those refuse for insufficient_data, while 546 props are
gradeable. So 8 props IS the board, structurally -- not a small sample of
it. Logged as its own finding; the cap is a separate order.
For a statistically meaningful delta I used 11 real historical boards
(n=328, board sizes 14-57): 79.9% of rows move, mean 5.16 places per
board, TOP READ CHANGES ON 9 OF 11 BOARDS. The re-ordering holds at real
board size. Query committed.
FLIPPED:
- rankGrades drops its edge key (safe for every sport: removes a
non-predictive tiebreak without putting p_win in front).
- selectTopGrades leads on forecast_rank, edge key removed.
- flattenToEdgeBoard sorts on forecastRank, not edge -- this board had
edge as its PRIMARY key, so the whole mobile board was ordered by a
quantity measured not to predict.
- forecast_rank threaded onto strip props.
Sports whose model is not built supply no forecast_rank, so their boards
fall through to the unchanged grade chain -- the fallback is the guard.
ROLLBACK ARMED: boards sort by forecast_rank WHEN PRESENT, so
FORECAST_RANK=0 reverts every surface on the next response -- no deploy,
no client release.
Edge is still computed, stored, carried and displayed as a labelled
diagnostic. Retired from ranking, not deleted.
Eight superseded tests updated to strictly stronger INVERSE properties --
they now fail if edge is ever re-introduced as a ranking key, which the
originals could not detect.
Gates: 4,045 tests / 323 suites green; next build exit 0.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QJs13VsyiSKYQP6rj3NNmc
161 lines
6.7 KiB
Markdown
161 lines
6.7 KiB
Markdown
# RANK ON p_win — CHALLENGER DELTA + EDGE RETIREMENT
|
||
|
||
**Date:** 2026-08-01 · **Live ordering byte-identical** · challenger measured on
|
||
live prod grades · edge retired from decisions and from display quality-signalling.
|
||
|
||
**Gates:** 4,039 tests / 323 suites green · `next build` exit 0 · delta recorded ·
|
||
live sorts untouched.
|
||
|
||
---
|
||
|
||
## WHY (the measurement that dictates this)
|
||
|
||
| instrument | corr with outcome, n=200 settled MLB |
|
||
|---|---:|
|
||
| **`p_win`** | **+0.26** |
|
||
| `p_win − fair_prob` (v1 single-book ruler) | −0.010 |
|
||
| `p_win − fair_prob` (v2 consensus ruler) | −0.022 |
|
||
|
||
**Subtracting the market destroys the signal, under both rulers.** A quantity
|
||
that does not predict must not rank, gate, or decide.
|
||
|
||
---
|
||
|
||
## THE CHALLENGER — `rankByForecast`
|
||
|
||
**Order:** takeable-gated `p_win` → grade tier → confidence → stable input order.
|
||
**No edge term anywhere** (a test flips edge from −99 to +99 and asserts the
|
||
order does not move).
|
||
|
||
**`p_win` leads and the letter follows — deliberately.** The grade letter measured
|
||
**r ≈ 0.005** against outcomes and is **inverted** (B 52.4% < C 56.9%), while
|
||
`p_win` measures **+0.26**. Leading with the letter would sort the board by the
|
||
weaker signal and use the stronger one only to break ties.
|
||
|
||
The takeable gate is unchanged and mandatory: raw `p_win` crowns −300 chalk.
|
||
|
||
### One thing worth stating precisely
|
||
|
||
**Isotonic calibration is a MONOTONE transform, so ranking on raw `p_win` and on
|
||
calibrated `p_win` produce the SAME ORDER.** Calibration matters when `p_win` is
|
||
*displayed* or *thresholded* — it cannot change a ranking. The order asked to
|
||
"rank on calibrated p_win"; for ranking specifically, that is a no-op relative to
|
||
raw. Recorded in the code so nobody re-derives it.
|
||
|
||
---
|
||
|
||
## THE DELTA (live prod grades, nothing flipped)
|
||
|
||
`GET /api/internal/ranking-delta` · `live_ordering_unchanged: true`
|
||
|
||
| | MLB | WNBA |
|
||
|---|---:|---:|
|
||
| graded props | 8 | 25 |
|
||
| `p_win` coverage | 100% | 100% |
|
||
| **props that move** | **7/8 (87.5%)** | **25/25 (100%)** |
|
||
| mean \|move\| | 2.5 places | 4.1 places |
|
||
| max move | 5 | 12 |
|
||
| top-10 overlap | 80% | 80% |
|
||
| **top read changes** | **YES** | **YES** |
|
||
|
||
MLB #1: `corey seager | hits 1.5 under` → `jake burger | hits 0.5 over`
|
||
(Seager falls 1 → 6).
|
||
|
||
**This is a large re-ordering, not a tweak.** Nearly every row moves and the
|
||
headline read changes.
|
||
|
||
**Caveat, stated because it matters:** MLB's slate had only **8 graded props** at
|
||
measurement time. The percentages are real but the sample is a single small
|
||
slate — re-run the endpoint on a full slate before the flip. It is one call.
|
||
|
||
---
|
||
|
||
## PER-SPORT DOCTRINE — ENFORCED IN CODE, NOT IN A COMMENT
|
||
|
||
**WNBA moves the most (100% of rows, mean 4.1 places) and must NOT adopt this.**
|
||
|
||
**CORRECTED 2026-08-01:** WNBA does **not** "abstain" and is **not**
|
||
"anti-predictive". The −0.12 that produced those words was **NBA-template
|
||
machinery run on WNBA data** — WNBA has never had its own archetypes, variables,
|
||
conditions or calibration. That is an **unbuilt model's expected failure, not a
|
||
verdict on the sport.** WNBA is **NOT BUILT YET**, held out until its own model
|
||
exists; its build is queued after MLB.
|
||
|
||
The live consequence is the same either way — an unbuilt sport must not rank on a
|
||
signal not shown to hold for it — which is why the guard code is unchanged.
|
||
|
||
A comment would not have stopped a future flip from applying this globally, so:
|
||
|
||
```js
|
||
const FORECAST_RANKED_SPORTS = Object.freeze(new Set(['mlb']));
|
||
```
|
||
|
||
`ranksOnForecast(sport)` gates the `forecast_rank` stamp, and tests assert **no
|
||
sport inherits MLB's result** — a sport joins only by passing its **own** holdout
|
||
(honest calibration AND surviving resolution).
|
||
|
||
**The WNBA number above is informational only.** It is in the report to show what
|
||
the guard is preventing.
|
||
|
||
---
|
||
|
||
## WHAT CHANGED, WHAT DID NOT
|
||
|
||
### Changed now (not rankings, so not gated on the flip)
|
||
|
||
- **`altLineScanner.compareToBookImplied`** no longer returns
|
||
`value_detected: edge > 0`. **Edge is still computed and returned** — losing the
|
||
record would be worse than mis-using it — but the verdict is `null` with
|
||
`value_basis: 'retired:edge_does_not_predict'`.
|
||
- **`scanAltLines`** no longer filters to `edge > 0` nor calls the survivor
|
||
`optimal_line`. The whole ladder returns, ranked, labelled
|
||
`price_gap_diagnostic_unvalidated`. **The module has zero callers** (verified) —
|
||
unwired like `mlbGrader.js`; left in place and made honest rather than deleted.
|
||
- **`MobileEdgeBoard.EdgeCell`** no longer renders green for positive edge and red
|
||
for negative. Two things were wrong with that: green/red **is** a quality claim
|
||
on a quantity that does not predict, and **ROW-GRAMMAR reserves red for
|
||
settled-negative only** (miss/dead/stale/faded) — a negative diagnostic is not a
|
||
settled loss. Now neutral mono with a diagnostic tooltip; the column header
|
||
reads **`MKT GAP · DIAGNOSTIC`**. **The number is still shown** — no display went
|
||
blank.
|
||
- **`DeskShowcase`** edge colour neutralised for the same reason.
|
||
|
||
### An honest asymmetry I did not paper over
|
||
|
||
**Ranking props against each other must not use edge. Choosing between RUNGS of
|
||
the same prop is inherently price-relative** — ranking rungs by model probability
|
||
alone would always pick the lowest line, because P(over 0.5) > P(over 2.5) by
|
||
construction. So the gap stays the rung key in `scanAltLines`, **explicitly
|
||
labelled unvalidated**, rather than being replaced by something that would look
|
||
principled and be degenerate.
|
||
|
||
### NOT changed (challenger-first)
|
||
|
||
- **`rankGrades`** — the incumbent (grade-first, edge as 4th key) is untouched,
|
||
and tested as untouched.
|
||
- **Every live sort** still calls the incumbent. `selectTopGrades`,
|
||
`flattenToEdgeBoard`, `topGradedService` — all byte-identical.
|
||
- **`forecast_rank`** is stamped additively on MLB snapshot grades, **before**
|
||
`stripModelPrice`, so every tier would receive the correct order without the
|
||
paid values (the `topGradedService` precedent — an ordinal travels where the
|
||
magnitude cannot). **Nothing sorts by it yet.**
|
||
- **Edge stays stored** in the ledger, as required.
|
||
|
||
---
|
||
|
||
## THE FLIP, WHEN YOU WANT IT
|
||
|
||
One decision, three edits: `selectTopGrades` and `flattenToEdgeBoard` sort by
|
||
`forecast_rank` when present; `rankGrades` drops its edge key. **Re-run the delta
|
||
on a full MLB slate first** — 8 props is not a slate.
|
||
|
||
## PINNACLE — LOGGED, NOT ENSHRINED
|
||
|
||
Per the order: **"market-not-sharp" is PENDING-RECOVERY, not a confirmed permanent
|
||
limitation.** The question for PropLine is logged in `BLOCKERS.md`:
|
||
|
||
> *Why did Pinnacle MLB player-prop coverage stop on 2026-07-31?* Captures ran
|
||
> 103,940 over the prior 10 days, then 4,022 → 0 while every other book continued.
|
||
|
||
Until answered, the plan must not record "no sharp anchor exists" as permanent.
|