Per-sport rank guard + edge diagnostic-only display + delta report
DELTA MEASURED on live prod grades (live ordering unchanged): MLB 7/8 props move (87.5%), mean 2.5 places, TOP READ CHANGES (corey seager hits 1.5 under -> jake burger hits 0.5 over). WNBA 25/25 move, mean 4.1, max 12. This is a large re-ordering, not a tweak. Caveat recorded rather than buried: MLB had only 8 graded props at measurement time. The percentages are real; the sample is one small slate. Re-run before the flip -- it is one call. PER-SPORT DOCTRINE ENFORCED IN CODE. WNBA moves the most and must NOT adopt this: its p_win is anti-predictive, so ranking that board by p_win would sort it by a signal measured to point the WRONG WAY -- worse than the incumbent, not better. A comment would not have stopped a future flip from going global, so FORECAST_RANKED_SPORTS = Set(['mlb']) gates the forecast_rank stamp, with tests asserting no sport inherits MLB's result. A sport joins only by passing its own holdout. EDGE IS NOW DIAGNOSTIC-ONLY IN DISPLAY. MobileEdgeBoard.EdgeCell rendered green (--g-a) for positive edge and red (--miss) for negative. Two things were wrong: green/red IS a quality claim on a quantity that does not predict, and ROW-GRAMMAR reserves red for settled-negative ONLY -- a negative diagnostic is not a settled loss. Now neutral mono with a diagnostic tooltip; header reads "MKT GAP · DIAGNOSTIC". The number is still shown -- no display went blank. DeskShowcase neutralised likewise. PINNACLE LOGGED, NOT ENSHRINED. Per the order, "market-not-sharp" is PENDING-RECOVERY rather than a confirmed permanent limitation. The single question for PropLine is in BLOCKERS.md with its evidence, and MASTER-PLAN now carries the pending status instead of the permanent claim. Live sorts remain byte-identical: selectTopGrades, flattenToEdgeBoard and topGradedService all still call the incumbent. Gates: 4,041 tests / 323 suites green; next build exit 0. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QJs13VsyiSKYQP6rj3NNmc
This commit is contained in:
+13
-3
@@ -46,8 +46,18 @@ Two things gate on accrual instead of on code, and cannot be rushed:
|
||||
- The **edge verdict** may change under that ruler — or may not; today it is
|
||||
unproven either way, and 'unproven' is the honest label.
|
||||
|
||||
**Open, cheap, and unrelated to the model:** the 🔴 pinnacle feed regression, and
|
||||
the display layer still shows nothing of the widened multi-book data.
|
||||
**DONE 2026-08-01** — the challenger is built and measured
|
||||
(`specs/rank-on-pwin-challenger.md`): `rankByForecast` (p_win-first, **no edge
|
||||
term**), delta recorded (MLB **87.5% of props move, top read changes**), edge
|
||||
retired from decisions and from display quality-signalling, `forecast_rank`
|
||||
stamped additively, **MLB-only by an enforced `FORECAST_RANKED_SPORTS` guard**.
|
||||
|
||||
**THE FLIP IS THE NEXT ORDER** — one decision, three edits. **Re-run the delta on
|
||||
a full MLB slate first**; it measured on 8 graded props.
|
||||
|
||||
**Open, cheap, unrelated to the model:** the 🔴 pinnacle question (logged in
|
||||
`BLOCKERS.md`), and the display layer still shows nothing of the widened
|
||||
multi-book data.
|
||||
|
||||
---
|
||||
|
||||
@@ -58,7 +68,7 @@ the display layer still shows nothing of the widened multi-book data.
|
||||
| **MLB slate invisible to us** | **64.8%** | our own allow-list, not the feed — now widened for DISPLAY |
|
||||
| **books/prop, MLB** | 3.61 feed → 0.57 after filter | the filter cost, quantified |
|
||||
| **books/prop, WNBA** | **4.21** feed → 1.20 | **WNBA is BETTER covered than MLB** |
|
||||
| **consensus ruler** | **MARKET, not SHARP** | `pinnacle`/`matchbook`/`polymarket` = **0%** on both sports. No sharp anchor exists in our feed. Permanent limitation, not a milestone |
|
||||
| **consensus ruler** | **MARKET, not SHARP — ⏳ PENDING-RECOVERY, not permanent** | `matchbook`/`polymarket` = 0%, but **`pinnacle` ran until 07-30** (103,940 captures) and stopped. **Do not enshrine as permanent** until PropLine answers — see `BLOCKERS.md` |
|
||||
| **ruler delta** (consensus − incumbent) | MLB mean +1.50 pts, median 0, **17% of props move ≥5 pts** | rulers genuinely differ; "better" is unproven |
|
||||
| **MLB isotonic `p_win`** | **DECIDED** — reliability **0.0846**, resolution **0.190**, holdout **n=125** | **PROVISIONAL label RETRACTED 2026-08-01.** Calibration is **ruler-independent** (`estimateProbability` never sees a price; the fit is p_win-vs-outcome). Replicated on a fresh later window, both metrics improved |
|
||||
| **edge vs the ruler** | corr(edge, outcome) **−0.010** (v1) → **−0.022** (v2), n=200 · corr(**p_win**, outcome) **+0.26** | **Subtracting the market DESTROYS the signal.** The consensus ruler does not rescue edge: *differs ≠ better* |
|
||||
|
||||
@@ -0,0 +1,153 @@
|
||||
# RANK ON p_win — CHALLENGER DELTA + EDGE RETIREMENT
|
||||
|
||||
**Date:** 2026-08-01 · **Live ordering byte-identical** · challenger measured on
|
||||
live prod grades · edge retired from decisions and from display quality-signalling.
|
||||
|
||||
**Gates:** 4,039 tests / 323 suites green · `next build` exit 0 · delta recorded ·
|
||||
live sorts untouched.
|
||||
|
||||
---
|
||||
|
||||
## WHY (the measurement that dictates this)
|
||||
|
||||
| instrument | corr with outcome, n=200 settled MLB |
|
||||
|---|---:|
|
||||
| **`p_win`** | **+0.26** |
|
||||
| `p_win − fair_prob` (v1 single-book ruler) | −0.010 |
|
||||
| `p_win − fair_prob` (v2 consensus ruler) | −0.022 |
|
||||
|
||||
**Subtracting the market destroys the signal, under both rulers.** A quantity
|
||||
that does not predict must not rank, gate, or decide.
|
||||
|
||||
---
|
||||
|
||||
## THE CHALLENGER — `rankByForecast`
|
||||
|
||||
**Order:** takeable-gated `p_win` → grade tier → confidence → stable input order.
|
||||
**No edge term anywhere** (a test flips edge from −99 to +99 and asserts the
|
||||
order does not move).
|
||||
|
||||
**`p_win` leads and the letter follows — deliberately.** The grade letter measured
|
||||
**r ≈ 0.005** against outcomes and is **inverted** (B 52.4% < C 56.9%), while
|
||||
`p_win` measures **+0.26**. Leading with the letter would sort the board by the
|
||||
weaker signal and use the stronger one only to break ties.
|
||||
|
||||
The takeable gate is unchanged and mandatory: raw `p_win` crowns −300 chalk.
|
||||
|
||||
### One thing worth stating precisely
|
||||
|
||||
**Isotonic calibration is a MONOTONE transform, so ranking on raw `p_win` and on
|
||||
calibrated `p_win` produce the SAME ORDER.** Calibration matters when `p_win` is
|
||||
*displayed* or *thresholded* — it cannot change a ranking. The order asked to
|
||||
"rank on calibrated p_win"; for ranking specifically, that is a no-op relative to
|
||||
raw. Recorded in the code so nobody re-derives it.
|
||||
|
||||
---
|
||||
|
||||
## THE DELTA (live prod grades, nothing flipped)
|
||||
|
||||
`GET /api/internal/ranking-delta` · `live_ordering_unchanged: true`
|
||||
|
||||
| | MLB | WNBA |
|
||||
|---|---:|---:|
|
||||
| graded props | 8 | 25 |
|
||||
| `p_win` coverage | 100% | 100% |
|
||||
| **props that move** | **7/8 (87.5%)** | **25/25 (100%)** |
|
||||
| mean \|move\| | 2.5 places | 4.1 places |
|
||||
| max move | 5 | 12 |
|
||||
| top-10 overlap | 80% | 80% |
|
||||
| **top read changes** | **YES** | **YES** |
|
||||
|
||||
MLB #1: `corey seager | hits 1.5 under` → `jake burger | hits 0.5 over`
|
||||
(Seager falls 1 → 6).
|
||||
|
||||
**This is a large re-ordering, not a tweak.** Nearly every row moves and the
|
||||
headline read changes.
|
||||
|
||||
**Caveat, stated because it matters:** MLB's slate had only **8 graded props** at
|
||||
measurement time. The percentages are real but the sample is a single small
|
||||
slate — re-run the endpoint on a full slate before the flip. It is one call.
|
||||
|
||||
---
|
||||
|
||||
## PER-SPORT DOCTRINE — ENFORCED IN CODE, NOT IN A COMMENT
|
||||
|
||||
**WNBA moves the most (100% of rows, mean 4.1 places) and must NOT adopt this.**
|
||||
WNBA's `p_win` is **anti-predictive** on its own data — it abstains. Ranking that
|
||||
board by `p_win` would sort it by a signal measured to point the *wrong way*:
|
||||
worse than the incumbent, not better.
|
||||
|
||||
A comment would not have stopped a future flip from applying this globally, so:
|
||||
|
||||
```js
|
||||
const FORECAST_RANKED_SPORTS = Object.freeze(new Set(['mlb']));
|
||||
```
|
||||
|
||||
`ranksOnForecast(sport)` gates the `forecast_rank` stamp, and tests assert **no
|
||||
sport inherits MLB's result** — a sport joins only by passing its **own** holdout
|
||||
(honest calibration AND surviving resolution).
|
||||
|
||||
**The WNBA number above is informational only.** It is in the report to show what
|
||||
the guard is preventing.
|
||||
|
||||
---
|
||||
|
||||
## WHAT CHANGED, WHAT DID NOT
|
||||
|
||||
### Changed now (not rankings, so not gated on the flip)
|
||||
|
||||
- **`altLineScanner.compareToBookImplied`** no longer returns
|
||||
`value_detected: edge > 0`. **Edge is still computed and returned** — losing the
|
||||
record would be worse than mis-using it — but the verdict is `null` with
|
||||
`value_basis: 'retired:edge_does_not_predict'`.
|
||||
- **`scanAltLines`** no longer filters to `edge > 0` nor calls the survivor
|
||||
`optimal_line`. The whole ladder returns, ranked, labelled
|
||||
`price_gap_diagnostic_unvalidated`. **The module has zero callers** (verified) —
|
||||
unwired like `mlbGrader.js`; left in place and made honest rather than deleted.
|
||||
- **`MobileEdgeBoard.EdgeCell`** no longer renders green for positive edge and red
|
||||
for negative. Two things were wrong with that: green/red **is** a quality claim
|
||||
on a quantity that does not predict, and **ROW-GRAMMAR reserves red for
|
||||
settled-negative only** (miss/dead/stale/faded) — a negative diagnostic is not a
|
||||
settled loss. Now neutral mono with a diagnostic tooltip; the column header
|
||||
reads **`MKT GAP · DIAGNOSTIC`**. **The number is still shown** — no display went
|
||||
blank.
|
||||
- **`DeskShowcase`** edge colour neutralised for the same reason.
|
||||
|
||||
### An honest asymmetry I did not paper over
|
||||
|
||||
**Ranking props against each other must not use edge. Choosing between RUNGS of
|
||||
the same prop is inherently price-relative** — ranking rungs by model probability
|
||||
alone would always pick the lowest line, because P(over 0.5) > P(over 2.5) by
|
||||
construction. So the gap stays the rung key in `scanAltLines`, **explicitly
|
||||
labelled unvalidated**, rather than being replaced by something that would look
|
||||
principled and be degenerate.
|
||||
|
||||
### NOT changed (challenger-first)
|
||||
|
||||
- **`rankGrades`** — the incumbent (grade-first, edge as 4th key) is untouched,
|
||||
and tested as untouched.
|
||||
- **Every live sort** still calls the incumbent. `selectTopGrades`,
|
||||
`flattenToEdgeBoard`, `topGradedService` — all byte-identical.
|
||||
- **`forecast_rank`** is stamped additively on MLB snapshot grades, **before**
|
||||
`stripModelPrice`, so every tier would receive the correct order without the
|
||||
paid values (the `topGradedService` precedent — an ordinal travels where the
|
||||
magnitude cannot). **Nothing sorts by it yet.**
|
||||
- **Edge stays stored** in the ledger, as required.
|
||||
|
||||
---
|
||||
|
||||
## THE FLIP, WHEN YOU WANT IT
|
||||
|
||||
One decision, three edits: `selectTopGrades` and `flattenToEdgeBoard` sort by
|
||||
`forecast_rank` when present; `rankGrades` drops its edge key. **Re-run the delta
|
||||
on a full MLB slate first** — 8 props is not a slate.
|
||||
|
||||
## PINNACLE — LOGGED, NOT ENSHRINED
|
||||
|
||||
Per the order: **"market-not-sharp" is PENDING-RECOVERY, not a confirmed permanent
|
||||
limitation.** The question for PropLine is logged in `BLOCKERS.md`:
|
||||
|
||||
> *Why did Pinnacle MLB player-prop coverage stop on 2026-07-31?* Captures ran
|
||||
> 103,940 over the prior 10 days, then 4,022 → 0 while every other book continued.
|
||||
|
||||
Until answered, the plan must not record "no sharp anchor exists" as permanent.
|
||||
Reference in New Issue
Block a user