Order Zero Phases 1-3: keyed verification, ruler_version boundary, report

PHASE 1 (measured on the live prod feed with the real key):

- WNBA is NOT thin at the feed -- 4.21 books/prop vs MLB's 3.61. It was
  allow-list-starved exactly as MLB was. This removes one candidate
  explanation for its anti-predictive result; it does not explain it, and
  WNBA stays abstaining.
- We cannot see 64.8% of the MLB slate at all (zero admitted books).
- Exchanges are real (smarkets 27%, novig 22%, kalshi 15% on MLB) but
  pinnacle, matchbook and polymarket measured 0% on BOTH sports. There is
  no sharp anchor for player props. The consensus is a MARKET consensus,
  not a SHARP one -- recorded as a permanent limitation, not a milestone.
- DFS is the trap, quantified: prizepicks covers 82% of MLB props, the
  highest in the feed. Admitting it "for breadth" would have looked like
  the biggest available win. Permanently excluded.
- Endpoints: /context WORKS and is FREE (umpire, roof, pitcher handedness,
  lineup confirmation -- richer than what we hand-built). /odds/closing and
  /movement are REDACTED (full structure, zero prices). /results and
  /exports/resolved-props are 403.
- The $19/mo question is answered: soccer IS graded, ~15 competitions in 30
  days (MLS 41k, Liga MX 15k, Brasileirao 12k, UCL/Europa/Conference). Our
  "soccer grades into a void" is a Pro-tier problem, not a data problem.
  NBA is absent because it is July -- seasonal, not inferable either way.

PHASE 2 delta, corrected: MLB mean +1.50 pts, median 0, p90 +10.0, 17.0%
of comparable props move >=5 pts, one-directional (the incumbent prices
the over below the exchange-inclusive consensus). WNBA symmetric and
tight. The median prop does not move -- the change is a right-skewed
minority. That the rulers DIFFER is established; that the new one is
BETTER is not, and that is the re-run.

PHASE 2 item 6: ledger_entries.ruler_version applied to prod, 1,384
existing rows backfilled to v1_first_book (a statement of fact -- every
row to date was produced by the first-book rule). ledgerService stamps
CURRENT_RULER_VERSION on new rows. Never pool edge or CLV across it.

Repo migration numbering lags prod; 025_ledger_ruler_version.sql records
the DDL for review.

PHASE 3: MLB isotonic p_win remains PROVISIONAL -- calibrated against
v1_first_book, does not promote until re-run on the consensus ruler.

NOT LIVE, deliberately: ALLOWED_BOOKS unchanged, served slate
byte-identical, CURRENT_RULER_VERSION still v1_first_book, no live path
calls consensusRuler.

Gates: 4,022 tests passed / 322 suites; next build exit 0.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QJs13VsyiSKYQP6rj3NNmc
This commit is contained in:
Kev
2026-07-31 23:52:28 -04:00
parent c38db1ad65
commit 1372e6bcf7
4 changed files with 332 additions and 2 deletions
+37
View File
@@ -0,0 +1,37 @@
-- 025_ledger_ruler_version.sql — Order Zero Phase 2 item 6 (APPLIED 2026-08-01)
--
-- Stamp the RULER that produced fair_prob_lock so pre- and post-change rows are
-- never pooled. fair_prob_lock is the denominator of every edge and CLV number;
-- changing how it is derived means a value computed before the change is NOT the
-- same measurement as one computed after it. Arithmetically forced, not policy —
-- the same reasoning as model_version.
--
-- v1_first_book — the incumbent: normalizeProps applies ALLOWED_BOOKS, then
-- gradeSlateService.dedupeProps keeps the FIRST surviving row
-- per player+stat+line. Whichever admitted book PropLine
-- happened to list first became the entire "market".
-- v2_consensus — median de-vigged fair_prob across >=2 REFERENCE books
-- posting BOTH sides at the SAME line. NOT LIVE.
--
-- Backfill to v1_first_book is a statement of fact: every row written to date
-- was produced by the first-book rule.
--
-- NOTE: repo migration numbering lags prod (the repo stops at 024 while prod
-- carries later ones applied via the Supabase MCP). This file records the DDL
-- for review; prod already has it.
ALTER TABLE public.ledger_entries
ADD COLUMN IF NOT EXISTS ruler_version text;
UPDATE public.ledger_entries
SET ruler_version = 'v1_first_book'
WHERE ruler_version IS NULL;
ALTER TABLE public.ledger_entries
ALTER COLUMN ruler_version SET DEFAULT 'v1_first_book';
COMMENT ON COLUMN public.ledger_entries.ruler_version IS
'Which fair-probability ruler produced fair_prob_lock. NEVER pool edge or CLV across differing values — the denominator changed. v1_first_book = first admitted book (incumbent); v2_consensus = median across >=2 reference books at the same line.';
CREATE INDEX IF NOT EXISTS idx_ledger_entries_ruler_version
ON public.ledger_entries (ruler_version);
+21 -2
View File
@@ -322,8 +322,27 @@ Two overturn standing beliefs: **"NBA/WNBA/soccer have no free settled feed"** m
be a **$19/mo** problem rather than a data problem; and we hand-built probable be a **$19/mo** problem rather than a data problem; and we hand-built probable
pitchers / depth charts / lineup confirmation that `/context` serves free. pitchers / depth charts / lineup confirmation that `/context` serves free.
**Caveat:** these are *documented*, not verified for our tier — that needs the > **VERIFIED 2026-08-01 with the real key** — `specs/order-zero-consensus-ruler.md`.
PropLine key (prod-only, absent locally). Do not plan on them as facts yet. > `/context` **WORKS, FREE** (umpire, roof, pitcher handedness, lineup
> confirmation — richer than what we hand-built). `/odds/closing` and
> `/movement` are **REDACTED** on our tier (full structure, **zero prices**) —
> my first pass wrongly called them "works" on a non-empty body. `/results` and
> `/exports/resolved-props` are **403**.
>
> **But the settlements exist to be bought:** `/markets/resolution-summary`
> shows **soccer graded across ~15 competitions in 30 days** (MLS 41k, Liga MX
> 15k, Brasileirão 12k, UCL/Europa/Conference…). **"Soccer grades into a void"
> is a $19/mo Pro-tier problem, not a data problem.** NBA is absent because it
> is July — seasonal, not a coverage gap, and not inferable either way.
>
> **WNBA is NOT thin at the feed** — 4.21 books/prop vs MLB's 3.61. It was
> allow-list-starved exactly as MLB was. This removes one candidate explanation
> for its anti-predictive result; it does not explain it, and WNBA stays
> abstaining.
>
> **No sharp anchor exists for props:** `pinnacle`, `matchbook` and `polymarket`
> all measured **0%** on both sports. The consensus ruler is therefore a MARKET
> consensus, not a SHARP one — stated as a permanent limitation, not a milestone.
## 10.2 What a real DATA AGGREGATOR has that we don't ## 10.2 What a real DATA AGGREGATOR has that we don't
+268
View File
@@ -0,0 +1,268 @@
# Order Zero — CONSENSUS RULER + ENDPOINT VERIFICATION
**Date:** 2026-08-01 · Phase 1 measured against the **live prod feed with the
real key**; Phase 2 built **challenger-first — the live ruler is byte-identical.**
**Gates:** tests 4,022 passed / 322 suites · `next build` **exit 0** · keyed
verification run on prod · challenger delta recorded.
---
## PHASE 1 — KEYED VERIFICATION
Run via `GET /api/internal/propline-verify` (internal-key gated, read-only, no
cache/ledger/grade writes). Breadth reuses `proplineAdapter.fetchRaw` — the
exact live request — so this measures what the pipeline actually receives.
### 1.1 WNBA is NOT thin at the feed. It was starved by our own allow-list.
MLB 3,138 props / 20 events · WNBA 160 props / 4 events. **Never pooled.**
| | MLB | WNBA |
|---|---:|---:|
| books/prop **in the feed** | **3.61** | **4.21** |
| books/prop **after `ALLOWED_BOOKS`** | **0.57** | **1.20** |
| props **INVISIBLE** (zero admitted books) | **2,033 (64.8%)** | 46 (28.8%) |
**WNBA has BETTER feed breadth than MLB (4.21 vs 3.61).** The answer to the
order's question is unambiguous: **WNBA was allow-list-starved, exactly as MLB
was — not thin at source.**
That does **not** by itself explain WNBA's anti-predictive result, and I am not
going to claim it does. It removes one candidate explanation ("no market to
measure against") and leaves the ruler bend as a live hypothesis to re-test
after the fix. **Nothing here promotes WNBA out of abstention.**
The headline number for MLB is not the mean — it is that **we cannot see 64.8%
of the MLB slate at all.**
### 1.2 Exchange reality — PARTIALLY confirmed, and two candidates are dead
Reference-book presence, as a share of props:
| book | MLB | WNBA | |
|---|---:|---:|---|
| `smarkets` | 27.1% | **0%** | exchange |
| `novig` | 22.2% | **46.3%** | exchange |
| `kalshi` | 14.9% | 8.8% | prediction market |
| `bovada` | 9.2% | 16.3% | offshore, PropLine's own no-vig fallback |
| **`pinnacle`** | **0%** | **0%** | the classic sharp anchor |
| **`matchbook`** | **0%** | **0%** | exchange |
| **`polymarket`** | **0%** | **0%** | prediction market |
**Confirmed:** exchanges do carry real props — `smarkets` 873 quotes, `novig`
730, `kalshi` 1,369 on MLB. The hypothesis was not empty.
**Three candidates are dead for player props:** `pinnacle`, `matchbook`,
`polymarket` returned **zero** on both sports. Pinnacle carries 144k markets
globally — just not these. **We have no sharp anchor for props.** Twice
measured, both sports.
### 1.3 The coverage curve — the number that decides the ruler
Strict eligibility: **≥2 reference books posting BOTH sides at the SAME line.**
One-sided quotes cannot be de-vigged; two books at different lines are not the
same market.
| reference policy | MLB n≥2 | WNBA n≥2 |
|---|---:|---:|
| `exchange_only` | 14.3% | 5.6% |
| `exchange_plus_sharp` (+pinnacle, bovada) | 14.3% | 5.6% |
| **`exchange_plus_us`** (+DK/FD/MGM/BR) | **28.1%** | **46.3%** |
| `takeable_only` | 12.6% | 30.6% |
Adding pinnacle and matchbook/polymarket changes **nothing** — they are 0%.
**Of props we can actually GRADE** (those with an admitted book), consensus is
available on **80.5% of MLB** (923/1,146) and **54.9% of WNBA** (79/144). That
is the operative number: the 15% figure is diluted by props we cannot see at all.
### 1.4 DFS is the trap, quantified
`prizepicks` covers **82.3%** of MLB props and `sleeper` **68.1%** of WNBA — the
highest coverage in the entire feed. Admitting DFS "for breadth" would have
looked like the single biggest improvement available and would have bent the
ruler in a new direction. Fixed-payout, deliberately shaded, flat juice.
**Permanently excluded from all pricing.**
### 1.5 Endpoint verification — works / partial / no, FOR OUR TIER
| endpoint | verdict | evidence |
|---|---|---|
| `/v1/sports` | ✅ **WORKS** | 54 sports, all active |
| `/v1/markets/resolution-summary` | ✅ **WORKS** | 2,085,840 props graded in 30d |
| `/events/{id}/context` (MLB) | ✅ **WORKS, FREE** | venue, `roof_type`, `is_indoor`, both probable pitchers **+ handedness**, `lineup_confirmed`, **home-plate umpire**, weather |
| `/events/{id}/context` (WNBA) | ⚠️ **PARTIAL** | 404 — MLB-only today, as documented |
| `/events/{id}/odds/closing` | ⚠️ **PARTIAL — REDACTED** | 200 with **2,102 outcomes and 0 prices** + `upgrade_url` |
| `/events/{id}/movement` | ⚠️ **PARTIAL — REDACTED** | 2,102 outcomes, **0 prices**, `steam: []` |
| `/events/{id}/results` | ❌ **NO** | 403 |
| `/exports/resolved-props` | ❌ **NO** | 403 on **all four** sport keys |
**A correction to my own first pass.** I initially classified `/odds/closing`
and `/movement` as **WORKS** because the bodies were non-empty. They are
**structure-only redactions** — full shape, zero prices, plus an `upgrade_url`.
`detectRedaction` now counts actual prices and downgrades works→partial, and a
test locks it. **A non-empty body is not proof of access.**
### 1.6 The $19/mo question — answered, with one honest gap
`/exports/resolved-props` is 403 for us, so we cannot **pull** settlements. But
`/markets/resolution-summary` proves they **exist to be bought** (30 days):
| sport | props graded | events |
|---|---:|---:|
| `baseball_mlb` | 1,798,850 | 363 |
| `basketball_wnba` | 152,960 | 72 |
| **soccer_mls** | **41,444** | 35 |
| **soccer_liga_mx** | 15,037 | 21 |
| soccer_brasileirao / world_cup / UCL / Europa / Conference / … | ~60,000 combined | ~340 |
| `mma_ufc` | 2,738 | 39 |
| tennis, golf, rugby | ~19,000 | ~1,150 |
**Soccer settles.** ~15 competitions graded in the last 30 days. Our standing
"soccer grades into a void" is a **Pro-tier ($19/mo) problem, not a data
availability problem.**
**NBA is absent — and that is seasonal, not a coverage gap.** It is July;
`/sports` lists `basketball_nba` as active. **I cannot confirm NBA settlement
until the season starts, and I am not going to infer it.**
---
## PHASE 2 — THE CONSENSUS RULER (challenger-first, NOT live)
### 2.1 The three-way split — `src/config/bookRoles.js`
One allow-list was answering two different questions: *"can we show this?"* and
*"can we price against this?"* Conflating them is what bent the ruler.
- **TAKEABLE** — a user can actually bet here. Best price, line shopping.
- **REFERENCE** — may price the ruler; never surfaced as a place to bet.
Selected policy: `exchange_plus_us`.
- **EXCLUDED** — DFS pick'em + offshore/intl. **Permanently** barred from
pricing.
**Deliberate deviation, stated plainly.** The order said *remove the 6 phantom
entries.* **I kept them.** They returned zero PropLine quotes, but PropLine is
not our only provider — the odds-api backup path may carry Caesars/bet365/etc.
A book that never appears is never matched, so keeping it costs nothing;
deleting it risks silently dropping real books on the backup with no upside.
Recorded in `PHANTOM_ON_PROPLINE` rather than enacted as a deletion. **Reverse
this if you'd rather.**
**The honest limitation, recorded in the config itself:** including soft US
books makes this a **MARKET** consensus, not a **SHARP** one. It is a large
improvement on one arbitrary book. It is **not** the sharp closing line the
category's leaders are measured against — and §1.2 says we cannot build that
one, because no sharp book prices these props.
### 2.2 The rule — `src/services/consensusRuler.js`
1. Group by the **SAME LINE**. Different lines are never averaged.
2. **TWO-SIDED REFERENCE quotes only.** A one-sided price cannot be de-vigged.
3. **MEDIAN** of per-book de-vigged `fair_prob` — median, so one stale exchange
cannot drag the ruler.
4. **n≥2**, or fall back to single-book **and label it** (`source:
'single_book'`, stamped v1). **Never silently mixed** — a column holding both
is two rulers wearing one name.
5. No reference quote at all → `null`, `source: 'none'`. Never fabricated.
### 2.3 The challenger delta — and a correction I caught mid-run
**My first delta run was wrong and I stopped it.** I modelled the incumbent as
first-row-wins over the **raw** feed and it reported that an EXCLUDED book was
"the market" on **69% of MLB prop-lines, prizepicks alone at 47%.**
**That is false.** `normalizeProps` applies `ALLOWED_BOOKS` **before**
`dedupeProps` runs, so DFS books never reach the incumbent. The allow-list, for
all the coverage it costs, **does** keep DFS out of the ruler. Overstating the
incumbent's badness would have been as dishonest as understating it — and more
persuasive. Corrected, tested, and re-run.
**Corrected results** (`allowedBooks` respected — incumbent is 100% admitted
books, no DFS):
| | MLB | WNBA |
|---|---:|---:|
| prop-line groups | 6,422 | 853 |
| gradeable (has an admitted book) | 1,146 (17.8%) | 144 (16.9%) |
| consensus available | 995 | 87 |
| comparable (both rulers) | 923 | 79 |
| incumbent books | betmgm 586 · dk 538 · fd 16 · br 6 | dk 90 · fd 54 |
**Delta (consensus incumbent, probability points, over side):**
| | MLB | WNBA |
|---|---:|---:|
| mean | **+1.50** | 0.29 |
| median | 0 | 0 |
| p10 / p90 | 1.6 / **+10.0** | 2.0 / +1.6 |
| median abs / p90 abs | 0.5 / **11.95** | 0.55 / 2.5 |
| disagree ≥2 pts | **24.8%** | 20.3% |
| disagree ≥5 pts | **17.0%** | 2.5% |
**Read it honestly: the median prop does not move at all.** The change is
concentrated in a **right-skewed minority** — on MLB, **17% of comparable props
move by 5+ probability points**, and the skew is one-directional (p90 +10.0 vs
p10 1.6): the incumbent systematically prices the OVER *below* the
exchange-inclusive consensus. WNBA is nearly symmetric and much tighter.
**What this does and does not establish.** It establishes the two rulers
genuinely differ, on a minority of props, in a consistent direction. It does
**not** establish that the new ruler is *better* — that requires re-running the
edge and CLV measurements against it, which is the next order.
### 2.4 `ruler_version` stamped at the boundary — **APPLIED**
`ledger_entries.ruler_version` (migration recorded as
`migrations/025_ledger_ruler_version.sql`; applied to prod). **1,384 existing
rows backfilled to `v1_first_book`** — a statement of fact, not an assumption:
every row to date was produced by the first-book rule. 629 of them carry a
`fair_prob_lock`. New rows stamp `CURRENT_RULER_VERSION` via `ledgerService`.
**NEVER pool edge or CLV across differing `ruler_version` values.** The
denominator changed; the numbers are not the same measurement. This is the same
no-pooling boundary as `model_version`, and it is **arithmetically forced**.
---
## PHASE 3 — CONSEQUENCE
**MLB isotonic `p_win` remains PROVISIONAL.** It was calibrated against the
`v1_first_book` ruler. **It does not promote until re-run against the consensus
ruler**, after this delta is reviewed. Recorded in MASTER-PLAN.
---
## WHAT IS AND IS NOT LIVE
**LIVE:** the internal verification endpoint; `bookRoles.js` and
`consensusRuler.js` as libraries; the `ruler_version` column and stamp.
**NOT LIVE — deliberately:** `ALLOWED_BOOKS` is **unchanged**, so the served
slate is byte-identical. `CURRENT_RULER_VERSION` is still `v1_first_book`. No
live path calls `consensusRuler`. No grade, cache or slate changed.
---
## THE DECISION THIS TEES UP
Widening `ALLOWED_BOOKS` to the reference set would make **64.8% of the MLB
slate visible** for the first time — that is the single largest product change
available, and it is a config edit.
But it is **not** free: those props would grade against a market we have never
measured our model on, and every edge number reset to `v2`. **That is the next
order's decision, not this one's** — and it should be taken after the ruler
re-run, not before it.
## OPEN
1. **Ruler policy sign-off** — `exchange_plus_us` selected off the curve. Confirm.
2. **Pro tier ($19/mo)** — buys settlement for **soccer** (proven), CLV via
`/odds/closing`, steam via `/movement`, and `/results`. Genuinely cheap
relative to what it unblocks.
3. **The 6 phantom books** — kept, against the order's wording, with reasoning
above. Reverse if you disagree.
4. **`/context` is free and richer than what we hand-built** — umpire, roof,
pitcher handedness, lineup confirmation. Wiring it is a later order.
+6
View File
@@ -45,6 +45,11 @@ const SETTLE_ATTEMPT_CAP = Number(process.env.SETTLE_ATTEMPT_CAP || 4);
const SETTLEMENT_VERSION = Number(process.env.SETTLEMENT_VERSION || 2); const SETTLEMENT_VERSION = Number(process.env.SETTLEMENT_VERSION || 2);
const settleSource = require('./settleSource'); const settleSource = require('./settleSource');
const MODEL_ERA_VERSION = process.env.MODEL_VERSION || 'engine1@2026-07-20'; const MODEL_ERA_VERSION = process.env.MODEL_VERSION || 'engine1@2026-07-20';
// Order Zero — which fair-probability RULER produced fair_prob_lock. It is the
// denominator of every edge and CLV number, so a value computed under one ruler
// is not the same measurement as one computed under another. Never pool across
// it. Stays v1_first_book until the consensus ruler is actually promoted.
const { CURRENT_RULER_VERSION } = require('../config/bookRoles');
/** Day's games for a sport (date-pinned since S57) — tells us what a player's /** Day's games for a sport (date-pinned since S57) — tells us what a player's
* ABSENCE means: DNP, postponed, or simply not final yet. */ * ABSENCE means: DNP, postponed, or simply not final yet. */
async function getScheduleFn(sport, date) { async function getScheduleFn(sport, date) {
@@ -306,6 +311,7 @@ function rowsFromSnapshot(sport, grades, oddsProps, nowIso) {
// labelled 'pre-retention-unknown' by migration 026; they cannot be // labelled 'pre-retention-unknown' by migration 026; they cannot be
// resolved retroactively. // resolved retroactively.
model_version: MODEL_ERA_VERSION, model_version: MODEL_ERA_VERSION,
ruler_version: CURRENT_RULER_VERSION,
game_id: gameIdFor(sp, prop, gameDate), game_id: gameIdFor(sp, prop, gameDate),
game_date: gameDate, game_date: gameDate,
}); });