diff --git a/specs/MASTER-PLAN.md b/specs/MASTER-PLAN.md index d70b95e..1ed70c8 100644 --- a/specs/MASTER-PLAN.md +++ b/specs/MASTER-PLAN.md @@ -183,9 +183,22 @@ scaffolding around that. Building all 23 orders and *not* closing this leaves a beautifully-built product that doesn't do the one thing it sells. **What closes it:** not code. **Sample and honest iteration.** The instrument -fields are ~10 days old (442 rows). At ~90 decided MLB rows/week, a defensible -verdict is **6–10 weeks out**. That clock cannot be shortened by engineering, and -any attempt to shorten it is the fabrication this whole session has been removing. +fields are ~10 days old (442 rows). At ~90 decided MLB rows/week, that is ~6–10 +weeks *of accrual*. + +> **CORRECTED 2026-07-31 — see §11.** The "6–10 weeks out" above quietly assumed +> the clock is **already running**. It is not. Those 442 rows measure a model +> with a bent single-book ruler and six disconnected layers — **a model that +> will not exist once Phase 1 lands.** They do not count toward the verdict and +> **must not be pooled** with post-completion rows. +> +> The clock starts at a **verified** "running as intended" gate, per sport. +> **6–10 weeks is the accrual duration, not the distance to the answer.** The +> distance to the answer is *build time + verification + 6–10 weeks.* +> +> Independently forced by arithmetic, not just discipline: the §10.1 ruler fix +> changes the denominator, so pre-fix and post-fix edge/CLV numbers are not the +> same measurement. ## 9.2 The projection — the actual engine — is thin and unvalidated The grade's only real inputs today are **l5/l20 averages, an opponent rank, rest @@ -253,6 +266,35 @@ in two places.** Fix the data breadth and both halves improve at once.* ## 10.1 The one finding that reframes everything +> **CORRECTED 2026-07-31 by Order Zero — `specs/order-zero-book-breadth-test.md`.** +> The original text below blamed a missing `regions`/`bookmakers` param. **That +> was wrong.** PropLine's OpenAPI contract states verbatim: `bookmakers` … +> **"Omitted = all books."** Omitting it is correct and always was. +> +> **The real cause is ours.** PropLine sends **18 books**; `oddsNormalizer` +> `ALLOWED_BOOKS` (11 entries) intersects them at **exactly 5** — which is +> precisely the "5 MLB books" the audit measured. We discard 13 of 18 ourselves, +> and 6 of our 11 allow-list entries don't exist at PropLine at all. +> +> Measured on real public data (MLB `pitcher_strikeouts`, 5 complete events): +> the feed carries **4.41 books/prop**; after our filter, **1.50** — and **12 of +> 34 props become invisible entirely** (zero allowed books). +> +> Also corrected: "73% single-book" is the **long tail** of deep/reliever props +> sole-posted by DraftKings or Bovada. **On the core props we actually grade, +> the market is 10–12 books wide.** +> +> And a real negative: **`pinnacle` appears on 0 of 40 MLB props.** The one +> sharp book in our allow-list contributes nothing here. The independent +> low-vig references that *are* present on 100% of core props are **exchanges** +> — `novig`, `smarkets`, `kalshi` (+ `matchbook`, `polymarket`). DFS pick'em +> (`prizepicks`/`underdog`/`sleeper`/`dabble`) also covers 100% but is **not a +> market price** and must never enter a consensus. +> +> **So this is not a test to run. It is a build we can do: split one allow-list +> into takeable / reference / excluded, and make `fair_prob_lock` a median +> consensus across reference books.** No param, no cost, no tier, no new source. + **Our "market" is often ONE book.** MLB props are **73% single-book** (2.18 audit). And `proplineAdapter` sends only `{ apiKey, markets }` — **no `regions`, no `bookmakers` param** (`:152`). We take PropLine's *default* response. @@ -264,17 +306,31 @@ That single fact causes four separate problems we have been treating as unrelate 3. **CLV is weak** — you cannot measure "beat the close" against one book's close. 4. **No steam/disagreement detection** — needs ≥2 books to even exist. -**So the highest-leverage unblocked action in the whole plan is a cheap API test: -does PropLine return more books with a `regions`/`bookmakers` param on our tier?** -It is one request. If yes, it upgrades the free product, the model's denominator, -and the CLV instrument simultaneously. +*(Items 1–4 stand. Only the **cause** was wrong — and the fix is cheaper than +the original diagnosis implied.)* + +### 10.1b We use 1 of PropLine's 29 endpoints + +Reading the full spec surfaced ten unused endpoints that map directly onto §10.2 +and §10.3 gaps — `/odds/closing` ("the canonical CLV helper"), `/odds/history`, +`/exports/odds-history`, `/movement` (steam across all 16 books), `/best-line`, +`/ev`, `/results` + `/exports/resolved-props` (**resolution across 33 sports**), +`/context` (**free**: probable pitchers, confirmed lineups, home-plate umpire, +first-pitch weather), `/markets/hit-rates`, `/players/{n}/trends`. + +Two overturn standing beliefs: **"NBA/WNBA/soccer have no free settled feed"** may +be a **$19/mo** problem rather than a data problem; and we hand-built probable +pitchers / depth charts / lineup confirmation that `/context` serves free. + +**Caveat:** these are *documented*, not verified for our tier — that needs the +PropLine key (prod-only, absent locally). Do not plan on them as facts yet. ## 10.2 What a real DATA AGGREGATOR has that we don't | capability | ours | gap | |---|---|---| -| **Book breadth** | 5 MLB / 2 WNBA, 73% single-book | the category runs 10–20. **Root gap (10.1)** | -| **True consensus / no-vig line** | single-book de-vig | needs breadth first | +| **Book breadth** | 5 admitted of **18 sent**; 4.41→1.50 books/prop after our own filter | **not a feed gap — one allow-list (10.1).** Core props are already 10–12 books wide | +| **True consensus / no-vig line** | single-book de-vig | **unblocked now**: median across reference books (exchanges + pinnacle), n≥2 or labelled fallback | | **Historical odds archive** | **STARTED** — `closing_captures` 844k rows, `lock_lines` (033) new, in-grade history capped at **24 points** | no full open→close series per prop. This is what makes CLV and backtesting real | | **Market breadth** | 11 live markets | the category ships 50+ (alt lines, combos, innings, quarters) | | **Alt-line ladders from books** | we *compute* a ladder; we don't *ingest* the books' | users shop rungs | @@ -321,8 +377,11 @@ the work. ## 10.5 If I had to order it for "top product" -1. **Book breadth test + consensus fair line** (10.1) — one API call to find out; - upgrades aggregator, model denominator and CLV together. +1. ~~Book breadth **test**~~ → **Book breadth FIX + consensus fair line** (10.1). + The test is done (Order Zero). It is now a build: split `ALLOWED_BOOKS` into + takeable / reference / excluded, make `fair_prob_lock` a median consensus. + Upgrades aggregator, model denominator and CLV together. **Gated on** exchange + prop-price validation + the WNBA measurement (needs the PropLine key). 2. **Opportunity projection** (10.3.2) — the biggest genuine modelling gain. 3. **Per-stat distributions** — connect `distribution.js`, prove per stat. 4. **Connect the built layers** (Phase 1) — each proven on held-out or left off. @@ -342,3 +401,98 @@ unfinished, but because **we have not measured it at adequate n on a market we c trust.** Book breadth + the odds archive + accrued settlements are what make that claim *possible*. Everything in §10 is in service of being able to make it — or of being able to say honestly that we can't. + + +--- + +# 11. THE ACCRUAL CLOCK — SEQUENTIAL, POST-COMPLETION + +*Kev's correction, 2026-07-31. Supersedes any "accrual runs in parallel with +building" framing anywhere in this document.* + +## 11.1 The causality + +You cannot meaningfully accrue until the product is **right and running as +intended** — model layers connected, ruler fixed (real consensus, not one soft +book), sports in, operating in the vision. + +**Only then does the accrual clock start, and only then does time produce a +verdict. Building faster shortens the time TO clock-start. It never runs the +clock.** + +The flawed assumption being removed: that today's half-connected model accrues +useful evidence while we build. It does not. That data measures a model that +**will not exist** after the layers are connected — a different model wearing the +same name. + +## 11.2 Measurement rule (non-negotiable) + +- Settled data accrued **before** completion does **NOT** count toward the edge + verdict. +- **NO POOLING across the completion boundary.** Pre-fix and post-fix are + different models — the same class of error as the model-version boundary, at + whole-model scale. +- The clock starts at **verified** "running as intended", not at "today". + +## 11.3 Two clocks — stated separately so neither corner-cuts + +**1. MLB-MODEL VERDICT clock.** Starts when MLB is genuinely complete: built +layers connected + honest consensus ruler + running as designed. Its accrued n +judges **MLB**. + +**2. FULL-PRODUCT TRACK RECORD clock.** Starts when the vision is running: +sports in, aggregator built, operating as intended. Its accrued n judges **the +product claim**. + +**Each subsequent sport gets its OWN clock**, starting when *that* sport's model +is complete — never when it is stubbed in. (Per-sport doctrine: a sport that +merely renders is not a sport that measures.) + +## 11.4 "Complete" — defined honestly + +A model/sport is **complete-enough-to-accrue** when its **built layers are +connected** and it runs against an **honest ruler** (real consensus), **operating +as designed** — *not* when every conceivable feature exists. + +This definition is doing real work in both directions: it blocks the corner-cut +("close enough, start counting") **and** it blocks never-ship ("one more +feature"). The full-product claim additionally requires the vision's sports + +aggregator running. + +## 11.5 The verification gate + +**Between build and clock.** Before any accrual counts, verify — not assume — +that the product is running as intended: + +- connected layers actually **fire** (present in the served payload, not merely + present in the repo) +- the ruler is a **real consensus** (`fair_prob_source: 'consensus_n'`, n≥2) +- the sport **settles correctly** (spot-checked against real box scores) +- surfaces are **honest** (no fabricated values; absent renders absent) + +**This gate is verified, not assumed.** No accrual line item runs during the +build phases. + +## 11.6 The 10-user track, reframed + +Real users are onboarded to a **complete** product, so their usage teaches about +the real thing rather than a half-built one. + +**We do NOT acquire users early to "start accrual."** That is the corner being +explicitly refused. + +## 11.7 The finish line + +``` +build orders complete + ↓ +running-as-intended VERIFIED (§11.5 gate) + ↓ +clock starts (per sport, two clocks, §11.3) + ↓ +verdict reported honestly — including if it says the edge is not there +``` + +That last clause is the whole point. A clock you are willing to stop early is not +a measurement, and a verdict you are only willing to publish if it is favourable +is not a verdict. diff --git a/specs/order-zero-book-breadth-test.md b/specs/order-zero-book-breadth-test.md new file mode 100644 index 0000000..ba03ea4 --- /dev/null +++ b/specs/order-zero-book-breadth-test.md @@ -0,0 +1,291 @@ +# Order Zero — BOOK-BREADTH TEST + +**Date:** 2026-07-31 · **Type:** measure-only · **Live adapter: UNTOUCHED** · +**Our API key: NOT USED** (measurement ran entirely on PropLine's *public, +unauthenticated* endpoints — zero quota spent, zero risk to the live feed). + +--- + +## VERDICT UP FRONT + +**The order's premise is disproven, and the finding is better than any of the +three anticipated outcomes.** + +The `bookmakers` param exists, but **omitting it already returns every book.** +We are not being starved by the feed. **PropLine sends 18 books; we discard 13 +of them ourselves**, in `src/utils/oddsNormalizer.js` `ALLOWED_BOOKS`. + +> **Outcome (d) — ALREADY OPEN.** The constraint is one allow-list in our own +> code. No param, no cost, no tier upgrade, no second data source. + +This is why Step 0 exists. Firing a guessed `bookmakers=` request would have +*restricted* the response and produced exactly the false negative the order +warned about — "doesn't open" — while the real cause sat in our repo. + +--- + +## STEP 0 — THE PARAM, CONFIRMED FROM THE AUTHORITATIVE SPEC + +Source: `https://api.prop-line.com/openapi.json` (200, 78KB, 29 paths) — the +OpenAPI contract itself, not a marketing page. + +`GET /v1/sports/{sport_key}/odds` parameters: + +| param | required | default | documented behaviour | +|---|---|---|---| +| `markets` | no | `h2h` | comma-separated market keys | +| `bookmakers` | no | *(none)* | **"Comma-separated bookmaker keys (e.g. 'draftkings,fanduel'). Omitted = all books. Matches the-odds-api's `bookmakers` param."** | +| `period` | no | *(none)* | full-game only unless set | + +**`Omitted = all books.`** Verbatim from the spec. The param is a *filter*, not +an expander. `proplineAdapter.js:152` omitting it is **correct** and always was. + +§10.1 of MASTER-PLAN said the missing param was the root cause. **That was +wrong.** Corrected in place. + +--- + +## STEP 1 — THE MEASUREMENT + +Two public endpoints, no auth, no key, no quota: + +1. `GET /v1/freshness` — per-bookmaker ingestion health = **the real book universe** +2. `GET /v1/exports/sample` — "last 7 days of MLB `pitcher_strikeouts` as CSV… + no auth required" — **real resolved props with book, line, price, resolution, + actual value, and closing price** + +### 1a. The book universe — 18 keys, all live + +`/v1/freshness`, sampled `2026-08-01T03:13Z`, stale threshold 1800s: + +| book | markets | staleness | in our `ALLOWED_BOOKS`? | +|---|---:|---:|---| +| draftkings | 487,182 | 16s | ✅ | +| **bovada** | **385,689** | 2s | ❌ dropped | +| betmgm | 166,171 | 119s | ✅ | +| betrivers | 151,845 | 12s | ✅ | +| pinnacle | 144,615 | 6s | ✅ | +| **novig** | 132,604 | 0s | ❌ dropped | +| prizepicks | 111,475 | 14s | ❌ dropped | +| unibet | 64,696 | 15s | ❌ dropped | +| **smarkets** | 39,113 | 8s | ❌ dropped | +| underdog | 33,632 | 66s | ❌ dropped | +| fanduel | 32,189 | 0s | ✅ | +| **kalshi** | 27,649 | 5s | ❌ dropped | +| **matchbook** | 24,682 | 5s | ❌ dropped | +| onexbet | 22,896 | 382s | ❌ dropped | +| **polymarket** | 17,347 | 1s | ❌ dropped | +| sleeper | 8,857 | 58s | ❌ dropped | +| tab_au | 6,183 | 45s | ❌ dropped | +| dabble | 4,409 | 51s | ❌ dropped | + +**Not one book is stale.** The feed is healthy; the loss is entirely ours. + +### 1b. The arithmetic that explains "5 MLB books" exactly + +`ALLOWED_BOOKS` (11) ∩ PropLine (18) = **5**: +`draftkings, fanduel, betmgm, betrivers, pinnacle`. + +**That is precisely the "5 MLB books" the 2.18 audit measured.** Cause +identified — it is a set intersection, not a coverage fact. + +Two separate defects fall out: + +- **13 of PropLine's 18 discarded by us.** +- **6 of our 11 allow-list entries do not exist at PropLine at all** — + `caesars, fanatics, bet365, hardrockbet, pointsbet, thescore`. They are inert + on the PropLine path. *(They are NOT dead code overall — the odds-api backup + path may still carry them. Do not delete them on this evidence.)* + +### 1c. Per-prop breadth — MLB `pitcher_strikeouts`, 6 events, 2026-07-31 + +1,000 rows. 5 complete events / 34 props used for breadth (the 6th event is +truncated by the export's 1,000-row cap and is excluded — reporting it would +manufacture single-book props out of a row limit). + +| | mean books/prop | single-book | props visible | +|---|---:|---:|---:| +| **PropLine sends (param omitted)** | **4.41** | 67.6% | 34 / 34 | +| **After `ALLOWED_BOOKS`** | **1.50** | 67.6% | **22 / 34** | + +**Our filter costs 2.9 books per prop and makes 12 of 34 props invisible +entirely** (zero allowed books → the prop does not exist to VYNDR). + +The distribution is **bimodal, and the shape is the real story**: + +- **13 "core" props** (probable starters) — **10–12 books each** +- **21 tail props** — sole-posted: 13 by `draftkings` alone, 13 by `bovada` + alone, 1 by `onexbet`. These are deep/reliever props no one else prices. + +The headline "67.5% single-book" is therefore **not** a statement about the +market for the props we actually grade. It is the long tail of one-book deep +props dragging the mean. **On the props that matter, the market is 10–12 books +wide.** + +Book presence across the 13 core props: + +| book | coverage | role | +|---|---|---| +| underdog, novig, sleeper, prizepicks, dabble, smarkets, kalshi | **13/13** | 3 exchanges + 4 DFS | +| draftkings, betmgm | **13/13** | ✅ ours | +| onexbet | 12/13 | offshore | +| fanduel | 12/13 | ✅ ours | +| betrivers | 10/13 | ✅ ours | + +### 1d. The sharp anchor — a real negative + +**`pinnacle` appears on 0 of 40 props.** It carries 144,615 markets globally but +**no MLB player props in this sample.** The one sharp book already in our +allow-list contributes **nothing** to MLB props. + +What *is* present on 100% of core props: **`novig`, `smarkets`, `kalshi`** — and +`matchbook`/`polymarket` elsewhere in the feed. These are **exchanges**: +order-book priced, so near-zero vig. Arguably a *better* fair-probability +reference than Pinnacle. + +**Stated as a hypothesis, not a fact:** exchange prop books are thinner than +their game-line books, and `kalshi`/`polymarket` are prediction markets whose +prop coverage is recent. Their prices must be **validated on real data before +being trusted as a ruler**, not assumed good because the theory is good. + +### 1e. Three buckets, not one list — DFS is not a market price + +`prizepicks, underdog, sleeper, dabble` cover 13/13 core props, but they are +**fixed-payout pick'em, not two-way markets.** Their lines are deliberately +shaded and their effective juice is flat. **They must never enter a consensus +fair price.** Admitting them "for breadth" would bend the ruler in a new +direction — the exact error this order exists to fix. + +### 1f. Cost and tier + +`/odds` returns **all books on the FREE tier**. Nothing about book breadth costs +money. Our quota (3 free keys × 1,000/day = 3,000/day) is unchanged by this. + +Tier gates apply only to the *analytical* endpoints (§ below): +Free $0/1k-day · Hobby $9/5k · Pro $19/25k · Streaming Lite $39/100k · +Streaming $79/1M · Enterprise custom. + +### 1g. ⚠️ WNBA — NOT MEASURED + +The public sample is **MLB `pitcher_strikeouts` only**. Every per-sport +alternative needs the API key, and **`PROPLINE_API_KEY_1/2/3` are absent from +the local `.env`** (prod/Coolify only). + +**Do not assume WNBA mirrors MLB.** WNBA's book set is likely thinner and its +exchange coverage thinner still — which would tie directly to its +anti-predictive result. **This measurement is outstanding and needs one thing: +a PropLine key.** + +--- + +## THE BIGGER FINDING — WE USE 1 OF 29 ENDPOINTS + +While confirming the param I read the full spec. Several §10 "gaps" are not gaps +in the data — they are endpoints we never called. + +| §10 gap | PropLine endpoint | tier | +|---|---|---| +| CLV instrument | **`/events/{id}/odds/closing`** — *"the canonical CLV helper"*, last snapshot at/before commence_time per (book, market, outcome) | Paid | +| Historical odds archive | `/events/{id}/odds/history` (`from`/`to`/`relative_to`/`interval`/`changes_only`) | Hobby+ | +| Bulk archive / backtest | `/exports/odds-history` — full tick firehose | Enterprise / one-time backfill | +| Steam detection | **`/events/{id}/movement`** — *"steam … computed across all 16 books we poll"* | Hobby+ | +| Line shopping | `/events/{id}/best-line` — best price per (market, player, line) + `all_prices` best-first | appears free | +| Cross-book +EV | `/events/{id}/ev` — no-vig via Pinnacle preferred, Bovada fallback | appears free | +| **Settlement for non-MLB sports** | `/events/{id}/results`, `/exports/resolved-props` (line, price, resolution, actual value, **closing price**) | Hobby+ / Pro+ | +| Park / weather / umpire / lineup | **`/events/{id}/context`** — probable pitchers, confirmed-lineup flag, home-plate umpire, first-pitch weather | **FREE** | +| Base rates | `/markets/hit-rates` | — | +| Player trend history | `/players/{name}/trends`, `/players/{name}/history` | Hobby+ | + +Two of these overturn standing beliefs in `CLAUDE.md`: + +1. **"NBA/WNBA/soccer have no free settled-result feed"** — PropLine *resolves + props across 33 sports* and exports them with actual values. Our + settle-nothing-but-MLB-and-WNBA constraint may be a **$19/mo** problem, not a + data-availability problem. +2. **We hand-built** probable pitchers, depth charts and lineup confirmation. + `/context` serves them, free, joined to the graded outcome. + +**Caveat, stated plainly:** these are *documented* capabilities. I have verified +the **`bookmakers` contract** (a spec-level guarantee) and the **book universe + +per-prop breadth** (real data). I have **not** verified what these endpoints +actually return for our tier — that needs the key. Do not plan on them as facts +yet. + +--- + +## STEP 3 — SCOPE OF THE RULER CHANGE *(scope only — NOT built)* + +**1. Split one allow-list into three roles.** Today `ALLOWED_BOOKS` answers +"can we show this?" and "can we price against this?" with the same list. Those +are different questions and conflating them is what bent the ruler. + +- `TAKEABLE_BOOKS` — a US user can actually bet here. Drives best-price, + line-shopping, `takeable`. *(today's list, minus nothing)* +- `REFERENCE_BOOKS` — the fair-probability ruler **only**; never surfaced as a + place to bet: `pinnacle, novig, smarkets, matchbook, kalshi, polymarket, bovada` +- `EXCLUDED_FROM_PRICING` — DFS pick'em (`prizepicks, underdog, sleeper, + dabble`) and offshore/intl (`onexbet, unibet, tab_au`). May be surfaced as a + separate DFS view; **never** in a consensus price. + +**2. `fair_prob_lock` becomes a consensus.** Per-book two-way de-vig +(`devigTwoWay`, unchanged), then the **median** across reference books — median, +not mean, so one stale exchange cannot drag the ruler. + +**3. Require n≥2 reference books, and label which ruler was used.** Below 2, +fall back to today's single-book de-vig and stamp +`fair_prob_source: 'consensus_n' | 'single_book'`. **Never silently mix the +two** — that pools two different rulers inside one column, the same error class +as pooling across a model-version boundary. + +**4. Stamp a `ruler_version` at the boundary.** Changing the denominator means +**every edge and CLV number measured to date was measured on the old ruler and +cannot be pooled with post-change rows.** + +**5. Gate the build on two things that do not exist yet:** + (i) exchange prop-price sanity validated on real data (§1d), and + (ii) the WNBA measurement (§1g). + +--- + +## STEP 4 — MLB ISOTONIC `p_win` IS PROVISIONAL + +**Declared explicitly:** the MLB isotonic `p_win` recalibration was measured +against the **bent single-book ruler**. It is **PROVISIONAL and GATED behind the +ruler fix.** It does **not** finalize or promote until re-run on the real +consensus ruler. + +**Plan sequencing edit:** MLB promotion is **downstream** of the ruler fix, not +before it. + +--- + +## WHAT CHANGED IN THE PLAN + +- §10.1 rewritten — the param premise was wrong; the allow-list is the cause. +- §10.2 — book breadth reclassified from "needs a new source" to "one file". +- §10.5 — item 1 is now a **build we can do**, not a test to run. +- Accrual clock corrected to **sequential / post-completion** with a + verification gate (separate ride-in, §11). + +**And the two interact:** the ruler change resets the denominator, so +pre-change settled rows cannot be pooled with post-change rows. That is an +*independent* derivation of Kev's sequential-clock rule — the no-pooling +boundary is forced by the arithmetic, not just by discipline. + +--- + +## REPRODUCE + +```bash +curl -s https://api.prop-line.com/openapi.json # the param contract +curl -s https://api.prop-line.com/v1/freshness # the 18-book universe +curl -s https://api.prop-line.com/v1/exports/sample # real resolved MLB props +``` + +All three are public. No key was used, printed, or committed. + +## OPEN — needs one thing from Kev + +**A PropLine API key** (it is in prod/Coolify, not in the local `.env`). With +it: the WNBA per-sport measurement, and live verification of the ten unused +endpoints above — including whether **$19/mo settles NBA and soccer.**