Book Comparison Phase 1-3(backend): fenced per-book store + honest gated crown
Per-book prices existed only transiently (odds cache, ~1h, raw names, grade-path
input); every grade-path persistence point collapses to one book. The
/api/books feature was built+mounted but non-functional (fed FLAT rows to a
GROUPED comparator -> always empty).
Phase 1: bookPriceStore captures per-book prices from `props` BEFORE dedupeProps,
keyed nameKey|stat, into bookprices:{sport} (SNAP_TTL) in snapshotService. Fenced:
reads props, writes its own key, read by nothing on the grade path. Grade proven
byte-identical (test + no-grade-path-reference grep test).
Phase 2: scripts/measure-book-spread.js reports same-line best-vs-worst spread
(cents + implied-prob pts), per sport, never pooled. Pre-registered crown
threshold: median >=8c OR >=2pp. Runs post-deploy on real data.
Phase 3 (backend): compareProp is honest-absent (single-book/flat -> no crown)
and the crown is gated (BOOK_CROWN_ENABLED, default OFF until Phase 2 clears).
/api/books repointed to the snapshot-locked store (fallback odds cache),
nameKey-matched; `source` field is the deploy fingerprint.
HELD unchanged: dedupeProps, snapshot dedup, selector, grade, champion,
challengers, ranking, edge_pct/ev_pct. UI routing of BookComparison + crown
treatment deferred to post-measurement (gated on Phase 2). Full suite 3834 green,
web build exit 0.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VsztNChZ7vEvSR61AuMhD1
This commit is contained in:
+90
-2
@@ -1,6 +1,49 @@
|
||||
# VYNDR — STATE OF THE WORLD
|
||||
### As of `4f3f433` (main, DEPLOYED + fingerprinted live), 2026-07-22. This file opens every future session. **Start with the CURRENT STATUS + OPEN ITEMS block below.**
|
||||
|
||||
---
|
||||
# 🎯 CLV REDIRECT — investigated 2026-07-26. FINDING: it was ALREADY BUILT.
|
||||
*The premise "CLV is dead, redirect it to closing_captures" is STALE. Nothing needed building.*
|
||||
- **Redirect EXISTS + WIRED:** `closingCapture.buildCaptureRows`→`closing_captures` (append-only,
|
||||
provenance: captured_at/book/line_type/both-prices/missed_reason) via `intradayRefreshService:221`
|
||||
+ internal endpoint; `ledgerService.attachClosingProb`→`closing_prob` (de-vigs both raw sides,
|
||||
write-once, honest-absent `market_unavailable_reason`) via `snapshotScheduler:310`.
|
||||
- **Honest-absent is already implemented + is the invalidation marker:** 651,624 capture rows
|
||||
(86% `missed`, no fabricated price); ledger has **59 genuine `closing_prob`, 870
|
||||
`market_unavailable_reason`**. Of 841 old-`clv` rows, **791 are already marked unavailable**;
|
||||
**745 (89%) of old `clv` = exactly 0/'flat'** (the fake "never captured" signal). Old broken
|
||||
`clv`/`closing_line` left in place, unread by the new instrument (but STILL consumed by ledger
|
||||
ROW_COLUMNS/UI — repointing display is a FUTURE order, not done here).
|
||||
- **0.4 GATE:** captures **differ from locked** (10/10 comparable rows moved) → NOT a
|
||||
captures==locked timing defect. BUT the capture **86%-misses** → CLV is STARVED: only **~10-50
|
||||
usable rows**, and only **10 have both `closing_prob` + `fair_prob_lock` (all WNBA, 0 MLB)**.
|
||||
- **LINE-LAG HYPOTHESIS → CANNOT DETERMINE.** The 10 WNBA comparable rows move ~1pp, roughly
|
||||
symmetric (mean CLV gap −0.008 over / −0.011 under). **The +4.57% MLB-C and the over/under
|
||||
asymmetry are NEITHER confirmed nor refuted by CLV** — there is essentially no MLB CLV data.
|
||||
- **The real open defect (separate order):** WHY 86% of captures miss (props leave the feed before
|
||||
the 45-min window? matching failure?). Until that's fixed the instrument stays starved.
|
||||
- Docs corrected: CLAUDE.md (WNBA settles, not MLB-only), STATE.md grade_11 (model_snapshots only).
|
||||
Orphan `next start -p 3111` (6-day-old, the "1 shell running") killed.
|
||||
|
||||
---
|
||||
# 📐 CALIBRATION DIAGNOSIS — measured 2026-07-26 (read-only; nothing changed)
|
||||
*Run on BOTH populations: unselected both-sides (model_snapshots, p_win per side) vs published selected (ledger). Never pooled across sport. Full detail in chat log of that order.*
|
||||
|
||||
**Feasibility caveats that govern every number below:**
|
||||
- p_win window ≈ **last 10 games** (`probabilityEstimator`, last-5 double-weighted) → **SE ≈ 0.16 at p=0.5** (noisy estimator).
|
||||
- Continuous p_win coverage is thin+recent (revived ~07-20): **~18 MLB games, ~5 WNBA games** of joinable calibration data. WNBA calibration is **effectively anecdotal (~5 games)**; MLB is suggestive not confident (~18 games).
|
||||
- Effective sample generally: MLB 31 games / 190 players, **WNBA 12 games / 77 players** — prop outcomes cluster within a game; do not treat rows as independent.
|
||||
|
||||
**Findings (VERIFIED where measured; sample caveats attached):**
|
||||
1. **Selection bias, measured directly** = both-sides→selected Brier: **MLB 0.212→0.233, WNBA 0.258→0.278 (~+0.02 each)**. Published grade is inflated ~1 sub-tier by the side-pick: mean idx **MLB C→C+**, **WNBA C+→B−**. Honest A-or-better ≈ 0 → the "caps at ~B+/A-" story survives (optimistic if anything).
|
||||
2. **MLB calibration** (both-sides): Reliability 0.010 (good), Resolution 0.047 (moderate), Uncertainty 0.249 → **discriminates + roughly calibrated**; middle deciles near-perfect. SELECTED side is **overconfident, growing with p** (+0.02 at p<.5 → **+0.19 at p≥.8**).
|
||||
3. **WNBA calibration** (both-sides): Brier 0.258 **> uncertainty 0.25 = worse than always-predicting-0.5**; Resolution 0.019 (near-noise), Reliability 0.027 (poor). Realized ~flat vs predicted. **Champion does not discriminate on WNBA** — but ~5 games, so unproven.
|
||||
4. **Null baselines (real prices):** **unders lose ~−14% flat both sports; overs near break-even** (MLB −2.8%, WNBA −1.4%). **Ledger strongly supports "no unders by default."** MLB model beats always-over (MLB-C +4.6% vs −2.8%); **WNBA model LOSES to always-over** (−5% vs −1.4%). Both within noise at 31/12 games.
|
||||
5. **ev_pct = p_win×decimal−1 uses the overconfident p_win** → inflated; hero ranks on it → picks the most-overconfident reads. No current metric reliably predicts ROI; price-aware EV requires recalibrated p_win first.
|
||||
|
||||
**BUILD FORK (declared before data; sample-tempered):** MLB → *resolution good / reliability poor-when-selected* → recalibration mapping + EV ranking — **but ~18 games can't validate a holdout yet (test half 15 games)**. WNBA → *resolution poor* → champion doesn't discriminate / price-aware replacement — **but ~5 games = accrue volume before concluding.** **CLV redirect ships next regardless.** Most robust actionable finding now: **kill default unders.**
|
||||
|
||||
|
||||
---
|
||||
|
||||
# 🧭 CURRENT STATUS + OPEN ITEMS — orientation block (2026-07-20, ~03:20 UTC)
|
||||
@@ -19,8 +62,12 @@ was verified live unless explicitly marked UNVERIFIED.*
|
||||
| 6 | ESPN team-stats parser fixed (`buckets is not iterable`, 0/15 → 15/15) | live refresh |
|
||||
|
||||
### ⚠️ Two honest qualifiers on the above
|
||||
- **A STILL DOES NOT EMIT IN PRODUCTION.** The 11-step grade is now *stored*
|
||||
(`grade_11`) and A/D are *arithmetically* reachable + locked by tests, but the
|
||||
- **A STILL DOES NOT EMIT IN PRODUCTION.** The 11-step grade is stored in
|
||||
`model_snapshots.grade_11` ONLY (CORRECTED 2026-07-26: it is NOT in
|
||||
`ledger_entries` — `_grade_11` is deleted at `gradeSlateService.js:97` before
|
||||
the ledger write; sub-tier resolution on a SETTLED ledger row is recoverable
|
||||
only by joining model_snapshots). A/D are *arithmetically* reachable + locked
|
||||
by tests, but the
|
||||
±1.0 opponent factor is still dead because `opp_rank_stat` is underivable from
|
||||
ESPN (see open item 6). Live boards remain **B/C only**.
|
||||
**→ The "A-RATED" marketing hold STANDS** (`AccuracyBadge` correctly falls
|
||||
@@ -1858,6 +1905,47 @@ Dockerfile must copy every dir the runtime reads.
|
||||
`web/src/app/terms/page.tsx` (6×) + `web/src/app/privacy/page.tsx` (5×).
|
||||
A test keeps unverified entity names out until replaced.
|
||||
|
||||
## BOOK COMPARISON — data layer shipped (per the build order)
|
||||
|
||||
**What shipped (backend + tooling; UI routing is the gated next step):**
|
||||
- **Phase 1 — fenced, snapshot-locked per-book store.** `src/services/
|
||||
bookPriceStore.js` captures per-book prices from the multi-book `props` array
|
||||
BEFORE `dedupeProps` runs, keyed by `nameKey|stat_type`, written to
|
||||
`bookprices:{sport}` at SNAP_TTL (24h) inside `snapshotService.runSnapshot`.
|
||||
STRUCTURAL FENCE: it only reads `props` and writes its own key; nothing on the
|
||||
grade path reads it. Proven byte-identical — the graded slate is unchanged with
|
||||
or without the capture (`tests/unit/bookPriceStore.test.js`, incl. a grep test
|
||||
asserting no grade-path file references `bookprices`).
|
||||
- **Phase 2 — the crown is threshold-gated.** `scripts/measure-book-spread.js`
|
||||
reports best-vs-worst same-line price spread (American cents AND implied-prob
|
||||
points), book-count histogram, %-zero, pinnacle presence — per sport, never
|
||||
pooled. PRE-REGISTERED: crown ships only if median ≥8¢ OR ≥2 implied-prob pts.
|
||||
Runs post-deploy against real captured data (Redis degraded locally → exits 0).
|
||||
- **Phase 3 (backend) — honest-absent + gated crown.** `bookComparisonService.
|
||||
compareProp` now renders single-book / flat-market props with NO crown; the
|
||||
crown fires only among ≥2 books at the SAME line with DIFFERING prices, and only
|
||||
when `BOOK_CROWN_ENABLED=1` (default OFF until Phase 2 clears). `/api/books/
|
||||
:sport` (a crown claim) returns [] while gated off; `/api/books/:sport/:player/
|
||||
:stat` returns the honest grid always. Route repointed to read `bookprices:*`
|
||||
(fallback: odds cache), matched by `nameKey`. Response `source` field is the
|
||||
deploy fingerprint (`bookprices` = new store serving).
|
||||
|
||||
**What it revealed:** the premise ("BookComparison.tsx is a dead component") was
|
||||
incomplete — the ENTIRE feature (service + `/api/books` routes + Next proxy) was
|
||||
built and mounted, but NON-FUNCTIONAL end-to-end: the route fed FLAT odds-cache
|
||||
rows to a comparator that expects GROUPED `{lines|books}`, so it always returned
|
||||
empty. Per-book prices were retained only transiently (odds cache, ~1h, raw
|
||||
names, grade-path INPUT); every grade-path persistence point collapses to one
|
||||
book. Fixed by the snapshot-locked store + route repoint.
|
||||
|
||||
**Gated next step (NOT this order-turn):** route `BookComparison.tsx` onto the
|
||||
prop card fed by the store, and design the crown treatment — AFTER Phase 2's
|
||||
measurement returns real spread numbers on prod (you can't honestly design a
|
||||
crown before you know it ships). `BOOK_CROWN_ENABLED` stays 0 until then.
|
||||
HELD unchanged: dedupeProps, snapshot dedup, the selector, the grade, champion,
|
||||
challengers, ranking, edge_pct/ev_pct. No push-to-book, no movement strip, no
|
||||
pinnacle edge test (data persisted only).
|
||||
|
||||
## HONEST OPEN ITEMS
|
||||
|
||||
- **Phase 4.5 — WNBA settlement via ESPN box scores. DUE ~Jul 24** (hard
|
||||
|
||||
Reference in New Issue
Block a user