builtbykev/vyndr
1
0
Fork 0
Code Issues Pull Requests Actions 5 Packages Projects Releases Wiki Activity

Verify the cap raise on prod: 7 -> 365 graded props

Browse Source 
Induced, not projected. DEFAULT_LIMIT=500 produced 365 graded props in
114s (was 7 in 16s) -- 52x the board. All 365 carry a unique forecast_rank
and ZERO leak p_win to anonymous callers, so the tier gating holds at 50x
the volume. Anon payload 220KB in 0.44s. Stat mix went from three stats to
ten. Health green.

Measured cost curve via the ?limit= bisect hook: 1->42s, 25->58s, 60->42s,
120->66s, 500->114s. About 42s of that is FIXED overhead (odds fetch,
roster logs, archetype classify, retention), paid whether we grade 1 prop
or 500 -- grading is the cheap part.

MY PRE-FLIGHT ESTIMATE WAS WRONG. I predicted ~72s from per-prop latency
measured in isolation, which ignored the fixed cost. Real figure 114s.

A FALSE ALARM RECORDED because acting on it would have meant reverting a
fix that works: the first induced run 502'd at 13.4s and I hypothesised
load -- memory or a proxy timeout under 20x the work. Wrong. A limit=25 run
then 502'd in 2 seconds, which no amount of load explains, and both
recovered on retry. The 502s were the deploy rolling, not the cap.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QJs13VsyiSKYQP6rj3NNmc
This commit is contained in:
Kev
2026-08-01 02:22:30 -04:00
parent 8d052131c5
commit 11b0139481
2 changed files with 42 additions and 4 deletions
Download Patch File Download Diff File Expand all files Collapse all files
specs/grade-cap-and-refusal-diagnosis.md
+41 -4
View File
@@ -123,11 +123,48 @@ cap stops discarding 95.7% of the slate — not because the bar moved.
---
## EXPECTED EFFECT
## VERIFIED EFFECT (induced on prod, not projected)
At the measured 55% grade rate on 585 unique props, the MLB board should go from
**~7 graded props to roughly 300**, at a cost of about **70 seconds per snapshot
run, five times a day.**
| | before | after |
|---|---:|---:|
| **graded props on the MLB board** | **7** | **365** |
| ledger rows per run | 7 | 365 |
| retention rows | 50 | 1,000 |
| snapshot duration | 16s | **114s** |
| anon `/api/snapshot/mlb` payload | ~6 KB | **220 KB, 0.44s** |
**52× the board.** Every one of the 365 carries a `forecast_rank` (1–365, all
unique) and **zero leak `p_win` to anonymous callers** — the gating still holds
at 50× the volume.
Stat mix, previously three stats, now ten: hits 141 · total_bases 78 · rbi 42 ·
runs 41 · walks 31 · strikeouts 12 · outs 11 · earned_runs 4 · hits_allowed 3 ·
doubles 2.
### Measured cost curve (induced runs, `?limit=` bisect hook)
| limit | grades | wall |
|---:|---:|---:|
| 1 | 0 | 42s |
| 25 | 7 | 58s |
| 60 | 39 | 42s |
| 120 | 63 | 66s |
| **500** | **365** | **114s** |
**~42s of that is FIXED overhead** (odds fetch, roster logs, archetype classify,
retention) — it is paid whether we grade 1 prop or 500. Grading is the cheap
part. **My pre-flight estimate of "~72s" was built from per-prop latency measured
in isolation and ignored that fixed cost; the real figure is 114s.** Wrong by
~40s, in the direction that mattered least, but wrong.
### A false alarm I chased, and what it actually was
The first induced run **502'd at 13.4s**, and did not complete in background. I
hypothesised load — memory or a proxy timeout under 20× the work. **That was
wrong.** A `limit=25` run then 502'd in **2 seconds**, which no amount of load
explains, and both recovered on retry: the 502s were **the deploy rolling**, not
the cap. Recorded because the load hypothesis was plausible enough to have been
acted on, and acting on it would have meant reverting a fix that works.
**Two things to watch after deploy, stated in advance rather than discovered
later:**