c7cc8f5e529c3cecf90ce9442f8a12128497be69
35 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 builtbykev
|
c7cc8f5e52 |
Build the gate, run it, and find we were proving things on the wrong stat
PREMISE CORRECTION FIRST. statModel.js and correlateValidator.js do not exist in this repository. The validation spec's only prior form is src/services/python/blueprints/unconventional.py -- a Flask blueprint in the Python service that is offline in production, scoring NBA factors against a warehouse that was never populated -- and tests/unit/supplementSystems.test.js requires only fs and path while defining its own validateFactor inline at line 368. Those tests assert a re-implementation of the thresholds, not an implementation, which is exactly why they passed for months while nothing was connected. The diagnosis behind the order is right -- every challenger was measured without a gate -- but the cause is that there was no gate on the Node side to import. So it is built, to the exact spec. correlateValidator: n>=500, |r|>=0.15, p<0.05, Bonferroni across the sweep. The p-value is exact rather than approximated (t-transform through a regularized incomplete beta) and is verified in the suite against known values, because scipy is not available here. Pairs with an unknown side are dropped, never zero-filled -- a zero-fill inside a correlation does not add noise, it invents a point at the origin. THE RUN, hits, n=570, Bonferroni-8: every skill feature fails, and not narrowly. The strongest marginal correlation against the counter's residual is 0.062 against a 0.15 bar. That is an effect-size failure at a sample that would have found a real effect comfortably -- a clean, well-powered negative. The head-to-head agrees: value engine 0.0499 against the counter's 0.166, delta -0.116 with CI [-0.189, -0.043]. Not promoted. THE RUN, total bases, n=295: cannot be tested, and that is the finding. hard_hit_pct shows a marginal r of 0.153 -- above the threshold -- and exit velo 0.124, refused solely because n is 205 short of 500. It is the most encouraging number this work has produced, and it is what the physics predicts: contact quality governs extra bases, not whether a grounder finds a hole. We have been testing skill inputs on the one stat where they should not matter much. Two things the run forced. Feature verdicts are now PER STAT, because marking these DEAD sport-wide on hits evidence would have killed, for total bases, the features that look most alive there -- per-sport doctrine one level deeper. And the gate now reports r and p even when underpowered, because "not enough data yet" and "nothing here" demand opposite decisions and a bare refusal was hiding the best signal on the board. Next: build the compound TB projection (skillProjection still refuses total bases by design, since a deterministic bases-per-hit made P(TB>=2) identical to P(hits>=1)), accrue to n>=500, re-run this gate. Leave hits alone. 4,200 tests green (334 suites); web build exit 0; counter byte-identical. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W1sivYNqY2TS5ftykmHBU9 |
||
|
builtbykev
|
258d8a6655 |
The skill engine: built, gated by construction, and Stage A honestly lost
Built src/services/model/ -- the forward, archetype-selected, skill-based projection, as a challenger. The champion is untouched. featureRegistry makes "earn its place or it's out" structural rather than aspirational: CANDIDATE / PROVEN / DEAD per feature per sport, liveFeatures() returns PROVEN only, promotion requires n>=200 with positive lift and a CI excluding zero, and there is deliberately no override argument. It ships with exactly ONE proven feature -- the incumbent counter, because it is the only one with a measurement. A test asserts that with only PROVEN features allowed the projection returns null, so an unproven model cannot reach a user by accident. The three champion adjustment layers are registered DEAD with their reasons so they cannot be silently rebuilt. skillProjection is a PA outcome tree: K and BB combined by log5 odds-ratio against league (both identities unit-tested), then archetype-weighted contact quality against contact allowed, then Binomial(PA, p_hit) mixed over a PA distribution. Archetype is a FEATURE SELECTOR, not a nudge -- BOMBER reads barrels at 0.50 and ground-ball speed at 0.00, GHOST inverts it -- and a test locks that the same hitter read two ways moves more than 0.15. STAGE A: IT LOSES. Out-of-sample on 570 settled hits props with 91.9% opposing-pitcher coverage, resolution 0.0499 against the champion's 0.166, delta -0.116 with CI [-0.189, -0.043]. It is not selective either: its eight most confident picks hit 50%, a lift of -0.065. Not promoted. The gate did its job on its first real test, which is the point of having built it that way. Two false starts, both recorded because they nearly produced a wrong verdict: statcast_aggregates stores PERCENTAGES, so raw rows made bip = 1-29.6-17.1 and refused 568 of 576 -- the honest-absent guards made a units bug loud instead of silent, and the conversion now lives at one chokepoint. And the first run resolved an opposing pitcher for 1 of 570 rows, because ledger team/opponent are NULL, so it would have reported "skill-v1 loses" while measuring a batter-only model with no matchup in it at all. The verdict above is from the corrected run. The loss is real but partial: park was passed as 1.0, handedness and opportunity_drift never fired, PA is season-PA over a constant, and the skill profiles carry no recency at all while the champion has a last-5 term. Also fixed: the Statcast nightly refresh was unreachable code. It sat inside tick() below "if (!HOURS_UTC.includes(h)) return" while testing h === 11, so it had never run once; the aggregates were 13 days stale and both of its alerts were in the same dead branch. It now runs on its own tick, and the test that passed happily throughout -- it only checked the string existed -- is replaced by one that asserts it is not behind the guard. 4,182 tests green (333 suites); web build exit 0. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W1sivYNqY2TS5ftykmHBU9 |
||
|
builtbykev
|
d8bf7765db |
Decompose the champion: its whole edge is a hit-rate counter
READ-ONLY. src/ and web/ untouched; 4,159 tests still green. WHAT THE CHAMPION IS. probabilityEstimator is five lines of arithmetic: the empirical frequency of (stat > THIS line) over the game log, blended 0.6/0.4 with the last-5 frequency, then +/-0.03 opponent, +/-0.015 home/away, a cv>0.40 pull toward 0.50, and a clamp to [0.10, 0.95]. It reads three features. featureCache retains a dozen more that p_win never touches. THE ABLATION IS EXACT, NOT A REFIT. Every adjustment is closed-form from stored features and the consistency step is linear, so each layer subtracts algebraically out of the stored p_win -- no re-estimation, no re-fetch, no lookahead possible. Per stat, paired bootstrap: removing ALL THREE adjustments changes resolution by NOTHING on every stat hits -0.0059 total_bases -0.0015 rbi +0.0106 runs +0.0130 walks +0.0008 and rbi's home/away is mildly HARMFUL (+0.0053, CI excludes zero). So ~100% of the champion's resolution is base+recency: how often this player has cleared this number lately. Everything else is decoration. A CORRECTION. Pooled, the champion resolves 0.46; per stat it is 0.196 (hits) to 0.499 (rbi). Pooling stats with different base rates inflates correlation, so 0.46 should not be quoted as the champion's resolution. Last session's paired differences remain valid; only the absolute level was inflated. THE BIGGEST LOSS IS NOT A MISSING FEATURE -- IT IS THE CLAMP. 358 of 1,741 settled rows (20.6%) sit on the boundary, so the model emits a constant there and cannot rank a fifth of the book at all. And that constant hides two opposite failures: 0.900 covers home_runs-under truly winning 99.5% (9.5pts under-confident) next to hits-under truly winning 51.9% (38.1pts over- confident). PROB_CEIL=0.95 makes the 99.5% case inexpressible. Global over-prediction is +3.5pts, +7.6 on total_bases. None of this needs new data. ONE REAL MISSING-WEIGHTING LEAD: opportunity_drift, residual corr +0.156 on hits and +0.145 on total_bases -- it REPEATS across independent stats, unlike the weather hits on TB which sit inside the expected false-positive count (70 tests at alpha .05 expects 3-4). And we already compute it: arch-v1's opportunity axis uses it and extracts nothing (delta +0.0001). Wrong implementation, not a missing feature -- opportunity must scale the rate, not nudge the probability. ARCHETYPE IS UNMEASURABLE, NOT REFUTED. Only 2 of 41 labels (BOMBER, GHOST) reach n>=40 settled rows and every mean residual straddles zero. That is "we have not measured it", and it does not license acting in either direction. Why every challenger has failed is now legible: the ladder and hits-v1 REPLACE the frequency question with a fitted distribution; the environment axis adds inputs the champion ignores. Asking the frequency question at the traded line is the thing that works. Flagged, not fixed: model_snapshots.outcome is NULL on all 22,032 rows -- the retention table built for exactly this replay was never settled, so labels had to be joined from ledger_entries. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W1sivYNqY2TS5ftykmHBU9 |
||
|
builtbykev
|
3ba3dd28f3 |
Scoreboard every challenger; diagnose the 429 as odds-api, not PropLine
PROMOTE-THE-EARNED. Nothing was promoted, because nothing earned it -- not because the bar was held high. Measured on the same bar that refuted hits-v1: own rows only, direction-aligned, paired bootstrap, promote only on a CI excluding zero. arch-v1 n=1741 delta 0.0000 CI[-0.0050,+0.0054] inconclusive contact-v1 n=1055 delta +0.0008 CI[-0.0052,+0.0069] inconclusive proj-v1.1 n=1664 delta -0.0301 CI[-0.0543,-0.0060] reliably WORSE matchup/tb-v1/hits-v1 n=0 genuinely pending (rows dated 08-02+) arch-v1 is the interesting one: it MOVED 76% of rows by 2.5 points on average and resolution is identical to the champion to four decimals, on the moved rows too. That is active movement carrying no information -- a finding, not a pending verdict. These are true prospective holdouts: arch-v1 and contact-v1 wrote p_win at grade time into their own columns before the game. Nothing recomputed. THE 429, read-only. The premise was that we re-pull the full picture every slot and blow the quota. Measured: PropLine is at 5 calls of 3,000/day -- 0.17%. One snapshot is ONE PropLine call per sport, all markets comma-joined. There is no request-pattern problem, so a change-based pull cannot fix it and no tier upgrade is needed. The 429 is odds-api: 478/500 MONTHLY, blocked at 95%. oddsService falls through silently when PropLine returns empty, and the backup's quota gate throws the error -- so an empty slate is indistinguishable from an outage and the message names the wrong provider. Flagged for its own order. Could NOT verify PropLine movement endpoints: docs are auth-gated and the keys are production-only. Not asserted either way. The movement-as-data argument stands on its own merits and should be justified that way, not as a quota fix it isn't. Book-breadth invariant written down: we never discard books. All are kept and shown (DISPLAY_BOOKS = MODEL + REFERENCE + DFS); DFS pick'em is excluded from PRICING only, because a fixed-payout shaded number is not a market price. Verified this is already what bookRoles.js does. Champion byte-identical; every challenger stays wired. 4,159 tests green (332 suites); web build exit 0. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W1sivYNqY2TS5ftykmHBU9 |
||
|
builtbykev
|
07626de3de |
hits-v1: built on the right structure, measured honestly, REFUTED
Hits was diagnosed as a family mismatch: 84% of hits rows trade at 0.5, so the stat rides on P(0), and a negative binomial has unbounded support and no notion of opportunity at all. hits-v1 models it as the bounded conversion it is -- N ~ the player's empirical at-bat distribution, hits|N ~ Binomial(N,q), with the multiplier scaling q (conversion) and never N (opportunity). STEP 0 confirmed the inputs before the model existed: 30/30 real ledger players, 100% combined-input coverage. Every read goes through knownRate -- a row with no atBats is dropped, never counted as a 0-at-bat game. It FIRES: 158/159 hits props (99.4%) on the live production snapshot, through the real attachProjection path. Scoping by book IDENTITY rather than price shape kept 94 out-of-promotion-band props on the board, 93 of them modelled -- 59% that a price rule would have deleted. And it LOST. Point-in-time replay (game log truncated strictly before each row's game_date, real grade-time multiplier), hits-only, direction-aligned, n=242: resolution champion 0.195 / ladder 0.048 / hits-v1 0.026. Paired bootstrap on the same rows: hits-v1 - ladder = -0.022, CI95 excluding zero. Not promoted. The value is in what it eliminates. The family was wrong AND the mean was not the constraint -- hits-v1 moved the line-0.5 mean 0.554 -> 0.581 toward a 0.598 base rate while resolution fell. What is left is per-prop discrimination: the ladder's inputs, not its distribution. The pre-registered fallback is recorded as WRONG rather than deleted. It said hits might be genuinely low-resolution for anyone; the champion scores 0.276 on the identical 189 rows, so there is real signal and the ceiling claim was the comfortable reading, not the honest one. Its own control refuted it, and that control was already in hand when the branch was written. hits-v1 stays wired as a challenger writing its own ledger columns so the forward accrual can confirm the backtest. Champion, ladder, ranking, calibration, reference ruler and the four accruing verdicts are byte-identical -- the diff has zero deleted lines. Tests 4,156 green (332 suites); web build exit 0. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W1sivYNqY2TS5ftykmHBU9 |
||
|
builtbykev
|
e29ab6fd6a |
Takeable enforcement: verified on real rows, 1,006 tagged, re-stamp call ready
PART 1 verified by inducing the REAL rowsFromSnapshot over REAL lock_lines
rows from prod. Three cases, 0 non-takeable anchors:
Narvaez (dabble/kalshi/prizepicks/smarkets, NO takeable book)
-> book=null, price=null, takeable=null [honest absent]
Schwarber(bovada/dabble/novig/PINNACLE before draftkings)
-> draftkings +102 [pinnacle SKIPPED, proving TAKEABLE not MODEL]
Ohtani (dabble/onexbet before draftkings) -> draftkings -266
Narvaez is the case that matters: pre-fix he was stamped dabble +104
takeable=true; he is now honestly absent.
A HARNESS BUG RECORDED: my first verification pulled live /api/odds/mlb,
which returned {"error":"Odds data temporarily unavailable"}. The script
read that as 0 props and printed "all from takeable books? true" -- a
VACUOUSLY TRUE pass. I caught it only because I also printed the book list
and it was empty. Same family as the silent-false traps: a probe that finds
nothing looks identical to a probe that finds nothing wrong.
PART 2: 1,006 rows tagged via the purpose-built quarantine_reason at ROW
level with three sub-cases (recoverable_same_line 936, no_takeable_quote
49, takeable_line_differs 21). getModelAggregate ALREADY excluded
quarantined rows, so the public record and the n>=20 gate were clean
automatically; all five committed holdout scripts now carry the exclusion
explicitly.
PART 3 -- the re-stamp call is now fact-based. The takeable LOCK-TIME price
is recoverable for 936/1,006 (93.0%) from lock_lines, the correct
instrument. Only 431 appear in closing_captures, which is the wrong timing
for a lock price anyway.
LINE CONTAMINATION ANSWERED (previously unverified): the stored line
MATCHES a takeable book's line on 936 (93.0%), DIFFERS on 21 (2.1%), and is
unverifiable on 49 (4.9%) where no takeable book quoted the prop at all.
That makes it cleanly row-level: re-stamp the 936 as an honest JOIN and
recover 886 pending rows for the holdouts, or leave all 1,006 excluded.
Either way the 21 + 49 stay out -- re-stamping those would invent a lock
price, or a line, we never captured. Nothing re-stamped; Kev's call.
Gates: 4,111 tests / 330 suites green; next build exit 0.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QJs13VsyiSKYQP6rj3NNmc
|
||
|
builtbykev
|
eabf3b5bcf |
tb-v1: model total_bases as a compound outcome (challenger)
Current ladder (proj_p_over_line) and champion p_win are BYTE-IDENTICAL. tb-v1 writes alongside them, on total_bases props only. STEP 0 -- components confirmed on real data, not assumed. statsapi has no singles field, but hits - doubles - triples - homeRuns reproduces stored totalBases EXACTLY on a real 10-game log. So the decomposition is exact, not an approximation. THE MODEL. Each component gets its own per-game Poisson rate; TB is their weighted sum, and the PMF is built by exact convolution rather than simulated (TB support is small). It inherits the SAME combined multiplier proj-v1.1 computes, so the two models differ only in STRUCTURE. Why this is the fix: with identical mean TB of 1.0, a pure-HR hitter and a pure-singles hitter get P(TB>=4) of 0.221 vs 0.019 -- a 12x difference an NB on TB alone cannot express, because it treats one home run as four events. A test asserts that separation, and asserts P(TB>=4) for a pure-HR hitter equals P(at least one HR) exactly. INDEPENDENCE IS AN APPROXIMATION AND IS LABELLED AS ONE: a plate appearance that becomes a double cannot also become a single, so the components are weakly negatively correlated and independent Poissons slightly overstate the tail. Closer to the truth than what it replaces; not a solved problem. HONEST-ABSENT throughout: fewer than 3 usable games, or no derivable component, returns null and the prop keeps the current ladder value. An inconsistent row (hits < extra-base hits) is SKIPPED rather than clamped to zero -- clamping would invent a plausible line out of a broken one. I HIT THE Number(null)===0 TRAP IN MY OWN CODE and a test caught it: a null rate passed a naive finite check and was treated as a measured zero, which is the difference between "this player never triples" and "we do not know his triple rate". Both tbPmf and tbMean now reject null/''/boolean strictly. Holdout committed: TB ROWS ONLY (49 of 437 settled -- averaging into other stats would hide the effect) and DIRECTION-ALIGNED, since the unaligned comparison is the artifact that accounted for 41% of the ladder's apparent loss. If tb-v1 does NOT improve, the family-mismatch hypothesis is wrong and the mean/similarity branch reopens -- recorded in the query header. Migration applied: proj_tb_p_over + proj_tb_meta, NULL-meaningful. Gates: 4,104 tests / 329 suites green; next build exit 0. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QJs13VsyiSKYQP6rj3NNmc |
||
|
builtbykev
|
9ebd77b68e |
Build the matchup/platoon axis: three joins fixed, axis now FIRES
The axis was already wired and firing on 0/634 prod rows. Three separate
absences kept it silent, and all three are now joined:
1. oppPitcherByTeam 0 -> the self-origin /api/schedule/mlb/pitchers route
returned nothing in prod. Added the statsapi probable-pitcher hydrate as
a fallback, mirroring the one the schedule step already uses. 29/30
team-sides, one free request.
2. handById 0 -> follows from (1); the batched people call now has ids.
3. bats 0/120 -> batter hand rode ONLY on statcast aggregate rows, which do
not cover the slate. The season player list we ALREADY fetch and cache
carries batSide on 1342/1342, so this is a join, not a fetch.
Switch-hitters ('S') are preserved as-is; platoonSplits decides what to
do with them, not the map.
Verified end-to-end against the live API: opp_declared 29,
pitchers_with_hand 29, batters_with_hand 1342, and a real read --
multiplier 0.966, L vs R, 287 observed PA, weight 0.324 -- composing
alongside environment in one challenger.
FALLBACK LADDER, and a deliberate deviation from the order. Shipped tier:
`batter_own_split` (the hitter's OWN vs-L/vs-R line, regressed toward HIS
OWN overall rate), labelled on every adjustment.
`league_generic` is deliberately NOT implemented. platoonSplits already
handles thin evidence by regressing toward the hitter's own rate, which
covers the thin case per-player; its own doc-comment argues a hitter with
no split evidence should get NO adjustment. A league split applied to such
a hitter models the LEAGUE, not the player -- the doctrine breach the order
itself names in the same step. Adding it would have produced more firing
rows and a weaker signal.
`archetype_x_archetype` is scoped, not built: it needs the opposing
starter classified per game, which is real work and a separate order. The
tier vocabulary is in place for it.
Honest-absent on every join: no starter, no pitcher hand, or no batter hand
-> NO matchup adjustment, never a fabricated neutral. A neutral multiplier
produces no adjustment row at all.
Holdout committed (scripts/matchup-axis-holdout.sql), filtered to
matchup-carrying rows, and it keeps MATCHUP'S OWN nudge visible rather than
only the combined challenger -- arch-v1 composes four axes into one
p_win_challenger, so a combined-only view could not tell which axis earned
the movement, or which one is dragging.
Champion p_win, ranking, calibration, the armed invariant and the two
accruing verdicts are untouched.
Gates: 4,093 tests / 328 suites green; next build exit 0.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QJs13VsyiSKYQP6rj3NNmc
|
||
|
builtbykev
|
48a2f764ac |
opportunity_drift: coverage 94%, collinearity PASSES, holdout n-blocked
STEP 1 -- input mapped and measured. opportunity_drift 94% coverage on 100 real props: 100% for batters (total_bases, hits, home_runs), 40-67% for pitchers, which is correct -- pitchers accumulate few at-bats so the ratio is genuinely undefined and ABSTAINS rather than being invented. STEP 2 -- THE COLLINEARITY GUARD PASSES DECISIVELY. Pearson r on n=94: drift vs l20_avg -0.020, vs l5_avg +0.027, vs ab_per_game -0.029. All essentially zero, so the axis is orthogonal to every existing projection input and carries information the projection does not already contain. That also validates the ratio-over-level decision EMPIRICALLY: ab_per_game is the same quantity over the same denominator as l20_avg, so the level would have been redundant. Dividing by the player's own baseline removed the collinearity -- r = -0.029 against the very quantity it is built from. STEP 3 -- live as a challenger, verified on prod over an induced 416-grade snapshot: 142 of 276 rows (51.4%) carry the opportunity axis, the challenger moved on 190 rows, mean |delta| 0.034, range -0.089..+0.108. Champion p_win and the live grade path are unchanged. STEP 4 -- HOLDOUT IS n-BLOCKED BY CONSTRUCTION and I am not manufacturing one. Settled rows carrying the axis: 0. Its first rows carry game_date 2026-08-01 -- games that have not been played. Running the test on rows the axis never touched would dilute the comparison with rows where challenger === champion by construction, making a null result look like a small positive one. Query committed for when n arrives; it filters to axis-carrying rows for exactly that reason, buckets before measuring reliability, and splits time-forward. BOTH metrics must improve or the axis is shelved. A MEASUREMENT TRAP RECORDED: the first prod run showed drift at 0% while ab_per_game read 94% -- indistinguishable from "the feature does not compute". It was the 120-second feature-vector cache serving payloads written by the previous image. A new feature field is invisible for one cache generation after deploy. I nearly reported it absent, having already confirmed atBats is present in the live statsapi payload and that the code produced drift = 1.05 locally on that exact data; the contradiction between those two facts is what saved it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QJs13VsyiSKYQP6rj3NNmc |
||
|
builtbykev
|
7140e62b65 |
MLB re-run vs consensus ruler: premise dissolved, isotonic DECIDED
MEASURE-ONLY. No promotion, no flip, no tier spend. Live path
byte-identical: CURRENT_RULER_VERSION still v1_first_book, model still
consumes MODEL_BOOKS only.
MANDATE 1'S PREMISE DOES NOT HOLD. The p_win calibration is
RULER-INDEPENDENT, confirmed two ways: estimateProbability takes
{gameLogs, line, statType, features} and never sees a market price, and
the calibration fits p_win against OUTCOMES. Reliability and resolution
are both p_win-vs-outcome measures, so fair_prob cannot enter either.
There is nothing to re-fit -- the ruler changes edge, CLV and takeable,
not calibration.
I RETRACT MY OWN LABEL. I declared the MLB isotonic result PROVISIONAL
"because it was measured against the bent ruler". That over-applied the
ruler caveat to a measurement the ruler never touched. The result was
never contaminated; it moves PROVISIONAL -> DECIDED, not by re-running but
because the gate I attached does not apply.
RAN THE GENUINELY RULER-DEPENDENT QUESTION INSTEAD -- does a median
consensus rescue EDGE? Timing held constant (both rulers at close; a
lock-time reconstruction joins only 43 rows, and mixing lock-incumbent
with close-consensus would confound WHEN with WHAT).
n=200 MLB settled rows: mean |ruler gap| 0.0085. corr(edge_v1, outcome)
-0.0101; corr(edge_v2, outcome) -0.0220; corr(p_win, outcome) +0.2598.
THE HEADLINE: p_win predicts outcomes at +0.26 while p_win minus the
market predicts nothing under EITHER ruler. Subtracting the market price
destroys the signal -- a direct empirical vindication of the identity now
at the top of CLAUDE.md. Market edge is not merely a poor criterion here;
it is a strictly worse instrument than the raw forecast.
CALIBRATION REFRESH (ruler-independent, but n grew 119 -> 250):
time-forward holdout n=125, reliability 0.0846 (was 0.0939), resolution
0.190 (was 0.123). Both hold and both improved on a fresh later window
the earlier fit never saw. Independent replication.
THE LIMITATION THAT BLOCKS A FULL VERDICT: closing_captures holds only
MODEL books -- exchange quotes were never stored, because normalizeProps
discarded them until yesterday. Mean 1.97 books in the historical join. So
this tested a US-books-median ruler, not the exchange-inclusive consensus
whose live delta showed p90 +10 points. That ruler is UNTESTABLE on
existing data at any n. Per Mandate 4's third outcome: inconclusive, not
forced.
SEPARATE FINDING -- LIVE FEED REGRESSION: pinnacle MLB captures went 4,022
-> 0 on 2026-07-31 and have not returned, while every other book continued
(103,940 captures in the prior 10 days). This also corrects an Order Zero
claim of mine: "no sharp anchor exists in our feed" was accurate for the
day measured but wrong generally -- pinnacle was there until 07-30 with
17,090 two-sided captures. line_type='sharp' is a label in closingCapture
via SHARP_BOOKS, not a separate provider. We had a sharp anchor and lost
it two days ago; not caused by anything in this session.
Both queries committed: scripts/ruler-comparison.sql,
scripts/pwin-timeforward.sql.
Gates: 4,028 tests / 322 suites green; next build exit 0.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QJs13VsyiSKYQP6rj3NNmc
|
||
|
builtbykev
|
6d87d7a33c |
report: p_win recalibration holdout — MLB qualifies on isotonic, WNBA abstains
Measure-only. p_win not flipped live, no grade rebuilt, no calibrator deployed. Per the doctrine, MLB and WNBA were fitted, selected and judged as SEPARATE models — and they reach opposite verdicts. No global instrument was fitted. METHOD: time-forward split per sport (earlier fits, later proves). Both instruments fitted on TRAIN only — single-parameter Platt and isotonic-with-pooling. Inputs p_win + outcome only; no market field, no closing value, no lookahead. Nothing about edge/CLV/beat-the-close enters any pass/fail line. MEASUREMENT CORRECTION made mid-run: the first pass reported mean|p - outcome| (~0.46-0.51), which is NOT calibration — it is noise-dominated individual error on 0/1 rows and would have made every instrument look identical. Reliability is only meaningful on BUCKETS (bucket mean predicted vs bucket actual rate, n-weighted), the metric T0 used. All reported numbers use the corrected metric. HOLDOUT RELIABILITY (lower better): MLB n=119/4 buckets — raw 0.1038, Platt 0.1120, ISOTONIC 0.0939. WNBA n=93/3 buckets — raw 0.1322, Platt 0.0491, isotonic 0.0667. HOLDOUT RESOLUTION: MLB raw 0.1388 -> Platt 0.1284 -> isotonic 0.1225. WNBA raw -0.1201 -> Platt +0.1269 -> isotonic +0.0322. Fitted Platt: MLB a=-0.381 b=+0.705; WNBA a=+0.040 b=-0.081. MLB QUALIFIES, MODESTLY — instrument selected BY HOLDOUT, not assumed: isotonic beats both raw and Platt, and Platt actually made MLB worse. Reliability improves 0.1038 -> 0.0939 (~10% relative, real but modest) and resolution SURVIVES (0.1388 -> 0.1225, not crushed). Both Mandate-3 conditions hold. WNBA ABSTAINS — its Platt result is the best number in the report and is REJECTED as a fake win. The fitted slope is b = -0.081, negative and near zero, so sigmoid(0.040 - 0.081*logit p) is nearly constant at ~0.51 for every input: it "calibrates" by discarding the prediction and emitting the base rate, which is exactly the failure Mandate 3 pre-registered. Its apparent resolution gain (-0.120 -> +0.127) is the sign flip, not skill — it would serve the opposite of its own forecast, fitted on n~96 of anti-signal. Isotonic says the same quietly (resolution collapses to +0.032). HONEST CEILING: MLB is a usable-but-unimpressive forecaster (holdout resolution ~0.12, reliability ~0.094, n=119); WNBA has no honest forecast today. Holdout n and bucket counts (4 and 3) suffice to reject WNBA and prefer isotonic for MLB, NOT to certify a letter ladder, and the T0 pathology is reduced rather than cured. CANNOT DETERMINE: per-archetype calibration (Mandate 3d) — bucket n falls below the reporting floor once split by sport AND archetype on 442 rows. Queries committed at scripts/pwin-calibration-holdout.sql. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QJs13VsyiSKYQP6rj3NNmc |
||
|
builtbykev
|
249b3e8235 |
report: grade diagnostic T0 — p_win is MISCALIBRATED, and it explains the inversion
STOPPED at the T0 gate as instructed. Nothing fixed, no recalibration applied, no grade touched. T1-T4 deliberately not run. T0 FIRES ON BOTH PRE-REGISTERED CONDITIONS. Condition 1 (mean |predicted-actual| > 0.05): MLB ~0.094, WNBA ~0.139. Condition 2 (monotonic slope): over-confidence GROWS with the prediction — MLB +0.034 -> +0.043 -> +0.084 -> +0.190 -> +0.189; WNBA +0.044 -> +0.109 -> +0.349. Worst cases: MLB predicted 0.842 actual 0.652 (n=23), predicted 0.917 actual 0.727 (n=11); WNBA predicted 0.730 actual 0.381 (n=21). WHY THIS EXPLAINS THE INVERSION, mechanically: p_win is over-stated and the overstatement SCALES with p_win, so p_win - fair_prob_lock is largest exactly where p_win is most inflated. Those props hit less than claimed, so the edge measure correlates negatively. The market was never the problem — fair_prob_lock is not a bent ruler, the thing subtracted from it is. It also explains why p_win ALONE still carries signal (+0.23 MLB): rank survives miscalibration, differences do not. This independently reconfirms the 2026-07-26 calibration finding (+0.02 at p<.5 -> +0.19 at p>=.8) on a newer, larger population, so it is structural rather than sampling noise. PART 0: P0a — only the GRADED side's fair prob is stored (fair_prob_lock; no opposite-side field), so T1's two-side-sum check cannot run and must use the stated no-vig recompute fallback. P0b — projection_locked_at exists as a timestamptz so T2 is potentially runnable, but distinctness from lock time was NOT verified because T0 gated it. Two cautions recorded before Part 2 runs: the top MLB buckets where the error is worst hold n=23 and n=11, so a flexible per-bucket correction would fit noise — isotonic with pooling or single-parameter Platt is safer; and calibration fixes magnitudes, so if the market is genuinely better the repaired edge may still land at ~0, which would be the honest ceiling and gets reported rather than graded around. Query committed at scripts/grade-calibration-t0.sql. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QJs13VsyiSKYQP6rj3NNmc |
||
|
builtbykev
|
ea1157d709 |
report: grade fix Part 1 — the p_win-vs-fair_prob rebuild is REFUTED by the data
STOPPED at the Part 1 gate. Nothing rebuilt, no grade changed, no cutover. THE FINDING: grading on p_win vs fair_prob does not work. All three candidate edge formulations correlate NEGATIVELY with outcomes, on both sports, overall, and in both time splits (n=432 decided rows carrying p_win AND fair_prob_lock): ALL n=432 champ -0.0016 p_win ALONE +0.1221 additive -0.0615 ratio -0.1161 logodds -0.0438 MLB n=240 champ +0.0984 p_win ALONE +0.2278 additive -0.0336 ratio -0.1350 logodds -0.0124 WNBA n=192 champ -0.1143 p_win ALONE -0.0842 additive -0.1326 ratio -0.1281 logodds -0.1243 Subtracting the market's lock-time fair probability destroys and inverts the signal. The plain reading: props where the model most disagrees with the market are LESS likely to hit — the market is better than the model, so "edge vs market" is anti-predictive here, while the raw probability retains some skill alone. WHAT DOES CARRY SIGNAL: p_win alone, MLB only, and it is modest. Time-forward split — TRAIN (07-21..07-26, n=120) r=0.2770; HOLDOUT (07-26..07-30, n=120) r=0.1647, with the additive edge negative in BOTH halves. So p_win survives forward validation directionally but the holdout is NOT significant (t~1.81, p~0.07). Suggestive, not proven. WNBA MUST ABSTAIN: every measure negative including p_win itself (-0.084). Forcing one threshold across both sports would make a coin-flip sport look sharp, which the order forbids. LOOKAHEAD GUARD SATISFIED: fair_prob_lock is the lock-time field, populated on 432 decided rows, range 0.145-0.713. closing_prob (415 rows) is the CLOSE and was NOT used in any correlation — using it would have manufactured a correlation. SAMPLE REALITY: 1103 decided rows but only 432 carry both instrument fields, so a per-sport train/holdout split leaves ~120 per half — enough to show direction, not to certify a letter ladder. I did not tune toward a win: three pre-registered candidates were tested and all three failed; picking a fourth because the first three lost is the overfitting the order guards against. Recommended instead: grade MLB on p_win alone with WNBA abstaining and label it modest/accruing (A-RATED hold stays); or wait ~6 weeks for n~500 MLB; or investigate WHY the market-relative edge inverts, which is the more valuable question. Both queries committed at scripts/grade-correlation-proof.sql so no number here has to be taken on trust. Working settlement untouched; dead resolve endpoint not wired. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QJs13VsyiSKYQP6rj3NNmc |
||
|
builtbykev
|
e81c9b8c51 |
Book Comparison Phase 1-3(backend): fenced per-book store + honest gated crown
Per-book prices existed only transiently (odds cache, ~1h, raw names, grade-path
input); every grade-path persistence point collapses to one book. The
/api/books feature was built+mounted but non-functional (fed FLAT rows to a
GROUPED comparator -> always empty).
Phase 1: bookPriceStore captures per-book prices from `props` BEFORE dedupeProps,
keyed nameKey|stat, into bookprices:{sport} (SNAP_TTL) in snapshotService. Fenced:
reads props, writes its own key, read by nothing on the grade path. Grade proven
byte-identical (test + no-grade-path-reference grep test).
Phase 2: scripts/measure-book-spread.js reports same-line best-vs-worst spread
(cents + implied-prob pts), per sport, never pooled. Pre-registered crown
threshold: median >=8c OR >=2pp. Runs post-deploy on real data.
Phase 3 (backend): compareProp is honest-absent (single-book/flat -> no crown)
and the crown is gated (BOOK_CROWN_ENABLED, default OFF until Phase 2 clears).
/api/books repointed to the snapshot-locked store (fallback odds cache),
nameKey-matched; `source` field is the deploy fingerprint.
HELD unchanged: dedupeProps, snapshot dedup, selector, grade, champion,
challengers, ranking, edge_pct/ev_pct. UI routing of BookComparison + crown
treatment deferred to post-measurement (gated on Phase 2). Full suite 3834 green,
web build exit 0.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VsztNChZ7vEvSR61AuMhD1
|
||
|
builtbykev
|
e809a0eb3c |
Backtest harness — the validator, built refusal-first
Phase 0 gate PASSED: the join is clean. No FK exists; the natural key (sport, player_key, stat, line, side, game_date) yields 283 clean 1:1 joins with ZERO ambiguity. game_id is NOT usable — 400/550 snapshot rows carry UNK@UNK because home/away names weren't threaded into the grader until Order 1.6. Non-joining rows are EXPECTED, not errors: retention stores both sides plus refusals; the ledger keeps only the graded side. Outcomes are NOT denormalized — ledger_entries stays the source of truth. BUILT TEST-FIRST, and the first property proven is the REFUSAL, not the math. Below threshold the harness emits INSUFFICIENT with n and the shortfall and NO rate anywhere in the payload, so a downstream renderer cannot surface one by accident. A test asserts the payload contains no hit_rate number at all. - Wilson intervals (correct at the n we actually have, unlike the normal approximation which emits negative lower bounds). - Strata NEVER mix sport or model_version. - Denominator excludes quarantined, void, unrecoverable, pending, push — asserted by test. - Monotonicity refuses to RANK buckets whose intervals overlap; it reports "not distinguishable on this sample". - Probability calibration (Brier + reliability) also respects the threshold: a thin sample returns status INSUFFICIENT and a NULL score. - Replay seam reads the STORED feature vector only. A row whose input was never retained is UN-BACKTESTABLE, never scored with substituted current data. Identity replay reproduces the live prediction exactly. The tests caught a real bug in my own code: `Number(null) === 0` let a null p_win through as a confident 0% forecast — this codebase's signature fabrication bug, inside the harness whose entire purpose is refusing invented numbers. Fixed with a strict null guard. FIRST LIVE RUN — the correct, passing output: VERDICT: INSUFFICIENT_HISTORY (can_validate=false) 283 joined -> 35 scored (120 quarantined, 124 pending, 4 terminal) C n=18 (short by 2), B n=17 (short by 3) strata: mlb 7, wnba 28 — never mixed migration 028 adds harness_results (append-only trend log; INSUFFICIENT rows are expected and correct) and opsWatch.harnessStaleAlarm pages if the harness stops running — a validator that isn't running looks exactly like one that keeps passing. Suite 283/3403 green, build exit 0. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SmNjJAwEnqHPtXbvSZR8kA |
||
|
builtbykev
|
b33612675d |
Heal execute: quarantine markers, re-enable DNP voiding, two exclusion scopes
Order 2 Phases 2 + 4. Pre-heal rollback point secured first: vyndr-20260720-093821.dump (856,890 bytes) VERIFIED ON THE BOX, not just exit 0. MIGRATION 027 — two DISTINCT exclusion scopes, deliberately separate: - quarantine_reason: the row's GRADE is untrustworthy (wrong_opponent_grade). The row REMAINS a real public settled result — the bet happened, the outcome is real — but it must never train or validate, so getModelAggregate now excludes it from the denominator alongside void/unrecoverable. - analysis_flags: the row is VALID for settlement and the record but unattributable for PER-GAME analysis (doubleheader dates). Explicitly NOT filtered from aggregates. Collapsing these would either wrongly drop 166 doubleheader rows from the record or wrongly keep 25 wrong-opponent grades inside model validation. Tests assert both directions, including that analysis_flags is NOT filtered. Also adds re_settled_at + settlement_source to model_snapshots. DNP VOIDING RE-ENABLED — reversing my own Order 1.5 disable, with scrutiny, because its premise was FALSE. Order 1.5 assumed a missing player row meant the row's DATE was wrong. The Phase 0 dry-run disproved it: across every bindable row the stored date matched a real game (MIS-DATED: 0), and the players I had cited as counter-evidence were genuine DNPs on their true dates (Freeman 07-18; Kwan/Hedges/Davis 07-17 — their teams played, they did not). The evidence is positive: games FINAL + no line in a full-season log = no bet existed. I got this wrong twice tonight in opposite directions; the dry-run is what caught it. Recording the reasoning in the code so the next reader sees why the flag flipped back. Suite 282/3386 green, build exit 0. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SmNjJAwEnqHPtXbvSZR8kA |
||
|
builtbykev
|
c4c9b97604 |
Off-box backup: pin the host key, guarantee the remote dir, page on failure
PHASE 1 — HOST KEY STATICALLY PINNED. ssh-keyscan -p 23 returned an ED25519 key whose fingerprint EQUALS the out-of-band value SHA256:XqONwb1S0zuj5A1CDxpOSuD2hnAArV1A3wKY7Z3sdgM, so it is safe to pin. scripts/storagebox_known_hosts now carries that verified line and ships to the container (Dockerfile already COPYs scripts/). backup-db.sh uses StrictHostKeyChecking=yes + UserKnownHostsFile=<pin> instead of accept-new, which was trust-on-first-use and would have accepted an impostor on the very first run. A missing pin file REFUSES the push rather than silently falling back. Never weakened to accept-new/=no//dev/null — a test asserts that on executable lines. PHASE 1b — REMOTE DIR GUARANTEED. The box has only .ssh/, and rsyncing a file into a missing parent either fails or silently writes the dump AS the directory name — one file, overwritten nightly, reading as "backups exist" while retaining exactly one. Uses rsync --mkpath when available, else an explicit remote mkdir -p ahead of the push. PHASE 2b — FAILED OFF-BOX PUSH IS NOW LOUD. Off-box is required, so the failed-push path pages at "urgent" (was "low"/deferred) and the script emits a machine-readable OFFBOX_OK=1/0/deferred that POST /api/internal/backup/run surfaces as a distinct offbox_ok field. Exit code deliberately still reflects ON-BOX durability — a good on-box dump must not raise a false total-failure alarm. Surfacing the truth, not manufacturing a failure. No key material is echoed anywhere; only the PUBLIC host key is committed. Suite 280/3338 green, build exit 0. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SmNjJAwEnqHPtXbvSZR8kA |
||
|
builtbykev
|
d3ffa1b8c2 |
Retention: model_snapshots live + base64 SSH key support
RETENTION (Phase 2, priority zero). History starts compounding tonight. migration 025 model_snapshots — APPLIED to prod. Append-only, one row per graded prop PER SIDE PER CYCLE, with a unique index on (snapshot_id, player_key, stat, line, side) so a retried cycle cannot duplicate. RLS on, service-role writes only. What it captures that the ledger never did: - features jsonb — the model's INPUTS. Without these a backtest can only grade our own homework; with them any future model can be replayed against the exact conditions this one faced. - REFUSALS (refused + refusal_reason). The ledger drops them, so a gate refusing props that would have WON is invisible — unmeasurable lost edge. Captured via a new onGraded hook in gradeSlateService that fires with BOTH sides before any filtering. - grade_11, the pre-collapse grade. The 4-letter map throws away the entire live C-/C/C+/B- range. - model_version + code_sha on every row. ledger_entries mixes pre/post-fix grades with no marker and cannot be separated retroactively. - p_win / ev_pct / fair_odds / takeable / value — none of which any permanent store held. Wiring: analyzeViaEngine1 attaches _features/_grade_11 (underscore = internal); gradeSlateService fires onGraded then STRIPS them so they never reach a cache or API payload; snapshotService builds rows and persists best-effort. Retention reuses the LEDGER's dateET/gameIdFor helpers so rows share the ledger's natural key exactly — otherwise the settle pass could never join outcomes onto them. Rows are written BEFORE the empty- slate early return: a slate that refused everything is exactly the case worth recording. CONTRACT HELD: retention is injectable and every path is caught. persist() returns errors, never throws; a missing Supabase client is SKIPPED, not an error. A retention failure can never break a snapshot. BACKUP: backup-db.sh now accepts BACKUP_SSH_KEY as base64 (recommended — survives env-var newline mangling, which is how injected SSH keys usually break silently) OR raw PEM, detected by decoding and looking for the PEM header. Verified both forms detect correctly against a real generated key. Suite 279/3325 green, build exit 0. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SmNjJAwEnqHPtXbvSZR8kA |
||
|
builtbykev
|
ef7f17610f |
Backup: durable on-box volume, off-box DEFERRED, and a real read-back check
BACKUP_DIR is now a persistent volume (/app/backups), so the dump already survives redeploys — the container-ephemeral risk that made this urgent is closed. Storage Box SSH auth is not sorted yet, so the off-box push is explicitly DEFERRED rather than failing: - gated on BACKUP_OFFBOX=1 (plus BACKUP_REMOTE and BACKUP_SSH_KEY); until then the script logs "off-box push DEFERRED" and exits clean. - if an enabled push DOES fail, it is a LOW-priority "deferred" notice, not a failure — the durable on-box dump succeeded, and calling that an incident would train us to ignore backup alerts. Adds the read-back check, because a backup nobody has read is a hope: countRowsInDump() runs `pg_restore --data-only --table=X -f -` and counts the rows between `FROM stdin;` and the terminating `\.`, proving the archive CONTAINS the data rather than merely parsing. Needs no Postgres server, so it runs inside the API container. Validated against a real pg_dump from a scratch Postgres: counted exactly 604 rows. GET /api/internal/backup/verify exposes it (newest dump in BACKUP_DIR, size, table, rows_in_dump). Unit tests inject spawn/fs so CI needs neither docker nor pg_restore. Suite 278/3310 green, build exit 0. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SmNjJAwEnqHPtXbvSZR8kA |
||
|
builtbykev
|
40aba37f83 |
Backup: env-injected SSH key, nightly off-box push, triggerable run
Closing the backup for real. Three changes, each fixing something that
would have made the Storage Box target fail or silently rot.
1. SSH KEY COMES FROM ENV, not from the container. Generating a keypair
inside the API container was the obvious move and it is wrong: the
container filesystem is ephemeral, so the key dies on the next
redeploy and the off-box push starts failing silently. backup-db.sh
now reads BACKUP_SSH_KEY (a Coolify secret), writes it to a 0600 temp
file per run, and removes it on exit via trap.
2. PORT 23, verified live. Hetzner Storage Box runs full OpenSSH on 23;
port 22 answers with mod_sftp (SFTP only). Banner-checked both against
u635423.your-storagebox.de. rsync now uses
-e "ssh -p ${BACKUP_SSH_PORT:-23} ... -i <key>"; the old invocation had
no -e at all and would have gone to 22.
3. OFF-BOX PUSH IS NIGHTLY, not Sundays-only. A weekly push meant up to
six days of dumps existed ONLY inside an ephemeral container, which is
the same as not existing. Alert copy updated to say exactly that when
the push fails or is skipped.
Also adds POST /api/internal/backup/run (internal-key gated) so a real
backup can be TRIGGERED and OBSERVED — it returns exit code, duration,
output tail, and whether the remote + ssh key are configured. The backup
can only run where SUPABASE_DB_URL and the Supabase route live (this
container), and there was no way to fire or inspect it without a shell.
Connectivity established this session: Storage Box reachable from the dev
box on 22/23; Supabase :5432 NOT reachable from WSL2 (so the dump must
run in-container, as designed); docker IS available locally, so the
restore-verify can run against a scratch Postgres using the real dump.
Suite 278/3305 green, build exit 0.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SmNjJAwEnqHPtXbvSZR8kA
|
||
|
builtbykev
|
b742230d94 |
Phase 1: ship the backup cron as CODE + a manual regrade trigger
FOUNDATION-FIRST re-order, phase 1 (tooling + safety). BACKUP (highest-severity open item) — INSTALLED, not re-proven. src/backupScheduler.js runs scripts/backup-db.sh nightly from inside the API container, armed at boot in server.js. The container already has SUPABASE_DB_URL, pg_dump and the Supabase route, so deploy == installed: no host crontab, no Coolify click. Arming is deliberately opt-OUT (armed whenever SUPABASE_DB_URL exists; BACKUP_CRON=0 kills it) because the S62 design was opt-in and nobody ever opted in — the DB went unbacked every night for weeks. A failed run pages high-priority ntfy; silence is the danger with backups. Durability is the one part still needing a human: the container FS is ephemeral, so a dump dies on redeploy unless BACKUP_REMOTE (off-box rsync) or BACKUP_DIR (persistent volume) is set. The scheduler detects that and pages a WARNING at boot rather than letting an undurable backup read as "backed up". Runbook rewritten to lead with the code path. MANUAL REGRADE TRIGGER — scripts/run-snapshot.js, runnable via docker exec with no VYNDR_INTERNAL_KEY and no new HTTP surface. Runs the SAME snapshotService.runSnapshot the cron runs (including the team-stats refresh that powers opp_rank_stat), supports `all` and `--settle`, and prints the grade/confidence distribution plus p_win/ev_pct presence — which is the thing you actually want when verifying a grading change. ACCESS BLOCKER, logged honestly in specs/model-train.md: there is no VYNDR_INTERNAL_KEY in the local .env and SSH to the box times out from WSL2, so I can neither curl the internal endpoints (which already exist from S45) nor docker exec. The trigger is built and correct but only Kev can run it until a key or SSH access exists. This is the highest-leverage unblock for phases 2 and 3, which both need on-demand regrade+settle to verify anything. Also logged the standing cautions: CLV ledger stays private until backtest-proven; "self-improving model" is unsupported marketing until the loop closes; the engine is MLB/WNBA-calibrated and NFL/NBA/soccer need their own calibration before the hub grades them (scaling gate). Suite 277/3300 green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SmNjJAwEnqHPtXbvSZR8kA |
||
|
builtbykev
|
1a94ef5fcf |
Revive the dead probability layer + restore grade range ON MERIT
Folds re-sequenced steps 1+2 into one change (Kev's call): same bug
family — features wired to sources that return null.
THE PROBABILITY LAYER WAS DEAD IN PRODUCTION. p_win/ev_pct/kelly/
model_odds/value were absent on 0/8 live grades because
gameLogService.getGameLogs returns null for MLB by construction and
depends on the offline Python service for NBA/WNBA, so meta.gameLogs was
[] for every sport. This was the S46 bug in a second location — that fix
gave featureCache an MLB branch (why grades still worked) but never the
estimator. featureCache.getStatRows now supplies normalized rows
([{date,[statType]:v}], most-recent-first) for every sport, feeding the
estimator AND consistency AND game_count_in_7d from one fetch.
VERIFIED on real props: p_win 25/25 WNBA, 8/8 MLB (was 0).
GRADE RANGE, ON MERIT — never by rescaling (permanent founder ruling:
minting A's without new information is a relabelled B sold as an A and
corrupts an append-only ledger).
- refreshTeamStats wired into runSnapshot — it had ZERO production
callers, so opp_rank_stat was permanently null and a +/-1.0 factor
could never fire. Test-env no-op (opsNotify precedent).
- L20 made SYMMETRIC: both branches were delta +1.0, so the season
baseline could only ever ADD. No negative path was a structural reason
D was unreachable. New l20_contradicts_* carries -1.0.
- game_count_in_7d derived from real logged dates (heavy_workload_7d).
- NOT wired, deliberately, with reasons inline: teamId (no team_id
column; getFeatures reads it top-level; factor also needs a starter-id
list) and season_type (ESPN 2 = REGULAR season; threading it raw would
fire veteran_in_playoffs in July). Dead code dressed as a fix is the
thing we are removing, not adding.
CALIBRATION GUARD (found by verifying, not assuming): consistency CV is
NBA-tuned; for a Poisson-ish stat cv ~ 1/sqrt(mean), so any stat with
mean < 4 auto-classifies boom_bust. First verification run showed 8/8 MLB
props boom_bust — a blanket -1.0 that dropped the board to all-C. Floored
at CONSISTENCY_MIN_MEAN=4 -> 'unknown' below. Absent beats wrong. MLB
low-count stats therefore still get no consistency factor: honest, not
fixed. Scale-free index-of-dispersion classifier is the open follow-up.
CONFIDENCE IS NOT A PROBABILITY: payloads carry confidence_basis:
'grade_band'. Corrected mlb-grade-degradation.md — its "25/25
grade<->confidence agreement" is a TAUTOLOGY (confidence is derived FROM
the letter, so it would report 25/25 even if every grade were wrong), not
a validation. Removed dead mlbGrader.js (referenced only by its own test)
and the stale computeFeatures comment claiming a penalty that never ran.
VERIFICATION (scripts/verify-grade-range.js, real props/logs/engine):
WNBA 25 props B 68%->32%, C 32%->64%, D 0->1 (4%); 11-step spread went
from 2 steps to 5 (C/C+/B-/D). The D is earned: Angel Reese assists o2.5,
p_win 0.365. Nothing flooded — grades got HARDER. A did not emit locally
because opp_rank_stat needs the Redis cache only prod populates (local
ceiling +3.0 vs the +4.5 A needs); reachability is proven arithmetically
and locked in tests. Prod A-emission is the outstanding fingerprint.
MARKETING HOLD: "A-RATED" (AccuracyBadge, TopSignals) is unsupported
until that fingerprint. Confirmed honest fallbacks render today —
/api/ledger/accuracy has B and C buckets only, so the badge shows
"MODEL · 63% HIT" and TopSignals self-hides. Nothing fabricated ships.
Suite 276/3286 green, web build exit 0.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SmNjJAwEnqHPtXbvSZR8kA
|
||
|
builtbykev
|
c2c43cdc92 |
Task A — make the container backup-capable + validated dump + mechanism fingerprint
SUPABASE_DB_URL is set in Coolify on the API service, and this WSL2 box can't reach db.<ref>.supabase.co — so the backup runs INSIDE the API container, which has the env + Supabase network. Made that real: - Dockerfile: install postgresql-client (pg_dump/pg_restore) + rsync + bash in the runner image. - backup-db.sh: added an integrity fingerprint on every run — pg_restore --list must parse the archive AND find ledger_entries, else the run FAILS + pages (stronger than the size check; catches a corrupt/structureless dump). - BACKUP-RUNBOOK.md: rewritten for the container-exec reality — host cron does `docker exec <api> sh /app/scripts/backup-db.sh` (inherits env + network + pg_dump), or a Coolify Scheduled Task. Full restore-fingerprint steps included. MECHANISM FINGERPRINT (run locally, docker + pg16): seeded a ledger_entries table (137 rows) → ran backup-db.sh (dump + validate: 22 archive objects, ledger_entries present) → pg_restore into a scratch DB → 137 rows restored, exact match. The dump/validate/restore path is proven end-to-end; it's the same pg_dump/pg_restore that run in the container against Supabase. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
builtbykev
|
5f5c004416 |
Item 2 — nightly pg_dump backup script + runbook (Supabase free tier has none)
scripts/backup-db.sh: nightly full-DB pg_dump via the direct connection string, 14-day local rotation, weekly off-box rsync copy, ntfy alert on any failure + an undersized-dump guard (an empty dump is a silent failure). docs/BACKUP- RUNBOOK.md: the ONE env var Kev must set (SUPABASE_DB_URL — the direct db.<ref>.supabase.co:5432 URI, not the pooler), the cron line, the off-box target (Hetzner Storage Box via rsync, simplest for a Hetzner box), and the restore FINGERPRINT procedure (pg_restore into a scratch DB + count ledger_entries — proves it's a real, restorable backup). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
builtbykev
|
4bd770480a |
Grade-fix validation script + finding doc (before-state, blast-radius SQL, plan)
scripts/validate-grade-fix.js checks the live MLB snapshot for the three degradation signatures (projection=0, edge=100 cluster, grade/conf disagreement) — run after the next 14:00 UTC regrade to fingerprint the fix. The finding doc now records root causes, fixes (commits 888d103/9fc4edf), the blast-radius SQL (box can't reach Supabase directly), and the shared-path note for NBA/WNBA. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
builtbykev
|
2ae8a5697e |
Session 56: Full audit — PropLine + boxscore + pipeline + sport coverage (2289 tests)
Research (verified against live MLB Stats / ESPN / The Odds APIs): - specs/propline-audit.md — every stat_type mapped against our 4-layer pipeline; real MLB boxscore fields; sport coverage status; pipeline gap analysis. - specs/vyndr-roadmap.md — priority-ordered Sessions 57–64 + coverage targets. - scripts/propline-audit.js + specs/audit-data/ (raw capture). Headline bug: oddsNormalizer mapped batter_rbis → 'rbis' while the whole grade/feature/outcome chain keys on 'rbi' — every PropLine RBI prop silently failed to grade AND settle. Fixed (+ regression test). Phase 4 — wired missing MLB stats end-to-end: - PropLine MLB markets 6 → 12 (+runs, walks, doubles, earned_runs, hits_allowed, outs — same request, no extra quota). - doubles/outs/triples added to featureCache + outcomeService MLB_LOG_FIELD and all three grade whitelists (analyze/scan/validation.py). Phase 6 — pipeline resilience: - opsNotify.js: ntfy alerts (never throws, test-disabled). Snapshot success/ stale/failure alerts; retry-once on hard odds error (not on empty slate). - Missed-cron watchdog (mostRecentExpectedSlot/isSnapshotOverdue); status probe now returns `overdue`. Coverage truth: MLB is the only end-to-end-live sport; outcome settlement is MLB-only (WNBA/NBA/soccer never settle) — documented as the #1 roadmap gap. Backend 2276 → 2289 tests (+13). Web build exit 0. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
builtbykev
|
6ab49d4c37 | Session 22: Tracker-driven quota guard, configurable cache TTL (1hr default), opt-in odds prewarmer (1505 tests) | ||
|
builtbykev
|
167996d99a |
Session 15: Intelligence hardening — park factors, weather, Tank01 prefetch, pace factors, signal audit, founder pricing fix (1405 tests)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
builtbykev
|
e5c45ecc8e | Session 10: Internal auth refactor, prefetch cascade keys, Sentry, welcome email (1286 tests) | ||
|
builtbykev
|
ad5ea8d5a8 | Session 7j: Soccer intelligence - 9 leagues, 11 signals, 6 traps, poller, prefetch, 131 new tests (1173 total) | ||
|
builtbykev
|
012c0ef47e | Session 7e: Grade adapter, normalize consolidation, ARCH-2 banners | ||
|
builtbykev
|
6f4a353de9 | Session 7d: Audit fixes - rate limiting, error leak, parallel parlays, analyze cache, bundle analyzer | ||
|
builtbykev
|
5c44922937 | Session 7b: Fix pipeline - body parser, Redis queueing, poller visibility, auto-start | ||
|
builtbykev
|
1fa04dc776 | Sessions 5-7a: 955 tests, deployment ready | ||
|
builtbykev
|
3da1b4242c |
feat: Feature 1.2 (NBA stats FastAPI service) + Feature 1.4 (database schema)
Feature 1.2: Python FastAPI microservice wrapping nba_api - GET /stats/season-avg, /stats/last-n, /stats/splits, /players/search - Redis caching (24hr/1hr/6hr/7day), 0.6s rate limiting, PRA derived stat - 27 Python tests passing Feature 1.4: Complete Supabase database schema - 6 tables: users, picks, scan_sessions, bets, outcomes, performance - RLS enabled on all tables with auth.uid() policies - 3 triggers: auto-create user, updated_at, scan count reset - 37 schema validation tests passing - Migration SQL ready, pending manual apply (WSL2 DNS blocker) Total: 92 tests (65 Node.js + 27 Python), all passing Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |