929fd81940895b1e73087609c21b3fe88135a4ea
2 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 builtbykev
|
4d1803f6d7 |
Calibrate hits point-in-time: partial pass, and an honest ceiling of 0.667
Fitted the isotonic map on game_date < 2026-08-02 (n=589) and evaluated it on everything from that date forward (n=383). The map never saw the evaluation rows, which is the only thing that makes the result mean anything -- fitting and evaluating on the same rows always looks perfectly calibrated, because the map is reciting the answers it was built from. It works, on most of the distribution. Held-out after correction: 0.477 comes back 0.506, 0.587 comes back 0.580, 0.667 comes back 0.603 -- against raw errors of +0.191, +0.279 and +0.246 in the same bins. Ordering survived, and that was verified pairwise rather than assumed, because a broken map would silently destroy the one thing this model does well. Two findings matter more than the pass. First, the honest ceiling is 0.667. Once the numbers are truthful this model has no 80%-plus hit reads at all -- the top of its range was miscalibration, not confidence. A four-leg ticket at the ceiling is 0.198, where the raw numbers implied 0.686. The high-floor parlay is a two-thirds-per-leg proposition, and that is the number to say out loud. Second, calibration is certified BY BAND rather than by a blanket flag. Held-out error was -0.029 and +0.007 through the middle but -0.167 at the bottom and +0.063 at the top: the model is trustworthy over most of its mass and untrustworthy at both edges. A single true/false would either throw away the 72% that works or ship the edges that do not. Only a probability inside a certified band is marked stackable, and that flag is what chainAcross requires before it will compound anything. The certified band is 0.40 to 0.60, n=276. A methodological catch on the way: my first pass condition demanded honest bins at 0.70 and above -- but honest calibration REMOVES those bins, since the ceiling drops to 0.667. The gate would have failed the repair for succeeding. It now tests the highest remaining band instead of a fixed threshold. Wired forward with the same discipline: calibrationService fits strictly before today, splits by time rather than at random, and returns null on thin history so that "no calibrator" means nothing is stackable rather than "trust the raw numbers". p_win is never mutated -- the calibrated value rides beside it as p_win_calibrated, because a calibration map is a correction to a forecast, not a different forecast, and the counter stays byte-identical. 4,275 tests green (339 suites); web build exit 0. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W1sivYNqY2TS5ftykmHBU9 |
||
|
builtbykev
|
9c5b968351 |
chaining-v1: the portable chain, and the gate that blocks the parlay surface
The order's own prerequisite for the hit-parlay surface was to verify the hit probability is calibrated. It is not, and the failure is exactly the shape that destroys a parlay. Measured on 972 settled hits props: the model is monotonically over-confident at the top and flat above 0.70. Predicted 0.911 comes back 0.630. Predicted 0.844 comes back 0.630. Predicted 0.747 comes back 0.605. There is no discrimination at all in the range a parlay is built from, and the error runs in the flattering direction. Four "91%" legs are 0.686 by the model and 0.157 in fact -- a 4.4x overstatement that compounds with every leg added. Single props survive a calibration error of that size. A parlay multiplies it. So chainAcross REFUSES to compound atoms not marked calibrated, and refusing is the feature rather than a limitation: a ticket built on these numbers would be confidently wrong in the direction the user pays for. calibration.js provides the reliability table, the gate (tolerance 0.05, weighted to the high end because that is where tickets live) and an isotonic fit. Isotonic is the honest repair here because it is monotone: the model's ordering survives untouched while the numbers move to what actually happened. The fitted map says 0.65 -> 0.594, 0.85 -> 0.639, 0.91 -> 0.639. chain.js is the portable core -- base events plus context, through a chain function, into a PLUGGABLE aggregator: across players for a compound ticket, up to the team for expected scoring. The sport-specific parts are inputs rather than code paths, so basketball plugs in as content. The archetype redistribution hook is there now, dormant in baseball because a nine-run lead does not change who bats next, and live in basketball where a blowout fades the star and feeds the bench. Two judgement calls worth naming. Treating same-game legs as independent errs in the FLATTERING direction, since they share pitcher, park and weather -- so correlation shifts the compound toward the weakest leg, bounded, and is labelled an approximation rather than a joint distribution. And market divergence does NOT downgrade confidence: it flags a contested script whose props are either the best or the worst on the board, and which one is unknown until settled. Internal inconsistency does downgrade it, because per-entity reads failing to sum to the team read means one of them is wrong and we do not know which. Not built: the independent game-script projection. It needs proven team-level atoms and out-of-sample validation against actual margins, and no atom has passed the gate yet. Building it now would produce something plausible rather than something proven, which is the failure mode this whole programme exists to avoid. 4,269 tests green (339 suites); web build exit 0; counter and frozen clusters byte-identical. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W1sivYNqY2TS5ftykmHBU9 |