builtbykev/vyndr
1
0
Fork 0
Code Issues Pull Requests Actions 5 Packages Projects Releases Wiki Activity

chaining-v1: the portable chain, and the gate that blocks the parlay surface

Browse Source 
The order's own prerequisite for the hit-parlay surface was to verify the hit
probability is calibrated. It is not, and the failure is exactly the shape that
destroys a parlay.

Measured on 972 settled hits props: the model is monotonically over-confident
at the top and flat above 0.70. Predicted 0.911 comes back 0.630. Predicted
0.844 comes back 0.630. Predicted 0.747 comes back 0.605. There is no
discrimination at all in the range a parlay is built from, and the error runs
in the flattering direction. Four "91%" legs are 0.686 by the model and 0.157
in fact -- a 4.4x overstatement that compounds with every leg added.

Single props survive a calibration error of that size. A parlay multiplies it.
So chainAcross REFUSES to compound atoms not marked calibrated, and refusing is
the feature rather than a limitation: a ticket built on these numbers would be
confidently wrong in the direction the user pays for.

calibration.js provides the reliability table, the gate (tolerance 0.05,
weighted to the high end because that is where tickets live) and an isotonic
fit. Isotonic is the honest repair here because it is monotone: the model's
ordering survives untouched while the numbers move to what actually happened.
The fitted map says 0.65 -> 0.594, 0.85 -> 0.639, 0.91 -> 0.639.

chain.js is the portable core -- base events plus context, through a chain
function, into a PLUGGABLE aggregator: across players for a compound ticket, up
to the team for expected scoring. The sport-specific parts are inputs rather
than code paths, so basketball plugs in as content. The archetype
redistribution hook is there now, dormant in baseball because a nine-run lead
does not change who bats next, and live in basketball where a blowout fades the
star and feeds the bench.

Two judgement calls worth naming. Treating same-game legs as independent errs
in the FLATTERING direction, since they share pitcher, park and weather -- so
correlation shifts the compound toward the weakest leg, bounded, and is labelled
an approximation rather than a joint distribution. And market divergence does
NOT downgrade confidence: it flags a contested script whose props are either the
best or the worst on the board, and which one is unknown until settled.
Internal inconsistency does downgrade it, because per-entity reads failing to
sum to the team read means one of them is wrong and we do not know which.

Not built: the independent game-script projection. It needs proven team-level
atoms and out-of-sample validation against actual margins, and no atom has
passed the gate yet. Building it now would produce something plausible rather
than something proven, which is the failure mode this whole programme exists to
avoid.

4,269 tests green (339 suites); web build exit 0; counter and frozen clusters
byte-identical.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W1sivYNqY2TS5ftykmHBU9
This commit is contained in:
Kev
2026-08-04 17:17:14 -04:00
parent a80a775fa0
commit 9c5b968351
5 changed files with 608 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/services/model/calibration.js
+156
View File
@@ -0,0 +1,156 @@
'use strict';
/**
* calibration — DOES 78% MEAN 78%?
*
* Resolution asks whether higher forecasts hit more often. Calibration asks
* whether the NUMBER is true. A model can rank perfectly and still be useless
* for anything compounded, and the two failures look nothing alike.
*
* ── WHY THIS GATES THE PARLAY SURFACE ────────────────────────────────────
* Measured on 971 settled MLB hits props (2026-08-04):
*
* predicted 0.456 → actual 0.493 (under-confident)
* predicted 0.645 → actual 0.614
* predicted 0.747 → actual 0.605 over by 0.142
* predicted 0.844 → actual 0.630 over by 0.214
* predicted 0.911 → actual 0.630 over by 0.281
*
* Above 0.70 the model is FLAT at ~63% — it has no discriminating power there at
* all, and it is severely over-confident. Single props survive this; a parlay
* does not. Four "90%" legs:
*
* model 0.91^4 = 0.686 reality 0.63^4 = 0.157
*
* A 4.4x overstatement, and it compounds with every leg. Errors that are
* survivable one at a time multiply when chained, which is why chaining must be
* GATED on calibration rather than merely warned about.
*
* ── THE FIX IS A MAP, NOT A MODEL CHANGE ─────────────────────────────────
* Isotonic (monotone) regression from predicted → realized, fitted on settled
* outcomes. It preserves the model's ORDERING — the ranking it does have is
* untouched — while correcting the numbers to what actually happened. This is
* the honest repair for a model that ranks better than it counts.
*
* POINT-IN-TIME: a calibration map must be fitted on outcomes that PRECEDE the
* prop it is applied to, or it has seen the answer. `fitIsotonic` takes whatever
* it is given; the caller is responsible for the cut, and `calibrationReport`
* carries the window so the cut is visible.
*/
const { knownNumber } = require('../../utils/known');
const MIN_BIN = 15; // below this a bin is noise, not a reading
const MIN_TOTAL = 200; // below this there is no calibration to speak of
/** Max tolerable |predicted − actual| in any populated bin, for the gate. */
const MAX_BIN_ERROR = 0.05;
/**
* Reliability table: bin the forecasts and compare each bin's mean prediction
* with what actually happened in it.
*/
function reliability(rows, bins = 10) {
const buckets = Array.from({ length: bins }, () => ({ n: 0, sumP: 0, wins: 0 }));
for (const r of rows || []) {
const p = knownNumber(r && r.p);
const won = knownNumber(r && r.won);
if (p === null || won === null) continue; // absent, never assumed
const idx = Math.min(bins - 1, Math.max(0, Math.floor(p * bins)));
const b = buckets[idx];
b.n += 1; b.sumP += p; b.wins += won > 0 ? 1 : 0;
}
return buckets
.map((b, i) => (b.n === 0 ? null : {
bin: i,
n: b.n,
mean_predicted: b.sumP / b.n,
actual: b.wins / b.n,
error: b.sumP / b.n - b.wins / b.n,
}))
.filter(Boolean);
}
/**
* THE GATE. Is this forecast safe to compound?
*
* Deliberately strict, and deliberately weighted toward the HIGH end: a parlay
* is built out of confident legs, so an error at 0.9 is the one that actually
* costs money. A model may be perfectly usable for single props and still fail
* here — those are different questions and this returns the chaining answer.
*/
function isCalibrated(rows, opts = {}) {
const minBin = opts.minBin ?? MIN_BIN;
const maxErr = opts.maxBinError ?? MAX_BIN_ERROR;
const table = reliability(rows, opts.bins ?? 10).filter((b) => b.n >= minBin);
const total = (rows || []).length;
if (total < (opts.minTotal ?? MIN_TOTAL)) {
return { calibrated: false, reason: 'insufficient_sample', n: total, table };
}
if (table.length === 0) {
return { calibrated: false, reason: 'no_populated_bins', n: total, table };
}
const worst = table.reduce((a, b) => (Math.abs(b.error) > Math.abs(a.error) ? b : a));
const highEnd = table.filter((b) => b.mean_predicted >= 0.70);
const worstHigh = highEnd.length
? highEnd.reduce((a, b) => (Math.abs(b.error) > Math.abs(a.error) ? b : a))
: null;
const ok = Math.abs(worst.error) <= maxErr;
return {
calibrated: ok,
reason: ok ? null : 'bin_error_exceeds_tolerance',
n: total,
max_bin_error: worst.error,
worst_bin: worst,
worst_high_confidence_bin: worstHigh,
tolerance: maxErr,
table,
};
}
/**
* Isotonic (pool-adjacent-violators) regression: predicted → realized, monotone
* non-decreasing. Returns the fitted step points.
*
* Monotone by construction, so the model's ORDERING survives untouched — only
* the numbers move. That is the point: we are repairing what it counts, not
* what it ranks.
*/
function fitIsotonic(rows, opts = {}) {
const pts = (rows || [])
.map((r) => ({ p: knownNumber(r && r.p), won: knownNumber(r && r.won) }))
.filter((r) => r.p !== null && r.won !== null)
.sort((a, b) => a.p - b.p);
if (pts.length < (opts.minTotal ?? MIN_TOTAL)) return null;
// Each observation starts as its own block; merge while monotonicity is violated.
const blocks = pts.map((r) => ({ sumY: r.won > 0 ? 1 : 0, n: 1, lo: r.p, hi: r.p }));
const stack = [];
for (const b of blocks) {
stack.push({ ...b });
while (stack.length > 1) {
const top = stack[stack.length - 1];
const prev = stack[stack.length - 2];
if (prev.sumY / prev.n <= top.sumY / top.n) break;
stack.pop(); stack.pop();
stack.push({ sumY: prev.sumY + top.sumY, n: prev.n + top.n, lo: prev.lo, hi: top.hi });
}
}
return stack.map((b) => ({ lo: b.lo, hi: b.hi, value: b.sumY / b.n, n: b.n }));
}
/** Apply a fitted map. Outside the fitted range, the nearest block governs. */
function applyIsotonic(map, p) {
const x = knownNumber(p);
if (x === null || !Array.isArray(map) || map.length === 0) return null;
if (x <= map[0].hi) return map[0].value;
for (const b of map) if (x >= b.lo && x <= b.hi) return b.value;
for (let i = 0; i < map.length - 1; i += 1) {
if (x > map[i].hi && x < map[i + 1].lo) return map[i + 1].value;
}
return map[map.length - 1].value;
}
module.exports = {
reliability, isCalibrated, fitIsotonic, applyIsotonic,
MIN_BIN, MIN_TOTAL, MAX_BIN_ERROR,
};
src/services/model/chain.js
+226
View File
@@ -0,0 +1,226 @@
'use strict';
/**
* chain — THE PORTABLE BASE-EVENT CHAIN.
*
* One idea, read two ways:
*
* base_events + situational_context → chain_fn → AGGREGATOR
*
* ACROSS players → compound ticket probability (parlay)
* UP to the team → expected scoring / game-script
*
* The atoms are the same either way. A hitter's P(hit) is a leg of a ticket when
* read across, and a component of his team's expected runs when read up. That is
* the whole reason this is one machine rather than two: an improvement to an
* atom improves everything that reads it, and an inconsistency between the two
* readings is a signal that something is wrong.
*
* ── WHAT IS PLUGGABLE, AND WHY ───────────────────────────────────────────
* Baseball fills this now; basketball must plug in WITHOUT a rebuild. So the
* sport-specific parts are inputs, not code paths:
*
* atoms — which base events exist for this sport
* context — the situational modifiers
* chainFn — how an atom becomes a per-entity probability
* aggregator — ACROSS (compound) or UP (sum to team)
* redistribute — the archetype-redistribution HOOK: a blowout moves
* involvement between archetypes (fades the star, feeds
* the bench). DORMANT in baseball — a nine-run lead does
* not change who bats next — and LIVE in basketball,
* where it is most of the edge. The hook exists here so
* basketball is content rather than a rewrite.
*
* ── CALIBRATION IS A HARD PRECONDITION, NOT A WARNING ────────────────────
* Errors that are survivable one at a time MULTIPLY when chained. Measured on
* real hits props: at a predicted 0.911 the realized rate is 0.630, so four such
* legs are 0.686 by the model and 0.157 in fact — a 4.4x overstatement that
* compounds with every leg added. So `chainAcross` REFUSES to compound atoms
* that are not marked calibrated. Refusing is the feature; a parlay built on
* uncalibrated probabilities is the single most harmful thing this product could
* ship, because it is confidently wrong in the direction the user pays for.
*
* ── SELF-CHECK ───────────────────────────────────────────────────────────
* Reading the same atoms two ways gives a free consistency test. If the
* per-player reads do not sum to the team read, one of them is wrong and we do
* not yet know which — so the correct output is LOW CONFIDENCE, pre-game, rather
* than a confident number from whichever path we happened to trust. A model that
* knows which of its own calls to distrust can grade selectively.
*/
const { knownNumber } = require('../../utils/known');
/** An atom that cannot be read is ABSENT — never a zero probability. */
function usableAtoms(atoms) {
return (atoms || []).filter((a) => a && knownNumber(a.p) !== null);
}
/**
* CHAIN ACROSS — compound probability of every leg landing.
*
* @param {Array} atoms [{ id, p, calibrated, gameId, entityId, ... }]
* @param {object} opts
* correlation(a, b) → 0..1 shared-variance estimate between two legs
* requireCalibrated (default TRUE) — see the header
* @returns {object|null} refusal is explicit and reasoned, never a silent 0
*/
function chainAcross(atoms, opts = {}) {
const requireCalibrated = opts.requireCalibrated !== false;
const legs = usableAtoms(atoms);
if (legs.length === 0) return { ok: false, reason: 'no_usable_atoms' };
if (requireCalibrated) {
const uncal = legs.filter((l) => l.calibrated !== true);
if (uncal.length > 0) {
return {
ok: false,
reason: 'uncalibrated_atoms',
detail: 'compounding multiplies calibration error; a leg whose stated probability is not its realized rate makes the ticket confidently wrong',
uncalibrated: uncal.map((l) => l.id),
legs: legs.length,
};
}
}
// Independent product first, then a correlation discount. Same-game legs share
// the pitcher, the park and the weather, so treating them as independent
// OVERSTATES the ticket — the error runs in the flattering direction, which is
// exactly the one to be careful about.
const corrFn = typeof opts.correlation === 'function' ? opts.correlation : () => 0;
let independent = 1;
for (const l of legs) independent *= Math.min(1, Math.max(0, Number(l.p)));
let pairs = 0;
let corrSum = 0;
for (let i = 0; i < legs.length; i += 1) {
for (let j = i + 1; j < legs.length; j += 1) {
const c = Number(corrFn(legs[i], legs[j]));
if (Number.isFinite(c)) { corrSum += Math.min(1, Math.max(0, c)); pairs += 1; }
}
}
const meanCorr = pairs > 0 ? corrSum / pairs : 0;
// Positive correlation makes the JOINT more likely than independence implies
// (legs tend to land together), so the adjustment moves toward the weakest leg
// — bounded, and stated as an approximation rather than a derivation.
const weakest = Math.min(...legs.map((l) => Number(l.p)));
const compound = independent + meanCorr * (weakest - independent);
return {
ok: true,
legs: legs.length,
independent_probability: round4(independent),
mean_pairwise_correlation: round4(meanCorr),
compound_probability: round4(Math.min(1, Math.max(0, compound))),
cross_game_legs: new Set(legs.map((l) => l.gameId)).size,
correlation_caveat: 'pairwise mean, applied as a bounded shift toward the weakest leg — an approximation, not a joint distribution',
};
}
/**
* CHAIN UP — sum atoms to a team-level expectation (expected scoring).
*
* `redistribute` is the archetype hook. It receives the atoms and the context
* and may return a reweighted set — dormant in baseball, live in basketball.
*/
function chainUp(atoms, opts = {}) {
let legs = usableAtoms(atoms);
if (legs.length === 0) return { ok: false, reason: 'no_usable_atoms' };
let redistributed = false;
if (typeof opts.redistribute === 'function') {
const out = opts.redistribute(legs, opts.context || {});
if (Array.isArray(out) && out.length > 0) { legs = usableAtoms(out); redistributed = true; }
}
const weightOf = (a) => {
const w = knownNumber(a.weight);
return w === null ? 1 : w; // absent weight contributes once, not zero
};
const expected = legs.reduce((s, a) => s + Number(a.p) * weightOf(a), 0);
return {
ok: true,
contributors: legs.length,
expected_value: round4(expected),
redistributed,
};
}
/**
* SELF-CHECK — do the two readings of the same atoms agree?
*
* A divergence does not tell us WHICH reading is wrong, so the honest output is
* a confidence downgrade rather than a correction. Pre-game, that is exactly
* what the product needs: the model flagging its own suspect calls.
*/
function selfCheck({ perEntity, teamRead, marketRead = null, tolerance = 0.15 } = {}) {
const flags = [];
const sum = (perEntity || []).reduce((s, a) => {
const p = knownNumber(a && a.p);
return p === null ? s : s + p * (knownNumber(a.weight) ?? 1);
}, 0);
const team = knownNumber(teamRead);
let internalDivergence = null;
if (team !== null && (perEntity || []).length > 0) {
const denom = Math.max(1e-9, Math.abs(team));
internalDivergence = Math.abs(sum - team) / denom;
if (internalDivergence > tolerance) {
flags.push({
flag: 'INTERNAL_INCONSISTENCY',
detail: `per-entity reads sum to ${round4(sum)} against a team read of ${round4(team)}`,
consequence: 'one of the two is wrong and we do not know which — every prop drawing on these atoms is LOW CONFIDENCE',
});
}
}
// Divergence from the market is NOT an error signal — it is the point of
// projecting independently. It flags the game script as one where our read and
// the market's disagree, which is where a mispricing would live IF we are
// right. It never means we are right.
let marketDivergence = null;
const mkt = knownNumber(marketRead);
if (mkt !== null && team !== null) {
marketDivergence = team - mkt;
if (Math.abs(marketDivergence) > tolerance * Math.max(1, Math.abs(mkt))) {
flags.push({
flag: 'SCRIPT_DIVERGES_FROM_MARKET',
detail: `our projection ${round4(team)} vs market ${round4(mkt)}`,
consequence: 'the game script is contested — props downstream of it are either the best or the worst on the board, and which one is unknown until settled',
});
}
}
return {
per_entity_sum: round4(sum),
team_read: team,
internal_divergence: internalDivergence === null ? null : round4(internalDivergence),
market_divergence: marketDivergence === null ? null : round4(marketDivergence),
tolerance,
flags,
confidence: flags.some((f) => f.flag === 'INTERNAL_INCONSISTENCY') ? 'LOW' : 'NORMAL',
};
}
/**
* PROPAGATION — a settled outcome corrects the atom responsible, and therefore
* every reading that shares it.
*
* Shrinkage-weighted by sample: one game moves a 400-observation atom barely at
* all and a 5-observation atom a lot, which is the difference between learning
* and chasing noise. `priorWeight` is the number of observations the existing
* estimate is worth.
*/
function propagate(atom, observation, opts = {}) {
const prior = knownNumber(atom && atom.p);
const obs = knownNumber(observation && observation.won);
if (prior === null || obs === null) return atom; // nothing to learn from
const n = knownNumber(atom && atom.n) ?? 0;
const priorWeight = knownNumber(opts.priorWeight) ?? Math.max(1, n);
const updated = (prior * priorWeight + (obs > 0 ? 1 : 0)) / (priorWeight + 1);
return { ...atom, p: round4(updated), n: n + 1, last_updated_from: observation.id ?? null };
}
const round4 = (v) => (Number.isFinite(v) ? Math.round(v * 10000) / 10000 : v);
module.exports = { chainAcross, chainUp, selfCheck, propagate, usableAtoms };