Read integrity, as-of context, and the shadow matchup resolve (A1-A7)

Seven orders of measurement-first repair. The served grade does not move.

A0/A1 — the unordered page walk returned the right COUNT and the wrong ROWS:
410-617 of 2,490 duplicated with an equal number never returned, while
rows.length matched the server exactly. safePaginate orders on a real unique
key, verifies the tuple at runtime, and THROWS on a query error instead of
treating it as end-of-data. Both hits PROVES are withdrawn: they were drawn
through that reader, and defense_by_direction's distinct-n was likely below
the gate floor all along.

A2/A2b — rolled across every reader: 11 FAIL -> 0. Composite keys pulled from
pg_index (the context tables are dated-composite and had no single unique
column). The unordered helper is deleted, not parked.

A3 — ledgerService and retentionService defaulted the SAME env var to
DIFFERENT versions, so no ledger row ever carried the marker eligibility
requires. One source now. model_snapshots settlement moved onto the cron:
15,484 -> 28,894 settled, repaired-champion 0 -> 7,556.

A4 — hitsFactorContext takes an as-of cutoff. Refusal over reconstruction: no
row at-or-before the date means the factor does not apply, never the nearest
row. Live path unchanged, proven 400/400 on real rows.

A5 — factor_inputs freezes what the factor READ, never the multiplier, so an
audit can recompute and check. It also recorded the finding: the three hits
factors have NEVER fired. prop.opponent and prop.opposing_pitcher are read by
the resolver and written by nothing.

A6/A7 — matchupKeys resolves those keys from the posted lineup plus the
schedule's probable pitchers, and fires the factors into a SHADOW freeze:
248 fires on 308 props, 245 of which would move the grade. The served
forecast is untouched. specs/a8-shadow-factor-gate.md pre-registers the test
that decides whether they ever go live.

Nothing is turned on. CALIBRATION_DEPLOYED stays []. Both verdicts stay
withdrawn. 4,772 tests / 371 suites green, web build exit 0, read-integrity
harness 34/34.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Kev
2026-08-11 22:49:56 -04:00
parent 387ae4d54e
commit f61ec6b391
49 changed files with 4874 additions and 308 deletions
+139
View File
@@ -0,0 +1,139 @@
'use strict';
/**
* Fix A2 — the learning readers do not regress to the unordered walk.
*
* WHAT THIS TEST IS AND IS NOT. It does NOT assert that a reader is correct —
* correctness is measured by `scripts/read-integrity.js` against an ordered
* control on live data, and a source grep can never stand in for that (the
* meta-scar rule, specs/read-integrity-harness.md §2).
*
* What it DOES assert is structural: the two tables measured corrupt
* (`ledger_entries`, `model_snapshots`) are never read through the legacy
* `page()` helper again, the scripts stay importable without executing, and the
* named READS the harness measures still exist. Those are the properties that,
* if they silently regressed, would make the next harness run measure the wrong
* thing.
*/
const fs = require('fs');
const path = require('path');
const SCRIPTS = [
'prove-hit-factors', 'prove-tb-factors', 'cluster-prove',
'tb-solo-and-interactions', 'build-grade-bands', 'proven-status',
'champion-ablation',
// A2b — the context readers and the two write-scripts
'prove-runs-rbi', 'pitcher-prove-k', 'prove-park-weather', 'skill-v1-stagea',
'stagea-gate-run', 'backfill-context', 'reconstruct-game-environment',
'calibrate-hits',
];
/** Every table these scripts page through must have a recorded unique key. */
const KEYED_TABLES = ['ledger_entries', 'model_snapshots', 'statcast_aggregates',
'batter_spray', 'team_defense', 'platoon_splits', 'park_dimensions',
'hitter_opportunity', 'lineup_context', 'game_context'];
const CORRUPTED_TABLES = ['ledger_entries', 'model_snapshots'];
/**
* READS DELIBERATELY LEFT ON THE LEGACY WALK — named, with the reason.
*
* The A2 order scoped the fix to the eleven readers measured FAIL and said
* explicitly to leave the siblings that measured 0%. Those siblings are listed
* here rather than excluded by a loosened regex, so the gap is VISIBLE and
* tracked instead of invisible.
*
* They are not safe by construction — they are clean by plan. The same query on
* `ledger_entries` measured 16.5% one minute and 24.8% the next, so a 0% reading
* is a fact about today, not a property. These should move in a follow-up.
*/
const KNOWN_LEGACY_READS = {
// A2b closed the last exceptions: the three reads A2 left on the legacy walk
// are converted, and the unordered helper is DELETED from every script rather
// than parked beside the safe one — a dead broken helper is an invitation.
};
const read = (name) => fs.readFileSync(path.join(__dirname, '..', '..', 'scripts', `${name}.js`), 'utf8');
describe.each(SCRIPTS)('scripts/%s.js', (name) => {
const src = read(name);
const allowed = KNOWN_LEGACY_READS[name] || [];
it('never reads a keyed table through an unordered walk', () => {
for (const table of KEYED_TABLES) {
// `page(sb, 'ledger_entries'` — the unordered walk. `pageSafe(...)` is fine.
expect(src).not.toMatch(new RegExp(`[^e]page\\(\\s*sb,\\s*'${table}'`));
}
expect(allowed).toHaveLength(0);
});
it('carries NO unordered page() helper at all — the landmine is deleted', () => {
const code = src.replace(/\/\/.*/g, '').replace(/\/\*[\s\S]*?\*\//g, '');
const defs = code.match(/async function page\(/g) || [];
for (const _ of defs) {
// A surviving helper must itself order; an unordered one must not exist.
expect(code).toMatch(/paginate\(/);
}
});
it('looks its key up from the schema rather than assuming id', () => {
expect(src).toMatch(/uniqueKeyFor/);
});
it('routes its fixed reads through safePaginate, not a hand-rolled walk', () => {
expect(src).toMatch(/require\('\.\.\/src\/utils\/safePaginate'\)/);
expect(src).toMatch(/async function pageSafe\(/);
// pageSafe must delegate — a second walk implementation is the thing the
// A2 order explicitly forbade.
expect(src).toMatch(/return paginate\(/);
});
it('exports the named READS the harness measures', () => {
const mod = require(path.join('..', '..', 'scripts', name)); // must not execute
expect(mod.READS).toBeDefined();
for (const fn of Object.values(mod.READS)) expect(typeof fn).toBe('function');
});
it('does NOT run main() on require — importing it must not hit the network or write', () => {
// prove-hit-factors and friends write to mc_test_ledger via
// tl.recordAndCount, which is the Bonferroni denominator. An unguarded
// entrypoint would let the harness inflate the correction just by measuring.
expect(src).toMatch(/if \(require\.main === module\)/);
expect(src).not.toMatch(/^main\(\)\.catch/m);
});
});
describe('the calibration services load rows through safePaginate', () => {
it.each([
['calibrationService', '../../src/services/model/calibrationService'],
['lowParamService', '../../src/services/model/lowParamService'],
])('%s exports loadSettledRows', (label, modPath) => {
const mod = require(modPath);
expect(typeof mod.loadSettledRows).toBe('function');
});
it.each([
['calibrationService', 'src/services/model/calibrationService.js'],
['lowParamService', 'src/services/model/lowParamService.js'],
])('%s no longer swallows a query error as end-of-data', (label, rel) => {
const src = fs.readFileSync(path.join(__dirname, '..', '..', rel), 'utf8');
// The :117 defect, in CODE. Anchored to line-start so the comment that
// documents the defect does not count as the defect.
expect(src).not.toMatch(/^\s*if \(error \|\| !data/m);
expect(src).toMatch(/paginate\(/);
});
});
describe('A2 did not open anything it was not meant to', () => {
it('CALIBRATION_DEPLOYED is still empty — no stat serves a calibrated number', () => {
const snapshotService = require('../../src/services/snapshotService');
expect(snapshotService.CALIBRATION_DEPLOYED).toEqual([]);
});
it('both hits verdicts remain withdrawn', () => {
const wv = require('../../src/services/model/withdrawnVerdicts');
expect(wv.isWithdrawn('mlb', 'hits', 'defense_by_direction')).toBe(true);
expect(wv.isWithdrawn('mlb', 'hits', 'pitcher_contact_profile')).toBe(true);
});
});