Software watches coverage now, and ancestry becomes a product contract

Three closeout items, traced before building.

THE OBSERVER WAS DIAGNOSTICS, NOT MONITORING. Traced from the deployed tree:
exactly two callsites, both manual internal routes, and nothing in the scheduler
or ops path consumed it. A persistent lineage failure could have sat unnoticed
until a human asked.

The monitor now runs on the scheduler's per-minute tick, throttled to 30
minutes. It is placed there rather than after a snapshot on purpose: an
in-snapshot audit structurally cannot report that no snapshot ran, which is the
failure mode that matters most, and it would run under peak write contention
where the audit already demonstrably times out. It reads only the durable
observer and never `attachLineage`'s counters, and every failure path is
swallowed — a monitor that can take down the pipeline it watches is worse than
no monitor.

HEALTHY IS SILENCE; EVERYTHING ELSE SPEAKS. `coverageAlarm` is pure, so the
policy is testable and cannot drift into the scheduler. AUDIT_UNAVAILABLE says
"could not be measured — the audit did not run", deliberately worded so it can
never be read as "coverage is zero": those are different claims and collapsing
them is how a monitor starts lying in the reassuring direction. Alerts dedupe on
(health, cohort) so a standing fault states itself once and a NEW cohort with
the same fault speaks again.

ANCESTRY BECOMES A PRODUCT CONTRACT. It was internal-only. `GET
/api/ancestry/ledger/:id` (requireAuth, rate-limited) plus the Next proxy that
makes it browser-reachable, keyed on the LEDGER ROW ID — a stable identifier the
ledger API already returns — rather than a raw natural key exposed because it
was convenient. Its own router, so `routes/ledger.js` stays free of lineage
entirely and the grade-badge guard keeps its teeth. Every response declares
`authority: LOCAL, authority_scope: ANCESTRY_ONLY`.

THREE TEETH CAME BACK GREEN AND ALL THREE WERE MY TESTS, NOT SAFE DEFECTS.
The badge guard was CASE-SENSITIVE, so `LINEAGE_ANCESTRY` and `readAncestry`
walked straight past it. `try/finally` is valid JavaScript, so removing the
monitor's catch produced no load error and nothing asserted the containment.
And the multi-date cohort check was a grep for `.gte('game_date'` that the
head query satisfied on its own. All three replaced with behavioural tests,
including a scheduler double whose fake client HONOURS its filters — a
pass-through would have made a narrowed cohort walk look correct.

Suite 398/5,522/0 · tsc 0 · web build 0 · teeth 14/14 and 15/15.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CQJeAG8vcDoL5zkiaJyVb8

src/app.js

@@ -125,6 +125,10 @@ app.use('/api/alerts', alertsRoutes);
 app.use('/api/bets', betsRoutes);
 // Session 49 — per-user onboarding preferences (auth-gated, user_metadata).
 app.use('/api/preferences', require('./routes/preferences'));
+// ADDITIVE ancestry contract — lineage is locally authoritative for Read
+// ancestry and nothing else. Its own router so the ledger router stays free of
+// lineage entirely, which is what keeps the grade-shift badge guard meaningful.
+app.use('/api/ancestry', require('./routes/ancestry'));
 app.use('/api/stripe', stripeRoutes);
 app.use('/api/stats', statsRoutes);
 app.use('/api/props', propsRoutes);

src/routes/ancestry.js

@@ -0,0 +1,80 @@
+/**
+ * READ ANCESTRY — the narrow PRODUCT contract for the one truth lineage owns.
+ *
+ * LOCAL AUTHORITY, AND ONLY LOCAL. For this contract lineage IS the source of
+ * truth: which published Read came first, what superseded what, whether a later
+ * observation revised or recaptured the same claim. That says nothing about the
+ * live slate (Redis), the model, the market, the Ledger or settlement, all of
+ * which keep their own authorities untouched.
+ *
+ * IT IS NOT THE GRADE-SHIFT BADGE. `revised_from_grade` answers "did the
+ * model's letter change"; a lineage REVISION answers "did the published claim
+ * change", which includes price. Different questions, different fields,
+ * different route — deliberately not bolted onto the ledger response.
+ *
+ * The identifier is the LEDGER ROW ID: a stable product identifier the ledger
+ * API already returns, rather than a raw database natural key exposed because
+ * it happened to be convenient.
+ */
+
+const express = require('express');
+const { createRateLimit } = require('../middleware/rateLimit');
+const { requireAuth } = require('../middleware/auth');
+const readAncestry = require('../services/read/readAncestry');
+
+const router = express.Router();
+router.use(createRateLimit({ max: 60, windowMs: 60 * 1000 }));
+
+const LEDGER_COLUMNS = 'id, sport, game_date, player_key, player_name, stat, side, line, game_id';
+
+/**
+ * GET /api/ancestry/ledger/:id
+ *
+ * Ancestry for the published Read behind one ledger row. Additive: no existing
+ * response changes, and nothing here is consulted by any other surface.
+ */
+router.get('/ledger/:id', requireAuth, async (req, res) => {
+  const id = Number.parseInt(req.params.id, 10);
+  if (!Number.isFinite(id) || id <= 0) {
+    return res.status(400).json({ error: 'invalid ledger id' });
+  }
+  try {
+    const { getSupabaseServiceClient } = require('../utils/supabase');
+    const supabase = getSupabaseServiceClient();
+    if (!supabase) {
+      return res.status(503).json({ state: readAncestry.ANCESTRY_STATE.LINEAGE_UNAVAILABLE, reason: 'store unavailable' });
+    }
+    const { data, error } = await supabase
+      .from('ledger_entries').select(LEDGER_COLUMNS).eq('id', id).limit(1);
+    if (error) return res.status(500).json({ error: error.message });
+    if (!data || data.length === 0) {
+      return res.status(404).json({ state: readAncestry.ANCESTRY_STATE.NOT_FOUND, reason: 'no such ledger row' });
+    }
+    const row = data[0];
+    const ancestry = await readAncestry.ancestryForRead({
+      sport: row.sport,
+      game_date: row.game_date,
+      player_key: row.player_key,
+      stat: row.stat,
+      side: row.side,
+      line: row.line,
+    });
+    return res.json({
+      ledger_id: row.id,
+      player_name: row.player_name,
+      stat: row.stat,
+      side: row.side,
+      line: row.line,
+      game_date: row.game_date,
+      sport: row.sport,
+      ancestry,
+      // Said on every response. This contract owns ancestry and nothing else.
+      authority: 'LOCAL',
+      authority_scope: 'ANCESTRY_ONLY',
+    });
+  } catch (e) {
+    return res.status(500).json({ error: e.message });
+  }
+});
+
+module.exports = router;

src/services/lineageCoverage.js

@@ -183,6 +183,99 @@ const AUDIT_COLUMNS = [
  * published and recorded nothing surfaces as MISSING_COVERAGE rather than as
  * silence.
  */
+/**
+ * Audit ONE NAMED COHORT — the exact snapshot_id that just completed, across
+ * every game_date it spans.
+ *
+ * The snapshot_id is cohort IDENTITY, not a success claim: expected and covered
+ * are still derived here from durable state, and `attachLineage`'s counters are
+ * never consulted. Passing the id is what makes this "the cohort that just
+ * completed" rather than "whatever is newest", which the multi-date bug showed
+ * are not the same question.
+ */
+async function auditCohort(sport, snapshotId, deps = {}) {
+  const sp = String(sport || '').toLowerCase();
+  const unavailable = (reason) => Object.freeze({
+    audit_available: false, reason, sport: sp, snapshot_id: snapshotId || null,
+    health: HEALTH.AUDIT_UNAVAILABLE,
+  });
+  if (!snapshotId) return unavailable('no_snapshot_id');
+  try {
+    const getClient = deps.getClient || require('../utils/supabase').getSupabaseServiceClient;
+    const supabase = getClient();
+    if (!supabase) return unavailable('no_supabase_env');
+    const nowIso = deps.now ? deps.now() : new Date().toISOString();
+    // Same index discipline as the latest-cohort audit: every read bounded on
+    // game_date, because `snapshot_id` carries no index of its own.
+    const anchor = deps.gameDate || nowIso.slice(0, 10);
+    const lo = shiftDate(anchor, -2);
+    const hi = shiftDate(anchor, 2);
+    const { paginate } = require('../utils/safePaginate');
+    const rows = await paginate(() => supabase
+      .from('model_snapshots')
+      .select(AUDIT_COLUMNS)
+      .eq('sport', sp)
+      .eq('snapshot_id', snapshotId)
+      .gte('game_date', lo)
+      .lte('game_date', hi), { key: 'id', label: 'lineageCoverage.auditCohort' });
+    const dates = [...new Set(rows.map((r) => r.game_date))].sort();
+    const audit = auditRows(rows, {
+      snapshot_id: snapshotId, sport: sp, game_date: dates.join(','),
+      code_sha: rows[0] ? rows[0].code_sha : null,
+      captured_at: rows[0] ? rows[0].captured_at : null,
+    });
+    return Object.freeze({ ...audit, date_window: [lo, hi], game_dates_audited: dates });
+  } catch (e) {
+    return unavailable(e && e.message ? e.message : String(e));
+  }
+}
+
+/**
+ * Should the periodic monitor run now? Pure, so the cadence is testable without
+ * a clock or a scheduler.
+ */
+function coverageDue(lastAtMs, nowMs, everyMs = 30 * 60 * 1000) {
+  if (!Number.isFinite(nowMs)) return false;
+  if (!Number.isFinite(lastAtMs)) return true;
+  return nowMs - lastAtMs >= everyMs;
+}
+
+/**
+ * What, if anything, to say about an audit — pure, so the alert policy is
+ * testable and cannot drift into the scheduler.
+ *
+ * HEALTHY is silence. Everything else speaks, and AUDIT_UNAVAILABLE speaks
+ * DIFFERENTLY from MISSING_COVERAGE: "we could not measure" and "there is
+ * nothing there" are different claims, and collapsing them is how a monitor
+ * starts lying in the reassuring direction.
+ *
+ * Deduped on (health, snapshot_id) so a standing condition states itself once
+ * rather than paging every cycle.
+ */
+function coverageAlarm(prev, audit) {
+  const a = audit || {};
+  const health = a.health || HEALTH.AUDIT_UNAVAILABLE;
+  const key = `${health}|${a.snapshot_id || 'none'}`;
+  if (health === HEALTH.HEALTHY || health === HEALTH.NO_ELIGIBLE_CLAIMS) {
+    return { alert: false, key, message: null, priority: null };
+  }
+  if (prev === key) return { alert: false, key, message: null, priority: null, deduped: true };
+  const where = `${(a.sport || '?').toUpperCase()} cohort ${String(a.snapshot_id || 'unknown').slice(0, 8)}`;
+  const messages = {
+    [HEALTH.MISSING_COVERAGE]: `Lineage coverage MISSING on ${where} — ${a.expected_keys} published Reads, ${a.covered_keys} recorded. The slate published; the record did not. Product is unaffected.`,
+    [HEALTH.PARTIAL_COVERAGE]: `Lineage coverage PARTIAL on ${where} — ${a.covered_keys}/${a.expected_keys} recorded (${a.coverage_pct}%), ${a.missing_keys} missing. Product is unaffected.`,
+    [HEALTH.INVALID_GRAPH]: `Lineage graph INVALID on ${where} — ${a.invalid_partial_actions} partial actions, ${a.graph_defects} defects, ${a.extra_keys} unexpected. Product is unaffected.`,
+    [HEALTH.STALE]: `Lineage coverage STALE — newest audited cohort ${where} is ${Math.round((a.age_ms || 0) / 3600000)}h old. Coverage has stopped advancing.`,
+    [HEALTH.AUDIT_UNAVAILABLE]: `Lineage coverage COULD NOT BE MEASURED (${a.reason || 'unknown'}). This is not a coverage result; the audit did not run.`,
+  };
+  return {
+    alert: true,
+    key,
+    message: messages[health] || `Lineage coverage ${health} on ${where}.`,
+    priority: health === HEALTH.AUDIT_UNAVAILABLE ? 'default' : 'high',
+  };
+}
+
 /** ISO date offset by whole days, for the query-plan window below. */
 function shiftDate(isoDate, days) {
   const d = new Date(`${String(isoDate).slice(0, 10)}T00:00:00Z`);
@@ -278,5 +371,5 @@ async function auditLatestCohort(sport, deps = {}) {
 module.exports = {
   HEALTH, VALID_ACTION_FIELDS, AUDIT_COLUMNS,
   isValidAction, expectedKeys, coveredKeys, graphDefects, classify, shiftDate,
-  auditRows, auditLatestCohort,
+  auditRows, auditLatestCohort, auditCohort, coverageDue, coverageAlarm,
 };

src/snapshotScheduler.js

@@ -158,9 +158,46 @@ function startSnapshotScheduler(opts = {}) {
     } catch { /* the pulse must never break the scheduler */ }
   };
 
+  // ── AUTOMATIC INDEPENDENT LINEAGE COVERAGE MONITOR ──────────────────────
+  //
+  // A protected endpoint that finds MISSING_COVERAGE only after a human asks is
+  // diagnostics, not monitoring. This runs on the per-minute cadence like the
+  // overdue watchdog, throttled, and it is INDEPENDENT of the lineage writer in
+  // both directions: it never sees `attachLineage`'s counters, and it does not
+  // need a snapshot to have completed in this process — so it still speaks when
+  // the writer never ran at all, which an in-snapshot audit structurally cannot.
+  //
+  // It is NOT product infrastructure. Every failure path here is swallowed; a
+  // monitor that can take down the pipeline it watches is worse than no monitor.
+  const lineageCoverage = opts.lineageCoverage || require('./services/lineageCoverage');
+  const COVERAGE_EVERY_MS = Number.parseInt(process.env.LINEAGE_COVERAGE_EVERY_MS || '', 10)
+    || 30 * 60 * 1000;
+  let lastCoverageAtMs = null;
+  let lastCoverageAlertKey = null;
+  const coverageTick = async () => {
+    try {
+      const nowMs = now().getTime();
+      if (!lineageCoverage.coverageDue(lastCoverageAtMs, nowMs, COVERAGE_EVERY_MS)) return;
+      lastCoverageAtMs = nowMs;
+      const audit = await lineageCoverage.auditLatestCohort('mlb', { now: () => now().toISOString() });
+      const alarm = lineageCoverage.coverageAlarm(lastCoverageAlertKey, audit);
+      lastCoverageAlertKey = alarm.key;
+      console.log(`[lineage-coverage] mlb ${audit.health}`
+        + ` ${audit.covered_keys ?? '?'}/${audit.expected_keys ?? '?'}`
+        + ` cohort=${String(audit.snapshot_id || 'none').slice(0, 8)}`
+        + `${audit.reason ? ` reason=${audit.reason}` : ''}`);
+      if (alarm.alert) {
+        await notify(alarm.message, {
+          title: 'VYNDR lineage', priority: alarm.priority, tags: ['open_book'],
+        });
+      }
+    } catch { /* the monitor must never break the scheduler */ }
+  };
+
   const tick = async () => {
     await checkOverdue();
     await pulseTick();
+    await coverageTick();
     const d = now();
     if (d.getUTCMinutes() !== 0) return;
     const h = d.getUTCHours();
@@ -467,7 +504,7 @@ function startSnapshotScheduler(opts = {}) {
   console.log(`[settlement] armed — ${settleTags} outcomes + ledger settle pass runs FIRST at each snapshot slot (${HOURS_UTC.join(',')} UTC), idempotent re-runs`);
   // Session 8 — same verifiability rule: every watchdog states itself at boot.
   console.log(`[opsWatch] armed — settle alarms (throw + morning zero-settle), failure pager (${failureTracker.threshold} consecutive), quota daily check, pulse ${PULSE_HOUR_UTC}:00 UTC`);
-  return { interval, tick, refreshTick, pulseTick };
+  return { interval, tick, refreshTick, pulseTick, coverageTick };
 }
 
 module.exports = { startSnapshotScheduler, HOURS_UTC, mostRecentExpectedSlot, isSnapshotOverdue };