builtbykev
9809626c99
The previous bug made the recorder write nothing. The dangerous successor is a
recorder that writes half and looks healthy: persist() writes in chunks of 250
and STOPS AT THE FIRST FAILED CHUNK, so chunks committed before the failure are
already durable. Rows exist under the snapshot_id, captured_at is uniform, Redis
kept working — and the cohort is short.
So row presence was never completion evidence, and neither was a matching
timestamp. Completeness is now proven by the writer or not at all.
TERMINAL RETENTION STATES (retentionService.classifyPersist):
NOTHING_TO_PERSIST attempted 0 — a refusal-only slate is still a cycle
SKIPPED_NO_DATABASE no database configured; not a failure
COMPLETE attempted > 0, written === attempted, no error
FAILED_ZERO_WRITE written === 0 — first chunk failed
FAILED_PARTIAL 0 < written < attempted — a later chunk failed
FAILED_UNRESOLVED_ERROR counts look complete but an error is unresolved;
unreachable through today's loop, and kept because
the alternative is reporting COMPLETE holding an error
The invariant: any written < attempted with attempted > 0 is a FAILED cycle. A
partial cohort is never degraded success.
classifyPersist reads the EXACT persist() result and refuses anything else — it
never recomputes attempted or written, because a second calculation could
disagree with the writer and then the status would describe a cycle that did not
happen. persist() itself is byte-identical to 35da190.
`written` counts rows in COMMITTED CHUNKS, not database inserts: the upsert uses
ignoreDuplicates, so a re-run legitimately inserts far fewer rows than it writes.
Comparing written to count(*) will disagree by design. Documented, because that
mismatch is exactly what would be misread as a partial write.
VISIBILITY. The 35da190 alert condition was
`r.error || (!r.skipped && r.attempted > 0 && r.written === 0)` — it could not
see a partial cohort as a distinct state. It is now driven by terminal status,
so FAILED_PARTIAL alerts as loudly as a total failure and is labelled INCOMPLETE
and unusable as evidence. Best-effort is unchanged: the product continues and
the alert says so.
OBSERVABILITY. A successful cycle previously left only a console.log with no
snapshot_id, no code_sha and no terminal status, so completion could not be
established after the fact. `GET /api/internal/snapshot/status` now returns
`last_retention` per sport — sport, snapshot_id, attempted, written, status,
completed_at, code_sha, error_summary — taken verbatim from the persistence
result. Existing internal auth, read-only, counts and status only, no payloads.
No new table, no new route.
RELEASE-AUTHORIZED INSERT CONTRACT. The migration-derived contract is the
release authority; production is not. A prod-only column is DRIFT / RECORDED
DEBT and never becomes permission by existing. Verifier classifies: release
column missing in prod -> HARD FAILURE; prod-only -> drift warning; outbound key
outside the contract -> contract failure (enforced against the real upsert
payload). It is read-only and never rewrites the contract from live schema.
Live: release 64, prod 67, prod-only 3, missing in prod 0.
Six teeth, each with the injection verified present, against a green baseline:
1 written>0 as generic success -> 6 fail
2 later-chunk failure reports COMPLETE -> 5 fail
3 FAILED_PARTIAL does not alert -> 3 fail
4 status reports a recalculated count -> 1 fail
5 row presence treated as completion -> 1 fail
6 invalid outbound column reintroduced -> 4 fail
Restored byte-identically (retention b341cf16c1baa992, snapshot 81ab1bd7730dee89).
Two stale assertions updated rather than deleted, with the mechanism change
recorded: the alert-shape tests described the superseded written===0 condition,
and the runtime probe test pinned an exact import list.
Model and product preserved: analyzeViaEngine1, probabilityEstimator,
gradeSlateService, lineageCanaryConfig, eventIdentity, ledgerService,
calibration and chain all UNCHANGED; zero lineage/publication files touched;
zero cacheSet changes; zero web paths. Lineage stays OFF.
383 suites / 5,118 tests pass. web tsc exit 0.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CQJeAG8vcDoL5zkiaJyVb8
85 lines
2.5 KiB
JSON
85 lines
2.5 KiB
JSON
{
|
|
"table": "model_snapshots",
|
|
"generated_by": "scripts/generate-schema-contract.js",
|
|
"derived_from": "supabase/migrations/*.sql applied in order to a disposable postgres:15-alpine",
|
|
"note": "Insertable columns only (no generated/identity columns). This is the contract the retention writer must satisfy. Regenerate after any migration that touches model_snapshots; scripts/verify-schema-contract.js detects drift against a live database.",
|
|
"known_production_drift": {
|
|
"columns_in_production_not_in_migrations": [
|
|
"quarantine_reason",
|
|
"re_settled_at",
|
|
"settlement_source"
|
|
],
|
|
"explanation": "Present in production but not created by any committed migration - added out of band, same class as the migration-014 debt. The contract deliberately uses the MIGRATION-derived set, which is the stricter of the two: a writer that stays within it is valid against both.",
|
|
"classification": "PROD-ONLY COLUMN -> DRIFT WARNING / RECORDED DEBT (not release-authorized)"
|
|
},
|
|
"column_count": 64,
|
|
"columns": [
|
|
"actual_value",
|
|
"archetype",
|
|
"book",
|
|
"book_odds",
|
|
"canonical_event_id",
|
|
"captured_at",
|
|
"chain_shadow",
|
|
"change_type",
|
|
"claim_digest",
|
|
"claim_schema_version",
|
|
"code_sha",
|
|
"confidence",
|
|
"confidence_basis",
|
|
"created_at",
|
|
"cycle_hour_utc",
|
|
"devig_method",
|
|
"digest_algorithm_version",
|
|
"edge_pct",
|
|
"ev_pct",
|
|
"event_identity_method",
|
|
"event_identity_source",
|
|
"event_identity_version",
|
|
"event_occurrence",
|
|
"factor_inputs",
|
|
"fair_odds",
|
|
"fair_prob",
|
|
"features",
|
|
"game_date",
|
|
"game_id",
|
|
"grade",
|
|
"grade_11",
|
|
"id",
|
|
"line",
|
|
"lineage_action",
|
|
"lineage_state",
|
|
"lineage_version",
|
|
"model_version",
|
|
"opponent",
|
|
"outcome",
|
|
"over_odds",
|
|
"overround",
|
|
"p_win",
|
|
"player_key",
|
|
"player_name",
|
|
"projection",
|
|
"publication_id",
|
|
"published",
|
|
"published_at",
|
|
"read_id",
|
|
"read_natural_key",
|
|
"recaptures_id",
|
|
"refusal_reason",
|
|
"refused",
|
|
"revision_ordinal",
|
|
"settled_at",
|
|
"side",
|
|
"snapshot_id",
|
|
"sport",
|
|
"stat",
|
|
"supersedes_id",
|
|
"takeable",
|
|
"team",
|
|
"under_odds",
|
|
"value"
|
|
],
|
|
"authority": "RELEASE-AUTHORIZED model_snapshots INSERT CONTRACT",
|
|
"authority_note": "Derived from the migration chain, which is the release authority. Production is not. A column present in production but in no migration is DRIFT / RECORDED DEBT and is NOT release-authorized; it must never be copied into this contract from live schema."
|
|
}
|