builtbykev/vyndr
1
0
Fork 0
Code Issues Pull Requests Actions 7 Packages Projects Releases Wiki Activity
Files
d954e4d9525b2ad209508c8d9926f30f3168b510
vyndr/docs/SYSTEM-MANIFEST.md
T

30 KiB
Raw Blame History

VYNDR — System Manifest

Source of truth for every interface in the codebase as of Session 7c (958 tests across 73 suites, zero regressions). When this drifts from reality, update this file FIRST.

1. Architecture overview

                  ┌──────────────────┐
                  │  Next.js (3000)  │ ─── public web app + app/api/* proxies
                  └────────┬─────────┘
                           │ BACKEND_URL
                           ▼
   n8n cron ── POST ─► ┌──────────────────┐ ─► Supabase (RLS-enforced reads,
   PM2 pollers ──────► │  Express (3001)  │     service-role writes from API)
   Coolify healthcheck │                  │ ─► Redis (cache + locks)
                       └────────┬─────────┘
                                │
                                ▼
                ┌───────────────────────────────────┐
                │ External adapters (configured()-  │
                │   gated, rate-limited, breakered) │
                │ SharpAPI · OddsPapi · PropOdds ·   │
                │ ParlayAPI · CFBD · OpenRouter ·    │
                │ ESPN · MLB Stats API · Pinnacle    │
                └───────────────────────────────────┘

Two parallel grading paths coexist (audited in Session 7c):

  • Legacy path (still wired): routes/scan|analyze|bets|alerts → parlayScanService → propAnalyzer → grader.js → UnifiedOddsProvider → 9 book adapters (ESPN, Pinnacle, DraftKings, FanDuel, BetMGM, Caesars, PrizePicks, Covers, Rotowire) → processing/{LineShoppingEngine, MiddlesDetector, EVCalculator}.
  • New path (Sessions 6a-6c): routes/grading/pipeline → gradingOrchestrator → featureCache + trapDetection + consistencyScore + probabilityEstimator → engine1 → (A/B-tier only) engine2 via OpenRouter.

The two paths share grade_history, Redis, Supabase. They do not share the grading function itself. This duplication is documented as finding #ARCH-1 below.

2. Express API endpoints

Mounted in src/app.js. Auth column meanings:

  • public — no auth middleware
  • user — requireAuth (Supabase JWT bearer)
  • internal — requireInternal (X-VYNDR-Internal-Key + loopback IP)
  • pipeline — requirePipelineSecret (X-Pipeline-Secret)
  • stripe-sig — stripe.webhooks.constructEvent signature
Method Path Auth Body limit Handler
GET /api/health public n/a app.js (inline)
GET /api/odds/nba public 10mb routes/odds.js
GET /api/odds/ncaab public 10mb routes/odds.js
POST /api/analyze/prop public 10mb routes/analyze.js (demo scan)
POST /api/analyze/batch public 10mb routes/analyze.js ⚠ no rate limit
POST /api/scan/parlay user 10mb routes/scan.js
GET /api/movements public 10mb routes/movements.js
GET /api/alerts user 10mb routes/alerts.js
PATCH /api/alerts/:id/read user 10mb routes/alerts.js
POST /api/bets/quickslip user 10mb routes/bets.js
POST /api/bets/screenshot user 10mb routes/bets.js
POST /api/bets/screenshot/confirm user 10mb routes/bets.js
POST /api/bets/sync user 10mb routes/bets.js
PATCH /api/bets/:id/settle user 10mb routes/bets.js
GET /api/bets user 10mb routes/bets.js
GET /api/bets/performance user 10mb routes/bets.js
POST /api/stripe/checkout user 10mb routes/stripe.js
POST /api/stripe/webhook stripe-sig raw bytes routes/stripe.js
POST /api/stripe/portal user 10mb routes/stripe.js
GET /api/stripe/status user 10mb routes/stripe.js
GET /api/stats/parlays-graded public 10mb routes/stats.js
GET /api/stats/public public 10mb routes/stats.js
GET /api/stats/live public 10mb routes/stats.js
GET /api/props/joint-history user 10mb routes/props.js
POST /api/waitlist public 10mb routes/waitlist.js
POST /api/pipeline/refresh pipeline 10mb routes/pipeline.js
GET /api/pipeline/status public 10mb routes/pipeline.js
POST /api/share-card public 10mb routes/shareCard.js
POST /api/push/subscribe user 10mb routes/push.js
DELETE /api/push/unsubscribe user 10mb routes/push.js
POST /api/grading/resolve internal 10mb routes/grading.js
POST /api/grading/pipeline internal 10mb routes/grading.js
POST /api/grading/correct internal 256kb routes/corrections.js
GET /api/widget public 10mb routes/widget.js
OPTIONS /api/widget public n/a routes/widget.js

Next.js API routes (frontend app/api/*)

These are proxies or thin wrappers; they hit Express via BACKEND_URL or the Python service via NEXT_PUBLIC_NBA_SERVICE_URL.

  • /api/checkout (POST/GET) — NexaPay checkout
  • /api/games/[id] and /api/games/tonight — list / detail
  • /api/games/[id]/props — props for a game
  • /api/intelligence/feed — homepage live signals
  • /api/ledger, /api/ledger/accuracy — Ledger feed
  • /api/parlay/add-leg, /api/parlay/grade — proxy to /api/scan/parlay
  • /api/players/search — proxy to Python /players/search
  • /api/props/live, /api/props/most-parlayed, /api/props/top-graded
  • /api/scan — bare scan endpoint
  • /api/stats/parlays-graded, /api/stats/public — proxy
  • /api/user/profile, /api/user/scans, /api/user/recent-scans
  • /api/waitlist — proxy
  • /api/webhook/nexapay — NexaPay webhook

3. Environment variables

Source: grep -rn 'process\.env\.' src/ poller/ scripts/ web/src/. ✓ = documented in .env.example. ⚠ = used but optional (default falls back). Updated this session in Section 1 of Session 7c.

Core

Var Required Default Used By Doc?
NODE_ENV no (none) various ✓
PORT no 3001 src/server.js implicit
BASE_URL yes http://localhost:3001 stripeService, widget ✓
FRONTEND_ORIGINS yes (none) app.js (CORS) ✓
VYNDR_INTERNAL_KEY yes (none) routes/grading, corrections ✓

Supabase

Var Required Used By Doc?
SUPABASE_URL yes both clients ✓
SUPABASE_ANON_KEY yes getSupabaseClient ✓
SUPABASE_SERVICE_ROLE_KEY yes getSupabaseServiceClient ✓
SUPABASE_SERVICE_KEY (legacy) no fallback only (7b) ✓ commented
SUPABASE_DB_URL yes scripts/backup.sh, migrations ✓
SUPABASE_DB_PASSWORD no legacy backup form ✓ commented
NEXT_PUBLIC_SUPABASE_URL yes Next.js client ✓
NEXT_PUBLIC_SUPABASE_ANON_KEY yes Next.js client ✓
NEXT_PUBLIC_SITE_URL yes Next.js metadata ✓

Web ↔ Backend

Var Required Default Used By Doc?
NEXT_PUBLIC_API_URL yes http://localhost:3001 web/src/lib/api.ts ✓ (added 7c)
NEXT_PUBLIC_NBA_SERVICE_URL yes http://localhost:8000 web/api/players/search ✓ (added 7c)
BACKEND_URL yes http://localhost:3001 web/services/odds-cache.ts ✓ (added 7c)

Stripe + Payments

Var Doc?
STRIPE_SECRET_KEY ✓
STRIPE_WEBHOOK_SECRET ✓
STRIPE_PRICE_ANALYST ✓
STRIPE_PRICE_ANALYST_FOUNDER ✓
STRIPE_PRICE_DESK ✓
STRIPE_PRICE_DESK_FOUNDER ✓
FOUNDER_CODES ✓
FOUNDER_CODE_EXPIRY ✓
NEXAPAY_API_URL ✓ (added 7c)
NEXAPAY_API_KEY ✓ (added 7c)
NEXAPAY_WEBHOOK_SECRET ✓ (added 7c)

Push

Var Doc?
VAPID_PUBLIC_KEY ✓
VAPID_PRIVATE_KEY ✓
VAPID_SUBJECT ✓
NEXT_PUBLIC_VAPID_PUBLIC_KEY ✓

Odds adapters

Var Doc?
SHARPAPI_KEY ✓
SHARPAPI_BASE_URL ✓
ODDSPAPI_KEY ✓
ODDSPAPI_BASE_URL ✓
PARLAYAPI_KEY ✓
PARLAYAPI_BASE_URL ✓
PARLAYAPI_PULL_RATE_MS ✓
PROPODDS_KEY ✓
PROPODDS_BASE_URL ✓
CFBD_KEY ✓
CFBD_BASE_URL ✓
PINNACLE_API_KEY ✓ commented (legacy)
PINNACLE_API_BASE ✓ commented (legacy)
ODDS_API_KEY ✓ commented (legacy)

Engine 2

Var Doc?
OPENROUTER_API_KEY ✓
OPENROUTER_PRIMARY_MODEL ✓
OPENROUTER_FALLBACK_MODEL ✓
OPENROUTER_BASE_URL ✓
OPENROUTER_REFERER ✓
OPENROUTER_TITLE ✓
ENGINE2_BATCH_SIZE ✓
ENGINE2_ENABLED ✓

Redis + Python

Var Doc?
REDIS_URL ✓
PYTHON_SERVICE_URL ✓
NBA_SERVICE_URL ✓ commented (legacy alias)
NBA_STATS_URL ✓ commented (legacy alias)
EVOLUTION_SERVICE_URL ✓ commented (legacy)
VYNDR_PYTHON ✓ commented (legacy alias)

Distribution + Analytics

Var Doc?
TELEGRAM_BOT_TOKEN ✓
TELEGRAM_CHANNEL_ID ✓
DISCORD_WEBHOOK_DAILY ✓
DISCORD_WEBHOOK_RESULTS ✓
DISCORD_WEBHOOK_ALERTS ✓
DISCORD_WEBHOOK_RARE ✓
RESEND_API_KEY ✓ (added 7c)
RESEND_FROM_EMAIL ✓ (added 7c)
NEXT_PUBLIC_POSTHOG_KEY ✓ (added 7c)
NEXT_PUBLIC_POSTHOG_HOST ✓ (added 7c)

Poller-set (per-process via PM2)

Var Default Doc?
SPORT (none) ✓ commented
POLL_INTERVAL 60000 ✓ commented
BUFFER_MS 30000 ✓ commented
VYNDR_API_URL http://localhost:3001 ✓ commented
OFF_HOURS_POLL_MS hardcoded 5min not env

Backup + Ops

Var Doc?
NTFY_PORT ✓
NTFY_TOPIC ✓
BACKUP_RETENTION_DAYS ✓
PIPELINE_SECRET ✓ commented
PUBLIC_SHARE_CARD_BASE ✓ commented
API_BASE_URL ✓ commented (alias of BASE_URL)
ENGINE ✓ commented (legacy)

4. Supabase tables

Source: grep -rn "\.from('[a-z_]*')" src/ poller/.

Tables created by VYNDR migrations

Table Migration Read by Written by
user_profiles 011, 015 AuthContext, stripeService mirror, accuracyTracker handle_new_user, stripeService
parlay_leg_frequency 011 parlay analytics scan flow
scan_history 011 user scans listing scanRoutes
odds_cache 014 oddsService oddsService, n8n pipeline
line_history 014 lineMovementService lineMovementService
cascade_alerts 014 cascadeService, intelligence feed cascadeService
player_stats_cache 014 propAnalyzer propAnalyzer
grade_history 014/16/18 Ledger, resolution route, weight adjuster, accuracy orchestrator, resolution, engine2, corrections
prop_correlations 014 correlationEngine correlationEngine
player_id_map 014→016 populate-player-ids, oddsPapiAdapter populate-player-ids
push_subscriptions 015 webPush push routes, webPush prune
closing_lines 016 clvTracker, oddsPapiAdapter oddsPapiAdapter.batchCapture
resolution_results 016 clvTracker, weightAdjuster, corrections resolution route
historical_props 017 trap historical signal (future) pull-parlayapi-history
line_snapshots 017 lineMovement intel sharpApiAdapter (fire-and-forget)
ref_profiles 017 refSignals scrape-sports-reference
coach_profiles 017 coachSignals scrape-sports-reference, seed file
game_ref_assignments 017 refSignals ops manual entry
engine1_weights 018 weightAdjuster weightAdjuster
accuracy_tracking 018 accuracyTracker, Ledger accuracy page accuracyTracker
user_notifications 013 notifications route notification senders

Tables present in DB but NOT in VYNDR migrations (BetonBLK era)

bets, daily_scan, discrepancy_reliability_scores, joint_outcomes, lineup_role_profiles, model_predictions_extended, outcomes, performance, picks, player_role_activations, player_role_profiles, prediction_registry, scan_sessions.

VYNDR code reads/writes only the subset still in use:

  • bets — routes/bets.js, betService.js
  • joint_outcomes — routes/props.js (joint-history)
  • model_predictions_extended — propAnalyzer
  • performance — performanceService
  • picks — parlayScanService
  • scan_sessions — routes/scan.js

Shared tables (BetonBLK + VYNDR)

  • users — both eras
  • waitlist / waitlist_signups — created in 012 (latter is canonical)

5. Redis keys

Source: grep -rn "cacheSet\|cacheGet\|redis\.set".

Key pattern TTL Set by Read by
odds:{sport}:{gameId}:player_props 60s (300s stale) sharpApiAdapter sharpApiAdapter, orchestrator
odds:{sport}:{gameId}:game 60s sharpApiAdapter sharpApiAdapter
parlayapi:hist:{sport}:{p}:{stat}:{n} 24h parlayApiAdapter trapDetection (future)
parlayapi:close:{sport}:{date} 24h parlayApiAdapter pull script
parlayapi:checkpoint:{sport}:{season} 30d pull-parlayapi-history pull-parlayapi-history
propodds:{sport}:{game}:{player}:{stat} 90s (300s stale) propOddsAdapter propOddsAdapter
cfbd:teamstats:{team}:{year} 6h cfbdAdapter cfbdAdapter
cfbd:usage:{player}:{team}:{year} 6h cfbdAdapter cfbdAdapter
cfbd:talent:{team}:{year} 6h cfbdAdapter cfbdAdapter
cfbd:lines:{team}:{year} 6h cfbdAdapter cfbdAdapter
gamelogs:{sport}:{player}:{n} 4h gameLogService featureCache, consistency
team_stats:{sport}:{teamAbbr} 24h teamStatsCache featureCache, engine1
features:{sport}:{playerId}:{stat}:{game} 2m featureCache orchestrator
injuries:{sport}:{teamId} 2h injuryParser featureCache
game:{gameId}:status 36h poller poller
game:{gameId}:resolution_lock 5min (NX) poller poller (dedupe)
game:{gameId}:live_status 1h poller frontend badges (future)
poller:{sport}:heartbeat 3min poller ops monitoring
vyndr:line_baseline:{...} 24h lineMovementService lineMovementService
(cascadeService player-set key) 24h cascadeService cascadeService
(oddsService legacy game cache) 5min oddsService routes/odds
(schemeClassifier cache) varies schemeClassifier schemeClassifier

6. External API dependencies

Service Adapter Env vars Rate budget Used by
SharpAPI sharpApiAdapter.js SHARPAPI_KEY, SHARPAPI_BASE_URL 10/min orchestrator
OddsPapi oddsPapiAdapter.js ODDSPAPI_KEY, ODDSPAPI_BASE_URL 5/min poller (tip-off CLV)
ParlayAPI parlayApiAdapter.js PARLAYAPI_KEY 5/min pull script
PropOdds propOddsAdapter.js PROPODDS_KEY 3/min trap consensus
CFBD cfbdAdapter.js CFBD_KEY 10/min college features
OpenRouter openRouterAdapter.js OPENROUTER_API_KEY + model env 20/min, 1000/day engine2
ESPN scoreboard poller, teamStatsCache, injuryParser none 2/min (limiter) poller, refresh job
MLB Stats API poller (gamePk feed) none 2/min poller (mlb final)
Pinnacle (legacy) PinnacleAdapter.js PINNACLE_API_KEY, PINNACLE_API_BASE tunable UnifiedOddsProvider
DraftKings (legacy) DraftKingsAdapter.js none tunable UnifiedOddsProvider
FanDuel/BetMGM/Caesars/PrizePicks/Covers/Rotowire (legacy) each *Adapter.js none tunable UnifiedOddsProvider
Sports-Reference HTML scripts/scrape-sports-reference.js REF_HTML_FILE, COACH_HTML_FILE (optional) 1 req / 5s scraper
Resend (email) web/src/services/email.ts RESEND_API_KEY, RESEND_FROM_EMAIL n/a transactional email
NexaPay web/src/services/nexapay.ts NEXAPAY_* n/a checkout fallback
PostHog web/src/lib/analytics.ts NEXT_PUBLIC_POSTHOG_KEY/HOST n/a browser analytics

7. File dependency graph (entry points)

Express entry: src/server.js

server.js
  └─ app.js
       ├─ routes/{odds,analyze,scan,movements,alerts,bets,stripe,stats,
       │   props,waitlist,pipeline,shareCard,push,grading,corrections,widget}
       └─ middleware/{auth, mission}

routes/grading.js  (POST /pipeline)
  └─ services/intelligence/gradingOrchestrator
       ├─ adapters/sharpApiAdapter
       ├─ services/intelligence/featureCache
       │     ├─ teamStatsCache · refSignals · coachSignals
       │     ├─ injuryParser · lineMovement · gameLogService
       │     └─ lineupSignals
       ├─ services/intelligence/trapDetection
       │     └─ lineMovement
       ├─ services/intelligence/consistencyScore
       ├─ services/intelligence/probabilityEstimator
       ├─ services/intelligence/engine1
       └─ services/intelligence/engine2
              └─ adapters/openRouterAdapter

routes/grading.js  (POST /resolve)
  └─ services/training/jsonlLogger
  └─ services/distribution/{webPush, telegram, discord}
  └─ services/intelligence/{clvTracker, accuracyTracker, weightAdjuster}

routes/scan.js  (POST /parlay  — legacy path, still wired)
  └─ services/parlayScanService
        └─ services/propAnalyzer
              └─ services/grader   ← legacy grading (see ARCH-1)
              └─ services/UnifiedOddsProvider
                    └─ adapters/{ESPN,Pinnacle,DraftKings,FanDuel,BetMGM,
                                 Caesars,PrizePicks,Covers,Rotowire}
                    └─ services/processing/{LineShoppingEngine,
                                            MiddlesDetector, EVCalculator}

Poller entry: poller/poller.js

poller.js
  ├─ src/config/sports
  ├─ src/utils/redis (cacheGet/cacheSet/lock)
  ├─ src/utils/rateLimiter (espnLimiter, mlbLimiter)
  └─ src/services/adapters/oddsPapiAdapter (tip-off CLV capture)

No circular imports detected.

8. Findings

ARCH — Architecture

  • [ARCH-1] Dual grading paths. Severity: Medium. The legacy parlayScanService → propAnalyzer → grader → UnifiedOddsProvider chain still serves /api/scan/parlay, /api/analyze/*, and /api/bets/*. The new gradingOrchestrator → engine1 → engine2 chain serves /api/grading/pipeline. Both write to grade_history with different column sets and factor models. Scope to fix: ~1 session — choose one as canonical, migrate the routes, retire the other adapter set.

  • [ARCH-2] Two circuit-breaker / rate-limiter modules. Severity: Low (documented in 6a). src/services/{circuitBreaker.js, rateLimiter.js} (keyed registry, legacy) coexist with src/utils/rateLimiter.js (factory, new). Consolidation is purely cosmetic — both work. Scope: half-session.

SEC — Security

  • [SEC-1] /api/analyze/batch has no auth or rate limit. Severity: High. A malicious caller can blast through prop-analyzer credits. Recommended fix: require auth OR add an IP-keyed rate limiter (10 req/min). Scope: ~30 min.

  • [SEC-2] routes/shareCard.js leaks err.message via detail:. Severity: Medium. Lines 185 and 206 expose upstream error messages to public callers. Recommended fix: drop detail from public responses; log to stderr only. Scope: ~10 min.

  • [SEC-3] Several console.log calls in production code. Severity: Low. All have [VYNDR] / [redis] / [poller-X] prefixes and represent OPERATIONAL status (server starting, poller tick summary), not debug. Per project convention console.log is acceptable for operational events. Documenting so future audits don't re-flag.

DEAD — Dead code

  • [DEAD-1] No truly orphaned files. Every adapter in src/services/adapters/ has at least one consumer (legacy ones via UnifiedOddsProvider, new ones via orchestrator/poller). Removing any would break either /api/scan or /api/grading/pipeline.

  • [DEAD-2] grader.js is LIVE. Imported by propAnalyzer.js → parlayScanService.js → routes/scan.js. Kept until ARCH-1 is resolved.

DUP — Duplicates

  • [DUP-1] normalizeName() exists twice. Severity: Low.

    • src/services/intelligence/trapDetection.js
    • scripts/populate-player-ids.js Implementations are near-identical (NFD strip, lowercase, suffix removal). Recommended fix: extract to src/utils/normalize.js; the script can require('../src/utils/normalize'). Skipped this session because the script is intentionally self-contained for remote execution.

  • [DUP-2] oddsToImplied etc. only live in src/utils/odds.js. Confirmed not duplicated despite the oddsNormalizer.js neighbor — the two files serve different purposes (math primitives vs book-level prop shape).

LEGACY — Documentation / naming

  • [LEG-1] BetonBLK references. Severity: Informational. Confined to BUILD-STATE.md historical notes, the FOUNDER_CODES allowlist (intentional grandfathering — documented in stripeService comment), and tests that assert the grandfathered code still validates. No action.

TODO

  • [TODO-1] PrizePicks / DraftKings / Pinnacle adapter TODOs. Severity: Low. Tracked in specs/data-pipeline-books.md. Each has a comment line referencing the spec. Keep until the corresponding feature ships.

PERF — Performance

  • [PERF-1] /api/analyze/* lacks Redis cache. Severity: Medium. Each call re-runs analyzeProp() end-to-end. Recommended fix: cache the result by (player, stat, line, direction, sport) for 60s. Scope: ~30 min.

  • [PERF-2] routes/scan.js resolves parlays one prop at a time. Sequential await analyzeProp() inside a for-loop. For 6-leg parlays this is ~6x latency vs Promise.allSettled. Recommended fix: parallelize independent props.

  • [PERF-3] Bundle analysis not run this session. The ANALYZE=true flag would require @next/bundle-analyzer which is not installed. Recommended fix: install bundle-analyzer dev dep

    • run once + decide whether to ship a hidden audit/page route.

Frontend ↔ Backend contract

All frontend API paths discovered are either:

  • Direct Express paths confirmed in src/app.js (e.g. /api/scan/parlay, /api/waitlist), or
  • Next.js app/api/*/route.ts proxies that internally fetch ${BACKEND_URL}/....

No broken contracts found — every fetched path maps to an existing handler. Spot-checked: /api/players/search (Next → Python), /api/scan (Next → Express), /api/intelligence/feed (Next direct DB).

9. How to update this manifest

When you add a route, table, env var, or external API:

  1. Add the row to the appropriate section above.
  2. Re-run the discovery greps from this session's prompt to confirm nothing else changed implicitly.
  3. If you remove a table or env var, mark it [REMOVED Session NX] instead of deleting the row outright — keeps audit trail.

Last updated: Session 7c.

Reference in New Issue View Git Blame Copy Permalink