feat: Unraid deploy, dealer-flow heatmap, confluence zones, 13F capture
Ship Node production images, Unraid compose, and Gitea CI/CD (test then push registry images; cron script if no runner). Rebuild dealer flow as a heatmap-first map with integrity gates and chart helpers. Add confluence zone rules, session clock, capture evidence, and tighter 13F/queue/options paths, plus the matching UI and tests.
This commit is contained in:
Investor Flow Build
+22
-7
@@ -71,7 +71,7 @@ All vendors (Yahoo, X, FRED, SEC, Reddit) have short rate limits. The system is
|
||||
| Incremental candles | Warm symbols re-fetch ~14d lookback, not full 10y every tick |
|
||||
| Poison quarantine | Delisted / not-found symbols fail permanent; not requeued; demand cleared |
|
||||
| Per-kind drain budgets | Quotes cannot starve symbol meta / candles forever |
|
||||
| Observability | `queue.health()`: cooldowns, `pendingByKind`, demand size, SPY candle lag, `dataPlaneHealthy` |
|
||||
| Observability | `queue.health()`: vendorGate+queue_state cooldowns, `pendingByKind`, demand size, SPY candle lag, `dataPlaneHealthy` |
|
||||
|
||||
**Anti-patterns (do not reintroduce):** N parallel Yahoo charts on a click; live `quoteSummary` without cache on every panel open; treating 429 as a 2s job retry that keeps hammering the same edge; calling `subscribe` on every page view (inflates refcount - use `ensureInDemand`); `dealerMap.get` calling Yahoo directly; frontend looping expiries to paint Dealer Flow.
|
||||
|
||||
@@ -83,7 +83,7 @@ All vendors (Yahoo, X, FRED, SEC, Reddit) have short rate limits. The system is
|
||||
- Request path: SQLite recompute + schedule-on-miss; max 4–6 nearest expiries; 15m map TTL; daily `dealer_map_snapshots` for velocity.
|
||||
- Integrity: pure `dealerMapIntegrity` hard/soft checks (missing expiries/OI/greeks fail; delay does not); `data_quality` kind `dealer_map`; replay via `dealerMapReplay` + `scripts/dealer-map-replay.ts` (as-of chain ts).
|
||||
- SEC institutional (alert-critical): SC-first fetch seeds `sec:cusip:SYMBOL`; offline curated CUSIP registry (`cusipRegistry`) so resolve does not depend solely on EFTS; 13F prefers EFTS CUSIP pagination, and on EFTS 403/outage falls back to **reverse 13F** (`reverse13fRefresh`: prior holders + tracked funds + major managers via `data.sec.gov`); `SecFetchAdapter` **throws** on hard resolve failure so queue retries (no silent done); `requeueUnhealthySecSymbols` caps heal requeues per tick.
|
||||
- **Dealer GEX sign convention:** maps are stored as classic OI GEX (`classic_call_pos_put_neg`: call +, put −). Request path can re-express as `dealer_inventory` (full GEX/VEX sign flip - Heatseeker-style dealer short when customers long) via `withExposureConvention` / `dealerMap.get({ convention })`. UI toggle: Classic | Dealer (HS).
|
||||
- **Dealer GEX sign convention:** maps are stored as classic OI GEX (`classic_call_pos_put_neg`: call +, put −). Request path can re-express as `dealer_inventory` (full GEX/VEX sign flip - Heatseeker-style dealer short when customers long) via `withExposureConvention` / `dealerMap.get({ convention })`. First-paint UI is GEX | VEX only; Method drawer has Customer book | Dealer inventory.
|
||||
- **Vendor rate-limit enforcement (hard requirement, all sources + future):** process-wide `vendorGate` with **open registration** (`registerVendorIntegration`). Built-in families: yfinance, sec, fred, finra, nasdaq, reddit, x, llm. New vendors must register family + bind `source_kind` before `AdapterQueue` construction (throws otherwise). Prefer `VendorSourceAdapter` / `defineVendorAdapter` so `fetchOne` is auto-gated. HTTP via `vendorFetch` / `secHttp`; SDKs via `withVendorGate`. CI guard bans bare `fetch(` in adapters/services. See `docs/VENDOR_INTEGRATIONS.md`.
|
||||
- LLM: per-user OpenAI-compatible `base_url` + encrypted key + model (`userLlm.*`); not OpenAI-only.
|
||||
- Dealer Flow plain-English notes: in-app `dealerFlowExplainNotes.ts` only (L0/L1). Optional offline scripts harvest X handles and distill into that file **and** write a personal Obsidian vault copy - the app never reads Obsidian at runtime.
|
||||
@@ -268,16 +268,31 @@ ADR-0006/0008 still govern intent; implementation is incomplete.
|
||||
|
||||
## 11. Deployment
|
||||
|
||||
Local dev (typical):
|
||||
|
||||
```bash
|
||||
# Local dev (typical)
|
||||
cd app/server && npm run dev # :3001
|
||||
cd app && npm run dev # :3000
|
||||
|
||||
# Compose
|
||||
docker compose up --build
|
||||
```
|
||||
|
||||
Secrets: `.env.example` documents session secret, SEC operator email, X cookie fields, Reddit, FRED, LLM URL.
|
||||
Production is two Node containers (not Bun). Unraid is the intended host; git + Gitea Actions already live at `unraid.local:3003/transnet/investor-flow`.
|
||||
|
||||
```bash
|
||||
# Build-from-git (laptop or Unraid checkout)
|
||||
docker compose --env-file .env up -d --build
|
||||
```
|
||||
|
||||
| Piece | Path |
|
||||
|---|---|
|
||||
| Backend image | `app/server/Dockerfile` (Node 22, `node --experimental-strip-types`) |
|
||||
| Frontend image | `app/Dockerfile` (Next standalone, proxies `/api` via `IFLOW_BACKEND_URL`) |
|
||||
| Unraid operator guide | `docs/DEPLOY_UNRAID.md` |
|
||||
| Gitea CI | `.github/workflows/ci.yml` |
|
||||
| Gitea CD (registry push) | `.github/workflows/cd.yml` |
|
||||
| act_runner | `deploy/unraid/act-runner-compose.yml` |
|
||||
| Cron CD (no runner) | `deploy/unraid/user-scripts/sync-and-up.sh` |
|
||||
|
||||
Secrets: `.env.example` and `deploy/unraid/.env.example`. Production refuses to start without `IFLOW_SESSION_SECRET`. Persist SQLite on a cache/single-disk path (`IFLOW_DATA_DIR`), not `/mnt/user`.
|
||||
|
||||
---
|
||||
|
||||
|
||||
Reference in New Issue
Block a user