Files
investor-flow/docs/FUNCTIONAL_DESIGN.md
Investor Flow Build 7426103d73
CI / Test (push) Canceled after 0s
CI / Build and push (push) Canceled after 0s
Isolate Yahoo drain from SEC backlog and protect the guest account.
Yahoo and SEC drain on separate workers so hung EDGAR jobs cannot freeze watchlist prices. Queue health splits yahooHealthy from institutional backfill, watchlist snapshots only enqueue missing quotes, and first subscribe seeds quote/candles/symbol. Hide the anonymous system user from Admin so it cannot be deleted.
2026-09-09 09:55:21 -04:00

319 lines
22 KiB
Markdown
Raw Permalink Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Investor Flow - Functional Design (as built)
**Audit date:** 2026-08-18
**Source of truth for status:** this file + code under `app/`. Older slice SPECs and HANDOFF.md are historical.
**Product intent:** Multi-tenant **investment management** workbench (portfolio, risk, monitor, research context) that helps users become better at market awareness and risk management *while managing*. Not a course app. Legal posture for copy/outputs: educational publisher (ADR-0007) - process and math, never buy/sell advice.
**What changed since the 2026-07-23 audit (high level):**
- Process loop is server-persisted (`/plan`, `/theses`, `/journal`).
- Per-user **module access** (research / execution / analytics / settings / admin) plus workspace density.
- **Confluence** (M22) and **Price Corridor** live at `/confluence`.
- **Tracked Funds / Mirror** (M21) live at `/funds` and `/funds/[id]`.
- Symbol Search Index (ADR-0011), short interest (Yahoo + NASDAQ + FINRA), analyst ratings, classification watchlists.
- Alert producers that were stubs are now registered (rotation, regime, thesis, unlock, portfolio risk, VIX, 13F, confluence, mirror).
- Dealer Flow heatmap + integrity + Study Desk remain live; Han-style day script on the map.
- App ships to Unraid as two containers via Gitea Actions (tests, then registry images). Operator guide: [`DEPLOY_UNRAID.md`](./DEPLOY_UNRAID.md).
---
## Status legend
| Status | Meaning |
|--------|---------|
| **Live** | Backend + UI end-to-end usable for the intended job |
| **Backend-ready** | Engine and/or tRPC exist with tests; little or no product UI |
| **UI-local** | Screen exists but state is browser-only (localStorage / Zustand), not multi-device / multi-tenant |
| **Stub** | Surface or procedure exists but incomplete (empty timeline, thin universe, etc.) |
| **Missing** | Designed in domain model; not implemented as a product path |
---
## 1. User jobs and surfaces
### 1.1 Identity, access, onboarding
| Capability | Status | Notes |
|------------|--------|-------|
| Email/password signup | **Live** | First-user path is `/welcome`. New users get `status=pending_approval`; no session until admin approves. Pending-approval panel lives on the same desk; no session cookie issued while pending. |
| Login + session cookie | **Live** | Non-active users rejected at login and `protectedProcedure`. TOTP step surfaces when required (`Two-factor code required or invalid.`) with a 6-digit input that retries `auth.login(email, password, totp)`. Header CTAs (Sign in / Create account) visible when logged out. |
| Forgot / reset password | **Live (operator)** | No email reset. Welcome desk explains the path. Operator CLI `src/cli/reset-password.ts` (no session). Logged-in admin can reset another user under `/admin/users`. Signed-in users change their own password in Settings. |
| First-visit redirect | **Live** | Unsigned users without the `iflow-welcome-seen` localStorage flag are redirected from `/` to `/welcome`. Skip sets the flag and goes to `/`. |
| TOTP 2FA + backup codes | **Live** | Settings page (enabled / confirmed). Login desk handles the interactive code step. |
| GitHub / Google OAuth | **Backend-ready** | Router complete; env-dependent; OAuth users default `active` |
| Onboarding wizard | **Live** | Workspace interview shown on `/welcome` after first successful login if `!onboarded`. Steps: experience / goal / horizon / jargon comfort, then `api.onboarding.complete`. Dispatches `workspace-profile-changed`, pushes to `/portfolio`. |
| Workspace density | **Live** | `focused` \| `standard` \| `full` filters nav, symbol Overview panels, strategy templates; editable in Settings |
| Per-user module access | **Live** | Admin assigns `research`, `execution`, `analytics`, `settings`, `admin`. Default for new users: research + settings. `FeatureGate` redirects if the module is off. |
| Admin approve/reject | **Live** | `/admin/users` |
| Settings / profile | **Live (signed-in only)** | `/settings` shows account card, 2FA enrollment/confirmation, workspace density, and user-configured OpenAI-compatible LLM endpoint. Guest users are redirected to `/welcome?mode=signin`. |
Two independent gates apply to almost every destination:
1. **Module access** (who is allowed to use a family of screens).
2. **Workspace density** (how much chrome a given user wants to see).
### 1.2 Research workbench (desktop)
| Surface | Route | Status | Data path |
|---------|-------|--------|-----------|
| Symbol overview | `/` | **Live** | `market.snapshot`, ticker context, company meta, fund-holdings strip, dealer levels strip, density-gated analyst ratings / short interest / X feed / filings / options |
| Multiple watchlists | sidebar | **Live** | Create/delete/rename/reorder/move symbol; dropdown selector; local-first autocomplete (`symbols.search`); hybrid add (known match vs "not in SEC registry - add anyway?") |
| Classification watchlists | sidebar | **Live** | Lists can be `user` or derived (`sector` / `thematic` / `style` / `region`) with `class_key` / `class_label` |
| Charts | `/chart-lab` | **Live** | candles, indicators (incl. EMA 9/21), institutional buy markers; equity volume-by-price profile (visible range, from OHLC bars) |
| Institutional dashboard | `/institutional` | **Live** | dashboard rollup, flow, insider stream, filings; Q-o-Q / M-o-M |
| Filings | `/filings` | **Live** | EDGAR index + 13F + Form 4 detail |
| Options DD | `/options-dd` | **Live** | chain + greeks (read-only teaching surface) |
| Dealer Flow | `/dealer-flow` | **Live** | Heatmap-first IA: GEX/VEX is the only map toggle (convention lives in Method); reading / method / study in drawers; point-in-time book clock; integrity gates (incomplete vs degraded vs complete); IV hygiene; keep last good map; as-of ET; Han-style day script; `dealer-map-replay` CLI + Admin → Queue **Dealer map integrity** buttons; Layer-0 + optional L1 |
| Study Desk | (drawer on `/dealer-flow`) | **Live** | Educational setups from map; hist rank; log; auto-grade; scorecard; promote to journal draft; CSV export |
| Mentor ledger | (Study Desk tab) | **Live (MVP)** | Local import from harvest → confirm drafts → path-match grade + mentor scorecard (not buy signals) |
| Signal Confluence | `/confluence` | **Live** | Picture quality + slot evidence grid + reliability scorecard + signal timeline + recent entry/exit zones (ADR-0012) |
| Price Corridor | `/confluence` (tab) | **Live** | Valuation corridor snapshot, corridor watchlist, corridor-method backtest ledger |
| Market outlook | `/market-outlook` | **Live** | Beginner language. Condition strip; stronger/weaker-vs-market map; seasonality + calendar; rank snapshots; auto leadership check; truck/factory; macro notes. True ETF *fund flow* still later. COT is consumed as a confluence slot, not a dedicated outlook panel. |
| Social / X feed | on overview (density `full`) | **Live** | DB-backed 30d history + admin X credentials/accounts |
| Symbol search | header + watchlist | **Live** | `symbols.search` against local `symbols` table (SEC `company_tickers.json` seed + yfinance hydration). No live vendor search on the request path (ADR-0009 / ADR-0011). |
| Command palette + nav | shell | **Live** | ⌘K, g-key shortcuts, theme presets. Palette covers core pages; some newer routes (Confluence, Funds, Dealer Flow) are sidebar-only. |
### 1.3 Book (portfolio + risk + funds)
| Surface | Route | Status | Notes |
|---------|-------|--------|-------|
| Portfolio book | `/portfolio` | **Live** | Unified holdings + live marks + add/update/remove (`HoldingsBookView`). Optional guided setup via `?strategyId=`. `/monitor` redirects here. |
| Option legs book | `/portfolio`, `/risk` | **Live (MVP)** | `OptionLegsPanel`; `portfolio.optionLegs` / add / remove; posture folds premium-at-risk + CSP cash reserve into exposure/flags |
| Risk posture + sizing | `/risk` | **Live** | `risk.posture` + `sizing.compute` + RiskPosturePanel. Drawdown / cluster / option-risk commentary. **Halt persist is not on this path today** (`risk.posture` returns `halted: false`); `halt_state` table + drawdown alert producer still exist. |
| Tracked funds index | `/funds` | **Live** | List / enable / add / delete tracked funds; 13F sync; capture ingest |
| Fund live book + mirror | `/funds/[id]` | **Live** | Live book (13F + X captures/claims) + mirror diff vs user holdings (ADR-0010: replication math, not advice) |
### 1.4 Process / journal workflow
| Surface | Route | Status | Gap |
|---------|-------|--------|-----|
| Guided start | `/guided-start` | **Live** | Density-filtered strategy presets + short quiz; can fork a preset into the user's strategies |
| Decision plan | `/plan` | **Live** | Server `trades.save`; optional thesis create |
| Theses | `/theses` | **Live** | CRUD on `theses` table |
| Journal / close | `/journal` | **Live** | `trades.list` / `close` + reflection + stats |
| Strategies | `/strategies` | **Live** | Preset catalog + fork; custom authoring still light |
| Strategy lab | `/lab` | **Live** | Backtest equity curve + evaluate latest; portfolio backtest API exists |
| Position considerations | `/exits` | **Live** | `derisking.suggest` + dividend health + close-position helper; EmotionLogger mounted here |
| Screener | `/screener` | **Live** | Filter + strategy screen over watchlist |
| Reports | `/reports` | **Live** | HTML research notes (`reports.generate`) |
| Daily focus / goals | `/daily-focus` | **UI-local** | localStorage goals; density `full` |
| Emotion logger | `/exits` + API | **Partial** | API live; used on Position considerations; journal close uses a reflection field instead |
| Sizing math | `/risk` (calculator) | **Live** | `sizing.compute` + UI cascade (math only, ADR-0007) |
| Derisking suggestions | `/exits` | **Live** | Productized from `derisking.suggest` |
| Thesis monitor | (no screen) | **Stub** | `thesisMonitor.assess` / `timeline` (empty events / empty timeline). Thesis *CRUD* is live on `/theses`. |
Legacy routes `/trade-plan`, `/execution`, and `/trade-closure` are gone. `/mobile` and `/monitor` redirect to `/portfolio`.
### 1.5 Discovery & strategy lab
| Capability | Status | Notes |
|------------|--------|-------|
| Filter screener (M15a) | **Live** | `/screener` + `screener.filter` over watchlist |
| Strategy screener (M15b) | **Live** | `/screener` strategy mode when density ≥ standard |
| Strategy authoring | **Partial** | presets + fork; custom authoring still light |
| Backtest | **Live** | `/lab` equity curve + evaluate latest; `backtest.runPortfolio` for allocation paths |
| Sector confirmation cross-link | **Stub** | `sectorCrosslink.confirm` with empty universe in practice |
| Screener saved filters | **Schema only** | `screener_filters` / `saved_filters` tables exist; incomplete product loop |
### 1.6 Confluence signal engine (M22)
| Capability | Status | Notes |
|------------|--------|-------|
| Slot catalog (34 slots, 6 families) | **Live** | technical, institutional, macro, seasonal, flows, sentiment. ADR-0007 explain text (evidence, never directives). |
| Rack evaluation + picture quality | **Live** | Redundancy-aware evidence; quality labels (strong/moderate/weak bullish/bearish, mixed, sparse) |
| System racks | **Live** | Seeded on boot: Full Confluence, Technical Momentum, Macro + Flows + Sentiment |
| User racks | **Live** | `confluence.saveRack` |
| Closed-loop signal history | **Live** | Fires logged; 4-week follow-through resolver; reliability weights |
| Entry/exit zone rules | **Live** | Weekly walk-forward derivation on GICS sector ETFs + SPY; last 3 entry + 3 exit windows on the page |
| Incremental replay | **Live** | Hourly tick + boot fill of ~3y lookback, budgeted |
| Confluence change alerts | **Live** | `confluence_change` producer (picture improved / deteriorated) |
| CFTC COT adapter | **Live (data)** | Used by `cotPositioning` slot; no standalone COT screen |
| 15-symbol learning universe | **Live** | Pinned into demand set on startup (PLTR, NVDA, AMD, AAPL, MSFT, SMH, XOM, JPM, UNH, COST, AMZN, CAT, LMT, LIN, NEE + SPY) |
### 1.7 Mirror portfolio (M21)
| Capability | Status | Notes |
|------------|--------|-------|
| Tracked fund CRUD | **Live** | CIK + manager + optional X handle; enable/disable |
| 13F sync into fund records | **Live** | `funds.adminSync13f` |
| X capture / trade-claim ingest | **Live** | `funds.adminIngestCaptures` from cached X posts |
| Live book | **Live** | Most recent record per symbol; 13F vs capture vs claim labeled; evidence URL |
| Mirror diff | **Live** | User-entered capital base (default $200k) + optional floor; share/value delta to match disclosed weights (ADR-0010) |
| Mirror alerts | **Live** | `fund_capture`, `fund_13f`, `mirror_diff` producers |
| Manager Form 4 as fund book | **Missing (by design)** | Personal insider activity is not part of the disclosed book |
### 1.8 Options convexity sleeve (M17)
| Capability | Status | Notes |
|------------|--------|-------|
| Options DD (read-only) | **Live** | Teaching panel |
| Option legs book + risk contribution | **Live (MVP)** | On `/portfolio` + `/risk` |
| Unlock ladder + payoff API | **Removed** | `optionsConvexity` router is gone. `ConvexityGate` is a stub (unlock system removed; strategies are not gated by a sleeve ladder). |
| Sleeve risk budget enforcement | **Missing** | Combined stock+options ceiling is not enforced as a product path |
### 1.9 Macro (M18)
| Capability | Status | Notes |
|------------|--------|-------|
| FRED series / key admin | **Live** | Admin X Accounts page also manages FRED key; series warmed by queue schedule (`fred`, daily) |
| Truck sales + manufacturing PMI charts | **Live** | Market Outlook |
| Sector rotation heatmap-style panel | **Live** | Relative-strength vs SPY, not measured fund flows |
| Custom rotation ETFs | **Live** | `market.addCustomEtf` / remove / list |
| Regime classify / history | **Backend-ready** | `macro.regimeClassify`, `regimeHistory` |
| Portfolio-impact commentary | **Live** | `macro.commentary` via Market Outlook |
| Full economic calendar UI | **Stub** | `macro.calendar` returns cached events (often empty) |
### 1.10 Alerts
| Capability | Status | Notes |
|------------|--------|-------|
| Alert list / ack / unacked / clear-all | **Live** | `/alerts` + notification bell |
| Alert subscriptions + per-type toggles | **Live** | Ticker / list / global inheritance; `alerts.listTypes` / `toggleType` |
| Email delivery (SMTP) | **Live** | Nodemailer, `/admin/smtp`, outbox table, 60s drain, rate limit 1/hr per type/symbol/user |
| SSE push | **Missing** | Design called for SSE; poll/list only |
**Producers (registered at backend boot):**
| Type | Frequency | Status |
|------|-----------|--------|
| `informed_buy`, `informed_sell` | batched (10 min) | **Live** |
| `new_13da` | batched | **Live** |
| `new_13f` | batched | **Live** |
| `vix_level` | realtime (30s loop) | **Live** |
| `rotation_incipient`, `regime_shift` | batched | **Live** |
| `conviction_unlock` | batched | **Live** |
| `thesis_broken`, `thesis_weakening` | batched | **Live** (depends on thesis rows + monitor input; monitor events still thin) |
| `cluster_breach`, `drawdown_halt`, `asymmetry_warning` | batched | **Live** |
| `confluence_change` | batched | **Live** |
| `fund_capture`, `fund_13f`, `mirror_diff` | batched | **Live** (lazy register) |
Admin default: every catalog type is seeded ON for `is_admin=1` (idempotent).
### 1.11 Mobile companion
| Capability | Status | Notes |
|------------|--------|-------|
| Dedicated `/mobile` app | **Redirect** | Redirects to Portfolio; main shell is the phone surface |
| Mobile shell (iPhone Air) | **Live** | Fixed `dvh` chrome, safe-area insets, bottom tabs (Portfolio · Risk · Research · Alerts · More). CSS-first breakpoints so mobile never flashes desktop chrome. |
| More sheet | **Live** | Density- and module-filtered secondary destinations |
| Mobile lists sheet | **Live** | Header list icon opens portfolio + watchlist bottom sheet for symbol switch |
| Portfolio on phone | **Live** | Card list + 2×2 snapshot under `md`; wide table from `md` up |
| Rotation / institutional on phone | **Live** | Card rows with key horizons; full tables from `md` up with sticky first column |
| Mobile polling hygiene | **Live** | Snapshot / alerts / market condition pause while document is hidden |
### 1.12 Admin / operator
| Capability | Status |
|------------|--------|
| Users, sessions, reset password, enable/disable/delete | **Live** | System sentinel `anonymous@investor-flow.local` (guest book owner, id `anonymous`) is hidden from the list and cannot be deleted. |
| Per-user module assignment | **Live** |
| Pending approval queue | **Live** |
| Adapter queue health, global pause/resume, **per-source pause/stop/start**, retry, schedules, error stacks | **Live** | Cooldown column ticks 429 pauses; blank is not-paused. Unhealthy Yahoo shows backlog/notes. T0/focused quotes drain first. |
| SEC queue fetch + lint holders/insiders + data quality | **Live** |
| Dealer map integrity (live or replay) | **Live** |
| Alert producer run status | **Live** | `admin.alertStatus` |
| X credentials, accounts, prune | **Live** |
| FRED key | **Live** |
| FINRA download URL | **Live** | Admin-configurable; `finra-bulk` schedule not auto-seeded (historical 403) |
| SMTP config (host, port, auth, test) | **Live** | `/admin/smtp` |
| Audit log | **Live** |
| Server restart control | **Live** | Meaningful on a laptop process; less so inside Unraid (`restart: unless-stopped` is the operator path) |
| GDPR export | **Backend-ready** | admin API only |
### 1.13 Reports
| Capability | Status |
|------------|--------|
| HTML research note generator | **Live** | `/reports` + `reports.generate` (symbol, portfolio, watchlist, risk posture, rotation) |
---
## 2. Functional architecture (intended jobs)
```
Job A - Monitor (mobile + alerts)
alerts.list/ack · portfolio.holdings · rotation chip · VIX / 13D / insider
Job B - Research (desktop workbench)
watchlist → active symbol → market + SEC + institutional + options + social
market outlook (macro + rotation)
dealer flow map + study desk
confluence picture + corridor
Job C - Process (planned investor loop)
thesis + confluence + sizing → plan → execute → emotion → close
[plan / theses / journal persist server-side; daily focus still UI-local]
Job D - Discover
filter/strategy screener → open in workbench
guided-start presets → fork → optional portfolio setup
Job E - Mirror (M21)
tracked fund live book → mirror diff vs user book (math, not advice)
```
---
## 3. Design principles still in force
Unchanged from domain model / ADRs:
1. **Primary Rule (ADR-0007)** - education, not advice; Primary-Rule lint test on curated strings.
2. **Analyst Voice (ADR-0005)** - Alfred/Druckenmiller blend where LLM copy is used.
3. **P6 plain English / P7 teach mechanics** - in-context cockpit labels (not a curriculum product).
3b. **Adaptive workspace density** - experience interview sets focused/standard/full; user can change anytime.
3c. **Module access** - operator decides which families a user can open; density then filters within those families.
4. **Local-first multi-tenant (ADR-0001)** + shared cache dedupe (ADR-0004).
5. **LLM data provenance (ADR-0006)** - sensitive data stays on local-class providers; full gateway module is still thin vs original design. Per-user OpenAI-compatible endpoint is live for Dealer Flow explain.
6. **Anti-gamification** - no confetti / outcome celebration.
7. **Rate-limit-first data plane (ADR-0009)** - stale UI beats vendor stampede.
8. **Mirror math, not advice (ADR-0010)** - user states the replication goal; app computes the delta.
9. **Symbol Search Index (ADR-0011)** - local-first autocomplete; SEC seed owns issuer identity.
10. **Confluence is evidence (ADR-0012)** - picture-quality labels, never directives.
---
## 4. Explicit non-goals (still)
- Brokerage execution or custody
- Investment adviser recommendations
- Mobile-first full research workbench
- Undefined-risk options strategies for beginners
- Training on user portfolio/thesis content via external LLMs
- Treating a tracked fund's personal Form 4 as the fund's disclosed book
---
## 5. Gap priority (product value)
| P | Gap | Why it matters | Status |
|---|-----|----------------|--------|
| P0 | Fix failing MacroRegime commentary tests | Suite health | **Done** (2026-07-18) |
| P0 | Expose SizingEngine + RiskEngine on tRPC + M20 UI | Risk job | **Done** (`/risk`) |
| P1 | Persist trade plan / thesis / journal server-side | Process loop | **Done** (`/plan`, `/theses`, `/journal`) |
| P1 | Frontend client coverage for backend-only routers | APIs unreachable from SPA | **Mostly done** - `lib/trpc.ts` now covers the product routers; leftovers are thin (gdprExport, some macro helpers) |
| P1 | Wire emotion logger UI to `emotionLogger` API | Dual paths confuse | **Partial** - live on `/exits`; journal uses reflection |
| P2 | Strategy Lab + backtest UI | Learning surface | **Done** (`/lab`, `/strategies`) |
| P2 | Screener UI (filter + strategy) | Discovery job | **Done** (`/screener`) |
| P2 | Derisking + thesis monitor product surfaces | Process depth | Derisking **done** (`/exits`); thesis monitor still **stub** |
| P2 | Options convexity sleeve UI | M17 incomplete | Legs **live**; unlock ladder **removed**; budget not enforced |
| P2 | Confluence + corridor | Evidence picture | **Done** (M22) |
| P2 | Mirror portfolio | Fund-first job | **Done** (M21) |
| P3 | Reports UI | Export pedagogy | **Done** (`/reports`) |
| P3 | Wire remaining alert producers | Awareness loop | **Done** (see §1.10) |
| P3 | SSE alerts / continuous event producers | Ops polish | **Open** |
| P3 | Full LLM Gateway as designed | Provenance enforcement completeness | **Open** |
| P3 | Persist risk halt onto `risk.posture` | Circuit breaker product path | **Open** (table + producer exist; posture hardcodes `halted: false`) |
| P3 | Persist daily-focus goals | Multi-device process | **Open** |
| P3 | Saved screener filters product loop | Repeatable discovery | **Open** |
---
## 6. Related docs
- Tech design (modules, stack, API map, CI/CD): [`TECH_DESIGN.md`](./TECH_DESIGN.md)
- Unraid operator / CI/CD process: [`DEPLOY_UNRAID.md`](./DEPLOY_UNRAID.md)
- Vendor integration rules: [`VENDOR_INTEGRATIONS.md`](./VENDOR_INTEGRATIONS.md)
- Domain glossary: [`../CONTEXT.md`](../CONTEXT.md)
- ADRs: [`adr/`](./adr/)
- Living wiki page: Obsidian `investor-flow.md` (orientation; this file wins for status)