Commit Graph
19 Commits
Author SHA1 Message Date
Investor Flow Build 76f60dc766 feat: Phase 2 NASDAQ adapter for days-to-cover + 24mo history
- Add nasdaq SourceKind + nasdaqShortinterest KindHandler (kv_cache with nasdaq: prefix)
- Create NasdaqAdapter: calls api.nasdaq.com/quote/SYM/short-interest, parses settlementDate/sharesShort/daysToCover/avgVolume, returns 24mo history; handles non-NASDAq gracefully
- Register NasdaqAdapter in server index.ts adapters map
- Update shortInterest procedure: reads both yfinance + nasdaq caches in parallel, reconciles (NASDAQ for daysToCover/sharesShort, Yahoo for %float/shortRatio and NYSE fallback)
- Update ShortInterestPanel: display daysToCover, settlement date, source badges, 6-month history table, updated disclaimer
- Update trpc.ts client type for merged response shape
2026-07-25 12:59:43 -04:00
Investor Flow Build 605dc7f3d1 feat: short interest panel Phase 1 Yahoo Finance
- Add short_interest TtlClass (24h) to CacheRepository.ts with shortInterestHandler KindHandler (kv_cache JSON pattern), register in HANDLERS and del switch case
- Add case 'shortinterest' to YFinanceAdapter.fetchOne using quoteSummary(defaultKeyStatistics): parses sharesShort, shortPercentOfFloat, shortRatio, dateShortInterest, sharesShortPriorMonth, floatShares, sharesOutstanding
- Add institutional.shortInterest tRPC procedure (cache-read pattern, returns stale flag + provenance)
- Add api.institutional.shortInterest to frontend trpc client
- Create ShortInterestPanel.tsx: mirrors AnalystRatings pattern with loading/error/retry states, formatted shares/ratio/percentage display with source date
- Mount ShortInterestPanel on overview page after AnalystRatings
2026-07-25 12:09:21 -04:00
Investor Flow Build 5b9f770aa4 Phase 5: alert subscriptions UI + multiple watchlists + queue fixes
CI / Test & Type-Check (push) Canceled after 0s
UI:
- /alerts page: event history with acknowledge, subscription create/manage with toggle
- /admin/smtp: SMTP config form (host, port, auth, test)
- Watchlist sidebar: dropdown selector for multiple watchlists, create/delete
- Sidebar: alerts count badge, SMTP link under admin
- Mobile tab nav: alerts tab added
- Client trpc.ts: all new API methods + types

Backend:
- watchlists.listByWatchlist procedure + listSymbolsByWatchlist repo fn
- yfinance min-interval 1500->2000ms to reduce Edge 429s
- Fixed e.date.slice error in yfinance-adjustments with typeof guard
- Removed defunct BITF from watchlist+queue
- Cleared 83 failed + 12 backoff queue jobs

Docs:
- FUNCTIONAL_DESIGN.md: alerts + multiple watchlists + SMTP documented
- TECH_DESIGN.md: new modules, tRPC procs, routes updated
2026-07-23 20:51:47 -04:00
Investor Flow Build e262187c3c fix: backfill symbol_demand for sidebar-added symbols + analyst ratings schema fix
- Add await ctx.cache.subscribe() to addSymbol mutation so symbols
  added via the sidebar get registered in symbol_demand and yfinance
  jobs are queued immediately
- Backfill PEP, WYNN, STZ, CELH into symbol_demand + adapter_queue
- Upgrade yahoo-finance2 3.15.3 -> 3.15.4 and pass validateResult:false
  to quoteSummary() to handle Yahoo schema drift
- Add error detail logging for analyst ratings schema failures
- Update .gitignore with common ignores
2026-07-23 18:02:24 -04:00
Investor Flow Build 5ef2b2f060 feat(sec-lint): lint+backfill system for SEC data gaps (B1-B4)
- SecLintAdapter implements SourceFetch, runs via shared queue/drain loop
- Two new SourceKinds: sec-lint-holders, sec-lint-insiders (weekly schedules)
- No-op cache handlers so drain->cache.set doesn't throw on lint keys
- tRPC admin.queueLint(symbol, kind) — run lint for one symbol, returns LintResult
- tRPC admin.queueLintAll(kind) — backfill ALL watched symbols at once
- tRPC admin.dataQualityList() — query data_quality rows (filterable by symbol/kind)
- InstitutionalDashboard: 'Lint holders' button + status badge in detail panel header
- Admin queue page: Data Quality section with per-row status badges, 'Lint all' buttons
- DEFAULT_RATE_MS includes 167ms (~6 req/s) for lint kinds matching EDGAR limiter
2026-07-12 20:19:12 -04:00
Investor Flow BuildandClaude ca385c1960 feat(queue): adapter queue overhaul — pause/resume, error capture, scheduling
- Pause/resume persisted in queue_state; drain loop checks isPaused() and skips
- Error capture: every failed/backoff attempt logged to queue_errors with the
  full stack trace; admin queue page expands a failed job to stream it
- Retry controls: retryJob(key), retrySource(kind), clearDone(olderThanMs)
- Per-source scheduling: queue_schedules table + 30s enqueueDueSchedules loop
  (seed defaults sec-fetch 24h, yfinance 5min); admin UI lists/adds/deletes
- Startup recovery: interrupted in_flight jobs reset to pending on boot
- fix(edgar): archive URLs use the filer CIK (accession-number prefix), not the
  company CIK — resolves Cloudflare 429 that left SEC backfills sparse/empty
- Migration: add queue_errors, queue_schedules, queue_state tables plus error/
  scheduled_for columns on adapter_queue (idempotent ALTER on startup)

Verified: full sec-fetch backfill now succeeds for all watched symbols
(NVDA 14,675 institution filings, CIFR 274 insider txns, TSLA 6,011, etc.).
503 backend tests pass.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-07-12 12:55:59 -04:00
Investor Flow Build 55f07e6b42 fix: wire data adapters, fix OptionsAdapter v3, build Research pages
- Fix OptionsAdapter for yahoo-finance2 v3 API ({ date: expiry } instead of positional arg)
- Fix institutional.flow and insiderStream endpoints to query correct DB tables
- Fix anonymous user INSERT (pw_hash column, not password_hash)
- Create secDataFetcher module for 13F/Form 4 data population
- Add institutional and marketOutlook endpoints to frontend trpc client
- Build out Chart Lab, Institutional, Market Outlook pages with real components
- Add focus-visible rings, aria-labels, hex tokens, shape icons across all panels
- Integrate EmotionLogger, MobileTabNav, useKeyboardShortcuts
- Add localStorage persistence for emotion logs
- Normalize spacing, type scale, hover effects across all components
2026-07-05 08:50:12 -04:00
Investor Flow Build 1007ab4ed5 fix (ornith-35): watchlistRepository double-encoding bug — single JSON.stringify, 13/13 tests pass 2026-06-30 17:54:01 -04:00
Investor Flow Build 97607e0bd4 slice 6 fix (ornith-35): form4_tx double-.xml URL + correct Form 4 XML elements (rptOwnerCik/rptOwnerName/rptOwnerRelationship). qwopus re-review pending 2026-06-30 13:56:29 -04:00
Investor Flow Build c2476ab137 slice 6 EdgarAdapter: fix full_text_search ETag caching (was not writing etags back); test asserts revalidation on 2nd call. 16/16 green (ornith-35) 2026-06-30 13:40:19 -04:00
Investor Flow Build cf1d7c7302 slice 6 EdgarAdapter 13f_holdings + form4_tx (ornith-35): edgarXmlFetch + regex parsers, ETag/rate-limit/cache, ADR-0007 2026-06-30 13:33:51 -04:00
Investor Flow Build cfc762e952 slice 15a OptionsAdapter (ornith-35): expiry_dates + options_chain (calls/puts/IV/greeks), yf2 lazy singleton, options_snapshot/intraday TTL, ADR-0007
Cross-review by qwopus35b pending.
2026-06-30 13:03:30 -04:00
Investor Flow Build e1e028b418 slice 16 X-cookie + Reddit adapters (qwopus35b): cashtag_search/trusted_timeline (1 req/3s, cookie-expiry->FAILED+degraded), Reddit public-JSON, 7d cache, FakeLLM, crowd-sentiment-not-edge caveat
Cross-review by ornith-35 pending. 17 new tests (40 adapter total).
2026-06-30 10:08:25 -04:00
Investor Flow Build 9f58d36566 slice 6a EdgarAdapter (ornith-35): filings_index/company_facts/filer_cik_meta/full_text_search + edgarFetch (UA, 8 req/s, ETag/If-Modified-Since, 304 no-op)
Cross-review by qwopus35b pending (after tests 6b).
2026-06-30 10:02:50 -04:00
Investor Flow Build e6b7219dd2 slice 4d (omlx/ornith-35): backfill integration test (first-track 4 keys -> drain -> candles+adjustments; rerun no-op)
Slice 4 (yfinance-backfill-permanent-ohlcv) COMPLETE: parseAdjustments + adjustments
kind + candles 10y backfill + adjustments fetchOne + subscribe queues backfill. 102/102
tests green. Implemented by local ornith-35, reviewed by orchestrator.
2026-06-29 22:09:12 -04:00
Investor Flow Build 2a99e008e3 slice 4c (omlx/ornith-35): YFinanceAdapter candles 10y backfill + adjustments fetchOne 2026-06-29 21:58:39 -04:00
Investor Flow Build 190e9500a9 slice 4a (omlx/ornith-35): parseAdjustments (chart events -> price_adjustments) + 12 tests
First dispatch to the LOCAL ornith-35 (omlx) — ~3min vs ~8min for remote. Pure fn
parses yf2 chart events (splits/dividends, array or map) into PriceAdjustment[]; 12
tests incl. NVDA 5y fixture (2 splits + 26 dividends). Reviewed + approved.
2026-06-29 21:50:27 -04:00
Investor Flow Build 82079b3e66 slice 1e-1f: tRPC router (auth + market.snapshot) + node:http server
auth.signup/login/logout/me with signed HMAC session cookies + scrypt hashing;
market.snapshot mega-endpoint (quote+candles+sector). node:http server mounts
tRPC at /api/trpc + /health + background drain loop. Live-verified: real NVDA
194.97/65 candles/Technology served after stale-while-revalidate drain. 36 tests green.
2026-06-29 17:40:42 -04:00
Investor Flow Build 1962ecc740 slice 1a-1d: DB + CacheRepository + YFinance adapter + AdapterQueue
Node 26 + node:sqlite backend (zero native deps; runtime glue adapted from
Bun-spec design, deep-module architecture unchanged). 29 tests green across
schema/CacheRepository/YFinance-parse/AdapterQueue-dedupe.
2026-06-29 17:28:57 -04:00