- Fix default base URL: https://cdn.finra.org/equity/regsho/daily (was www.finra.org/sites/default/files)
- Fix filename pattern: CNMSshvol{YYYYMMDD}.txt (was CAshvol with year/month subdir)
- Fix parser: actual file columns are Date|Symbol|ShortVolume|ShortExemptVolume|TotalVolume|Market
(was incorrectly expecting Market|Symbol|... — first col is Date, last col is Market)
- Dedupe by symbol (volume is total across all listed exchanges, not per-exchange)
- Store combined market codes (e.g. 'B,Q,N') as exchange string
- Use BEGIN/COMMIT instead of db.transaction() (node:sqlite DatabaseSync doesn't have transaction())
- Update finra_config defaults in schema.sql + migration + admin fallback
- Add 6s server-side timeout to Yahoo Finance quoteSummary (Promise.race) so the backend responds with stale data instead of hanging indefinitely. Stale cache is served on error (router.ts:1895-1898), matching ADR-0009.
- Hoist YahooFinance to a module-level singleton (getYf()) so the crumb/cookie is fetched once, not per call. Matches YFinanceAdapter pattern.
- Add suppressNotices for yahooSurvey to reduce log noise.
- Add accession column to institution_filings (schema.sql + migration in client.ts) and thread it through both secDataFetcher.ts INSERT paths (13D/G and 13F-HR). Fixes recurring `no such column: accession` error in new13daProducer alert tick.
- new13daProducer.ts needs no changes - its SELECT accession query now works.
- Add await ctx.cache.subscribe() to addSymbol mutation so symbols
added via the sidebar get registered in symbol_demand and yfinance
jobs are queued immediately
- Backfill PEP, WYNN, STZ, CELH into symbol_demand + adapter_queue
- Upgrade yahoo-finance2 3.15.3 -> 3.15.4 and pass validateResult:false
to quoteSummary() to handle Yahoo schema drift
- Add error detail logging for analyst ratings schema failures
- Update .gitignore with common ignores
- Pause/resume persisted in queue_state; drain loop checks isPaused() and skips
- Error capture: every failed/backoff attempt logged to queue_errors with the
full stack trace; admin queue page expands a failed job to stream it
- Retry controls: retryJob(key), retrySource(kind), clearDone(olderThanMs)
- Per-source scheduling: queue_schedules table + 30s enqueueDueSchedules loop
(seed defaults sec-fetch 24h, yfinance 5min); admin UI lists/adds/deletes
- Startup recovery: interrupted in_flight jobs reset to pending on boot
- fix(edgar): archive URLs use the filer CIK (accession-number prefix), not the
company CIK — resolves Cloudflare 429 that left SEC backfills sparse/empty
- Migration: add queue_errors, queue_schedules, queue_state tables plus error/
scheduled_for columns on adapter_queue (idempotent ALTER on startup)
Verified: full sec-fetch backfill now succeeds for all watched symbols
(NVDA 14,675 institution filings, CIFR 274 insider txns, TSLA 6,011, etc.).
503 backend tests pass.
Co-Authored-By: Claude <noreply@anthropic.com>
totp.ts + backup-codes.ts produced by ornith (remote model, reviewed+approved);
router/context/schema integration by orchestrator (delicate edit-existing — ornith
timed out on a dispatch of that size in-session). 58/58 backend tests green.
2FA flow: enable2fa -> confirm2fa -> login now requires a valid TOTP.
Investor Flow Build