Replay of 80 as-of days per symbol was a sync CPU loop; boot and symbol-page reads ran it on the main thread, so login and every tRPC call hit the 8s timeout. Cap wall time, yield every as-of, and stop filling the learning ledger on boot.
Boot replay was a tight CPU loop that wedged the Unraid backend, so login hit the 8s client timeout. Yield every few as-of dates, shorten the boot ledger fill, and make /health a cheap liveness check.
Yahoo and SEC drain on separate workers so hung EDGAR jobs cannot freeze watchlist prices. Queue health splits yahooHealthy from institutional backfill, watchlist snapshots only enqueue missing quotes, and first subscribe seeds quote/candles/symbol. Hide the anonymous system user from Admin so it cannot be deleted.
First visit lands on /welcome instead of burying signup in Settings.
Add a host CLI to reset passwords without a session, plus Settings
change-password. Refresh as-built design docs and the Unraid operator guide.
Ship Node production images, Unraid compose, and Gitea CI/CD (test then
push registry images; cron script if no runner). Rebuild dealer flow as a
heatmap-first map with integrity gates and chart helpers. Add confluence
zone rules, session clock, capture evidence, and tighter 13F/queue/options
paths, plus the matching UI and tests.
Add operator pause/resume/stop/start for individual adapter sources,
persisted in queue_state. Pause freezes draining and retains jobs; stop
clears pending/backoff jobs and blocks re-enqueue until start. Wires admin
tRPC endpoints plus a clean two-row Actions layout in the queue admin page.
fund_capture alerts for a symbol with multiple capture records sharing one
as_of re-fired every batch cycle: the single comparison-state slot
ping-ponged between sibling evidence URLs, and the pre-check dedup key
never matched the stored buildDedupKey format. Track a seen-set of URLs in
comparison state and align dedup-key construction with buildDedupKey.
Also add alerts.clearAll (hard delete) with an immediate-refresh bell.
- Add cancelled flags to evaluation and scorecard effects so stale
responses from a previous symbol cannot overwrite current data
- Clear evaluation and scorecard state immediately in pickSymbol so
old data never flashes when switching symbols
- Add key={symbol} to ConfluenceChart to force clean remount on
symbol change
- Add searchQuery state to ConfluencePanel that tracks input value
- Pass value={searchQuery} and onChange={setSearchQuery} to SymbolAutocomplete
- Sync searchQuery when activeSymbol changes externally (from another panel)
- Update SymbolAutocomplete onChange to call externalOnChange in controlled mode
- Input now shows current symbol after pick and stays in sync
- Move symbol badge inline with input (flex items-end) so symbol and
rack inputs align at the bottom
- Move Signal Timeline chart above Slot Evidence table
- Deduplicate signal fires by slotId per day in chart data processing
and by slotName in tooltip rendering (prevents MACD Bullish etc
from appearing twice)
- New ConfluenceChart component renders daily price as area chart
- Overlays colored dot markers at each signal fire date (green=bull,
red=bear, amber=exit)
- Detects entry zones (2+ bull signals within 5 days) and exit zones
(2+ bear/exit signals) and shades them on the chart
- Custom tooltip shows price + which signals fired on hover
- Fetches candles and signal history in parallel on mount
- Placed between Slot Evidence and Reliability Scorecard sections
Returns signal fire history for a symbol (last 6 months) with slot
names and body types. Used by the upcoming confluence chart to plot
signal markers on price data.
- Tab label, panel title, ariaLabel, and explanatory text all updated
- Add visible symbol badge below autocomplete input so user always sees
what symbol they're viewing after picking from search results
- Remove value prop from SymbolAutocomplete (was making input read-only)
- Use inputClassName with app design tokens (bg-surface-sunken, border-line,
rounded-lg) to match SymbolHeader and other pages
- Make symbol input, watchlist input, and rack select all use same
border/radius/padding/font-size tokens
- Both panels now consistent with each other and the rest of the app
- Replace plain text inputs with SymbolAutocomplete (existing component)
in both CorridorPanel and ConfluencePanel
- Add explanatory text to both panels describing what the visualization
shows and how to interpret results
- Fix symbol reset bug: panels now sync with global activeSymbol store
so symbol persists across tab switches and page reloads
- Add 'Run evaluation now' button when no corridor snapshot exists,
wired to existing runEvaluationNow mutation
Fix stuck adapter_queue jobs (ASTS/IRE/IREN stuck pending forever):
1. fetchSpec early-return paths (cooldown checks) now update job status
to 'backoff' with last_attempt set and 30s backoff_until, instead of
returning without any status change. Prevents jobs from being
re-processed every drain cycle indefinitely.
2. Wrap adapter.fetchOne() in 30s Promise.race timeout. A hung HTTP
request no longer blocks the entire per-source promise chain
forever, preventing all subsequent jobs for that source.
Also includes corridor/confluence feature, tiered quote schedules,
cache improvements, and HoldingsBookView refinements.
ADR-0012 documents the 34-slot confluence signal engine architecture: slot
catalog, redundancy-aware rack evaluation, CandleProvider seam, closed-loop
signal history, and picture-change alert producer. CONTEXT.md gains a new
'Confluence Signal Engine (M22)' glossary section.
ConfluencePanel shows the current picture quality for a symbol+rack, a
color-coded evidence summary, a per-slot fired/not-fired evidence grid with
notes, and the reliability scorecard. Uses the existing LayoutShell and
CollapsibleSection patterns. ADR-0007: quality labels are evidence-based
descriptions, never buy/sell directives.
Confluence startup seed: pins the15-symbol research universe + SPY benchmark
into the permanent demand set, and creates three system rack presets (Full
Confluence, Technical Momentum, Macro+Flows+Sentiment) if absent. Called once
from index.ts; idempotent on every restart.
Pure, tested evaluator modules for the seasonal family (5 slots) and the
candle-computable technical family (15 slots). Uses the CandleProvider seam
(slice 8) and the indicator functions from slice 1. ADR-0007 evidence-only
notes on every assessment.
CANDLE_PROVIDER resolves a symbol's candles per-granularity (1d/1wk) from the
shared cache, with a reversible fold-in of the freshest live quote so a
mid-session evaluation sees the current price. Pure fold logic + todayIso are
unit-tested; CacheCandleProvider is a thin cache-backed shim. Evaluators use
this seam instead of cache.get inline, so a future realtime/replay source can
slot in without touching slot logic.
Parse the annual FinFutYY.txt zip (leveraged-funds long/short + OI) into a
CotSeries. Registers the 'cot' source_kind, 'cot_weekly' TTL (7d), the 'cftc'
vendor family (1 req/1.5s pacing), and the adapter in the queue registry.
Uses the file-header column names so parser is robust to layout changes.
New functions in indicators.ts:
- sma: Simple Moving Average
- crossover: golden/death cross detector
- roc: Rate of Change (momentum)
- timeSeriesMomentum: 12-1 month momentum (Moskowitz et al.)
- relativeStrength: symbol vs benchmark momentum differential
- realizedVol: annualized realized volatility from log returns
- volatilityPercentile: percentile rank of current vol vs history
- fibLevels: Fibonacci retracement and extension levels
- fibCluster: proximity check for Fib confluence zones
New file volumeByPrice.ts:
- buildVolumeProfile: ported from frontend volumeProfile.ts
- Identical algorithm: bin clamping, uniform volume split, POC/VAH/VAL
All functions are pure (zero deps, no I/O). Tests added for all new
functions with edge cases, error handling, and numerical correctness.
The stale-while-revalidate pattern queues background fetches via the AdapterQueue,
but with 227+ pending jobs, short interest keys take too long to process.
Users saw empty panels on every symbol because the fetch never completed before
the response returned.
Fix:
- shortInterest procedure: when Yahoo + NASDAQ caches are both empty, fetch
directly via the adapters, store in cache, and return — all within the request.
Subsequent requests hit the cache. Bypasses the clogged queue entirely.
- FinraBulkAdapter: guard against per-symbol keys (non-'latest' and non-date IDs)
by returning a skipped result instead of trying to download a file named
after the ticker.
- finraShortInterest handler: return isStale=false when no data is in the table,
so per-symbol reads never trigger queue fetches (FINRA is schedule-only).
Router stale computation now excludes FINRA from the combined stale flag.
- Add yfinance:shortinterest: and nasdaq:nasdaqShortinterest:
to subscribe() initial queue list so short interest data is fetched proactively
when a symbol is first viewed, rather than only on stale read
- FINRA remains bulk-schedule only via queue_schedules
- Fix default base URL: https://cdn.finra.org/equity/regsho/daily (was www.finra.org/sites/default/files)
- Fix filename pattern: CNMSshvol{YYYYMMDD}.txt (was CAshvol with year/month subdir)
- Fix parser: actual file columns are Date|Symbol|ShortVolume|ShortExemptVolume|TotalVolume|Market
(was incorrectly expecting Market|Symbol|... — first col is Date, last col is Market)
- Dedupe by symbol (volume is total across all listed exchanges, not per-exchange)
- Store combined market codes (e.g. 'B,Q,N') as exchange string
- Use BEGIN/COMMIT instead of db.transaction() (node:sqlite DatabaseSync doesn't have transaction())
- Update finra_config defaults in schema.sql + migration + admin fallback
- Add finra-bulk case to enqueueDueSchedules() — queues a single
finra-bulk:finraShortinterest:latest key per cycle
- Register default twice-monthly schedule (14-day interval) in seedDefaultSchedules()
- Removes need for any external cron; the existing 30s scheduler loop handles it
- Add short_interest TtlClass (24h) to CacheRepository.ts with shortInterestHandler KindHandler (kv_cache JSON pattern), register in HANDLERS and del switch case
- Add case 'shortinterest' to YFinanceAdapter.fetchOne using quoteSummary(defaultKeyStatistics): parses sharesShort, shortPercentOfFloat, shortRatio, dateShortInterest, sharesShortPriorMonth, floatShares, sharesOutstanding
- Add institutional.shortInterest tRPC procedure (cache-read pattern, returns stale flag + provenance)
- Add api.institutional.shortInterest to frontend trpc client
- Create ShortInterestPanel.tsx: mirrors AnalystRatings pattern with loading/error/retry states, formatted shares/ratio/percentage display with source date
- Mount ShortInterestPanel on overview page after AnalystRatings
- Add 6s server-side timeout to Yahoo Finance quoteSummary (Promise.race) so the backend responds with stale data instead of hanging indefinitely. Stale cache is served on error (router.ts:1895-1898), matching ADR-0009.
- Hoist YahooFinance to a module-level singleton (getYf()) so the crumb/cookie is fetched once, not per call. Matches YFinanceAdapter pattern.
- Add suppressNotices for yahooSurvey to reduce log noise.
- Add accession column to institution_filings (schema.sql + migration in client.ts) and thread it through both secDataFetcher.ts INSERT paths (13D/G and 13F-HR). Fixes recurring `no such column: accession` error in new13daProducer alert tick.
- new13daProducer.ts needs no changes - its SELECT accession query now works.
- trpcQuery/trpcMutate now throw 'Request timed out (8s)' instead of
generic 'The operation was aborted.' so users know WHY the request failed
- AnalystRatings adds cancelled guard (race condition fix) so stale
promise results from symbol changes are ignored (matches OverviewPanel
pattern)
- Add Retry button on error state for timeout or other failures
- useEffect cleanup returns cancelled flag setter
- Add await ctx.cache.subscribe() to addSymbol mutation so symbols
added via the sidebar get registered in symbol_demand and yfinance
jobs are queued immediately
- Backfill PEP, WYNN, STZ, CELH into symbol_demand + adapter_queue
- Upgrade yahoo-finance2 3.15.3 -> 3.15.4 and pass validateResult:false
to quoteSummary() to handle Yahoo schema drift
- Add error detail logging for analyst ratings schema failures
- Update .gitignore with common ignores
- SecLintAdapter implements SourceFetch, runs via shared queue/drain loop
- Two new SourceKinds: sec-lint-holders, sec-lint-insiders (weekly schedules)
- No-op cache handlers so drain->cache.set doesn't throw on lint keys
- tRPC admin.queueLint(symbol, kind) — run lint for one symbol, returns LintResult
- tRPC admin.queueLintAll(kind) — backfill ALL watched symbols at once
- tRPC admin.dataQualityList() — query data_quality rows (filterable by symbol/kind)
- InstitutionalDashboard: 'Lint holders' button + status badge in detail panel header
- Admin queue page: Data Quality section with per-row status badges, 'Lint all' buttons
- DEFAULT_RATE_MS includes 167ms (~6 req/s) for lint kinds matching EDGAR limiter
- Pause/resume persisted in queue_state; drain loop checks isPaused() and skips
- Error capture: every failed/backoff attempt logged to queue_errors with the
full stack trace; admin queue page expands a failed job to stream it
- Retry controls: retryJob(key), retrySource(kind), clearDone(olderThanMs)
- Per-source scheduling: queue_schedules table + 30s enqueueDueSchedules loop
(seed defaults sec-fetch 24h, yfinance 5min); admin UI lists/adds/deletes
- Startup recovery: interrupted in_flight jobs reset to pending on boot
- fix(edgar): archive URLs use the filer CIK (accession-number prefix), not the
company CIK — resolves Cloudflare 429 that left SEC backfills sparse/empty
- Migration: add queue_errors, queue_schedules, queue_state tables plus error/
scheduled_for columns on adapter_queue (idempotent ALTER on startup)
Verified: full sec-fetch backfill now succeeds for all watched symbols
(NVDA 14,675 institution filings, CIFR 274 insider txns, TSLA 6,011, etc.).
503 backend tests pass.
Co-Authored-By: Claude <noreply@anthropic.com>
- Fix OptionsAdapter for yahoo-finance2 v3 API ({ date: expiry } instead of positional arg)
- Fix institutional.flow and insiderStream endpoints to query correct DB tables
- Fix anonymous user INSERT (pw_hash column, not password_hash)
- Create secDataFetcher module for 13F/Form 4 data population
- Add institutional and marketOutlook endpoints to frontend trpc client
- Build out Chart Lab, Institutional, Market Outlook pages with real components
- Add focus-visible rings, aria-labels, hex tokens, shape icons across all panels
- Integrate EmotionLogger, MobileTabNav, useKeyboardShortcuts
- Add localStorage persistence for emotion logs
- Normalize spacing, type scale, hover effects across all components
Chart Lab data path: market.candles (cached OHLCV by timeframe) + market.indicators
(EMA/RSI/relativeVolume via analysis/indicators.ts). By ornith-35 (local, ~75s);
reviewed by orchestrator (128/128 tests, no regression). Conflict-free: impl=ornith-35,
reviewer=orchestrator.
Local ornith-35 dispatch (~2min). +PriceAdjustment, +adjustmentsHandler
(price_adjustments table, permanent), subscribe now queues candles+adjustments on
first demand. Surgical, 99/99 tests, no regression.
First dispatch to the LOCAL ornith-35 (omlx) — ~3min vs ~8min for remote. Pure fn
parses yf2 chart events (splits/dividends, array or map) into PriceAdjustment[]; 12
tests incl. NVDA 5y fixture (2 splits + 26 dividends). Reviewed + approved.
totp.ts + backup-codes.ts produced by ornith (remote model, reviewed+approved);
router/context/schema integration by orchestrator (delicate edit-existing — ornith
timed out on a dispatch of that size in-session). 58/58 backend tests green.
2FA flow: enable2fa -> confirm2fa -> login now requires a valid TOTP.
auth.signup/login/logout/me with signed HMAC session cookies + scrypt hashing;
market.snapshot mega-endpoint (quote+candles+sector). node:http server mounts
tRPC at /api/trpc + /health + background drain loop. Live-verified: real NVDA
194.97/65 candles/Technology served after stale-while-revalidate drain. 36 tests green.
Investor Flow Build