feat: Phase 3 FINRA bulk adapter + Phase 4 three-way merge with discrepancy flagging

Phase 3 — FINRA bulk short-interest ingest:
- Add finra_short_interest table to schema (per-symbol, per-settlement-date, per-exchange, with short/exempt/total volume, avg daily vol, days to cover)
- Create FinraIngestService: downloads FINRA consolidated pipe-delimited file from configurable base URL, parses Market|Symbol|ShortVolume|ShortExemptVolume|TotalVolume, stores in finra_short_interest table
- Create FinraBulkAdapter: SourceFetch that calls downloadAndIngestFinra, registers as finra-bulk source kind with finraShortinterest cache handler
- finraShortinterest handler reads latest settlement row per symbol from finra_short_interest table (no per-symbol kv_cache write; data is bulk-ingested)
- Register in index.ts adapter map + HANDLERS + del case

Phase 4 — three-way merge with discrepancy detection:
- shortInterest tRPC procedure now reads all 3 caches (yfinance, nasdaq, finra-bulk) in parallel
- Reconciliation hierarchy: FINRA (shares short) > NASDAQ > Yahoo
- daysToCover: NASDAQ (specific) > FINRA (computed) > Yahoo (short ratio fallback)
- settlementDate: FINRA > NASDAQ > Yahoo
- Discrepancy detection: compares sharesShort across available sources, flags >10% difference with discrepancyPct + discrepancyBetween
- Updated ShortInterestPanel: FINRA source badge, discrepancy warning banner, three-source disclaimer
- Updated trpc.ts client type for new shape
This commit is contained in:
Investor Flow Build
2026-07-25 13:03:49 -04:00
parent 76f60dc766
commit 3fa6b32916
8 changed files with 308 additions and 15 deletions
+40 -10
View File
@@ -1900,27 +1900,54 @@ const institutionalRouter = router({
return { symbol, ratings: result.ratings, consensus: result.consensus };
}),
/** Short interest reconciled across Yahoo Finance + NASDAQ API.
/** Short interest reconciled across Yahoo Finance + NASDAQ API + FINRA.
* Yahoo covers all US-listed tickers (basic: shares short, % float, short ratio).
* NASDAQ covers NASDAQ-listed only but adds days-to-cover + 24mo history.
* Reconciliation: prefer NASDAQ's sharesShort/daysToCover when available (richer),
* fall back to Yahoo for NYSE/other tickers. */
* FINRA covers all exchanges via consolidated twice-monthly files.
* Reconciliation hierarchy (most → least authoritative for each field):
* sharesShort: FINRA > NASDAQ > Yahoo
* daysToCover: NASDAQ (specific) > FINRA (computed) > Yahoo (ratio)
* % of Float: Yahoo only (has floatShares)
* Discrepancy detection: flags >10% difference between top-2 available sources. */
shortInterest: publicProcedure
.input(z.object({ symbol: z.string().min(1) }))
.query(async ({ ctx, input }) => {
const symbol = input.symbol.toUpperCase();
const [yfEntry, ndqEntry] = await Promise.all([
const [yfEntry, ndqEntry, frEntry] = await Promise.all([
ctx.cache.get<Record<string, unknown>>(`yfinance:shortinterest:${symbol}`),
ctx.cache.get<Record<string, unknown>>(`nasdaq:nasdaqShortinterest:${symbol}`),
ctx.cache.get<Record<string, unknown>>(`finra-bulk:finraShortinterest:${symbol}`),
]);
const yf = yfEntry.value;
const ndq = ndqEntry.value;
const fr = frEntry.value;
// Prefer NASDAQ's sharesShort + daysToCover when available (NASDAQ-listed);
// otherwise fall back to Yahoo (covers NYSE/other tickers).
const sharesShort = ndq?.sharesShort ?? yf?.sharesShort ?? null;
const daysToCover = ndq?.daysToCover ?? null;
const settlementDate = ndq?.settlementDate ?? null;
// Collect sharesShort from all sources for discrepancy detection
const sharesSources: Array<{ source: string; sharesShort: number | null }> = [
{ source: 'finra', sharesShort: (fr?.shortVolume as number | undefined) ?? null },
{ source: 'nasdaq', sharesShort: ndq?.sharesShort as number | null },
{ source: 'yfinance', sharesShort: yf?.sharesShort as number | null },
];
const availableSources = sharesSources.filter((s) => s.sharesShort !== null);
let discrepancyPct: number | null = null;
let discrepancyBetween: string | null = null;
if (availableSources.length >= 2) {
const sorted = [...availableSources].sort((a, b) => (b.sharesShort ?? 0) - (a.sharesShort ?? 0));
const hi = sorted[0].sharesShort!;
const lo = sorted[sorted.length - 1].sharesShort!;
if (lo > 0) {
const pct = Math.abs(hi - lo) / lo;
if (pct > 0.10) {
discrepancyPct = Math.round(pct * 100);
discrepancyBetween = `${sorted[0].source} vs ${sorted[sorted.length - 1].source}`;
}
}
}
const sharesShort = fr?.shortVolume ?? ndq?.sharesShort ?? yf?.sharesShort ?? null;
const daysToCover = ndq?.daysToCover ?? fr?.daysToCover ?? yf?.shortRatio ?? null;
const settlementDate = fr?.settlementDate ?? ndq?.settlementDate ?? yf?.dateShortInterest ?? null;
const history = ndq?.history ?? null;
return {
@@ -1936,11 +1963,14 @@ const institutionalRouter = router({
daysToCover,
settlementDate,
history,
discrepancyPct,
discrepancyBetween,
sources: {
yfinance: { available: !!yf, stale: yfEntry.isStale },
nasdaq: { available: !!ndq, stale: ndqEntry.isStale },
finra: { available: !!fr, stale: frEntry.isStale },
},
stale: yfEntry.isStale && ndqEntry.isStale,
stale: yfEntry.isStale && ndqEntry.isStale && frEntry.isStale,
};
}),
});