fix: correct FINRA URL, filename format, parser column order, and default base URL
- Fix default base URL: https://cdn.finra.org/equity/regsho/daily (was www.finra.org/sites/default/files) - Fix filename pattern: CNMSshvol{YYYYMMDD}.txt (was CAshvol with year/month subdir) - Fix parser: actual file columns are Date|Symbol|ShortVolume|ShortExemptVolume|TotalVolume|Market (was incorrectly expecting Market|Symbol|... — first col is Date, last col is Market) - Dedupe by symbol (volume is total across all listed exchanges, not per-exchange) - Store combined market codes (e.g. 'B,Q,N') as exchange string - Use BEGIN/COMMIT instead of db.transaction() (node:sqlite DatabaseSync doesn't have transaction()) - Update finra_config defaults in schema.sql + migration + admin fallback
This commit is contained in:
@@ -698,6 +698,6 @@ CREATE INDEX IF NOT EXISTS idx_finra_settlement ON finra_short_interest(settleme
|
||||
-- FINRA download configuration (admin-configurable URL).
|
||||
CREATE TABLE IF NOT EXISTS finra_config (
|
||||
id TEXT PRIMARY KEY DEFAULT 'singleton',
|
||||
base_url TEXT NOT NULL DEFAULT 'https://www.finra.org/sites/default/files',
|
||||
base_url TEXT NOT NULL DEFAULT 'https://cdn.finra.org/equity/regsho/daily',
|
||||
updated_at TEXT NOT NULL
|
||||
);
|
||||
|
||||
Reference in New Issue
Block a user