feat: dealer flow, mirror portfolio (M21), options convexity, FINRA short interest, alert producers, vendor gate
CI / Test & Type-Check (push) Canceled after 0s
CI / Test & Type-Check (push) Canceled after 0s
Snapshot of in-progress module work across multiple slices: - Dealer Flow: dealerExposureEngine, dealerMapService, dealerMapExplain, dealerMapIntegrity, dealerMapReplay, dealerStudyEngine, hanStyleLevels - Mirror Portfolio (M21): fundRepository, captureIngest, mirrorAlertProducers, fund holdings strip, live book, position capture ingest - Options: BSM, NormalizedOptionSurface types, OptionsChainRouter, ConvexityGate, option legs panel - Alert producers: vixLevel, rotation, thesis, unlock, portfolioRisk, mirror (fund_capture, fund_13f, mirror_diff) - FINRA short interest adapter + queue integration - SEC company tickers adapter + ingest (symbol search index seed) - Vendor gate (rate-limit-first data plane, ADR-0009) - CUSIP registry, reverse 13F refresh, stock float service - LRU cache, portfolio backtest engine - Frontend: dealer-flow, funds, journal, lab, monitor, plan, portfolio, reports, screener, strategies, theses, guided-start, exits, more pages - Volume profile, workspace profile, visibility-aware poll - ADRs 0010 (mirror math not advice), 0011 (symbol search index) - VENDOR_INTEGRATIONS.md, END_USER_TEST.md - .gitignore: exclude DBs, .DS_Store, local config, agent scratch
This commit is contained in:
@@ -0,0 +1,110 @@
|
||||
# Adding a vendor integration
|
||||
|
||||
Rate limits are a **first-class product constraint** (ADR-0009). Every external
|
||||
vendor - existing and future - must share the same process-wide gate.
|
||||
|
||||
If a new integration can call the network without `vendorGate`, that is a bug.
|
||||
|
||||
## Required steps
|
||||
|
||||
### 1. Register the family (before any traffic)
|
||||
|
||||
```ts
|
||||
import { registerVendorIntegration } from '../services/vendorGate.ts';
|
||||
|
||||
registerVendorIntegration({
|
||||
family: 'polygon', // unique budget name
|
||||
sourceKinds: ['polygon'], // AdapterQueue source_kind(s)
|
||||
policy: {
|
||||
minIntervalMs: 200, // gap between completed calls
|
||||
maxInflight: 1, // 1 = single-flight
|
||||
drainJobBudget: 2, // max jobs per drain cycle
|
||||
hostPattern: 'polygon\\.io', // docs / optional default allowlist
|
||||
},
|
||||
});
|
||||
```
|
||||
|
||||
Call this at process startup (e.g. next to adapter registration in `index.ts`)
|
||||
or inside the adapter module top-level so importing the adapter registers it.
|
||||
|
||||
### 2. Extend SourceKind if needed
|
||||
|
||||
Add the string to `SourceKind` in `app/server/src/cache/CacheRepository.ts`.
|
||||
|
||||
### 3. Implement the adapter (prefer base class)
|
||||
|
||||
```ts
|
||||
import { VendorSourceAdapter, type FetchResult } from './SourceAdapter.ts';
|
||||
import { vendorFetch } from '../services/vendorGate.ts';
|
||||
|
||||
export class PolygonAdapter extends VendorSourceAdapter {
|
||||
readonly sourceKind = 'polygon' as const;
|
||||
|
||||
protected async fetchOneUngated(key: CacheKey): Promise<FetchResult> {
|
||||
// All HTTP:
|
||||
const resp = await vendorFetch('polygon', url, {
|
||||
hostAllowlist: /polygon\.io/i,
|
||||
});
|
||||
// Library SDKs:
|
||||
// return withVendorGate('polygon', () => client.get(...));
|
||||
// but VendorSourceAdapter already gates the whole job — do not double-gate
|
||||
// unless you make additional calls from a service outside fetchOne.
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Helpers:
|
||||
|
||||
| API | Use when |
|
||||
|-----|----------|
|
||||
| `VendorSourceAdapter` | New queue adapter (auto-gates `fetchOne`) |
|
||||
| `defineVendorAdapter({...})` | Tiny one-off adapter without a class |
|
||||
| `withVendorGate(family, fn)` | SDK / subprocess calls |
|
||||
| `vendorFetch(family, url, opts)` | Raw HTTP |
|
||||
| `secHttp` / `secFetch` | SEC only (UA + host rules) |
|
||||
|
||||
### 4. Register with AdapterQueue
|
||||
|
||||
```ts
|
||||
adapters.set('polygon', new PolygonAdapter());
|
||||
// Constructor throws if 'polygon' was not bindSourceKind'd.
|
||||
```
|
||||
|
||||
### 5. Schedule (optional)
|
||||
|
||||
Add default interval in `SCHEDULE_INTERVALS` / `sourceRatePolicy` if the source
|
||||
should refresh on a timer.
|
||||
|
||||
## What AdapterQueue enforces
|
||||
|
||||
- **Construction:** every adapter `source_kind` must have a family binding.
|
||||
- **Drain:** at most `drainJobBudget` jobs per family per cycle.
|
||||
- **429/403:** cools **all** source_kinds in that family + process-wide gate.
|
||||
- **Heal / schedules:** skip family while cooling.
|
||||
|
||||
## What CI enforces
|
||||
|
||||
`src/services/__tests__/vendorHttpGuard.test.ts`:
|
||||
|
||||
- Fails if bare `fetch(` appears under `adapters/`, `services/`, `macro/`, `mirror/`
|
||||
outside allowlisted gate modules.
|
||||
- Covers runtime registration of a fictional future vendor.
|
||||
|
||||
## Checklist for PR review
|
||||
|
||||
- [ ] `registerVendorIntegration` (or family + bind) present
|
||||
- [ ] `SourceKind` updated
|
||||
- [ ] No bare `fetch` / ungated SDK in the adapter
|
||||
- [ ] `hostAllowlist` set for HTTP
|
||||
- [ ] Rate-limit errors surface as thrown messages matching `isRateLimitError`
|
||||
- [ ] Stale/cached path preferred on UI (never stampede from clicks)
|
||||
|
||||
## Anti-patterns
|
||||
|
||||
| Don't | Do |
|
||||
|-------|-----|
|
||||
| `await fetch(vendorUrl)` in an adapter | `vendorFetch(family, url, { hostAllowlist })` |
|
||||
| New `TokenBucket` per adapter | Shared `vendorGate` policy |
|
||||
| Cool only one `source_kind` on 429 | Family cool-down (automatic if gated) |
|
||||
| Call vendor from tRPC handler live | Cache / queue / static fallback |
|
||||
| Skip registration "just for a prototype" | Register with strict policy even for prototypes |
|
||||
Reference in New Issue
Block a user