Investor Flow Build
ac94acf9e3
CI / Test & Type-Check (push) Canceled after 0s
Snapshot of in-progress module work across multiple slices: - Dealer Flow: dealerExposureEngine, dealerMapService, dealerMapExplain, dealerMapIntegrity, dealerMapReplay, dealerStudyEngine, hanStyleLevels - Mirror Portfolio (M21): fundRepository, captureIngest, mirrorAlertProducers, fund holdings strip, live book, position capture ingest - Options: BSM, NormalizedOptionSurface types, OptionsChainRouter, ConvexityGate, option legs panel - Alert producers: vixLevel, rotation, thesis, unlock, portfolioRisk, mirror (fund_capture, fund_13f, mirror_diff) - FINRA short interest adapter + queue integration - SEC company tickers adapter + ingest (symbol search index seed) - Vendor gate (rate-limit-first data plane, ADR-0009) - CUSIP registry, reverse 13F refresh, stock float service - LRU cache, portfolio backtest engine - Frontend: dealer-flow, funds, journal, lab, monitor, plan, portfolio, reports, screener, strategies, theses, guided-start, exits, more pages - Volume profile, workspace profile, visibility-aware poll - ADRs 0010 (mirror math not advice), 0011 (symbol search index) - VENDOR_INTEGRATIONS.md, END_USER_TEST.md - .gitignore: exclude DBs, .DS_Store, local config, agent scratch
111 lines
3.8 KiB
Markdown
111 lines
3.8 KiB
Markdown
# Adding a vendor integration
|
|
|
|
Rate limits are a **first-class product constraint** (ADR-0009). Every external
|
|
vendor - existing and future - must share the same process-wide gate.
|
|
|
|
If a new integration can call the network without `vendorGate`, that is a bug.
|
|
|
|
## Required steps
|
|
|
|
### 1. Register the family (before any traffic)
|
|
|
|
```ts
|
|
import { registerVendorIntegration } from '../services/vendorGate.ts';
|
|
|
|
registerVendorIntegration({
|
|
family: 'polygon', // unique budget name
|
|
sourceKinds: ['polygon'], // AdapterQueue source_kind(s)
|
|
policy: {
|
|
minIntervalMs: 200, // gap between completed calls
|
|
maxInflight: 1, // 1 = single-flight
|
|
drainJobBudget: 2, // max jobs per drain cycle
|
|
hostPattern: 'polygon\\.io', // docs / optional default allowlist
|
|
},
|
|
});
|
|
```
|
|
|
|
Call this at process startup (e.g. next to adapter registration in `index.ts`)
|
|
or inside the adapter module top-level so importing the adapter registers it.
|
|
|
|
### 2. Extend SourceKind if needed
|
|
|
|
Add the string to `SourceKind` in `app/server/src/cache/CacheRepository.ts`.
|
|
|
|
### 3. Implement the adapter (prefer base class)
|
|
|
|
```ts
|
|
import { VendorSourceAdapter, type FetchResult } from './SourceAdapter.ts';
|
|
import { vendorFetch } from '../services/vendorGate.ts';
|
|
|
|
export class PolygonAdapter extends VendorSourceAdapter {
|
|
readonly sourceKind = 'polygon' as const;
|
|
|
|
protected async fetchOneUngated(key: CacheKey): Promise<FetchResult> {
|
|
// All HTTP:
|
|
const resp = await vendorFetch('polygon', url, {
|
|
hostAllowlist: /polygon\.io/i,
|
|
});
|
|
// Library SDKs:
|
|
// return withVendorGate('polygon', () => client.get(...));
|
|
// but VendorSourceAdapter already gates the whole job — do not double-gate
|
|
// unless you make additional calls from a service outside fetchOne.
|
|
}
|
|
}
|
|
```
|
|
|
|
Helpers:
|
|
|
|
| API | Use when |
|
|
|-----|----------|
|
|
| `VendorSourceAdapter` | New queue adapter (auto-gates `fetchOne`) |
|
|
| `defineVendorAdapter({...})` | Tiny one-off adapter without a class |
|
|
| `withVendorGate(family, fn)` | SDK / subprocess calls |
|
|
| `vendorFetch(family, url, opts)` | Raw HTTP |
|
|
| `secHttp` / `secFetch` | SEC only (UA + host rules) |
|
|
|
|
### 4. Register with AdapterQueue
|
|
|
|
```ts
|
|
adapters.set('polygon', new PolygonAdapter());
|
|
// Constructor throws if 'polygon' was not bindSourceKind'd.
|
|
```
|
|
|
|
### 5. Schedule (optional)
|
|
|
|
Add default interval in `SCHEDULE_INTERVALS` / `sourceRatePolicy` if the source
|
|
should refresh on a timer.
|
|
|
|
## What AdapterQueue enforces
|
|
|
|
- **Construction:** every adapter `source_kind` must have a family binding.
|
|
- **Drain:** at most `drainJobBudget` jobs per family per cycle.
|
|
- **429/403:** cools **all** source_kinds in that family + process-wide gate.
|
|
- **Heal / schedules:** skip family while cooling.
|
|
|
|
## What CI enforces
|
|
|
|
`src/services/__tests__/vendorHttpGuard.test.ts`:
|
|
|
|
- Fails if bare `fetch(` appears under `adapters/`, `services/`, `macro/`, `mirror/`
|
|
outside allowlisted gate modules.
|
|
- Covers runtime registration of a fictional future vendor.
|
|
|
|
## Checklist for PR review
|
|
|
|
- [ ] `registerVendorIntegration` (or family + bind) present
|
|
- [ ] `SourceKind` updated
|
|
- [ ] No bare `fetch` / ungated SDK in the adapter
|
|
- [ ] `hostAllowlist` set for HTTP
|
|
- [ ] Rate-limit errors surface as thrown messages matching `isRateLimitError`
|
|
- [ ] Stale/cached path preferred on UI (never stampede from clicks)
|
|
|
|
## Anti-patterns
|
|
|
|
| Don't | Do |
|
|
|-------|-----|
|
|
| `await fetch(vendorUrl)` in an adapter | `vendorFetch(family, url, { hostAllowlist })` |
|
|
| New `TokenBucket` per adapter | Shared `vendorGate` policy |
|
|
| Cool only one `source_kind` on 429 | Family cool-down (automatic if gated) |
|
|
| Call vendor from tRPC handler live | Cache / queue / static fallback |
|
|
| Skip registration "just for a prototype" | Register with strict policy even for prototypes |
|