investor-flow/docs/VENDOR_INTEGRATIONS.md

Adding a vendor integration

Rate limits are a first-class product constraint (ADR-0009). Every external vendor - existing and future - must share the same process-wide gate.

If a new integration can call the network without vendorGate, that is a bug.

Required steps

1. Register the family (before any traffic)

import { registerVendorIntegration } from '../services/vendorGate.ts';

registerVendorIntegration({
  family: 'polygon',                    // unique budget name
  sourceKinds: ['polygon'],             // AdapterQueue source_kind(s)
  policy: {
    minIntervalMs: 200,                 // gap between completed calls
    maxInflight: 1,                     // 1 = single-flight
    drainJobBudget: 2,                  // max jobs per drain cycle
    hostPattern: 'polygon\\.io',        // docs / optional default allowlist
  },
});

Call this at process startup (e.g. next to adapter registration in index.ts) or inside the adapter module top-level so importing the adapter registers it.

2. Extend SourceKind if needed

Add the string to SourceKind in app/server/src/cache/CacheRepository.ts.

3. Implement the adapter (prefer base class)

import { VendorSourceAdapter, type FetchResult } from './SourceAdapter.ts';
import { vendorFetch } from '../services/vendorGate.ts';

export class PolygonAdapter extends VendorSourceAdapter {
  readonly sourceKind = 'polygon' as const;

  protected async fetchOneUngated(key: CacheKey): Promise<FetchResult> {
    // All HTTP:
    const resp = await vendorFetch('polygon', url, {
      hostAllowlist: /polygon\.io/i,
    });
    // Library SDKs:
    // return withVendorGate('polygon', () => client.get(...));
    // but VendorSourceAdapter already gates the whole job — do not double-gate
    // unless you make additional calls from a service outside fetchOne.
  }
}

Helpers:

API	Use when
VendorSourceAdapter	New queue adapter (auto-gates fetchOne)
defineVendorAdapter({...})	Tiny one-off adapter without a class
withVendorGate(family, fn)	SDK / subprocess calls
vendorFetch(family, url, opts)	Raw HTTP
secHttp / secFetch	SEC only (UA + host rules)

4. Register with AdapterQueue

adapters.set('polygon', new PolygonAdapter());
// Constructor throws if 'polygon' was not bindSourceKind'd.

5. Schedule (optional)

Add default interval in SCHEDULE_INTERVALS / sourceRatePolicy if the source should refresh on a timer.

What AdapterQueue enforces

• Construction: every adapter source_kind must have a family binding.
• Drain: at most drainJobBudget jobs per family per cycle.
• 429/403: cools all source_kinds in that family + process-wide gate.
• Heal / schedules: skip family while cooling.

What CI enforces

src/services/__tests__/vendorHttpGuard.test.ts:

• Fails if bare fetch( appears under adapters/, services/, macro/, mirror/ outside allowlisted gate modules.
• Covers runtime registration of a fictional future vendor.

Checklist for PR review

• registerVendorIntegration (or family + bind) present
• SourceKind updated
• No bare fetch / ungated SDK in the adapter
• hostAllowlist set for HTTP
• Rate-limit errors surface as thrown messages matching isRateLimitError
• Stale/cached path preferred on UI (never stampede from clicks)

Anti-patterns

Don't	Do
await fetch(vendorUrl) in an adapter	vendorFetch(family, url, { hostAllowlist })
New TokenBucket per adapter	Shared vendorGate policy
Cool only one source_kind on 429	Family cool-down (automatic if gated)
Call vendor from tRPC handler live	Cache / queue / static fallback
Skip registration "just for a prototype"	Register with strict policy even for prototypes