26 lines
2.6 KiB
Markdown
26 lines
2.6 KiB
Markdown
|
# ADR-0008: Default LLM provider = `ornith` (remote-local OpenAI-compatible)
|
||
|
|
|
||
|
|
Date: 2026-06-28
|
||
|
|
Status: Accepted (design phase)
|
||
|
|
|
||
|
|
## Context
|
||
|
|
The LLM Gateway (ADR-0005, ADR-0006) is provider-agnostic by design. The operator uses a remote-hosted LLM endpoint named `ornith` exposed via the standard OpenAI-compatible REST interface (`/v1/chat/completions`). It is "remote-local": remote in network location, local in trust posture (owned by the operator, not a third-party SaaS). For sensitive content (ADR-0006 Data-Classification Gate), `ornith` qualifies as the local-only provider because the operator owns it.
|
||
|
|
|
||
|
|
## Decision
|
||
|
|
- Default primary LLM provider = `ornith` (OpenAI-compatible REST endpoint).
|
||
|
|
- `llm_providers` table (Section 1 schema) seeded on first boot with:
|
||
|
|
- `id='ornith', name='ornith', base_url=<from env ORNITH_LLM_URL>, api_key=<from env ORNITH_LLM_API_KEY>, is_local=true, default_for_public=true, default_for_sensitive=true`
|
||
|
|
- A secondary fallback provider (e.g. local Ollama or another OpenAI-compatible endpoint) may be configured but is optional in v1.
|
||
|
|
- Env vars `ORNITH_LLM_URL` and `ORNITH_LLM_API_KEY` injected via Docker Compose secrets (slice 26); never written to the repo.
|
||
|
|
- `LLMGateway.dispatch` resumes honoring classifyPayload (ADR-0006): sensitive features route per `llm_providers.is_local`; `ornith` is `is_local=true`, so it serves thesis_monitor_l1 / sizing_explain / derisk_suggestion / macro_commentary as well as public features.
|
||
|
|
- Analyst Voice preamble (ADR-0005) + Primary-Rule (ADR-0007) passed to `ornith` on every dispatch regardless of feature.
|
||
|
|
|
||
|
|
|
||
|
|
## Implementation model (updated)
|
||
|
|
Ornith (35B MoE) is also the **primary implementation/coding model** for the loop-runner orchestrator, prioritized over Qwythos-9B (9B dense) for code generation quality. Qwythos-9B remains available as a local fallback. The orchestrator dispatches `pi --print --model remote/ornith:medium` for implementer ticks; reviews the diff itself; cross-checks with `remote/ornith` or `omlx/Qwythos-9B` as needed.
|
||
|
|
## Consequences
|
||
|
|
- Loop-runner implementer configures `ornith` in the FakeLLMGateway fixture path with a canned OpenAI-compatible shape, and real `LLMGateway` reads `llm_providers` + env at boot.
|
||
|
|
- Tests use FakeLLMGateway (no network) so `ornith` is never required for green tests; tests asserting prompt-hash cache + Analyst Voice preamble still pass.
|
||
|
|
- If `ornith` is unreachable at runtime, LLM features degrade gracefully (cached summary if available; "LLM source degraded" UI banner; cache-first rule preserved).
|
||
|
|
- ADR-0006's "local-only for sensitive" obligation is honored because `ornith` is operator-owned and configured `is_local=true`.
|