Files

T

CI / build (push) Has been cancelled

Details

Flatten model-divergence subtasks into 3 independent tasks

Replaced parent+subtask structure with 3 standalone tasks:
- mde-manifest-detection (foundational: models.json schema, detect_models.py)
- mde-interactive-enforcement (conflict matrix, --model args, audit, badges)
- mde-loop-enforcement (loop.json per-role model, {model} substitution, check-gate)

Parent archived to tasks/complete/ for history. Each task has its own
SPEC.md and is at research:awaiting_approval.

Rationale: subtasks are independently actionable with their own lifecycle.
Parent container added complexity without benefit.

2026-06-25 07:25:10 -04:00

4.9 KiB

Raw Blame History

SPEC — model-divergence-enforcement

Problem

The framework has no computational enforcement of model divergence for conflict-of-interest roles. status.py:1432-1436 enforces that the session playing reviewer differs from the implementer (via .state.implementer), but there is no enforcement that the model playing bug-finder differs from adversarial-bug-finder, or that the referee model differs from the implementer model. Same-model conflict-of-interest yields rubber-stamping.

Design (locked)

Full design is in design/framework/functional.md §4 and design/framework/technical.md §§2-5,13. This SPEC references those docs and does not repeat them.

Key decisions (from design session 2026-06-25)

Manifest: models.json with {default, advised, models:[{name, provider, context_window, location}]}.
Mode detection: 0-1 models → single-LLM (advisory once, then silent). 2+ → multi-LLM (hard block). Missing file → single-LLM (backward compatible).
Conflict matrix (locked): code_review≠implement; bug_find≠implement; adversarial_bug_find≠implement+bug_find; referee≠implement+bug_find+adversarial_bug_find; loop-verify≠loop-implement.
Auto-assignment: default model → next-available on conflict → user override via --transition --model or loop.json roles.<role>.model. Refuse only if no non-conflicting model exists.
State: .state.models per task recording which model filled which role.
Loop integration: loop.json per-role model field + {model} substitution in _invoke_harness.
Audit: new model_divergence category in --audit.
Dashboard: model badges on task cards.
Detection: scripts/detect_models.py probes opencode.json + localhost endpoints (8080/11434/1234/8000).

Scope

This is a parent task. It decomposes into 3 sequential subtasks:

Subtask 1: manifest+detection

models.json schema + loader
scripts/detect_models.py (probe opencode.json + localhost endpoints)
install.sh / update.sh / upgrade.sh integration (run detect_models after VRAM detection)
config.md ## Available Models section
prompts/onboarding.md Step 2d (model config)
Tests: test_model_divergence.py (manifest loading, single vs multi mode detection, missing file backward compat)

Subtask 2: interactive enforcement+audit

.state.models schema + writer
status.py --transition --model <name> (records model, checks conflict matrix in multi-LLM mode)
status.py --claim --model <name> (refuses on conflict)
CONFLICT_MATRIX constant + _check_conflict helper in status.py
status.py --audit model_divergence category
Dashboard model badges on task cards
Tests: test_model_divergence.py (conflict matrix, auto-assign, audit category, dashboard badges)

Subtask 3: loop enforcement+dashboard

loop.json per-role model field (optional, defaults to models.json default)
loop-runner.py _invoke_harness {model} substitution
status.py --check-gate model-divergence check (loop-verify ≠ loop-implement in multi-LLM mode)
Loop dashboard badges (model per role)
Tests: test_model_divergence.py (loop model binding, {model} substitution, check-gate halt)

Dependencies

Subtask 1 → no deps (foundational)
Subtask 2 → depends on subtask 1 (needs manifest loader)
Subtask 3 → depends on subtask 2 (needs .state.models + conflict matrix)

Out of scope

Rule agents (FW-2, FW-3) — separate backlog items that consume this feature
Agent tab redesign (FW-1) — separate backlog item, no dependency on this task
Model capability inspection (D8: framework never inspects capability/size/provider)
detect_models.py auto-writing models.json without user confirmation (detection is advisory; user confirms the manifest)

Success criteria

models.json missing → single-LLM mode, all model commands are no-ops, existing behavior unchanged.
models.json with 1 model → single-LLM mode, advisory printed once if advised: true, then silent.
models.json with 2+ models → multi-LLM mode, conflict matrix enforced on --transition --model and --claim --model.
--audit flags conflict-matrix violations as model_divergence category.
loop.json per-role model binding works; --check-gate halts on loop-verify = loop-implement in multi-LLM mode.
detect_models.py probes opencode.json + localhost endpoints and emits a candidate manifest.
pytest tests/ -v green; no regressions.

References

design/framework/functional.md §4 (Model-Divergence Enforcement)
design/framework/technical.md §§2-5 (manifest, state, flags, conflict matrix), §13 (loop integration)
design/framework/README.md (locked decisions F4, F5)
.agent.md Agent Configuration (6 phase roles)
scripts/status.py:1432-1436 (existing session-level reviewer≠implementer enforcement)
scripts/loop-runner.py:366-404 (_invoke_harness, needs {model} substitution)

4.9 KiB Raw Blame History