Files
automaton/tasks/add-loop-runner/VERDICT.md
T
Lap Tran bc7daf8590 Restore archived tasks, fix dashboard scroll-reset, bind ornith, add Playwright smoke test
- **Restore 82 completed tasks** from tasks/complete/ back to tasks/ top
  level (all <7 days old per the cleanup policy; premature bulk archive
  was fixed).
- **Dashboard: fix scroll-reset on auto-refresh** — renderBoard rebuilds
  the board via innerHTML every 2s, destroying each column-body's
  scrollTop. Now snapshots column-body scrollTop + board.scrollLeft +
  view.scrollTop before rebuild and restores after (matched by
  PHASE_GROUPS index).
- **Dashboard UI additions** (pre-existing unstaged work): approval
  section cards, transition buttons, inline artifact editor (textarea for
  writing missing SPEC/VERDICT/etc from the detail modal).
- **Bind ornith as Implement model** — config.md: Model explicit to
  omlx/Ornith-1.0-35B-4bit-mlx, context window 32768. Interactive
  autopilot already used ornith via opencode default; now explicit.
- **Fix cleanup stub** — automaton-cleanup.sh had a stale --project arg
  pointing at a pytest temp dir (test isolation leak). Rewired to point
  at ~/.automaton.
- **Fix plist-isolation test** — test asserted host plist doesn't exist,
  but a real install creates it. Now snapshots mtime before run, asserts
  unchanged after (only a write during the test counts as bleed).
- **New Playwright smoke test** (tests/test_dashboard_ui.py) — 2 tests:
  board renders tasks, column scroll survives auto-refresh tick.
  Verified the test fails without the scroll fix (scrollTop resets to 0).
  Skipped via importorskip when playwright is absent (main CI stays
  green).
- **Clarify SI loop scope in README** — new-project onboarding section
  documents the framework-scoped self-improvement loop and options
  (leave/pause/create project loop).
- **CHANGELOG** documents all changes including the known model-divergence
  gap (mde tasks marked complete but per-role model binding was never
  implemented).
2026-06-26 10:05:18 -04:00

3.7 KiB

VERDICT: add-loop-runner

Status: PASS

Task delivers scripts/loop-runner.py -- the per-tick engine that partners with the brakes layer (task 2). The runner is the only piece that ever invokes the user's harness (subprocess to loop.json harness.command); it never inspects model capability (D8) and never auto-approves (D4 -- the orchestrator role calls status.py --approve itself, the runner only spawns the role).

Requirement coverage

Req Status Tests
R1 entrypoint + clean exits delivered TestEntrypoint (2)
R2 11-step tick flow delivered TestTickFlow (5), TestOrchestratorOrdering (1), TestJsonOutput (1)
R3 daemon mode delivered TestDaemonMode (1)
R4 harness command substitution delivered TestHarnessSubstitution (1)
R5 context-floor guard (D13) delivered TestContextFloor (1)
R6 idempotence / no state advance on parse failure delivered TestVerifierParseFailure (5)
R7 tests (18 total) delivered per-class rows above
R8 out-of-scope items deferred delivered (none in code; docs note deferral)

Tests: 18 new. Full suite: 328 passed (was 310 + 18 new). No regressions.

Defense against the five loop deaths -- runtime enforcement

  • drift -> runner sees not-ok via --check-gate and SKIPs (drift_detected reason).
  • runaway -> runner's iteration_count increments only after gate passes; next tick's --check-gate halts at iterations_exhausted.
  • bad verifier -> score appended to history; next --check-gate halts verifier_failed when score plateaus. Parse-failure halts immediately. Idempotent (no state advance).
  • resource burn -> --check-gate halts budget_exhausted; runner never invokes the harness before then.
  • undetected halt -> runner SKIPs on any not-ok gate; tick log records SKIP with reason; --audit Cat-6 surfaces the halt across all loops.

Agnosticism preserved

  • Harness-agnostic: harness.command is a JSON list; any subprocess-capable harness works. Default opencode run is only a default; the user can swap it for claudia run, claude --prompt-file, a custom shell wrapper, or an SSH-remote harness command.
  • OS-agnostic: loop-runner.py --mode tick is pure Python; works on Linux, macOS, Windows. --mode daemon is the portable fallback for CI containers without cron/launchd/schtasks.
  • Model-agnostic: runner never inspects model size/provider. It only checks hardware context (vram_detect.py --loop-mode --json -- loop_mode_eligible). The 16k floor (D13) is enforced by the runner, not the gate, because available context is hardware state (per-tick), not loop state (cached).

Doc impact landed

  • AGENTS.md "Loop runner" bullet under State Enforcement -- Loops (v1).
  • README.md loop-runner one-liner.
  • CHANGELOG.md [unreleased] entry for add-loop-runner.

No code-doc mismatches.

Hardening items deferred (tracked in BUG_REPORT + ADVERSARIAL_BUG_REPORT)

  1. fcntl lock on .state.loop (A2/A7 TOCTOU; same item as add-status-brakes A6) -> v1.1.
  2. parse_verdict score clamp + pass string coercion (O6 + A6) -> v1.1.
  3. outputs.retention in loop.json (O5) -> v1.1.

All three are explicit follow-ups; none block this task.

Resolution

PASS -- proceed to complete. Task 3 is the runtime half of the loop v1 foundation. With task 2 (brakes) + task 3 (runner) both shipped, the framework can run a single tick end-to-end against any configured harness. Remaining tasks (4 goal-mode, 5 blast-radius-scheduler, 6 templates-onboarding, 7 self-improvement-loop) add work sources, worktree plumbing, usable templates + prompts, and the default-on self-improvement loop. Tasks 8 and 9 are infrastructure cleanup.