Files
automaton/tasks/runnable-test-suite/subtasks/make-tests-runnable/SPEC.md
T
Lap Tran bc7daf8590 Restore archived tasks, fix dashboard scroll-reset, bind ornith, add Playwright smoke test
- **Restore 82 completed tasks** from tasks/complete/ back to tasks/ top
  level (all <7 days old per the cleanup policy; premature bulk archive
  was fixed).
- **Dashboard: fix scroll-reset on auto-refresh** — renderBoard rebuilds
  the board via innerHTML every 2s, destroying each column-body's
  scrollTop. Now snapshots column-body scrollTop + board.scrollLeft +
  view.scrollTop before rebuild and restores after (matched by
  PHASE_GROUPS index).
- **Dashboard UI additions** (pre-existing unstaged work): approval
  section cards, transition buttons, inline artifact editor (textarea for
  writing missing SPEC/VERDICT/etc from the detail modal).
- **Bind ornith as Implement model** — config.md: Model explicit to
  omlx/Ornith-1.0-35B-4bit-mlx, context window 32768. Interactive
  autopilot already used ornith via opencode default; now explicit.
- **Fix cleanup stub** — automaton-cleanup.sh had a stale --project arg
  pointing at a pytest temp dir (test isolation leak). Rewired to point
  at ~/.automaton.
- **Fix plist-isolation test** — test asserted host plist doesn't exist,
  but a real install creates it. Now snapshots mtime before run, asserts
  unchanged after (only a write during the test counts as bleed).
- **New Playwright smoke test** (tests/test_dashboard_ui.py) — 2 tests:
  board renders tasks, column scroll survives auto-refresh tick.
  Verified the test fails without the scroll fix (scrollTop resets to 0).
  Skipped via importorskip when playwright is absent (main CI stays
  green).
- **Clarify SI loop scope in README** — new-project onboarding section
  documents the framework-scoped self-improvement loop and options
  (leave/pause/create project loop).
- **CHANGELOG** documents all changes including the known model-divergence
  gap (mde tasks marked complete but per-role model binding was never
  implemented).
2026-06-26 10:05:18 -04:00

5.5 KiB

SPEC — make-tests-runnable

Parent: runnable-test-suite (see PARENT_SPEC.md).

Scope

Establish the green-test baseline the other sub-tasks depend on. Pin pytest, sweep docs/prompts for python → python3, add install snippet, verify the streak contract.

Files this sub-task touches (and ONLY these)

  • requirements.txt (NEW) — pin pytest==7.4.4 (last 7.x; supports Python 3.9+).
  • AGENTS.md — Build & Test Commands section: python → python3, reference requirements.txt.
  • README.md — every bare python → python3 in commands.
  • automaton/dashboard/README.md — same sweep.
  • prompts/orchestrate.md — same sweep on status.py invocations.
  • scripts/install.sh — append idempotent venv snippet (only create .venv if it doesn't already exist; only pip install -r requirements.txt inside .venv).
  • CHANGELOG.md — [unreleased] entry noting new requirements.txt, python3 requirement, venv install path.
  • automaton/dashboard/core/scope.py — add from __future__ import annotations at top (after module docstring) so PEP 604 annotations become lazy strings (PEP 563) and the file imports on Python 3.9.
  • automaton/dashboard/core/board.py — same from __future__ import annotations fix.
  • automaton/dashboard/ui/app.py — same fix (transitively imports scope/board).
  • Any other file under automaton/dashboard/ using X | Y annotation syntax evaluated at definition time — same from __future__ import annotations fix.

MUST NOT touch

  • scripts/vram_detect.py (subtask-2 owns it)
  • tests/test_vram_detect.py (subtask-3 owns it)
  • Any other test file (tests/*.py)
  • scripts/status.py, scripts/autopilot.py, or other runtime scripts
  • Logic in the dashboard files — ONLY add the from __future__ import annotations import line. Do NOT change function signatures, types, or behavior. The fix is purely a compatibility shim so annotations become strings (PEP 563).

Requirements (numbered)

  1. Create requirements.txt at repo root with exactly: pytest==7.4.4
  2. In AGENTS.md, change every python -m py_compile, python -m pytest, python -m automaton.dashboard to python3 -m .... Add a one-line note after the section title: "Install: pip3 install -r requirements.txt"
  3. In README.md, change every python ~/.automaton/scripts/status.py ... and python -m automaton.dashboard to python3 ....
  4. In automaton/dashboard/README.md, change every python -m automaton.dashboard to python3 -m automaton.dashboard.
  5. In prompts/orchestrate.md, change every python ~/.automaton/scripts/status.py to python3 ~/.automaton/scripts/status.py.
  6. In scripts/install.sh, append a venv block:
    # --- Python deps (idempotent) ---
    if [ ! -d ".venv" ]; then
      python3 -m venv .venv
    fi
    .venv/bin/pip install --quiet --upgrade pip
    .venv/bin/pip install --quiet -r requirements.txt
    
    Must be set -e-safe and re-runnable without errors.
  7. In CHANGELOG.md, add under [unreleased]:
    • Added: requirements.txt pinning pytest==7.4.4 for reproducible test runs.
    • Changed: All documented python invocations now read python3 (stock macOS / Windows Python ship as python3).
    • Added: scripts/install.sh now creates .venv/ and installs pytest into it.
  8. After edits, run rg -n "^python |\"python " AGENTS.md README.md automaton/dashboard/README.md prompts/orchestrate.md — expect ZERO matches for bare python commands.
  9. Run rg -l "^from __future__ import annotations" automaton/dashboard/ and confirm scope.py, board.py, and ui/app.py (and any other dashboard module using PEP 604 X | Y annotations) are listed. If a dashboard module uses PEP 604 at definition time and lacks the future import, add it. Do NOT add it to modules that don't use PEP 604 — keep the change surgical.
  10. Confirm on stock Python 3.9.6: python3 -c "import automaton.dashboard.core.scope, automaton.dashboard.core.board, automaton.dashboard.ui.app" exits 0 without TypeError.

Acceptance criteria

  1. pip3 install -r requirements.txt exits 0.
  2. python3 -m pytest tests/ -v exits 0 (N passed, 0 errors).
  3. python3 -m py_compile automaton/**/*.py automaton/dashboard/**/*.py scripts/*.py exits 0.
  4. bash -n scripts/*.sh exits 0.
  5. bash scripts/install.sh exits 0 and creates .venv/ containing pytest.
  6. rg sweep from requirement 8 returns zero matches.
  7. Streak: 10 consecutive python3 -m pytest tests/ -v runs all clean. Cap: 5 attempts. Reset on first failure.

Anti-spin rails

  • If a test fails for a reason OTHER than missing pytest, STOP and report — do not edit the failing test. It is subtask-3's job to add/fix tests; subtask-1 only makes existing tests runnable.
  • If python3 -m pytest tests/ -v fails because of a vram_detect.py import in test_vram_detect.py, STOP and report. That's a bug in subtask-2's scope.
  • Do NOT add pytest-cov, pytest-mock, tox, or any other dep.
  1. Write requirements.txt.
  2. pip3 install -r requirements.txt.
  3. Run python3 -m pytest tests/ -v. If it fails on missing pytest for OTHER reasons, stop and report (do not patch tests).
  4. Sweep docs/prompts with edit (batch by file; use replaceAll=true for the python → python3 substitution within each file).
  5. Append venv block to scripts/install.sh.
  6. Write CHANGELOG entry.
  7. Run streak verifier: for i in $(seq 1 10); do python3 -m pytest tests/ -q || break; done. If all 10 pass, done. Otherwise reset up to 5 times.