Files
automaton/tasks/runnable-test-suite/subtasks/make-tests-runnable/SPEC.md
T
Lap Tran f32f98575b
CI / build (push) Has been cancelled
Make test suite runnable from clean checkout + cross-platform vram_detect
runnable-test-suite (parent) — complete. Three sub-tasks all complete:

- make-tests-runnable: add requirements.txt pinning pytest==7.4.4, sweep all
  docs/prompts from bare 'python' to 'python3' (stock macOS/Windows ships
  python3), add idempotent .venv install block to scripts/install.sh, and add
  'from __future__ import annotations' to 3 dashboard modules using PEP 604
  union syntax at definition time so they import on Python 3.9+. The
  PEP 604 bug was caught by the streak verifier itself during implementation.

- vram-detect-cross-platform: scripts/vram_detect.py now branches on
  platform.system() for Linux/Darwin/Windows. macOS path uses
  system_profiler SPDisplaysDataType (Apple Silicon unified memory via
  sysctl hw.memsize; Intel Macs via 'VRAM (Total):'). Windows uses wmic
  path win32_VideoController get AdapterRAM with PowerShell fallback. Linux
  /proc/meminfo and nvidia-smi/lspci paths unchanged (regression test locks
  them). Added 14 local-LLM context-window entries (llama-3.1, qwen2.5,
  mistral, deepseek-r1/v3, glm-4/4.5, gemma-2, phi-3/4) with source-cited
  model cards. Added _probe_ollama_model() that runs 'ollama list' as a
  last-resort fallback. run_command() now wraps PowerShell cmdlets on
  Windows (['powershell', '-NoProfile', '-NoLogo', '-Command', ...]).

- vram-detect-cross-platform-tests: 11 new monkeypatched tests in
  tests/test_vram_detect.py covering Linux/Darwin/Windows branches for
  detect_ram and detect_gpu_vram, prefix-match for unknown model names,
  ollama probe, Windows PowerShell wrapper, and a LOCKED regression test
  for _detect_ram_linux(). All external subprocess/sysctl/wmic calls are
  mocked; no live hardware probes. Suite total: 235 passed, 0 errors.

Verified on this box: gpu_vram_gb 0 -> 32 on Apple M5 (32GB unified memory),
target context correctly jumped 12k -> 42k.

Subtask-2 implementation was authored by local LLM (gemma-4-26B-A4B-it
via headroom proxy @ localhost:8787). The 10-consecutive-clean-pass streak
verifier ran as the independent checker model (article #2/#9/#13 in
'WTF Is a Loop? Part 2'). One anti-spin rail fired: local LLM produced
inline branches where subtask-3 tests expected private _detect_ram_linux()
helper; extracted helper to match the test contract without weakening tests.

Parent + all 3 subtasks complete. Prior opencode-subagent implementation
of subtask-2 preserved in git stash for reference.
2026-06-21 18:28:15 -04:00

5.5 KiB

SPEC — make-tests-runnable

Parent: runnable-test-suite (see PARENT_SPEC.md).

Scope

Establish the green-test baseline the other sub-tasks depend on. Pin pytest, sweep docs/prompts for python → python3, add install snippet, verify the streak contract.

Files this sub-task touches (and ONLY these)

  • requirements.txt (NEW) — pin pytest==7.4.4 (last 7.x; supports Python 3.9+).
  • AGENTS.md — Build & Test Commands section: python → python3, reference requirements.txt.
  • README.md — every bare python → python3 in commands.
  • automaton/dashboard/README.md — same sweep.
  • prompts/orchestrate.md — same sweep on status.py invocations.
  • scripts/install.sh — append idempotent venv snippet (only create .venv if it doesn't already exist; only pip install -r requirements.txt inside .venv).
  • CHANGELOG.md — [unreleased] entry noting new requirements.txt, python3 requirement, venv install path.
  • automaton/dashboard/core/scope.py — add from __future__ import annotations at top (after module docstring) so PEP 604 annotations become lazy strings (PEP 563) and the file imports on Python 3.9.
  • automaton/dashboard/core/board.py — same from __future__ import annotations fix.
  • automaton/dashboard/ui/app.py — same fix (transitively imports scope/board).
  • Any other file under automaton/dashboard/ using X | Y annotation syntax evaluated at definition time — same from __future__ import annotations fix.

MUST NOT touch

  • scripts/vram_detect.py (subtask-2 owns it)
  • tests/test_vram_detect.py (subtask-3 owns it)
  • Any other test file (tests/*.py)
  • scripts/status.py, scripts/autopilot.py, or other runtime scripts
  • Logic in the dashboard files — ONLY add the from __future__ import annotations import line. Do NOT change function signatures, types, or behavior. The fix is purely a compatibility shim so annotations become strings (PEP 563).

Requirements (numbered)

  1. Create requirements.txt at repo root with exactly: pytest==7.4.4
  2. In AGENTS.md, change every python -m py_compile, python -m pytest, python -m automaton.dashboard to python3 -m .... Add a one-line note after the section title: "Install: pip3 install -r requirements.txt"
  3. In README.md, change every python ~/.automaton/scripts/status.py ... and python -m automaton.dashboard to python3 ....
  4. In automaton/dashboard/README.md, change every python -m automaton.dashboard to python3 -m automaton.dashboard.
  5. In prompts/orchestrate.md, change every python ~/.automaton/scripts/status.py to python3 ~/.automaton/scripts/status.py.
  6. In scripts/install.sh, append a venv block:
    # --- Python deps (idempotent) ---
    if [ ! -d ".venv" ]; then
      python3 -m venv .venv
    fi
    .venv/bin/pip install --quiet --upgrade pip
    .venv/bin/pip install --quiet -r requirements.txt
    
    Must be set -e-safe and re-runnable without errors.
  7. In CHANGELOG.md, add under [unreleased]:
    • Added: requirements.txt pinning pytest==7.4.4 for reproducible test runs.
    • Changed: All documented python invocations now read python3 (stock macOS / Windows Python ship as python3).
    • Added: scripts/install.sh now creates .venv/ and installs pytest into it.
  8. After edits, run rg -n "^python |\"python " AGENTS.md README.md automaton/dashboard/README.md prompts/orchestrate.md — expect ZERO matches for bare python commands.
  9. Run rg -l "^from __future__ import annotations" automaton/dashboard/ and confirm scope.py, board.py, and ui/app.py (and any other dashboard module using PEP 604 X | Y annotations) are listed. If a dashboard module uses PEP 604 at definition time and lacks the future import, add it. Do NOT add it to modules that don't use PEP 604 — keep the change surgical.
  10. Confirm on stock Python 3.9.6: python3 -c "import automaton.dashboard.core.scope, automaton.dashboard.core.board, automaton.dashboard.ui.app" exits 0 without TypeError.

Acceptance criteria

  1. pip3 install -r requirements.txt exits 0.
  2. python3 -m pytest tests/ -v exits 0 (N passed, 0 errors).
  3. python3 -m py_compile automaton/**/*.py automaton/dashboard/**/*.py scripts/*.py exits 0.
  4. bash -n scripts/*.sh exits 0.
  5. bash scripts/install.sh exits 0 and creates .venv/ containing pytest.
  6. rg sweep from requirement 8 returns zero matches.
  7. Streak: 10 consecutive python3 -m pytest tests/ -v runs all clean. Cap: 5 attempts. Reset on first failure.

Anti-spin rails

  • If a test fails for a reason OTHER than missing pytest, STOP and report — do not edit the failing test. It is subtask-3's job to add/fix tests; subtask-1 only makes existing tests runnable.
  • If python3 -m pytest tests/ -v fails because of a vram_detect.py import in test_vram_detect.py, STOP and report. That's a bug in subtask-2's scope.
  • Do NOT add pytest-cov, pytest-mock, tox, or any other dep.
  1. Write requirements.txt.
  2. pip3 install -r requirements.txt.
  3. Run python3 -m pytest tests/ -v. If it fails on missing pytest for OTHER reasons, stop and report (do not patch tests).
  4. Sweep docs/prompts with edit (batch by file; use replaceAll=true for the python → python3 substitution within each file).
  5. Append venv block to scripts/install.sh.
  6. Write CHANGELOG entry.
  7. Run streak verifier: for i in $(seq 1 10); do python3 -m pytest tests/ -q || break; done. If all 10 pass, done. Otherwise reset up to 5 times.