Files
automaton/tasks/runnable-test-suite/subtasks/make-tests-runnable/IMPLEMENTATION.md
T
Lap Tran f32f98575b
CI / build (push) Has been cancelled
Make test suite runnable from clean checkout + cross-platform vram_detect
runnable-test-suite (parent) — complete. Three sub-tasks all complete:

- make-tests-runnable: add requirements.txt pinning pytest==7.4.4, sweep all
  docs/prompts from bare 'python' to 'python3' (stock macOS/Windows ships
  python3), add idempotent .venv install block to scripts/install.sh, and add
  'from __future__ import annotations' to 3 dashboard modules using PEP 604
  union syntax at definition time so they import on Python 3.9+. The
  PEP 604 bug was caught by the streak verifier itself during implementation.

- vram-detect-cross-platform: scripts/vram_detect.py now branches on
  platform.system() for Linux/Darwin/Windows. macOS path uses
  system_profiler SPDisplaysDataType (Apple Silicon unified memory via
  sysctl hw.memsize; Intel Macs via 'VRAM (Total):'). Windows uses wmic
  path win32_VideoController get AdapterRAM with PowerShell fallback. Linux
  /proc/meminfo and nvidia-smi/lspci paths unchanged (regression test locks
  them). Added 14 local-LLM context-window entries (llama-3.1, qwen2.5,
  mistral, deepseek-r1/v3, glm-4/4.5, gemma-2, phi-3/4) with source-cited
  model cards. Added _probe_ollama_model() that runs 'ollama list' as a
  last-resort fallback. run_command() now wraps PowerShell cmdlets on
  Windows (['powershell', '-NoProfile', '-NoLogo', '-Command', ...]).

- vram-detect-cross-platform-tests: 11 new monkeypatched tests in
  tests/test_vram_detect.py covering Linux/Darwin/Windows branches for
  detect_ram and detect_gpu_vram, prefix-match for unknown model names,
  ollama probe, Windows PowerShell wrapper, and a LOCKED regression test
  for _detect_ram_linux(). All external subprocess/sysctl/wmic calls are
  mocked; no live hardware probes. Suite total: 235 passed, 0 errors.

Verified on this box: gpu_vram_gb 0 -> 32 on Apple M5 (32GB unified memory),
target context correctly jumped 12k -> 42k.

Subtask-2 implementation was authored by local LLM (gemma-4-26B-A4B-it
via headroom proxy @ localhost:8787). The 10-consecutive-clean-pass streak
verifier ran as the independent checker model (article #2/#9/#13 in
'WTF Is a Loop? Part 2'). One anti-spin rail fired: local LLM produced
inline branches where subtask-3 tests expected private _detect_ram_linux()
helper; extracted helper to match the test contract without weakening tests.

Parent + all 3 subtasks complete. Prior opencode-subagent implementation
of subtask-2 preserved in git stash for reference.
2026-06-21 18:28:15 -04:00

7.5 KiB

IMPLEMENTATION — make-tests-runnable

Parent: runnable-test-suite (see PARENT_SPEC.md)

Summary

All 8 in-scope requirements were implemented. The test suite goes from "No module named pytest" to 198 passed, 3 collection errors. The 3 errors are a STOP-and-report trigger: PEP 604 union syntax (Path | None) in out-of-scope dashboard source files, incompatible with the stock macOS Python 3.9.6. These files are NOT in this sub-task's 7-file scope and were not touched.

Files changed

  • requirements.txt (NEW) — created at repo root with pytest==7.4.4.
  • AGENTS.md — python -m → python3 -m (7 occurrences); added install note *Install: pip3 install -r requirements.txt* after "## Build & Test Commands".
  • README.md — python ~/.automaton/scripts/status.py → python3 ... (16 occurrences); python -m automaton.dashboard → python3 -m automaton.dashboard.
  • automaton/dashboard/README.md — python -m automaton.dashboard → python3 -m automaton.dashboard (4 occurrences).
  • prompts/orchestrate.md — python ~/.automaton/scripts/status.py → python3 ... (17 occurrences).
  • scripts/install.sh — restructured early-exit exit 0 to if/else so the appended venv block is reachable on existing installs; appended idempotent venv block (python3 -m venv .venv + pip install requirements.txt).
  • CHANGELOG.md — added 3 entries under [unreleased]: two ### Added (requirements.txt, install.sh venv) and one ### Changed (python → python3).

Acceptance criteria

# Criterion Status
1 pip3 install -r requirements.txt exits 0 PASS
2 python3 -m pytest tests/ -v exits 0 (N passed, 0 errors) FAIL — exit 2, 3 collection errors
3 python3 -m py_compile automaton/**/*.py automaton/dashboard/**/*.py scripts/*.py exits 0 PASS
4 bash -n scripts/*.sh exits 0 PASS
5 bash scripts/install.sh exits 0 and creates .venv/ containing pytest PASS (.venv/bin/pytest = 7.4.4)
6 rg "^python |"python " sweep returns zero matches PASS (exit 1 = no matches)
7 Streak: 10 consecutive clean pytest tests/ -v runs BLOCKED by #2

Streak verifier result

Not run — blocked by acceptance #2. The suite never reaches a clean pass on the stock Python 3.9.6 interpreter, so a 10-run streak is impossible without fixing the out-of-scope PEP 604 syntax. Attempt 1 of 5 stopped at the STOP-and-report trigger.

STOP-and-report trigger

Trigger: Tests fail for a reason OTHER than missing pytest.

Failing tests (collection errors):

  • tests/test_app.py
  • tests/test_board.py
  • tests/test_scope.py

Root cause: PEP 604 union type syntax (X | None) evaluated at class/function definition time. This syntax requires Python 3.10+. The stock macOS CommandLineTools Python is 3.9.6.

Out-of-scope files containing the bug (NOT touched):

  • automaton/dashboard/core/scope.py:6 — def find_automaton_root(start: Path | None = None) -> Path | None:
  • automaton/dashboard/core/board.py:40 — def __init__(self, tasks: list[Task] | None = None, ...)
  • automaton/dashboard/ui/app.py:15 — transitive failure (imports scope)

Traceback (representative, test_scope.py):

tests/test_scope.py:7: in <module>
    from automaton.dashboard.core.scope import detect_scope, find_automaton_root
automaton/dashboard/core/scope.py:6: in <module>
    def find_automaton_root(start: Path | None = None) -> Path | None:
E   TypeError: unsupported operand type(s) for |: 'type' and 'NoneType'

Suggested fix (for whoever owns these files): Add from __future__ import annotations at the top of scope.py, board.py, and any other dashboard module using PEP 604 syntax. This makes annotations lazy (string-evaluated), restoring Python 3.9 compatibility without changing any type semantics. Alternatively, replace X | None with Optional[X] from typing.

Anomalies / scope notes

  1. install.sh restructure: The SPEC says "append a venv block" to install.sh. A literal append at the end would be unreachable because the script's early-exit (if [ -d "$FRAMEWORK_DIR" ]; then ... exit 0) fires before the end on any system where ~/.automaton already exists. To satisfy acceptance #5 ("creates .venv/ containing pytest"), the early-exit was converted from exit 0 to an else branch, and the venv block was appended after the closing fi so it runs unconditionally. The venv block text matches the SPEC exactly. The script remains set -e-safe and re-runnable.

  2. Python 3.13 available but not used: /opt/homebrew/bin/python3.13 exists on this system, but the parent SPEC mandates "Stock python3 + pip3 only" — stock is 3.9.6 from CommandLineTools. Using Homebrew Python would violate the parent constraint and mask the real bug (PEP 604 syntax in framework code).

  3. 198/201 tests pass with --continue-on-collection-errors: The 3 erroring tests are all dashboard tests that transitively import scope.py or board.py. No test under tests/ was edited. No out-of-scope source file was edited.

  4. Not vram_detect: The failure is NOT in test_vram_detect.py or vram_detect.py. That sub-task's scope is unaffected.

Scope extension: future-annotations fix

The Orchestrator extended this sub-task's scope to include the 3 dashboard source files previously reported as out-of-scope (STOP-and-report trigger above). The fix is purely a compatibility shim: add from __future__ import annotations as the first import line (after the module docstring) so PEP 604 X | Y annotations become lazy strings (PEP 563) and the files import on stock Python 3.9.6. No signatures, types, or behavior were changed.

Files patched

  • automaton/dashboard/core/scope.py — added from __future__ import annotations after docstring (PEP 604 at find_automaton_root(start: Path | None = None) -> Path | None).
  • automaton/dashboard/core/board.py — added from __future__ import annotations after docstring (PEP 604 at KanbanBoard.__init__(self, tasks: list[Task] | None = None, ...)).
  • automaton/dashboard/ui/app.py — added from __future__ import annotations after docstring (PEP 604 at _get_review_path(self, task_name: str) -> Path | None; also transitively imports scope/board).

A rg sweep of automaton/dashboard/ for PEP 604 union syntax found no other dashboard modules using X | Y at definition time — only the three files above. No spurious future-imports were added to modules that don't need it.

Test results after the fix

  • python3 -c "import automaton.dashboard.core.scope, automaton.dashboard.core.board, automaton.dashboard.ui.app; print('IMPORTS_OK')" → IMPORTS_OK (no TypeError on Python 3.9.6).
  • python3 -m pytest tests/ -v → 224 passed, 0 errors (up from 198 passed / 3 collection errors).

Streak result

STREAK_COMPLETE attempt=1 clean=10/10 — 10 consecutive clean python3 -m pytest tests/ -q passes on the first attempt, no resets needed.

Acceptance criteria (re-checked after extension)

# Criterion Status
1 pip3 install -r requirements.txt exits 0 PASS
2 python3 -m pytest tests/ -v exits 0 (N passed, 0 errors) PASS — 224 passed, 0 errors
3 python3 -m py_compile ... exits 0 PASS
4 bash -n scripts/*.sh exits 0 PASS
5 bash scripts/install.sh exits 0 and creates .venv/ containing pytest PASS
6 rg sweep returns zero matches PASS (exit 1 = no matches)
7 Streak: 10 consecutive clean pytest runs PASS — 10/10 on attempt 1