60 lines
3.0 KiB
Markdown
60 lines
3.0 KiB
Markdown
|
# Parent Task: runnable-test-suite
|
||
|
|
|
||
|
|
This is the parent SPEC for the runnable-test-suite task. Each sub-task references
|
||
|
|
this for context, scope boundaries, and the parent acceptance contract.
|
||
|
|
|
||
|
|
## Parent Goal
|
||
|
|
|
||
|
|
Make `python3 -m pytest tests/ -v` pass from a clean checkout of `~/.automaton`,
|
||
|
|
with deterministic Python deps pinned and docs that reflect the actual interpreter
|
||
|
|
on stock macOS/Windows/Linux. In the same wave, make `scripts/vram_detect.py`
|
||
|
|
cross-platform (macOS, Windows, Linux) — the existing version was developed on
|
||
|
|
Cachyos and only fully works on Linux.
|
||
|
|
|
||
|
|
## Parent Acceptance Contract
|
||
|
|
|
||
|
|
1. `pip3 install -r requirements.txt` succeeds on stock macOS CommandLineTools + pip3.
|
||
|
|
2. `python3 -m pytest tests/ -v` exits 0 from a clean clone, zero `error` lines.
|
||
|
|
3. **Streak verifier:** 10 consecutive clean `python3 -m pytest tests/ -v` runs with
|
||
|
|
no edits between runs. A single failure resets the count. Cap: 5 attempts.
|
||
|
|
4. `python3 -m py_compile automaton/**/*.py automaton/dashboard/**/*.py scripts/*.py` exits 0.
|
||
|
|
5. `bash -n scripts/*.sh` exits 0.
|
||
|
|
6. `rg "^python " AGENTS.md README.md automaton/dashboard/README.md prompts/orchestrate.md`
|
||
|
|
returns zero matches for a bare `python ` command.
|
||
|
|
7. `python3 scripts/vram_detect.py` on Darwin prints `gpu_vram_gb > 0` (was 0 before).
|
||
|
|
8. `python3 scripts/vram_detect.py` JSON shape identical to before on Linux/Cachyos.
|
||
|
|
|
||
|
|
## Sub-tasks
|
||
|
|
|
||
|
|
- `make-tests-runnable` — Wave 1, parallel-ok
|
||
|
|
- `vram-detect-cross-platform` — Wave 1, parallel-ok
|
||
|
|
- `vram-detect-cross-platform-tests` — Wave 2, depends on `vram-detect-cross-platform`
|
||
|
|
|
||
|
|
Parent is complete ONLY when ALL three sub-tasks pass and the streak verifier above
|
||
|
|
runs 10 consecutive clean passes.
|
||
|
|
|
||
|
|
## Anti-spin rails (from the source article)
|
||
|
|
|
||
|
|
- The streak verifier IS the independent checker model from Boris's loop. The
|
||
|
|
worker (local LLM) does not grade its own homework.
|
||
|
|
- If a test is genuinely broken (not just import-failing due to missing pytest),
|
||
|
|
STOP and report. Do not patch the test to make it pass. An agent that grades
|
||
|
|
itself will delete the failing test and call it done.
|
||
|
|
- Unknown model name → fail open with `0`. Wrong-context detection is worse than none.
|
||
|
|
- No new third-party deps beyond `pytest`. Pure stdlib for `vram_detect.py`.
|
||
|
|
|
||
|
|
## Hardware/VRAM context
|
||
|
|
|
||
|
|
- Detected by `vram_detect.py` on this box: **32GB RAM, no GPU, model unknown**
|
||
|
|
(because `vram_detect.py` is broken on macOS — subtask-2 fixes that)
|
||
|
|
- Target context: 12k tokens, headroom 25%, max peak per sub-task: 9k tokens.
|
||
|
|
- Sub-task peak estimates all fit within 9k. No further decomposition.
|
||
|
|
|
||
|
|
## Constraints / non-goals (parent)
|
||
|
|
|
||
|
|
- No `psutil`, `wmi`, `pywin32`, `tox`, `pytest-cov`, or other third-party deps.
|
||
|
|
- No removal of existing OpenAI/Anthropic entries in `MODEL_CONTEXT_WINDOWS`.
|
||
|
|
- No changes to `status.py`, `autopilot.py`, or any other runtime script's behavior.
|
||
|
|
- No touching files under `tasks/` (those are state, not source).
|
||
|
|
- No Docker, no conda, no `pyenv`. Stock `python3` + `pip3` only.
|
||
|
|
- VRAM detection is additive — Linux/Cachyos output must NOT regress.
|