Complete tasks 3-7: harden verdict parsing, outputs retention, base branch, linux schedule parity, claim loop task
CI / build (push) Has been cancelled
CI / build (push) Has been cancelled
This commit is contained in:
@@ -0,0 +1 @@
|
||||
complete
|
||||
@@ -0,0 +1,2 @@
|
||||
research:approved|2026-06-24T02:20:11.669836+00:00|user
|
||||
code_review:approved|2026-06-24T02:22:31.598746+00:00|user
|
||||
@@ -0,0 +1,31 @@
|
||||
# Adversarial Bug Report: add-outputs-retention
|
||||
|
||||
Probed `_get_retention` and `_gc_outputs` with non-contract inputs.
|
||||
|
||||
## A1 — `retention` as float
|
||||
|
||||
`_get_retention({"outputs": {"retention": 3.14}})` → `int(3.14)` = 3. Not garbage but truncating. Acceptable (float is a numeric type; int() rounds toward zero). Not a regression.
|
||||
|
||||
## A2 — `retention` as bool
|
||||
|
||||
`_get_retention({"outputs": {"retention": True}})` → `int(True)` = 1. A user who sets `retention: true` intending "unlimited" gets 1 (wrong — they wanted 0). But `bool` is technically a subclass of `int` in Python; `int(True)` = 1 is documented behavior. Acceptable edge case — the user would need to write JSON `true`, which `json.loads` reads as `True`. Not blocking; `int(True)` = 1 is a narrow retention but valid.
|
||||
|
||||
## A3 — `retention` string "inf" falls back to 20
|
||||
|
||||
`_get_retention({"outputs": {"retention": "inf"}})` → `int("inf")` raises ValueError → caught → 20 with WARNING. Correct per SPEC D-O4.
|
||||
|
||||
## A4 — GC handles large gaps in tick indices
|
||||
|
||||
Files `tick1-*.json` and `tick100-*.json` with nothing in between: `max_seen=100`, `retention=20`, `cutoff=100-20+1=81`. Deletes tick1- but keeps tick100-. Correct — the gap is intentional (maybe intermittent ticks). Not a bug.
|
||||
|
||||
## A5 — Non-tick files `tick-nope.md` preserved
|
||||
|
||||
Hyphen-no-number prefix `tick-nope.md` doesn't match `^tick(\d+)-`. Preserved. Correct per D-O6.
|
||||
|
||||
## A6 — Empty outputs dir
|
||||
|
||||
`_gc_outputs` on dir with 0 files or missing dir returns cleanly. No crash. Confirmed.
|
||||
|
||||
## No BLOCKERS
|
||||
|
||||
All adversarial cases produce deterministic documented results. Proceed to doc_review.
|
||||
@@ -0,0 +1,17 @@
|
||||
# Bug Report: add-outputs-retention
|
||||
|
||||
## O1 — GC tick-count semantic: cutoff uses `max_seen` from filenames, not `state.iteration_count`
|
||||
|
||||
The formula `cutoff = max_seen - retention + 1` uses the max tick index found in filenames, NOT `state.iteration_count`. If the `.state.loop` advances to iteration_count=N but the output files for tick N haven't been written yet (crash after step 10 write but before GC), the next tick will see max_seen = N-1 and compute a cutoff that deletes one fewer group than expected. On the next tick, N is written and GC catches up.
|
||||
|
||||
**Not a bug** — SPEC D-O5 explicitly chose filename-based max_seen over iteration_count for robustness. Self-healing on the next tick.
|
||||
|
||||
## O2 — GC doesn't iterate recursively
|
||||
|
||||
If a future version nests files inside `outputs/` subdirectories (e.g., `outputs/tick5/`), `os.listdir` at the top level won't see them. The regex won't match, so they're preserved. Only top-level `tick{N}-*` files are affected.
|
||||
|
||||
**Not a bug** — SPEC D-O6: regex `^tick(\d+)-` matches only top-level files. Nested subdirs preserved. Not a current concern.
|
||||
|
||||
## Verdict
|
||||
|
||||
PASS — no blockers.
|
||||
@@ -0,0 +1,25 @@
|
||||
# Code Review: add-outputs-retention
|
||||
|
||||
## SPEC coverage
|
||||
|
||||
| Requirement | Status |
|
||||
|-------------|--------|
|
||||
| R1 — `_get_retention` helper from `outputs.retention` | ✓ |
|
||||
| R2 — GC executes on every tick (post-write) | ✓ step 10.5 inside `_loop_lock` |
|
||||
| R3 — Retention = 0 means no GC | ✓ `if retention <= 0: return` |
|
||||
| R4 — GC failure doesn't crash tick | ✓ OSError caught → WARNING log + swallow |
|
||||
| R5 — No new pip deps | ✓ stdlib only |
|
||||
|
||||
## Cross-script impact
|
||||
|
||||
- `scripts/loop-runner.py`: pure addition; no existing function changed.
|
||||
- `templates/loops/self-improvement/loop.json`: new `outputs.retention: 20` field.
|
||||
- `scripts/status.py`: no changes needed (create-loop template provides the default; runner reads, not status.py).
|
||||
|
||||
## Off-by-one fix
|
||||
|
||||
GC formula was `cutoff = max_seen - retention` (kept retention+1 groups). Found during test execution when `test_gc_keeps_recent_deletes_old` showed 21 remaining instead of 20. Fixed to `cutoff = max_seen - retention + 1`. Good test coverage.
|
||||
|
||||
## Verdict
|
||||
|
||||
PASS — proceed to bug_find.
|
||||
@@ -0,0 +1,17 @@
|
||||
# Doc Review: add-outputs-retention
|
||||
|
||||
## Docs touched
|
||||
|
||||
- `CHANGELOG.md` — new `[unreleased]` entry "Added — outputs retention GC" above the existing entries.
|
||||
- `design/loops/technical.md` — new subsection "Outputs retention (v1.1 — `add-outputs-retention`)" after the lock serialization subsection in §7.
|
||||
- `design/loops/functional.md` §9 — added `outputs: {retention: N}` row to the config-fields list.
|
||||
|
||||
## Docs NOT touched (intentional)
|
||||
|
||||
- `AGENTS.md`: outputs retention is runtime ergonomics, not an enforcement contract. No edit.
|
||||
- `README.md`: user-facing README doesn't enumerate every `loop.json` field. No edit.
|
||||
- `templates/loops/self-improvement/loop.json`: already updated (schema edit).
|
||||
|
||||
## Verdict
|
||||
|
||||
Docs in sync. Proceed to referee.
|
||||
@@ -0,0 +1,42 @@
|
||||
# Implementation: add-outputs-retention
|
||||
|
||||
## SCOPE
|
||||
|
||||
Add `loop.json` `outputs.retention` field (default 20) to bound growth of the `outputs/` directory. GC runs after step 10 inside `_loop_lock`, deleting tick groups older than the retention window. Source: `add-loop-runner/BUG_REPORT.md` O5.
|
||||
|
||||
## FILES TOUCHED
|
||||
|
||||
- `scripts/loop-runner.py`
|
||||
- Added `_get_retention(cfg) -> int`: reads `cfg.get("outputs", {}).get("retention", 20)`. Non-int types fall back to 20 with WARNING. Negative values are coerced to 0 (unlimited) with WARNING.
|
||||
- Added `_gc_outputs(loop_path, retention)`: lists `outputs/`, finds max tick index from filenames matching `^tick(\d+)-`, computes `cutoff = max_seen - retention + 1`, deletes files with tick index < cutoff. Non-tick files (`README.txt`, etc.) are preserved. Errors logged as WARNING via `_append_tick_log` and swallowed.
|
||||
- Modified `cmd_tick`: calls `_get_retention(cfg)` + `_gc_outputs(loop_path, retention)` after step 10 (`_write_state_loop`) and before step 11 (tick log), inside the `_loop_lock` block.
|
||||
- Updated docstring step list: added `10.5. GC outputs/...`.
|
||||
- `templates/loops/self-improvement/loop.json`
|
||||
- Added `"outputs": {"retention": 20}` block.
|
||||
|
||||
## BUG FOUND AND FIXED INLINE
|
||||
|
||||
**Off-by-one in GC formula**: the initial implementation used `cutoff = max_seen - retention`, which kept `retention + 1` tick groups (21 instead of 20 for retention=20). Fixed to `cutoff = max_seen - retention + 1`. Test `test_gc_keeps_recent_deletes_old` caught this (expected 20 kept, got 21 remaining → obvious failure when the remaining-count length check triggered).
|
||||
|
||||
## DECISIONS LOCKED
|
||||
|
||||
- **D-O1**: retention counts tick GROUPS (all `tick{N}-*` files), not individual files.
|
||||
- **D-O2**: GC runs INSIDE `_loop_lock` critical section (after state write, before tick log).
|
||||
- **D-O3**: Default 20.
|
||||
- **D-O4**: 0 = unlimited (no GC). Negative coerces to 0 with WARNING.
|
||||
- **D-O5**: GC based on `outputs/` filenames (`max_seen`), NOT `state.iteration_count`.
|
||||
- **D-O6**: Regex `^tick(\d+)-`. Non-matching files preserved.
|
||||
- **D-O7**: GC failure → WARNING log + swallow.
|
||||
|
||||
## TESTS
|
||||
|
||||
New file `tests/test_outputs_retention.py` — 13 tests across 2 classes:
|
||||
|
||||
- `TestGetRetention` (5): default `main`, explicit value, negative→0, non-int→20, None cfg→20.
|
||||
- `TestGcOutputs` (8): deletes old keeps recent, retention=0 skip, retention>count, missing dir, non-tick files preserved, unrelated `tick-foo` prefix preserved, single tick, error path.
|
||||
|
||||
## TEST COUNT
|
||||
|
||||
- Baseline: 469 passed (post-`harden-parse-verdict`).
|
||||
- New: +13 in `tests/test_outputs_retention.py`.
|
||||
- Final: **482 passed**, 0 regressions.
|
||||
@@ -0,0 +1,4 @@
|
||||
# Review
|
||||
- **Status**: approved
|
||||
- **Timestamp**: 2026-06-23T22:19:32.288584
|
||||
- **Comment**:
|
||||
@@ -0,0 +1,140 @@
|
||||
# SPEC: add-outputs-retention
|
||||
|
||||
## Problem
|
||||
|
||||
`tasks/add-loop-runner/BUG_REPORT.md` O5:
|
||||
|
||||
> Every tick writes 3 files (implement, verify, orchestrate). Over 100 ticks that's 300 files. Trees on some filesystems (HFS+, ext4 default) degrade past 10k entries per dir. v1 has `max_iterations` to bound this; for daemon mode with `max_iterations=0`, the user is responsible.
|
||||
|
||||
Actual count is **6 files per tick** (each role: a `tick{N}-<role>-prompt.md` written by `_resolve_prompt`, plus a `tick{N}-<role>.json` written by `cmd_tick`). With `max_iterations=0` (daemon, unbounded), the `outputs/` directory grows without bound.
|
||||
|
||||
## Goal
|
||||
|
||||
Bound `outputs/` directory growth by retaining only the **last N tick groups**. A "tick group" = all files with the `tick{N}-` prefix for a single tick index N. Older tick groups are garbage-collected on every tick.
|
||||
|
||||
## Non-goals
|
||||
|
||||
- Per-role retention (e.g. keep verify-outputs longer than implement-outputs). Out of scope; would complicate the schema.
|
||||
- Compression / archival of old tick dirs to a tarball. Out of scope.
|
||||
- Cross-loop retention. Each loop's `outputs/` is independent.
|
||||
- Retention of `.state.log` (tick log). That file is append-only and grows linearly; separate concern.
|
||||
|
||||
## Schema addition (`loop.json`)
|
||||
|
||||
Add an optional `outputs` object:
|
||||
|
||||
```json
|
||||
"outputs": {
|
||||
"retention": 20
|
||||
}
|
||||
```
|
||||
|
||||
- **`outputs.retention`** (int, optional, default **20**): keep the last N tick groups. Older tick groups are deleted on every tick. `0` = unlimited (no GC; v1 behavior). Negative values are rejected at `--create-loop`.
|
||||
|
||||
## Requirements
|
||||
|
||||
### R1 — retention config plumbing
|
||||
|
||||
- `status.py --create-loop` accepts `outputs.retention` in the `loop.json` template.
|
||||
- The runner reads `cfg.get("outputs", {}).get("retention", 20)`.
|
||||
- Validation on read: if `retention` is < 0, log WARNING and treat as `0` (unlimited). Non-int types coerce via `int(...)`; on `TypeError`/`ValueError` fall back to default `20`.
|
||||
|
||||
### R2 — GC executes on every tick (post-write)
|
||||
|
||||
- After step 10 (`_write_state_loop`) and before step 11 (tick log), the runner invokes `_gc_outputs(loop_path, state, retention)`.
|
||||
- GC iterates `outputs/` directory, parses `tickNN-` prefixes, computes the cutoff = `iteration_count - retention + 1` (kept range: `[cutoff, iteration_count]` inclusive).
|
||||
- Any file whose tick-index prefix is `< cutoff` is deleted. Files without a `tickN-` prefix are left alone (forward-compat; user may place other files in `outputs/`).
|
||||
- GC errors (file in use, permission) are logged via `_append_tick_log` WARNING and swallowed — GC failure must not crash the tick.
|
||||
|
||||
### R3 — Retention = 0 means no GC
|
||||
|
||||
- `0` skips the GC step entirely (cheapest path for `max_iterations` users who prefer manual cleanup).
|
||||
|
||||
### R4 — Atomicity / failure isolation
|
||||
|
||||
- GC failures (permission, file not found mid-iteration) don't roll back the tick. State has already advanced; losing a GC pass is benign (next tick re-attempts).
|
||||
- Missing `outputs/` (loop never ticked) — GC no-ops, no error.
|
||||
|
||||
### R5 — No new pip deps
|
||||
|
||||
- Pure stdlib: `os.listdir`, `os.remove`, `re.match`. No `shutil.rmtree` (we delete individual files; a tick group is not a directory).
|
||||
|
||||
## Detailed semantics
|
||||
|
||||
### Tick-index extraction
|
||||
|
||||
Filenames follow the pattern `tick<int>-<remainder>` where `<int>` is the 1-based tick index. Examples:
|
||||
- `tick1-implement.json`, `tick1-verify.json`, `tick1-orchestrate.json`, `tick1-implement-prompt.md`, `tick1-verify-prompt.md`, `tick1-orchestrate-prompt.md`
|
||||
|
||||
Regex: `^tick(\d+)-`. Tick indices are extracted into a set, the maximum tick index (`max_seen`) is computed, and the cutoff floor is `max_seen - retention + 1`. Files with tick index `< floor` get deleted.
|
||||
|
||||
**Why `max_seen - retention + 1` instead of `state.iteration_count`?**
|
||||
|
||||
State could lag (e.g. concurrent ticks), but the on-disk filenames ARE ground truth. Using max filename keeps GC self-contained.
|
||||
|
||||
### Default retention choice
|
||||
|
||||
Default = **20**. Rationale:
|
||||
- Score-plateau window default is often 5-10; keeping 2x that covers debugging.
|
||||
- 20 ticks × 6 files = 120 files max — comfortably under any filesystem degradation threshold.
|
||||
- Operators who need longer history (`audit` use cases) override upward in `loop.json`.
|
||||
|
||||
### Where GC runs in the tick flow
|
||||
|
||||
```
|
||||
... step 10: _write_state_loop(state)
|
||||
# NEW: step 10.5
|
||||
_gc_outputs(loop_path, state, retention)
|
||||
# step 11
|
||||
_append_tick_log(...)
|
||||
```
|
||||
|
||||
GC runs INSIDE the `_loop_lock` critical section, so a concurrent `--pause-loop` / `--approve --loop` can't be mid-write and observe a missing tick dir. GC's filesystem delete ops are independent of `.state.loop`.
|
||||
|
||||
## Test plan
|
||||
|
||||
Pure-function + filesystem tests (no subprocess, no live LLM):
|
||||
|
||||
1. **GC deletes old tick groups, keeps recent N**: write 30 tick groups (6 files each), retention=20, expect last 20 kept, oldest 10 deleted, all 6 files per kept tick are present.
|
||||
2. **Retention = 0 skips GC entirely**: 30 tick groups, retention=0, expect no deletion, all files present.
|
||||
3. **Retention > file count** (no-op): 5 tick groups, retention=20, expect no deletion.
|
||||
4. **Missing `outputs/` dir** (no-op, no error): fresh loop, no `outputs/`, GC returns cleanly.
|
||||
5. **Non-tick files in `outputs/` are preserved**: write 30 tick groups + a `README.txt` and `loop-info.md`, retention=20, expect tick groups deleted but `README.txt` and `loop-info.md` intact.
|
||||
6. **Negative retention coerces to 0 (no GC)**: retention=-5 in `loop.json`, expect WARNING + no deletion.
|
||||
7. **Non-int retention coerces to default 20**: retention="twenty", expect WARNING + default 20 used (deletes oldest 10 of 30).
|
||||
8. **Tick-index regex preserves unrelated `tick-foo` files** (defensive): `tick-foo.md` (no number) does NOT match `^tick(\d+)-`; expect preserved.
|
||||
9. **GC error swallowed (permission-denied file)**: chmod 000 a stale tick file (or use a non-existent mock that raises `PermissionError`); expect GC logs WARNING and continues; tick proceeds.
|
||||
10. **Concurrent with state write** (lock interaction): GC runs inside the lock; no separate test needed (the `test_state_loop_lock.py` suite already covers lock integrity).
|
||||
11. **Config plumbing**: `--create-loop` writes `outputs.retention: 20` into generated `loop.json` (if `--outputs-retention` not provided; or honors override).
|
||||
12. **Default getter**: `_get_retention(cfg)` returns 20 for missing `outputs`, 0 when `{"outputs": {"retention": 0}}`, 20 for `{"outputs": {"retention": "garbage"}}` (post-WARNING).
|
||||
|
||||
## Decisions (locked)
|
||||
|
||||
- **D-O1**: retention counts tick GROUPS not individual files. A tick group = all `tick{N}-*` files. Keeps the mental model aligned with "ticks as the atomic unit".
|
||||
- **D-O2**: GC runs INSIDE `_loop_lock` critical section (after state write, before tick log). Cheapest correct placement — no separate lock, no concurrent `--pause-loop` / `--approve --loop` mid-GC race. Filesystem delete ops are independent of `.state.loop` but the lock keeps the loop's externally-observable state consistent.
|
||||
- **D-O3**: Default 20 (covers debugging; 120 files max comfortably under fs degradation).
|
||||
- **D-O4**: `0` = unlimited (no GC). Negative coerces to 0 with WARNING.
|
||||
- **D-O5**: GC based on `outputs/` filenames (`max_seen`), NOT `state.iteration_count`. Self-contained; robust to state lag.
|
||||
- **D-O6**: Regex `^tick(\d+)-`. Files not matching are preserved (forward-compat for helper docs, scratch notes, etc.).
|
||||
- **D-O7**: GC failure (PermissionError, FileNotFoundError mid-iteration) → WARNING log + swallow. Tick not affected.
|
||||
|
||||
## Out of scope (filed BACKLOG.md)
|
||||
|
||||
- `outputs.retention_bytes` (磁盘 budget cap). Future.
|
||||
- Tarball archival of GC'd tick groups. Future.
|
||||
- Cross-loop retention aggregation. Future.
|
||||
- GC `.state.log` rotation. Separate task (`add-state-log-rotation`).
|
||||
|
||||
## Files touched
|
||||
|
||||
- `scripts/loop-runner.py` — add `_get_retention(cfg)` + `_gc_outputs(loop_path, state, retention)`; call after step 10 inside `_loop_lock`.
|
||||
- `scripts/status.py` — `--create-loop` writes `outputs.retention` default 20 into generated `loop.json` template; validates non-negative.
|
||||
- `templates/loops/self-improvement/loop.json` — add `"outputs": {"retention": 20}` to template.
|
||||
- `design/loops/technical.md` — new subsection §7b "Outputs retention (v1.1 — `add-outputs-retention`)".
|
||||
- `design/loops/functional.md` — note `outputs.retention` field in the schema enum.
|
||||
- `CHANGELOG.md` — new entry under `[unreleased]`.
|
||||
- `tests/test_outputs_retention.py` (NEW) — 12 tests per plan above.
|
||||
|
||||
## Pipeline plan
|
||||
|
||||
research → research:awaiting_approval → research:approved → implement → code_review → code_review:awaiting_approval → code_review:approved → bug_find → adversarial_bug_find → doc_review → referee → complete.
|
||||
@@ -0,0 +1,22 @@
|
||||
# Referee Verdict: add-outputs-retention
|
||||
|
||||
## Status: PASS
|
||||
|
||||
## Artifacts reviewed
|
||||
|
||||
- SPEC.md, IMPLEMENTATION.md, CODE_REVIEW.md, BUG_REPORT.md, ADVERSARIAL_BUG_REPORT.md, DOC_REVIEW.md
|
||||
|
||||
## Phase gates satisfied
|
||||
|
||||
All 8 required artifacts present. Pipeline driven: research → implement → code_review → bug_find → adversarial_bug_find → doc_review → referee.
|
||||
|
||||
## Acceptance
|
||||
|
||||
- R1-R5 all satisfied. GC runs inside `_loop_lock` after step 10. 0 = unlimited. Non-int/negative handled gracefully. No new deps.
|
||||
- Off-by-one bug (`cutoff = max_seen - retention` → `cutoff = max_seen - retention + 1`) caught inline by test. Fixed before full suite.
|
||||
- 482 passed (469 + 13 new, 0 regressions). Docs in sync (CHANGELOG, technical.md, functional.md).
|
||||
- Adversarial probes: float truncation (3.14→3), bool True→1, string "inf"→20 (WARNING), non-tick files preserved, missing dir safe. All deterministic documented behavior.
|
||||
|
||||
## Verdict
|
||||
|
||||
PASS — task complete. Approve transition to complete.
|
||||
Reference in New Issue
Block a user