Files

53 lines
3.5 KiB
Markdown
Raw Permalink Normal View History

# Verdict: Full-Codebase Audit (v2.0)
## Status: NEEDS_REVIEW
**Completion Date**: 2026-06-22
## Summary
Full-codebase audit of the automaton framework v2.0. The Bug Finder identified 7 bugs (score: 55); the Adversarial Bug Finder identified 3 additional bugs (score: 7). No overlapping or contradictory findings. The framework's core state machine, phase transitions, and approval gates work correctly. However, several bugs exist in the enforcement code itself (`status.py`), the guard registration script, and the dashboard — undermining the "state-enforced" guarantee in specific scenarios.
## Findings
### What Passed
- Core state machine: `.state` file reading, `_base_phase()` approval substate handling, and `--transition` validation are correct
- `_check_forbidden_artifacts()` correctly detects out-of-order artifacts per phase
- Dashboard static file serving has proper path-traversal protection (`resolve()` + `relative_to()`)
- Dashboard review file writing validates task names against `[A-Za-z0-9_-]+`
- POST body size limits (64KB) and review comment length limits (4096) are enforced
- Test suite: 235 tests pass
- Git hooks (pre-commit, post-commit, pre-push) have correct logic
### What Failed
- **Bug 1** (High): Category 3 audit is broken for ALL regular (non-framework) projects — task folder changes are always flagged as unauthorized
- **Bug 4** (High): Verdict inference uses `"PASS" in content` substring search, which can misclassify FAIL/NEEDS_REVIEW verdicts as complete
- **Bug 5** (High): `register-guards.sh` never successfully registers the OpenCode guard due to 3 compounding bugs (wrong filename, wrong config key, JSONC parsing)
- **Bug 7** (High): `--can-edit` file scope check uses string `startswith` without trailing separator, allowing edits in sibling directories
### What Needs Review
- **Bug 2** (Medium): `migrate-project.sh` `find` precedence silently skips `.md` files during migration
- **Bug 3** (Medium): `vram_detect.py` prefix matching gives wrong context windows for unknown models
- **Bug 6** (Medium): Dashboard ignores `.state` files, contradicting the "single source of truth" design
- **Bug 8** (Medium): Dashboard CORS `*` allows cross-origin writes from any website
- **Bug 9** (Low): Stale-task mtime proxy is unreliable after transitions
- **Bug 10** (Low): Artifact inference maps TEST_PLAN.md to `implement` instead of `test_design`
## Bug Finder vs Adversarial Bug Finder Comparison
- **Found by both**: None (reports are complementary by design)
- **Found only by Bug Finder**: Bugs 1–7 (enforcement logic, shell scripts, model lookup, guard registration)
- **Found only by Adversarial Bug Finder**: Bugs 8–10 (security, session tracking, phase inference edge case)
- **Contradictions**: None. The two reports are consistent and non-overlapping.
## Tasks for Review / Tie-Breaks
- None. No contradictions between Bug Finder and Adversarial Bug Finder. All 10 bugs are independently verified with reproduction steps.
## Remaining Issues
- Bugs 1, 4, 5, 7 (High severity) should be fixed before relying on the framework for production enforcement. Bug 5 in particular means the primary enforcement layer (harness pre-edit guard) is likely not registered for most users.
- Bug 6 means the dashboard display may not match the actual enforced state — users could make decisions based on stale dashboard info.
- Bug 7 is a security issue: the `--can-edit` scope check can be bypassed via sibling directory names.
## Score
+5 (NEEDS_REVIEW)
## Reviewer Comments