A comprehensive bug finder review of the agent-framework identified **18 bugs** (17 new + 1 re-reported from the previous review). All bugs have been fixed. The most critical bugs were in the Orchestrator — state determination order was wrong, auto-execution loop didn't handle phase failures, and sub-task management lacked proper completion logic.
- **Location**: `prompts/orchestrate.md`, "State Determination" section
- **Description**: The state determination checks from "most advanced state backward" but the order was inconsistent with the workflow.md state machine. The Orchestrator checked for IMPLEMENTATION.md (line 82) BEFORE checking for BUG_REPORT.md + SPEC.md (line 80), which meant if a task had both IMPLEMENTATION.md and BUG_REPORT.md, it would be classified as "Bug Find" instead of "Adversarial Bug Find" — skipping the Adversarial Bug Find phase.
- **Reproduction**: A task that has `BUG_REPORT.md`, `SPEC.md`, and `IMPLEMENTATION.md` would be classified as "Bug Find" instead of "Adversarial Bug Find".
- **Fix Applied**: Reordered the state determination to match the workflow.md exactly — from most advanced backward: VERDICT.md with PASS → Complete, VERDICT.md with FAIL/NEEDS_REVIEW → Human Intervention, DOC_REVIEW.md → Referee, ADVERSARIAL_BUG_REPORT.md + BUG_REPORT.md + SPEC.md → Doc Review, BUG_REPORT.md + SPEC.md without ADVERSARIAL_BUG_REPORT.md → Adversarial Bug Find, IMPLEMENTATION.md → Bug Find, etc. Also added overlapping conditions note.
- **Description**: The paragraph "In Autopilot mode, the Orchestrator auto-executes all phases until completion or human intervention:" appeared twice consecutively (copy-paste duplication).
- **Reproduction**: Read the file; observe the duplicated sentence.
- **Fix Applied**: Removed the duplicate sentence.
- **Location**: `prompts/orchestrate.md`, "New tasks from user input" section
- **Description**: When creating a new task from user input, the Orchestrator was creating the task folder with an empty `IMPLEMENTATION.md`. But the Orchestrator comment explicitly says "Do not pre-create it." This was inconsistent and could confuse the Research phase.
- **Reproduction**: Start a new task with "Research add user auth". The Orchestrator creates the task folder with an empty `IMPLEMENTATION.md`.
- **Fix Applied**: Added explicit note: "**Do not create an empty `IMPLEMENTATION.md` for new tasks** — this is inconsistent with the Orchestrator's own rule and can confuse the Research phase."
- **Location**: `prompts/orchestrate.md`, "Sub-Task Completion and Parent Task" section
- **Description**: When a sub-task reaches a terminal state, the Orchestrator doesn't verify that the VERDICT.md exists before checking its verdict. If a sub-task somehow reaches a terminal state without a VERDICT.md (e.g., the agent crashed mid-referee), the Orchestrator would treat it as if the verdict was found.
- **Reproduction**: Sub-task folder has `BUG_REPORT.md`, `ADVERSARIAL_BUG_REPORT.md`, `DOC_REVIEW.md` but no `VERDICT.md`. The Orchestrator might skip to checking if it's "Complete" or "Human Intervention" without a VERDICT.md.
- **Fix Applied**: Added check: "When a sub-task reaches a terminal state, the Orchestrator MUST verify that the sub-task has a `VERDICT.md` before evaluating the verdict. If the sub-task does not have a `VERDICT.md`, it is NOT in a terminal state."
- **Location**: `prompts/orchestrate.md`, "Sub-Task Completion and Parent Task" section
- **Description**: The Orchestrator says "When ALL sub-tasks are in terminal state: If ALL sub-tasks PASS: The parent task is Complete." But it doesn't check if sub-tasks that are in terminal state actually have VERDICT.md with PASS. It only checks if the verdict is PASS/FAIL/NEEDS_REVIEW.
- **Reproduction**: Parent task has 3 sub-tasks. Two have VERDICT.md with PASS. The third has DOC_REVIEW.md but no VERDICT.md (agent crashed). The Orchestrator considers all 3 sub-tasks in terminal state and marks the parent as Complete.
- **Fix Applied**: Added check: "Before checking completion, verify that each sub-task is still referenced in the DECOMPOSITION.md. If a sub-task is orphaned (no longer in the DECOMPOSITION.md), remove it from the parent's completion check."
- **Description**: When creating sub-task folders, the Orchestrator doesn't check if the DECOMPOSITION.md has sub-tasks with dependencies that span different waves. If a sub-task in Wave 2 depends on a sub-task in Wave 1, the Orchestrator should ensure Wave 1 sub-tasks are driven to completion before starting Wave 2.
- **Reproduction**: Parent task has Wave 1 (sub-task A, sub-task B) and Wave 2 (sub-task C depends on A and B). The Orchestrator creates all three sub-task folders and tries to run them all in parallel.
- **Fix Applied**: Added wave enforcement: "The Orchestrator MUST check sub-task dependencies and only start driving Wave 2 sub-tasks when all Wave 1 sub-tasks are in terminal state."
- **Location**: `prompts/orchestrate.md`, "Continue from existing tasks" section
- **Description**: When the user says "orchestrate" or "continue", the Orchestrator scans for the "most advanced task." But it doesn't distinguish between parent tasks and sub-tasks. A parent task with completed sub-tasks might be more "advanced" than a sub-task that's still in the Research phase.
- **Reproduction**: Parent task has 3 sub-tasks. Two sub-tasks are in the Research phase, one is in the Referee phase. The parent task has a VERDICT.md with FAIL. The Orchestrator picks the parent task instead of continuing the sub-task in the Referee phase.
- **Fix Applied**: Added: "prioritize sub-tasks over parent tasks" and "When choosing among sub-tasks in the same wave, prioritize those in later phases (e.g., Bug Find over Research) because they are closer to completion."
- **Description**: The auto-execution loop doesn't check for phase failures that are NOT verdicts — for example, if the agent crashes mid-phase or a phase doesn't produce the expected artifact. The loop assumes every phase produces its artifact and then checks the next phase.
- **Reproduction**: Bug Find phase produces an empty `BUG_REPORT.md`. The Orchestrator checks for the artifact, sees it exists, and proceeds to Adversarial Bug Find.
- **Fix Applied**: Added to the loop: "if phase artifact is empty or malformed: break (human intervention needed — artifact validation failed)"
- **Description**: The VRAM_CONFIG.md template uses `recommended_k` from the detection script output (which is in k units, e.g., "16" for 16k), but the template says "Target VRAM context: {from detection script or config.md override}" without specifying the unit.
- **Reproduction**: The detection script outputs "recommended_k: 16" and "max_peak_context_kb: 12000". The VRAM_CONFIG.md template uses "Target VRAM context: 16" (without the "k" suffix) and "Max peak context per sub-task: 12000" (without the "k" suffix), leading to ambiguity about units.
- **Fix Applied**: Clarified the units: "Target VRAM context: {value}k tokens (e.g., "16k")", "Max peak context per sub-task: {value}k tokens (e.g., "12k")", "This sub-task's estimated peak context: {value}k tokens (e.g., "10k")". Also added note: "The units must be clarified."
- **Description**: The PARENT_SPEC.md is supposed to contain "the parent task's SPEC.md content" and "any context the sub-task needs from the parent." But it doesn't include the sub-task's own scope/acceptance criteria from the DECOMPOSITION.md.
- **Reproduction**: Parent task's SPEC.md has 5 requirements. The DECOMPOSITION.md says sub-task A is only for requirements 1-2. The PARENT_SPEC.md only contains the parent's SPEC.md (all 5 requirements).
- **Fix Applied**: Changed the PARENT_SPEC.md content to include "The sub-task's own scope/acceptance criteria from the DECOMPOSITION.md" and added: "Do NOT include the parent task's full SPEC.md — this can cause circular references if the parent's SPEC.md references the sub-task's SPEC.md files."
### Bug 11: Orchestrator — No mechanism to handle sub-task failures in parent (HIGH — FIXED)
- **Severity**: High
- **Location**: `prompts/orchestrate.md`, "Sub-Task Completion and Parent Task" section
- **Description**: When a sub-task FAILs or NEEDS_REVIEW, the Orchestrator reports "human intervention is required" but doesn't create fix tasks for the failing sub-task.
- **Reproduction**: Sub-task A FAILs. The Orchestrator reports "human intervention is required." The parent task is stuck.
- **Fix Applied**: Added fix task creation for sub-tasks in Manual Mode: "Task name: `{parent-task-name}-fix-{sub-task-name}` (e.g., `add-user-auth-fix-auth-gateway`) — The Orchestrator creates the folder with an empty `IMPLEMENTATION.md` — The task starts at the **Bug Find** phase — The Orchestrator copies the sub-task's `SPEC.md`, `BUG_REPORT.md`, and `ADVERSARIAL_BUG_REPORT.md` (if they exist) into the new task folder"
- **Description**: The Orchestrator's VRAM detection priority says "Auto-detect via script: Run {project}/.agent-framework/scripts/vram_detect.sh". If the script is not available, it falls back to "Auto-detect via API config." But the Orchestrator doesn't check if the detection script exists before trying to run it.
- **Reproduction**: User starts a new task. The Orchestrator tries to run `{project}/.agent-framework/scripts/vram_detect.sh` but the script doesn't exist.
- **Fix Applied**: Added: "Check if `{project}/.agent-framework/scripts/vram_detect.sh` exists. If it does, run it..." and "If the detection script does not exist, skip to the next detection method."
- **Description**: Even if the detection script exists, it might fail (e.g., `nvidia-smi` is not installed, the GPU is busy, etc.). The Orchestrator doesn't handle script failures gracefully.
- **Reproduction**: The detection script exists but `nvidia-smi` is not installed. The script fails with an error.
- **Fix Applied**: Added: "If the detection script fails (e.g., `nvidia-smi` is not installed, the GPU is busy, etc.), check the exit code and fall back to the next detection method."
- **Description**: The Orchestrator says "The Orchestrator should aggregate sub-task verdicts when reporting the parent task's status." But it doesn't actually implement this aggregation.
- **Reproduction**: Parent task has 3 sub-tasks. Two PASS, one FAIL. The Orchestrator reports the parent task as "Complete" because it doesn't aggregate sub-task verdicts.
- **Fix Applied**: Added: "The Orchestrator MUST aggregate sub-task verdicts when reporting the parent task's status: If ANY sub-task FAILs or NEEDS_REVIEW, the parent task should be marked as **Human Intervention** regardless of whether the parent's own VERDICT.md says PASS."
### Bug 15: Orchestrator — No mechanism to handle sub-task "Tie-Breaks" (LOW — FIXED)
- **Severity**: Low
- **Location**: `prompts/referee.md`
- **Description**: The referee has "Tasks for Review / Tie-Breaks" but the Orchestrator doesn't have logic to handle sub-task tie-breaks.
- **Reproduction**: Sub-task A has tie-breaks. The Orchestrator doesn't create tie-break tasks for the sub-task.
- **Fix Applied**: Added: "If a sub-task has 'Tasks for Review / Tie-Breaks' in its VERDICT.md, the Orchestrator should create tie-break tasks for the sub-task: Task name: `{parent-task-name}-tiebreak-{sub-task-name}`"
### Bug 16: Orchestrator — State Determination doesn't check for empty artifacts (MEDIUM — FIXED)
- **Severity**: Medium
- **Location**: `prompts/orchestrate.md`, "State Determination" section
- **Description**: The Orchestrator checks for the existence of artifacts (e.g., "Has `SPEC.md`") but doesn't check if they are empty.
- **Reproduction**: Research phase produces an empty `SPEC.md`. The Orchestrator checks for `SPEC.md` and sees it exists, so it classifies the task as "Design" (optional) or "Implement".
- **Fix Applied**: Added "(non-empty)" after every artifact check: "Has `VERDICT.md` with `PASS` (non-empty)", "Has `DOC_REVIEW.md` (non-empty)", etc.
### Bug 17: Orchestrator — "Continue" doesn't prioritize sub-tasks in the same wave (MEDIUM — FIXED)
- **Severity**: Medium
- **Location**: `prompts/orchestrate.md`, "Continue from existing tasks" section
- **Description**: When the user says "orchestrate" or "continue", the Orchestrator should prioritize sub-tasks in the same wave that are still in progress.
- **Reproduction**: Wave 1 has sub-tasks A, B, and C. A is in the Research phase, B is in the Implement phase, and C is in the Bug Find phase. The Orchestrator picks A (Research phase) instead of C (Bug Find phase).
- **Fix Applied**: Added: "When choosing among sub-tasks in the same wave, prioritize those in later phases (e.g., Bug Find over Research) because they are closer to completion."
- **Description**: The section "Auto-Execution Rules (Autopilot Mode Only)" appears after the "Auto-Execution Loop" section and contains overlapping rules.
- **Reproduction**: Read the file; observe that the Auto-Execution Loop and Auto-Execution Rules sections contain overlapping rules.
- **Fix Applied**: Consolidated the Auto-Execution Loop and Auto-Execution Rules sections into a single section with additional rules for artifact validation, timeout, iteration limit, and sub-task parallel execution.
- **Description**: The auto-execution loop has no maximum iteration count or timeout. If the agent produces an artifact but doesn't output CONTRACT_MET, the loop will spin forever.
- **Fix Applied**: Added to the loop: "if iteration_count >= MAX_ITERATIONS (default: 10): break (human intervention needed — too many iterations)", "if total_time_elapsed >= MAX_TOTAL_TIME (default: 24 hours): break (human intervention needed — too much time elapsed)", "if phase_time_elapsed >= MAX_PHASE_TIME (default: 1 hour): break (human intervention needed — phase took too long)"
- **Description**: When the Orchestrator creates sub-task folders, it doesn't check if they already exist.
- **Fix Applied**: Added: "Check for existing sub-task folders: For each sub-task, check if the folder `tasks/{parent-task-name}/subtasks/{sub-task-name}/` already exists. If it does, skip the creation and report that the sub-task has already been created."
### Adversarial Bug 3: Orchestrator — Auto-detect VRAM can cause resource exhaustion (HIGH — FIXED)
- **Description**: If the VRAM detection script is run in a loop (e.g., the Orchestrator is invoked multiple times), it will repeatedly probe the GPU and RAM, causing performance degradation.
- **Fix Applied**: Added VRAM detection caching: "When the Orchestrator is invoked multiple times (e.g., the user says 'orchestrate' twice), it MUST cache the VRAM detection results and reuse them instead of running the detection script again."
- **Location**: `prompts/orchestrate.md`, "Sub-Task Completion and Parent Task" section
- **Description**: The Orchestrator doesn't check if there are orphaned sub-tasks — sub-tasks that were created by the Orchestrator but are no longer referenced in the DECOMPOSITION.md.
- **Fix Applied**: Added: "Check for orphaned sub-tasks: Before checking completion, verify that each sub-task is still referenced in the DECOMPOSITION.md. If a sub-task is orphaned (no longer in the DECOMPOSITION.md), remove it from the parent's completion check."
- **Description**: The auto-execution loop only drives one sub-task at a time, even when sub-tasks are in the same wave and can run in parallel.
- **Fix Applied**: Added: "Sub-task Parallel Execution: When sub-tasks are in the same wave and can run in parallel, the Orchestrator should drive them simultaneously instead of sequentially."
### Adversarial Bug 6: Orchestrator — State Determination can produce ambiguous states (MEDIUM — FIXED)
- **Location**: `prompts/orchestrate.md`, "State Determination" section
- **Description**: The state determination has multiple overlapping conditions that can produce ambiguous states.
- **Fix Applied**: Added: "Note on overlapping conditions: If a task has both `TEST_PLAN.md` and `DESIGN.md`, the Orchestrator should prioritize the more advanced state (TEST_PLAN.md → Implement) over the optional state (DESIGN.md → Test Design). Similarly, if a task has both `IMPLEMENTATION.md` and `BUG_REPORT.md`, the Orchestrator should prioritize the more advanced state (BUG_REPORT.md → Adversarial Bug Find) over the earlier state (IMPLEMENTATION.md → Bug Find)."
### Adversarial Bug 7: Orchestrator — Sub-task PARENT_SPEC.md can cause circular references (MEDIUM — FIXED)
- **Description**: The PARENT_SPEC.md contains the parent task's SPEC.md content. If the parent's SPEC.md references the sub-task's SPEC.md files, a circular reference is created.
- **Fix Applied**: Changed the PARENT_SPEC.md content to include "The sub-task's own scope/acceptance criteria from the DECOMPOSITION.md" and added: "Do NOT include the parent task's full SPEC.md — this can cause circular references if the parent's SPEC.md references the sub-task's SPEC.md files."
### Adversarial Bug 8: Orchestrator — Auto-detect VRAM can cause memory exhaustion (MEDIUM — FIXED)
- **Description**: If the detection script doesn't exist, the Orchestrator tries to read multiple config files to detect the model name. If the .env file is large, reading it could cause memory exhaustion.
- **Fix Applied**: Added: "Only read the specific lines needed (e.g., the model name line), not the entire file. Limit file reads to 10KB to prevent memory exhaustion."
- **Location**: `prompts/orchestrate.md`, "Sub-Task Completion and Parent Task" section
- **Description**: When a sub-task FAILs during the Referee phase (before producing BUG_REPORT.md and ADVERSARIAL_BUG_REPORT.md), the Orchestrator wouldn't have the bug reports needed to create a fix task.
- **Fix Applied**: Added: "If the sub-task failed during the Referee phase (before producing BUG_REPORT.md and ADVERSARIAL_BUG_REPORT.md), the Orchestrator still creates the fix task but only copies the artifacts that exist (BUG_REPORT.md if it exists, ADVERSARIAL_BUG_REPORT.md if it exists)."
- **Description**: If the DECOMPOSITION.md is updated after the Orchestrator has already created sub-task folders, the Orchestrator doesn't handle the update.
- **Fix Applied**: Added: "Check for DECOMPOSITION.md updates: Compare the DECOMPOSITION.md with the existing sub-task folders. If the DECOMPOSITION.md has been updated (new sub-tasks added or existing sub-tasks removed), update the sub-task folders accordingly."
- **Description**: The VRAM_CONFIG.md includes "Max peak context per sub-task: {from detection script or config.md override}" which is the global max peak context from the detection script. But it doesn't include the sub-task's own estimated peak context from the DECOMPOSITION.md.
- **Fix Applied**: Added to the VRAM_CONFIG.md template: "This sub-task's estimated peak context: {from DECOMPOSITION.md}k tokens (e.g., "10k")" and "Fits within VRAM: Yes/No"
