automaton/tasks/state-file-enforcement/SPEC.md

SPEC: State File Enforcement

Goal

Replace the fragile "check which artifacts exist" heuristic for determining task phase with an explicit .state file that is the single source of truth for a task's current phase.

Background

Currently, orchestrate.md and workflow.md determine a task's phase by checking which artifact files exist in the task folder (e.g., "if SPEC.md exists but not DESIGN.md, the task is in research phase"). This is unreliable because:

• Any agent can create any artifact file at any time, bypassing phase ordering
• File-existence checks are ambiguous (e.g., overlapping conditions when multiple artifacts exist)
• There's no authoritative record of what phase a task is in — every agent has to re-derive it

Requirements

1. .state file format

• Location: tasks/{task-name}/.state
• Content: a phase name (and optional approval status), one of:
  • new
  • research / research:awaiting_approval / research:approved
  • decomposition / decomposition:awaiting_approval / decomposition:approved
  • design / design:awaiting_approval / design:approved
  • test_design / test_design:awaiting_approval / test_design:approved
  • implement
  • bug_find
  • adversarial_bug_find
  • doc_review
  • referee
  • complete
  • human_intervention
• The base phase name (e.g., research) is used for backward compatibility and when approval is not applicable
• The :awaiting_approval sub-state means the phase artifact has been produced but not yet approved by the user
• The :approved sub-state means the user has given explicit approval to proceed
• Only phases with interactive sign-off requirements use sub-states: research, decomposition, design, test_design
• Phases without sign-off (implement, bug_find, adversarial_bug_find, doc_review, referee) use only the base name
• The file must be written atomically (write to .state.tmp, then rename to .state) to prevent partial reads
• The file must NOT be listed in task artifact checks — it is metadata, not a deliverable

2. State transitions

• Only the Orchestrator (or status.py) may write to .state
• Phase prompts must READ .state to confirm they are in the correct phase before acting
• Transition rules match the existing state machine in workflow.md:
  • new → research (when task folder is created)
  • research → research:awaiting_approval (when SPEC.md draft is presented for sign-off)
  • research:awaiting_approval → research:approved (when user says APPROVED)
  • research:approved → decomposition or design or implement (when SPEC.md is finalized)
  • decomposition → decomposition:awaiting_approval (when DECOMPOSITION.md draft is presented)
  • decomposition:awaiting_approval → decomposition:approved (when user says APPROVED)
  • decomposition:approved → sub-task research (when DECOMPOSITION.md is finalized)
  • design → design:awaiting_approval (when DESIGN.md draft is presented)
  • design:awaiting_approval → design:approved (when user says APPROVED)
  • design:approved → test_design or implement (when DESIGN.md is finalized)
  • test_design → test_design:awaiting_approval (when TEST_PLAN.md draft is presented)
  • test_design:awaiting_approval → test_design:approved (when user says APPROVED)
  • test_design:approved → implement (when TEST_PLAN.md is finalized)
  • implement → bug_find (when IMPLEMENTATION.md is produced — no approval needed)
  • bug_find → adversarial_bug_find (when BUG_REPORT.md is produced — no approval needed)
  • adversarial_bug_find → doc_review (when ADVERSARIAL_BUG_REPORT.md is produced — no approval needed)
  • doc_review → referee (when DOC_REVIEW.md is produced — no approval needed)
  • referee → complete (when VERDICT.md has PASS)
  • referee → human_intervention (when VERDICT.md has FAIL or NEEDS_REVIEW)

Approval gate enforcement:

• status.py --transition MUST REFUSE to transition past an :awaiting_approval sub-state
• Only status.py --approve can move from :awaiting_approval to :approved
• Only status.py --transition from :approved can move to the next phase
• This makes sign-off enforceable — an agent cannot skip approval and proceed

3. Backward compatibility

• If .state exists, it is authoritative
• If .state does not exist, fall back to the artifact-based heuristic (current behavior) and write .state with the inferred phase
• This ensures existing tasks without .state continue to work and get migrated on first access

4. Artifact validation rules (enforced by status.py)

Each phase has a defined set of artifacts that are ALLOWED and FORBIDDEN in the task folder. These rules are enforced by status.py --validate-folder and status.py --transition:

Forbidden artifacts per phase (artifacts from future phases — must NOT exist):

Phase	Forbidden artifacts
new	SPEC.md, DESIGN.md, DECOMPOSITION.md, TEST_PLAN.md, IMPLEMENTATION.md, BUG_REPORT.md, ADVERSARIAL_BUG_REPORT.md, DOC_REVIEW.md, VERDICT.md
research	DESIGN.md, DECOMPOSITION.md, TEST_PLAN.md, IMPLEMENTATION.md, BUG_REPORT.md, ADVERSARIAL_BUG_REPORT.md, DOC_REVIEW.md, VERDICT.md
decomposition	DESIGN.md, TEST_PLAN.md, IMPLEMENTATION.md, BUG_REPORT.md, ADVERSARIAL_BUG_REPORT.md, DOC_REVIEW.md, VERDICT.md
design	DECOMPOSITION.md, TEST_PLAN.md, IMPLEMENTATION.md, BUG_REPORT.md, ADVERSARIAL_BUG_REPORT.md, DOC_REVIEW.md, VERDICT.md
test_design	DECOMPOSITION.md, IMPLEMENTATION.md, BUG_REPORT.md, ADVERSARIAL_BUG_REPORT.md, DOC_REVIEW.md, VERDICT.md
implement	BUG_REPORT.md, ADVERSARIAL_BUG_REPORT.md, DOC_REVIEW.md, VERDICT.md
bug_find	ADVERSARIAL_BUG_REPORT.md, DOC_REVIEW.md, VERDICT.md
adversarial_bug_find	DOC_REVIEW.md, VERDICT.md
doc_review	VERDICT.md
referee	(none)
complete	(none)
human_intervention	(none)

Non-artifact files are never forbidden: .state, VRAM_CONFIG.md, PARENT_SPEC.md, REVIEW.md, .state.tmp. These are metadata and can exist at any phase.

Enforcement: When status.py --transition is called, it must check for forbidden artifacts BEFORE allowing the transition. If forbidden artifacts exist, the transition is refused with a clear error identifying the offending artifacts.

5. Orchestrator updates

• Update prompts/orchestrate.md:
  • State determination section must read .state first, falling back to artifact heuristic
  • After each phase transition, the Orchestrator must write .state with the new phase name
  • The .state file must be written BEFORE the orchestrator begins executing the next phase
• Update prompts/workflow.md:
  • Add .state as the canonical phase indicator
  • Note that artifact-based heuristic is a fallback only

5. Phase prompt updates

• Every phase prompt must include a precondition check:

  Read tasks/{task}/.state. If the phase does not match this prompt's phase, STOP and report.
  

• This is a 2-line addition to each prompt's "Read These Files" section

6. Task creation gate

• Task folders MUST be created via status.py --create-task (defined in status-script spec)
• A valid task folder has a .state file with new as the initial phase
• --validate-folder checks that task folders have a .state file; folders without .state were created manually and should be flagged
• --audit flags task folders without .state as violations: "Task '{task-name}' was created manually (no .state file). Use 'python ~/.automaton/scripts/status.py --create-task' to create tasks properly."

7. Artifact integrity

• .state is NOT an artifact — it should NOT appear in state determination logic that checks artifact files
• .state should be added to .gitignore patterns (or documented that it's transient metadata)

Acceptance Criteria

• .state file format is specified and documented (including approval sub-states)
• Approval sub-states defined for research, decomposition, design, test_design
• Approval transition rules defined (:awaiting_approval → :approved only via --approve)
• Transition past :awaiting_approval without approval is REFUSED by --transition
• Phases without sign-off (implement, bug_find, etc.) use base names only
• Atomic write mechanism is defined (write-to-tmp-then-rename)
• Fallback to artifact heuristic when .state doesn't exist is specified
• Forbidden artifacts per phase are defined and documented
• status.py --validate-folder enforces forbidden artifacts
• status.py --transition refuses transitions when forbidden artifacts exist
• status.py --validate-folder flags task folders without .state as manually created
• status.py --audit flags manually created task folders
• status.py --create-task is the only valid way to create task folders
• orchestrate.md updated to read/write .state
• workflow.md updated to reference .state as canonical
• All phase prompts include .state precondition check
• State transition rules match existing state machine
• .state is excluded from artifact-based state determination
• Sub-task .state files are scoped to sub-task folders

Non-Goals

• This spec does not cover the status.py script (separate task)
• This spec does not cover prompt restructuring with FORBIDDEN sections (separate task)
• This spec does not cover autopilot integration (separate task)