Files

T

Lap Tran bc7daf8590 Restore archived tasks, fix dashboard scroll-reset, bind ornith, add Playwright smoke test

- **Restore 82 completed tasks** from tasks/complete/ back to tasks/ top
  level (all <7 days old per the cleanup policy; premature bulk archive
  was fixed).
- **Dashboard: fix scroll-reset on auto-refresh** — renderBoard rebuilds
  the board via innerHTML every 2s, destroying each column-body's
  scrollTop. Now snapshots column-body scrollTop + board.scrollLeft +
  view.scrollTop before rebuild and restores after (matched by
  PHASE_GROUPS index).
- **Dashboard UI additions** (pre-existing unstaged work): approval
  section cards, transition buttons, inline artifact editor (textarea for
  writing missing SPEC/VERDICT/etc from the detail modal).
- **Bind ornith as Implement model** — config.md: Model explicit to
  omlx/Ornith-1.0-35B-4bit-mlx, context window 32768. Interactive
  autopilot already used ornith via opencode default; now explicit.
- **Fix cleanup stub** — automaton-cleanup.sh had a stale --project arg
  pointing at a pytest temp dir (test isolation leak). Rewired to point
  at ~/.automaton.
- **Fix plist-isolation test** — test asserted host plist doesn't exist,
  but a real install creates it. Now snapshots mtime before run, asserts
  unchanged after (only a write during the test counts as bleed).
- **New Playwright smoke test** (tests/test_dashboard_ui.py) — 2 tests:
  board renders tasks, column scroll survives auto-refresh tick.
  Verified the test fails without the scroll fix (scrollTop resets to 0).
  Skipped via importorskip when playwright is absent (main CI stays
  green).
- **Clarify SI loop scope in README** — new-project onboarding section
  documents the framework-scoped self-improvement loop and options
  (leave/pause/create project loop).
- **CHANGELOG** documents all changes including the known model-divergence
  gap (mde tasks marked complete but per-role model binding was never
  implemented).

2026-06-26 10:05:18 -04:00

3.5 KiB

Raw Permalink Blame History

Bug Report: harden-parse-verdict

Bug_find phase observations. Each non-blocking unless marked BLOCKER.

O1 — `bool(raw_pass.strip())` fallback for "0" / "1" strings

For {"pass": "0", "score": 0.5}, the new path strips → "0" → not "true"/ "false" → bool("0") = True (non-empty string is truthy).

Pre-fix: bool("0") was also True (same).

This is consistent with v1 — no behavior change. A verifier emitting pass: "0" intending "false" gets True in v1 AND in the hardened implementation. The SPEC says non-true/false strings fall through to bool(...), which is truthy for non-empty. Not a regression.

Not a bug — documented behavior per SPEC R1 + D-V1. If users want strict numeric-string handling, that's a separate future task (out of scope for v1.1 harden-parse-verdict).

O2 — `float("nan")` serializes back as `NaN` to `.state.loop`

When the verifier emits NaN as the score, parse_verdict returns score=0.5 (without writing to disk by itself). But this score is part of verdict which gets written via _write_state_loop(loop_path, state) into .state.loop as JSON. Since the clamp converts NaN to 0.5 BEFORE the verdict is stored, .state.loop gets 0.5, not NaN. No NaN leaks into the loop state.

Not a bug — confirmed via tracing: parse_verdict returns the clamped dict; cmd_tick then stores state["last_verdict"] = verdict (a clean dict with score: 0.5); the JSON round-trip is clean.

O3 — `_gate_score_plateau`'s threshold unaffected

_gate_score_plateau (status.py) reads score_history and decides a halt when last N scores are within some delta. With clamped scores, the plateau detection range is now strictly [0, 1] instead of [any, any]. Brief review:

Pre-fix: a verifier could emit score: 1.5 across N ticks; plateau detection sees a flat line at 1.5; halt fires. Expected behavior.
Post-fix: the same verifier's 1.5 clamps to 1.0 across N ticks; plateau sees flat line at 1.0; halt fires. Same outcome.
Pre-fix: a verifier emits alternating 0.9 and 1.1; plateau sees a bimodal history [0.9, 1.1, 0.9, 1.1] — NOT plateau (variation > epsilon).
Post-fix: alternating 0.9 and 1.0 (1.1 clamps to 1.0); plateau sees [0.9, 1.0, 0.9, 1.0] — still variation above a small epsilon — still NOT plateau. Same outcome in this scenario.

Edge case: a verifier emits all 1.0 and 0.99 (vs 1.0 and 1.0 clamped). The clamp DOES change plateau detection in this case — 1.0 1.0 1.0 looks more plateau-like than 1.0 0.99 0.99. Could cause halt earlier than prior. Documented as a desirable side effect (clamp reduces the verifier's untrustworthiness from inflating scores; plateau detection is more honest).

Not a bug — improved behavior. Documented in CHANGELOG.

O4 — Verifier prompt hasn't been updated

prompts/loop-verifier.md still asks the model to emit JSON with "pass": true/false and "score": 0.0-1.0. The runner now defensive-coerces, but the prompt's contract is unchanged. Was the prompt already JSON-typed-booleans-only? Let me check.

prompts/loop-verifier.md review: still says "Output: strict JSON, no prose" with example shape. The prompt explicitly tells the model to emit JSON booleans — no mention of string-typed pass. So the v1 contract was strict; the O6 finding was a defense-in-depth concern, not a present-fault. The harden task adds belt-and-suspenders without changing the contract.

Not a bug — the prompt remains authoritative. No edit needed.

Verdict

No blockers. Proceed to adversarial_bug_find.

3.5 KiB Raw Permalink Blame History