Restore archived tasks, fix dashboard scroll-reset, bind ornith, add Playwright smoke test
- **Restore 82 completed tasks** from tasks/complete/ back to tasks/ top level (all <7 days old per the cleanup policy; premature bulk archive was fixed). - **Dashboard: fix scroll-reset on auto-refresh** — renderBoard rebuilds the board via innerHTML every 2s, destroying each column-body's scrollTop. Now snapshots column-body scrollTop + board.scrollLeft + view.scrollTop before rebuild and restores after (matched by PHASE_GROUPS index). - **Dashboard UI additions** (pre-existing unstaged work): approval section cards, transition buttons, inline artifact editor (textarea for writing missing SPEC/VERDICT/etc from the detail modal). - **Bind ornith as Implement model** — config.md: Model explicit to omlx/Ornith-1.0-35B-4bit-mlx, context window 32768. Interactive autopilot already used ornith via opencode default; now explicit. - **Fix cleanup stub** — automaton-cleanup.sh had a stale --project arg pointing at a pytest temp dir (test isolation leak). Rewired to point at ~/.automaton. - **Fix plist-isolation test** — test asserted host plist doesn't exist, but a real install creates it. Now snapshots mtime before run, asserts unchanged after (only a write during the test counts as bleed). - **New Playwright smoke test** (tests/test_dashboard_ui.py) — 2 tests: board renders tasks, column scroll survives auto-refresh tick. Verified the test fails without the scroll fix (scrollTop resets to 0). Skipped via importorskip when playwright is absent (main CI stays green). - **Clarify SI loop scope in README** — new-project onboarding section documents the framework-scoped self-improvement loop and options (leave/pause/create project loop). - **CHANGELOG** documents all changes including the known model-divergence gap (mde tasks marked complete but per-role model binding was never implemented).
This commit is contained in:
@@ -0,0 +1,78 @@
|
||||
# Bug Report: harden-parse-verdict
|
||||
|
||||
Bug_find phase observations. Each non-blocking unless marked BLOCKER.
|
||||
|
||||
## O1 — `bool(raw_pass.strip())` fallback for "0" / "1" strings
|
||||
|
||||
For `{"pass": "0", "score": 0.5}`, the new path strips → "0" → not "true"/
|
||||
"false" → `bool("0")` = True (non-empty string is truthy).
|
||||
|
||||
Pre-fix: `bool("0")` was also True (same).
|
||||
|
||||
This is **consistent with v1** — no behavior change. A verifier emitting
|
||||
`pass: "0"` intending "false" gets `True` in v1 AND in the hardened
|
||||
implementation. The SPEC says non-`true`/`false` strings fall through to
|
||||
`bool(...)`, which is truthy for non-empty. Not a regression.
|
||||
|
||||
**Not a bug** — documented behavior per SPEC R1 + D-V1. If users want
|
||||
strict numeric-string handling, that's a separate future task (out of
|
||||
scope for v1.1 harden-parse-verdict).
|
||||
|
||||
## O2 — `float("nan")` serializes back as `NaN` to `.state.loop`
|
||||
|
||||
When the verifier emits `NaN` as the score, `parse_verdict` returns
|
||||
`score=0.5` (without writing to disk by itself). But this score is part
|
||||
of `verdict` which gets written via `_write_state_loop(loop_path, state)`
|
||||
into `.state.loop` as JSON. Since the clamp converts NaN to 0.5 BEFORE
|
||||
the verdict is stored, `.state.loop` gets `0.5`, not `NaN`. No NaN leaks
|
||||
into the loop state.
|
||||
|
||||
**Not a bug** — confirmed via tracing: `parse_verdict` returns the clamped
|
||||
dict; `cmd_tick` then stores `state["last_verdict"] = verdict` (a clean
|
||||
dict with `score: 0.5`); the JSON round-trip is clean.
|
||||
|
||||
## O3 — `_gate_score_plateau`'s threshold unaffected
|
||||
|
||||
`_gate_score_plateau` (status.py) reads `score_history` and decides a halt
|
||||
when last N scores are within some delta. With clamped scores, the plateau
|
||||
detection range is now strictly `[0, 1]` instead of `[any, any]`. Brief
|
||||
review:
|
||||
|
||||
- Pre-fix: a verifier could emit `score: 1.5` across N ticks; plateau
|
||||
detection sees a flat line at 1.5; halt fires. Expected behavior.
|
||||
- Post-fix: the same verifier's 1.5 clamps to 1.0 across N ticks; plateau
|
||||
sees flat line at 1.0; halt fires. Same outcome.
|
||||
|
||||
- Pre-fix: a verifier emits alternating `0.9` and `1.1`; plateau sees a
|
||||
bimodal history [0.9, 1.1, 0.9, 1.1] — NOT plateau (variation > epsilon).
|
||||
- Post-fix: alternating `0.9` and `1.0` (1.1 clamps to 1.0); plateau sees
|
||||
[0.9, 1.0, 0.9, 1.0] — still variation above a small epsilon — still NOT
|
||||
plateau. Same outcome in this scenario.
|
||||
|
||||
Edge case: a verifier emits all `1.0` and `0.99` (vs 1.0 and 1.0 clamped).
|
||||
The clamp DOES change plateau detection in this case — `1.0 1.0 1.0`
|
||||
looks more plateau-like than `1.0 0.99 0.99`. Could cause halt earlier than
|
||||
prior. Documented as a desirable side effect (clamp reduces the verifier's
|
||||
untrustworthiness from inflating scores; plateau detection is more
|
||||
honest).
|
||||
|
||||
**Not a bug** — improved behavior. Documented in CHANGELOG.
|
||||
|
||||
## O4 — Verifier prompt hasn't been updated
|
||||
|
||||
`prompts/loop-verifier.md` still asks the model to emit JSON with
|
||||
`"pass": true/false` and `"score": 0.0-1.0`. The runner now defensive-coerces,
|
||||
but the prompt's contract is unchanged. Was the prompt already
|
||||
JSON-typed-booleans-only? Let me check.
|
||||
|
||||
**`prompts/loop-verifier.md` review**: still says "Output: strict JSON, no
|
||||
prose" with example shape. The prompt explicitly tells the model to emit
|
||||
JSON booleans — no mention of string-typed `pass`. So the v1 contract was
|
||||
strict; the O6 finding was a defense-in-depth concern, not a present-fault.
|
||||
The harden task adds belt-and-suspenders without changing the contract.
|
||||
|
||||
**Not a bug** — the prompt remains authoritative. No edit needed.
|
||||
|
||||
## Verdict
|
||||
|
||||
**No blockers.** Proceed to adversarial_bug_find.
|
||||
Reference in New Issue
Block a user