The runner cleared the input with only 6 backspaces and fired one unverified
Enter, so a dropped keystroke left a half-typed prompt that corrupted the next
send and lingered after exit. Worse, online/grant detection counted chat events
via `capture-pane -S` — but this is a full-screen alt-screen app whose scrollback
returns stale/empty frames, so detection was unreliable and the restart loop kept
dismissing healthy-but-slow spawns into a churn cycle.
New bench/tui.py exposes verified primitives shared by the runner and a restart
CLID:
* clear_input / submit — backspace-clear and Enter until the input box reads
empty (the box has no line-editing; Ctrl-A/U/K arrive as literal letters)
* capture() now reads only the VISIBLE viewport (no -S) — the live screen is
the only trustworthy source
* agent_online() reads the present-tense clergy roster, not scrolled-away chat
* restart_agent() stops/starts/grants with a generous 180s online wait (cold
/ai start reloads the model and takes 60-90s on CPU) and retries only a
genuinely hung spawn
run.py now delegates send/clear/online-check to tui and clears the box on exit.
python bench/tui.py restart <model> # one-shot reliable restart+grant
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add bench/ — a 4-category × easy/medium/hard task matrix (shell, code, git,
multi) and a runner that drives the live TUI over tmux and grades each task by
a `podman exec` verify snippet (exit 0 == PASS), never by the model's
self-reported summary (which the weak CPU model fabricates). Tasks run in the
agent's real cwd with bare filenames so the suite measures task completion, not
the model's absolute-path discipline. Completion is detected off the viewport-
independent `is thinking…` footer (the TUI is full-screen, so capture-pane
scrollback is not chat history).
First baseline (qwen2.5:3b): 2/12 PASS, high variance. Surfaces the next
harness-addressable improvements — `<native>` tag leakage and bare
tool-call-as-text — now measurable against this suite. Findings doc updated.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
leetcrypt