Trilltechnician/hack-house
1
3
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

test(ai): robust TUI driving — fix stale-input + agent-restart keystrokes

Browse Source 
The runner cleared the input with only 6 backspaces and fired one unverified
Enter, so a dropped keystroke left a half-typed prompt that corrupted the next
send and lingered after exit. Worse, online/grant detection counted chat events
via `capture-pane -S` — but this is a full-screen alt-screen app whose scrollback
returns stale/empty frames, so detection was unreliable and the restart loop kept
dismissing healthy-but-slow spawns into a churn cycle.

New bench/tui.py exposes verified primitives shared by the runner and a restart
CLID:
  * clear_input / submit — backspace-clear and Enter until the input box reads
    empty (the box has no line-editing; Ctrl-A/U/K arrive as literal letters)
  * capture() now reads only the VISIBLE viewport (no -S) — the live screen is
    the only trustworthy source
  * agent_online() reads the present-tense clergy roster, not scrolled-away chat
  * restart_agent() stops/starts/grants with a generous 180s online wait (cold
    /ai start reloads the model and takes 60-90s on CPU) and retries only a
    genuinely hung spawn

run.py now delegates send/clear/online-check to tui and clears the box on exit.

    python bench/tui.py restart <model>   # one-shot reliable restart+grant

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
leetcrypt
2026-06-13 09:31:47 -07:00
parent c61413c648
commit d211a75616
2 changed files with 195 additions and 26 deletions
Download Patch File Download Diff File Expand all files Collapse all files
bench/run.py
+15 -26
View File
@@ -33,44 +33,28 @@ from pathlib import Path
sys.path.insert(0, str(Path(__file__).resolve().parent))
import tasks as T # noqa: E402
from tui import agent_online, capture, clear_input, send_command # noqa: E402
def sh(cmd, **kw):
return subprocess.run(cmd, capture_output=True, text=True, **kw)
def tmux(*args):
return sh(["tmux", *args])
def capture(target, lines=4000):
"""Full scrollback of the TUI pane as text."""
r = tmux("capture-pane", "-t", target, "-p", "-S", f"-{lines}")
return r.stdout
def podman(container, snippet):
"""Run a POSIX-sh snippet in the container; return (rc, output)."""
r = sh(["podman", "exec", container, "sh", "-c", snippet])
return r.returncode, (r.stdout + r.stderr).strip()
def agent_online(target, model):
return f"{model} (ai) online" in capture(target) or "online — ollama" in capture(target)
def is_thinking(buf):
return "is thinking" in buf
def send_task(target, model, prompt):
"""Clear the input box, then type the /ai command and submit it."""
# The TUI has no kill-line; clear with backspaces (harmless when empty).
tmux("send-keys", "-t", target, *(["BSpace"] * 6))
time.sleep(0.3)
tmux("send-keys", "-t", target, "-l", f"/ai {model} !{prompt}")
time.sleep(0.3)
tmux("send-keys", "-t", target, "Enter")
"""Robustly clear the input box, then type and submit the /ai command. The
box is fully backspace-cleared and the Enter is verified (see tui.py), so a
dropped keystroke can't leave a half-typed prompt to corrupt the next task."""
send_command(target, f"/ai {model} !{prompt}")
def scrape_summary(target, user):
@@ -197,11 +181,16 @@ def main():
print(f"running {len(selected)} task(s) against {args.model} "
f"(container={args.container}, tui={args.target})")
rows = []
for i, task in enumerate(selected, 1):
print(f"[{i}/{len(selected)}] {task.id} … ", end="", flush=True)
r = run_task(args, task)
print(f"{r['result']} ({r['secs']}s)")
rows.append(r)
try:
for i, task in enumerate(selected, 1):
print(f"[{i}/{len(selected)}] {task.id} … ", end="", flush=True)
r = run_task(args, task)
print(f"{r['result']} ({r['secs']}s)")
rows.append(r)
finally:
# Leave the input box empty so the next action (e.g. an agent restart)
# isn't corrupted by a lingering prompt.
clear_input(args.target)
print_table(rows)