Optimize the native tool-calling loop for qwen2.5:3b on CPU, where it
previously invented paths (/ai/bin/bash), ran scripts it never wrote, and
silently dropped valid actions. Three changes:
- NATIVE_SYSTEM rewritten directive: explicit write→chmod→run workflow,
relative paths only, never run an uncreated file, never guess interpreter
paths, fix the cause on non-zero exit.
- New _sandbox_facts() probe injects LIVE SANDBOX STATE (real cwd, bash
path, current files) into the system prompt so the model anchors to
ground truth instead of guessing.
- OllamaProvider recovers tool calls qwen emits as <tool_call>{json}</…>
TEXT in content (brace-balanced JSON scan), so a correct action isn't lost.
- Bump Ollama timeout 120→240s: the tool turn is non-streaming and a long
write_file can exceed a tighter cap on a contended CPU box.
Live-validated (podman/Kali): 0/3 incoherent → reliable write/run with
self-correction on exit=126 for both single- and multi-script tasks.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Implement the bounded native harness from docs/spec-native-harness.md §1.3 and
make it the default granted-!task path. The model runs host-side (no container→
host Ollama hop); only its tool calls exec in the sandbox.
providers.py:
- OllamaProvider.complete_with_tools(system, messages, tools) -> (text, calls):
one non-streaming /api/chat turn with a `tools` schema; parses message.tool_calls
(dict or JSON-string arguments). Caches tool capability (_tools_ok / supports_tools).
- ToolsUnsupported raised when the model rejects `tools` ("does not support tools").
bridge.py:
- NATIVE_SYSTEM + a 3-tool schema (run_shell / write_file / read_file), turn/byte caps.
- _run_native: seed transcript window + task → loop up to max_turns; exec each tool
call in the sandbox, feed captured output back as a `tool` message; stop on a plain
answer or the cap; stream per-call progress to chat. Degrades to _run_simple when the
provider has no complete_with_tools or the model rejects tools.
- _exec_prefix/_exec_capture/_exec_tool: <engine> exec into docker/podman/multipass/local;
paths passed as positional args + content via stdin (no shell interpolation); combined
stdout+stderr byte-capped + time-bounded. run_shell is the only intentional shell.
- Guards: DESTRUCTIVE run_shell commands are blocked (not run — no human in the loop;
use simple + /ai confirm for destructive intent); MAX_COMMANDS budget per task.
- _run_in_sandbox dispatches native|simple; default harness flipped to native.
__main__.py: default harness native (self-degrades to simple, so safe).
Offline-tested: full write/run/read loop on the local backend; destructive block
(rm -rf never executed); ToolsUnsupported → simple fallback. Live Ollama wire
validation deferred to Phase 3 bench (daemon was down). py_compile clean.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- /sbx save [--local] and new /sbx vmsave/vmsnaps: snapshot Docker,
Multipass, and VirtualBox; --local also writes a portable artifact
(docker .tar / VBox .ova) under hh-snapshots/ that survives pruning.
- sandbox reappears for anyone who leaves and rejoins (host replays
status + screen snapshot + ACL on Joined); SbxStatus ready handler is
now idempotent so it never wipes scrollback.
- received files auto-bridge into the hosted sandbox (ft::tar_path).
- AI agent: translate Ollama's cryptic 404 into model/host/fix guidance.
- bootstrap installs the AI layer (Ollama + default model) by default,
with consent gates; --no-ai opts out, --yes skips prompts.
- help menu lists the new save/vm flags.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Tier A/B/C wins for the CPU-only Ollama box (no GPU → optimize TTFT and
tokens/sec, not VRAM):
- Separate qwen2.5-coder provider for the sandbox `!task` path; chat keeps
the general model. Auto-selected when chat is Ollama and a coder build is
present, override with --code-model.
- OllamaProvider num_ctx default 8192→4096 (8192 was a GPU-mindset default
that inflates prefill/TTFT on CPU); expose num_thread; add --num-ctx,
--num-thread, --num-predict. token_budget default 3000→2000 to fit.
- OllamaProvider.stream() generator over Ollama's stream=True chat endpoint
(provider half of token streaming; agent/Rust rendering is a follow-up).
- Few-shot request→shell exemplars in SANDBOX_SYSTEM to anchor the small
model's fenced-command output.
- Matryoshka embedding truncation: OllamaEmbedder truncate_dim=256 (--embed-dim)
for faster pure-Python cosine and less RAM; query+stored share the dim.
- docs/ai-perf-plan.md records all 8 items with status and the server-side
env (OLLAMA_NUM_PARALLEL=1, keep_alive) that must be set where ollama serve runs.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Give the agent recall of things said beyond the verbatim window, without
breaking the RAM-only philosophy — nothing is persisted to disk.
- MemoryIndex: a capped, in-memory pool of embedded messages with pure-Python
cosine search (no numpy). Retains far more than the rolling transcript so old
lines can be surfaced on demand; oldest evicted past the cap to bound RAM.
- OllamaEmbedder: local embeddings via nomic-embed-text, on by default and
independent of the chat provider (reuses the Ollama host when chat is Ollama).
- Bridge: captured room messages (live + backfilled) are embedded on a
background worker so a slow embedder can't stall frame draining. On a /ai
question the agent retrieves top-k relevant lines, drops weak (<min_score) and
windowed-duplicate hits, and prepends them as a clearly-fenced "recalled
context" preamble — kept at user role, never elevated to system, so untrusted
room text informs without instructing. Falls back to recency-only if the
embedder is unreachable.
- CLI: --no-rag, --embed-model, --embed-host, --rag-top-k.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
OllamaProvider now sends keep_alive (default 30m) so the model stays resident
in VRAM between /ai calls instead of cold-reloading, and sets explicit options
(num_ctx 8192, num_predict 512) — Ollama otherwise caps context at 2048, which
would silently truncate the larger backfilled window.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Make connecting any model a config step, not a code change:
- models.toml named profiles (api_key_env names an env var, never the key)
- providers gain available_models(); add preflight + --list-models/--check
- /ai list and /ai models in-room; client probes local Ollama for
/ai models when no agent is running, and /ai list hints to summon one
- docs/providers.md provider guide + examples/echo_provider.py
- README: command table, AI section, layout updated
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add cmd_chat/agent: a headless client that joins a room via SRP, decrypts
broadcasts, and answers /ai <question> through a pluggable model provider
(ollama default + anthropic + openai-compatible + module:Class). Server and
zero-knowledge guarantees unchanged; the agent is just another encrypted client.
Also pin the lets-hack demo to a detached worktree of main (default) so running
it from dev still demos stable main without touching the working checkout.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
leetcrypt