Phase 4 (final) of the native-harness working-memory sprint. OllamaProvider
.complete_with_tools now returns (text, calls, usage), surfacing the response's
real prompt_eval_count/eval_count (free — already in the payload). The native loop
EMA-smooths real/estimate into self._tok_ratio (clamped [0.5,3.0]) and prunes
against native_token_budget / ratio, so context budgeting tracks the TRUE window
instead of the systematic bias of the len//4 char estimate. Providers that omit
counts leave the ratio at 1.0, so behaviour is unchanged where unavailable — a
free correctness win, no regression. RAM-only, no disk, no new network frames.
Scope note: touches only cmd_chat/agent/ + providers — disjoint from the parallel
feat(operator) work on this branch, so it merges/reverts independently by path.
Sprint: native-harness-working-memory (Phase 4/4) — see also 46e5620, dc6317f
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The first harness changes to move the benchmark off the 1/12 noise floor, on
the model that needs it most (qwen2.5:0.5b: 1/12 -> 4/12, 3/12).
- complete_with_tools now decodes the tool loop at temperature 0 (scoped; chat
keeps default sampling). At Ollama's default 0.8 the weak model sampled away
from the tool-call format into prose/fabrication; the nudge prompt changes
between turns so temp 0 still escapes a failed state on retry.
- Greedy decode made 0.5b's leak deterministic, exposing its real shape: not a
JSON object with a name key, but the name in a <tools> tag and the args in a
SEPARATE object — <tools>write_file</tools>{"path":…} — ~5 of 12 tasks/run.
_NAMED_TAG pairs the tag-name with the following args object, gated on the
known tool set so it still can't fabricate an action.
- Bridge recovers a ```bash block narrated in prose as a run_shell call,
non-destructive only (FENCE_DESTRUCTIVE guard); fires on prose-leak turns,
no-op where the model emits structured calls.
Ablation on 0.5b: structured-JSON-only 0/0 -> fenced+temp0 2/0 -> +split-tag
4/3. The lift is concentrated on the weakest model by design — a 3B emits
proper calls and fails on capability/content (unchanged at 1/12), which no
parser can fix. All recovery paths unit-checked for the positive shapes and
the negatives (prose / unknown tool / destructive block) they must ignore.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Generalise OllamaProvider._extract_text_tool_calls to recover a tool-call
JSON object regardless of how a small/quantized model wraps it — qwen's
<tool_call> tags, bare JSON, ```json fences, alternate tags (<tools>,
<function_call>), OpenAI {"function":{…}} nesting, and parameters-vs-arguments.
A new _coerce_call gates recovery on the known tool-name set from the tools
schema, so a stray JSON blob in prose (or a hallucinated make_dir) can never
be coerced into an action. 11-case unit check: 8 leak shapes recover, 3
negatives (prose / unknown tool / random config JSON) ignored.
Benchmark verdict (honest): this does NOT move the weak-CPU-model pass rate
— 3b went 2/1/0 of 12 across three passes (baseline 1/12, noise), 0.5b went
0/0 (baseline 1/12). A direct /api/chat probe shows the hypothesis was wrong
about the FORM of the leak: the weak models emit either malformed structured
tool_calls (write_file content:null) or a fenced bash block in prose with no
tool call at all — not JSON-as-text. The structured-JSON recovery is still a
correct, safe hardening for any model that does leak JSON; the real
weak-model lever (parse ```bash fences -> run_shell) is documented as an
explicit safety decision, not folded in here.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Optimize the native tool-calling loop for qwen2.5:3b on CPU, where it
previously invented paths (/ai/bin/bash), ran scripts it never wrote, and
silently dropped valid actions. Three changes:
- NATIVE_SYSTEM rewritten directive: explicit write→chmod→run workflow,
relative paths only, never run an uncreated file, never guess interpreter
paths, fix the cause on non-zero exit.
- New _sandbox_facts() probe injects LIVE SANDBOX STATE (real cwd, bash
path, current files) into the system prompt so the model anchors to
ground truth instead of guessing.
- OllamaProvider recovers tool calls qwen emits as <tool_call>{json}</…>
TEXT in content (brace-balanced JSON scan), so a correct action isn't lost.
- Bump Ollama timeout 120→240s: the tool turn is non-streaming and a long
write_file can exceed a tighter cap on a contended CPU box.
Live-validated (podman/Kali): 0/3 incoherent → reliable write/run with
self-correction on exit=126 for both single- and multi-script tasks.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Implement the bounded native harness from docs/spec-native-harness.md §1.3 and
make it the default granted-!task path. The model runs host-side (no container→
host Ollama hop); only its tool calls exec in the sandbox.
providers.py:
- OllamaProvider.complete_with_tools(system, messages, tools) -> (text, calls):
one non-streaming /api/chat turn with a `tools` schema; parses message.tool_calls
(dict or JSON-string arguments). Caches tool capability (_tools_ok / supports_tools).
- ToolsUnsupported raised when the model rejects `tools` ("does not support tools").
bridge.py:
- NATIVE_SYSTEM + a 3-tool schema (run_shell / write_file / read_file), turn/byte caps.
- _run_native: seed transcript window + task → loop up to max_turns; exec each tool
call in the sandbox, feed captured output back as a `tool` message; stop on a plain
answer or the cap; stream per-call progress to chat. Degrades to _run_simple when the
provider has no complete_with_tools or the model rejects tools.
- _exec_prefix/_exec_capture/_exec_tool: <engine> exec into docker/podman/multipass/local;
paths passed as positional args + content via stdin (no shell interpolation); combined
stdout+stderr byte-capped + time-bounded. run_shell is the only intentional shell.
- Guards: DESTRUCTIVE run_shell commands are blocked (not run — no human in the loop;
use simple + /ai confirm for destructive intent); MAX_COMMANDS budget per task.
- _run_in_sandbox dispatches native|simple; default harness flipped to native.
__main__.py: default harness native (self-degrades to simple, so safe).
Offline-tested: full write/run/read loop on the local backend; destructive block
(rm -rf never executed); ToolsUnsupported → simple fallback. Live Ollama wire
validation deferred to Phase 3 bench (daemon was down). py_compile clean.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- /sbx save [--local] and new /sbx vmsave/vmsnaps: snapshot Docker,
Multipass, and VirtualBox; --local also writes a portable artifact
(docker .tar / VBox .ova) under hh-snapshots/ that survives pruning.
- sandbox reappears for anyone who leaves and rejoins (host replays
status + screen snapshot + ACL on Joined); SbxStatus ready handler is
now idempotent so it never wipes scrollback.
- received files auto-bridge into the hosted sandbox (ft::tar_path).
- AI agent: translate Ollama's cryptic 404 into model/host/fix guidance.
- bootstrap installs the AI layer (Ollama + default model) by default,
with consent gates; --no-ai opts out, --yes skips prompts.
- help menu lists the new save/vm flags.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Tier A/B/C wins for the CPU-only Ollama box (no GPU → optimize TTFT and
tokens/sec, not VRAM):
- Separate qwen2.5-coder provider for the sandbox `!task` path; chat keeps
the general model. Auto-selected when chat is Ollama and a coder build is
present, override with --code-model.
- OllamaProvider num_ctx default 8192→4096 (8192 was a GPU-mindset default
that inflates prefill/TTFT on CPU); expose num_thread; add --num-ctx,
--num-thread, --num-predict. token_budget default 3000→2000 to fit.
- OllamaProvider.stream() generator over Ollama's stream=True chat endpoint
(provider half of token streaming; agent/Rust rendering is a follow-up).
- Few-shot request→shell exemplars in SANDBOX_SYSTEM to anchor the small
model's fenced-command output.
- Matryoshka embedding truncation: OllamaEmbedder truncate_dim=256 (--embed-dim)
for faster pure-Python cosine and less RAM; query+stored share the dim.
- docs/ai-perf-plan.md records all 8 items with status and the server-side
env (OLLAMA_NUM_PARALLEL=1, keep_alive) that must be set where ollama serve runs.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Give the agent recall of things said beyond the verbatim window, without
breaking the RAM-only philosophy — nothing is persisted to disk.
- MemoryIndex: a capped, in-memory pool of embedded messages with pure-Python
cosine search (no numpy). Retains far more than the rolling transcript so old
lines can be surfaced on demand; oldest evicted past the cap to bound RAM.
- OllamaEmbedder: local embeddings via nomic-embed-text, on by default and
independent of the chat provider (reuses the Ollama host when chat is Ollama).
- Bridge: captured room messages (live + backfilled) are embedded on a
background worker so a slow embedder can't stall frame draining. On a /ai
question the agent retrieves top-k relevant lines, drops weak (<min_score) and
windowed-duplicate hits, and prepends them as a clearly-fenced "recalled
context" preamble — kept at user role, never elevated to system, so untrusted
room text informs without instructing. Falls back to recency-only if the
embedder is unreachable.
- CLI: --no-rag, --embed-model, --embed-host, --rag-top-k.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
OllamaProvider now sends keep_alive (default 30m) so the model stays resident
in VRAM between /ai calls instead of cold-reloading, and sets explicit options
(num_ctx 8192, num_predict 512) — Ollama otherwise caps context at 2048, which
would silently truncate the larger backfilled window.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Make connecting any model a config step, not a code change:
- models.toml named profiles (api_key_env names an env var, never the key)
- providers gain available_models(); add preflight + --list-models/--check
- /ai list and /ai models in-room; client probes local Ollama for
/ai models when no agent is running, and /ai list hints to summon one
- docs/providers.md provider guide + examples/echo_provider.py
- README: command table, AI section, layout updated
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add cmd_chat/agent: a headless client that joins a room via SRP, decrypts
broadcasts, and answers /ai <question> through a pluggable model provider
(ollama default + anthropic + openai-compatible + module:Class). Server and
zero-knowledge guarantees unchanged; the agent is just another encrypted client.
Also pin the lets-hack demo to a detached worktree of main (default) so running
it from dev still demos stable main without touching the working checkout.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
leetcrypt