feat(ai): in-RAM semantic recall (RAG) for conversation context

Give the agent recall of things said beyond the verbatim window, without
breaking the RAM-only philosophy — nothing is persisted to disk.

- MemoryIndex: a capped, in-memory pool of embedded messages with pure-Python
  cosine search (no numpy). Retains far more than the rolling transcript so old
  lines can be surfaced on demand; oldest evicted past the cap to bound RAM.
- OllamaEmbedder: local embeddings via nomic-embed-text, on by default and
  independent of the chat provider (reuses the Ollama host when chat is Ollama).
- Bridge: captured room messages (live + backfilled) are embedded on a
  background worker so a slow embedder can't stall frame draining. On a /ai
  question the agent retrieves top-k relevant lines, drops weak (<min_score) and
  windowed-duplicate hits, and prepends them as a clearly-fenced "recalled
  context" preamble — kept at user role, never elevated to system, so untrusted
  room text informs without instructing. Falls back to recency-only if the
  embedder is unreachable.
- CLI: --no-rag, --embed-model, --embed-host, --rag-top-k.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
leetcrypt
2026-06-02 17:59:01 -07:00
parent 85fde59292
commit e5e1ad8dee
5 changed files with 236 additions and 51 deletions
+10 -6
View File
@@ -43,12 +43,16 @@ That is a RAM-only history the agent can backfill from on join at zero new cost.
8. **Tune Ollama `options`** — explicit `num_ctx` (so the larger window in #1/#2
is actually honored) and bounded `num_predict`. *(implementing)*
### Tier 2 — deeper context (next branch)
3. **In-RAM semantic retrieval (RAG, no disk).** Embed each captured message
with the already-present `nomic-embed-text`, hold vectors in a numpy array in
memory; on a `/ai` question retrieve top-k by cosine and prepend to the
recency window. Fully ephemeral.
4. **In-RAM hierarchical compaction.** When over budget, summarize the oldest
### Tier 2 — deeper context
3. **In-RAM semantic retrieval (RAG, no disk).** *(done)* Each captured message
is embedded with the already-present `nomic-embed-text` and held in a capped
in-memory `MemoryIndex` (pure-Python cosine, no numpy). On a `/ai` question
the agent embeds the query, retrieves top-k, drops weak/duplicate hits, and
prepends them as a clearly-fenced "recalled context" preamble (never system
role — keeps untrusted text from instructing). Embedding runs on a background
worker so it can't stall the recv loop; if the embedder is unreachable it
degrades to recency-only. Toggle with `--no-rag` / `--rag-top-k`.
4. **In-RAM hierarchical compaction.** *(staged)* When over budget, summarize the oldest
chunk into a single rolling `Msg("system", "earlier: …")` instead of dropping
it — the Claude Code auto-compaction pattern, kept in RAM.