feat(ai): in-RAM semantic recall (RAG) for conversation context
Give the agent recall of things said beyond the verbatim window, without breaking the RAM-only philosophy — nothing is persisted to disk. - MemoryIndex: a capped, in-memory pool of embedded messages with pure-Python cosine search (no numpy). Retains far more than the rolling transcript so old lines can be surfaced on demand; oldest evicted past the cap to bound RAM. - OllamaEmbedder: local embeddings via nomic-embed-text, on by default and independent of the chat provider (reuses the Ollama host when chat is Ollama). - Bridge: captured room messages (live + backfilled) are embedded on a background worker so a slow embedder can't stall frame draining. On a /ai question the agent retrieves top-k relevant lines, drops weak (<min_score) and windowed-duplicate hits, and prepends them as a clearly-fenced "recalled context" preamble — kept at user role, never elevated to system, so untrusted room text informs without instructing. Falls back to recency-only if the embedder is unreachable. - CLI: --no-rag, --embed-model, --embed-host, --rag-top-k. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
@@ -32,7 +32,7 @@ import sys
|
||||
|
||||
from .bridge import AgentBridge
|
||||
from .profiles import load_profiles, provider_from_profile
|
||||
from .providers import make_provider, preflight
|
||||
from .providers import OllamaEmbedder, make_provider, preflight
|
||||
|
||||
|
||||
def _build_provider(args, ap):
|
||||
@@ -80,6 +80,14 @@ def main() -> None:
|
||||
help="max prior messages fed to the model per reply")
|
||||
ap.add_argument("--token-budget", type=int, default=3000,
|
||||
help="approx token cap on the context window (whichever is smaller wins)")
|
||||
ap.add_argument("--no-rag", action="store_true",
|
||||
help="disable in-RAM semantic recall (recency-only context)")
|
||||
ap.add_argument("--embed-model", default="nomic-embed-text",
|
||||
help="Ollama model used to embed messages for recall")
|
||||
ap.add_argument("--embed-host", default=None,
|
||||
help="Ollama host for embeddings (default: chat host or $OLLAMA_HOST)")
|
||||
ap.add_argument("--rag-top-k", type=int, default=4,
|
||||
help="how many recalled messages to surface per reply")
|
||||
ap.add_argument("--list-models", action="store_true",
|
||||
help="list models the backend can serve, then exit")
|
||||
ap.add_argument("--check", action="store_true",
|
||||
@@ -112,11 +120,21 @@ def main() -> None:
|
||||
if not ok:
|
||||
print(f"⚠ preflight: {msg}", file=sys.stderr)
|
||||
|
||||
# In-RAM semantic recall is on by default and local (Ollama embeddings),
|
||||
# independent of which provider answers chat. Reuse the chat host if it's an
|
||||
# Ollama provider so a single --host/profile covers both.
|
||||
embedder = None
|
||||
if not args.no_rag:
|
||||
embedder = OllamaEmbedder(
|
||||
model=args.embed_model,
|
||||
host=args.embed_host or getattr(provider, "host", None),
|
||||
)
|
||||
|
||||
bridge = AgentBridge(
|
||||
args.server, args.port, name=args.name, provider=provider,
|
||||
password=args.password, insecure=args.insecure, no_tls=args.no_tls,
|
||||
system_prompt=args.system, context_window=args.context_window,
|
||||
token_budget=args.token_budget,
|
||||
token_budget=args.token_budget, embedder=embedder, rag_top_k=args.rag_top_k,
|
||||
)
|
||||
try:
|
||||
bridge.run()
|
||||
|
||||
Reference in New Issue
Block a user