Context manager

This commit is contained in:
K. Hodges 2026-06-05 01:54:01 -07:00
parent 3efb320f03
commit 37dca8954d
9 changed files with 1859 additions and 117 deletions

2
Cargo.lock generated
View File

@ -129,7 +129,7 @@ dependencies = [
[[package]] [[package]]
name = "exoshell" name = "exoshell"
version = "0.1.0" version = "0.2.0"
dependencies = [ dependencies = [
"async-trait", "async-trait",
"futures-util", "futures-util",

View File

@ -1,6 +1,6 @@
[package] [package]
name = "exoshell" name = "exoshell"
version = "0.1.0" version = "0.2.0"
edition = "2024" edition = "2024"
license = "GPL-3.0-or-later" license = "GPL-3.0-or-later"

View File

@ -26,7 +26,11 @@ Phase 1 is closed.
The current implementation supports the first shell-adjacent model chat milestone: a Rust CLI, OpenAI-compatible provider abstraction, PowerShell/POSIX shell-family selection, command-suggestion formatting, markdown transcripts, and a basic interactive REPL. The current implementation supports the first shell-adjacent model chat milestone: a Rust CLI, OpenAI-compatible provider abstraction, PowerShell/POSIX shell-family selection, command-suggestion formatting, markdown transcripts, and a basic interactive REPL.
The roadmap is tracked in [docs/PHASES.md](docs/PHASES.md). Phase 2 moves toward explicit context, modes, hotkeys, and stronger operator controls. The codebase also contains the initial Phase 1.5 context engine foundation: context entries, provenance metadata, priority and size estimates, a session context store, provider registry, default manual/file/command-output/directory-summary providers, deterministic pruning, and prompt-context rendering. REPL context commands are still planned.
The active milestone is Phase 1.5, the explicit context engine foundation tracked in [docs/tasks/phase15_context_tasks.md](docs/tasks/phase15_context_tasks.md). Phase 2 builds on that context engine with stances, safer command handling, hotkeys, and stronger operator controls.
The broader roadmap is tracked in [docs/PHASES.md](docs/PHASES.md) and [docs/tasks](docs/tasks).
## Running Locally ## Running Locally

View File

@ -432,6 +432,160 @@ Memory scopes may include:
* session * session
* ephemeral * ephemeral
---
## Context Philosophy
Context is a first-class system within Exoshell.
Most AI systems accumulate context implicitly through hidden memory, opaque retrieval systems, session state, telemetry, or behavioral profiling.
Exoshell rejects this model.
Context should be explicit.
The operator should always be able to answer:
* What information is being sent to the model?
* Where did it come from?
* Why is it present?
* How large is it?
* How can I remove it?
Context is not hidden machinery.
Context is instrumentation.
### Context As Cargo
Exoshell treats context as cargo carried into a conversation.
The operator chooses what to bring.
Examples include:
* files
* command output
* logs
* notes
* repository summaries
* notebook entries
* search results
* git state
Context should never silently appear.
The operator should deliberately attach information when it becomes useful.
### Inspectability
All active context should be visible.
The user should be able to inspect:
* source
* size
* type
* age
* priority
* inclusion status
before a request is sent to a model.
The system should make context visible enough that an operator can reason about it the same way they reason about shell pipelines.
### Provenance
Every context entry should retain provenance.
The operator should be able to determine:
* where context originated
* when it was added
* how it entered the session
Examples:
* manually pasted
* loaded from a file
* generated from git
* generated from search
* imported from a notebook
Context without provenance is difficult to trust.
### Explicit Inclusion
Possessing context and sending context are separate concepts.
A context entry may exist within a session without being included in model requests.
Operators should be able to:
* enable context
* disable context
* pin context
* prioritize context
* remove context
without destroying underlying information.
### Context Budgets
Context is a finite resource.
Exoshell should expose:
* approximate token usage
* approximate character usage
* pruning decisions
* compression decisions
Users should never be surprised by context loss.
If context must be reduced, the system should explain:
* what was removed
* why it was removed
* what remains
### Context Is Not Memory
Context and memory serve different purposes.
Context is active operational state.
Memory is retained knowledge.
Context is temporary, task-oriented, and immediately relevant.
Memory is persistent, searchable, and intentionally preserved.
Exoshell should maintain a clear boundary between the two.
### Operator Ownership
The operator owns context.
Not the model.
Not the application.
Not a hidden retrieval system.
Context should remain:
* visible
* editable
* removable
* exportable
* inspectable
at all times.
A practitioner should never need faith to understand what information Exoshell is using.
The system should make context obvious enough that trust emerges naturally from visibility.
Logging Philosophy Logging Philosophy

View File

@ -1,161 +1,465 @@
# Exoshell Phases # Roadmap
This roadmap turns the project vision into concrete build phases. The phases are intentionally ordered from the smallest useful shell-adjacent assistant toward the broader cognitive shell described in `README.md` and `docs/DESIGN.md`. Detailed milestone task lists live in `docs/tasks/`.
## Phase 1: Shell-Adjacent Model Chat Current ordering:
Goal: ship a usable terminal program that lets a user talk to a model from PowerShell, bash, zsh, and similar shells without replacing the shell. * Phase 1 is closed.
* Phase 1.5 establishes the explicit context engine foundation before Phase 2.
* Phase 2 integrates the context engine with stances, safer command handling, and operator controls.
* Phase 3 adds project awareness and operational memory on top of visible, user-controlled context.
Core features: # Phase 3 Tasks
- CLI entry point, likely `exoshell`, that runs as a normal terminal application. Phase 3 goal: provide useful project awareness and operational memory while keeping all context visible, inspectable, and user-controlled.
- Provider configuration for at least one OpenAI-compatible backend.
- Local-first-compatible model adapter interface, even if the first implementation targets one backend.
- Interactive chat loop with streaming responses.
- Basic command suggestion formatting that distinguishes prose, shell commands, and warnings.
- Explicit copy/paste-oriented workflow: Exoshell suggests commands, the user remains responsible for execution.
- Cross-platform terminal behavior on Windows, macOS, and Linux.
- Shell-family awareness for PowerShell versus POSIX-like shells.
- Basic session transcript saved as markdown.
- Minimal config file for provider, model, default shell family, and notebook location.
- Clear error handling for missing API keys, provider failures, and interrupted requests.
User experience: ## P3-001: Project Root Detection
- User should be able to run "exo this is a prompt" and get a response, the program exiting
- User should be able to run exo (without a prompt) and have an open session, or "exo this is a prompt --stay" (or a more appropriate flag) to keep the shell/conversation open
- User should be able to pipe (bash) or powershell equivalent into exo and have it treated as a prompt, with the additional context of the command run that sent the pipe, cwd, etc.
- For example, "ls | exo" should send a prompt similar to "{"command": "ls", "stdout": "foo bar baz", "cwd": "/foo/bar"}" and any relevant system prompts
Non-goals:
- PTY control of the user's shell. Status: planned.
- Hotkeys that inject commands into the shell.
- Ambient context collection.
- Tree-sitter repo indexing.
- Long-term memory.
- Stances and personalities beyond a fixed default behavior.
- Autonomous command execution.
Exit criteria: Outcome: Exoshell can identify and operate against a project root.
- A user can install/run Exoshell, configure a model backend, ask for help from a terminal, receive command suggestions appropriate to PowerShell or bash/zsh, and save the session transcript. Acceptance criteria:
- The implementation has a small automated test suite for provider abstraction, shell-family prompt behavior, config loading, and transcript writing.
## Phase 2: Context, Modes, and Operator Controls * Detect Git repositories.
* Detect project roots using common markers.
* User can override detected root.
* Current project root is visible.
* Tests cover nested repositories and overrides.
Goal: make Exoshell feel like an operational overlay instead of a generic chat client. ## P3-002: Project Context Model
Core features: Status: planned.
- Explicit context commands for adding files, command output, directory summaries, and pasted logs. Outcome: Exoshell has a structured representation of project information.
- Session-scoped context panel or context summary visible in the TUI.
- Stances such as `operator`, `audit`, `teach`, and `quiet`.
- Hotkeys for accepting, copying, explaining, and discarding suggested commands.
- Safer command suggestion UI with destructive-command detection and confirmation language.
- Context budget management with visible token or character estimates.
- Shell-specific command rendering and explanation for PowerShell and POSIX-like shells.
- Notebook improvements: title, timestamps, model metadata, commands suggested, and user-marked discoveries.
- Basic configuration profiles for different providers and shells.
Non-goals: Acceptance criteria:
- Passive screen watching. * Project metadata includes root path, repository type, language hints, and discovery timestamps.
- Full PTY embedding. * Metadata is serializable.
- Repository-wide semantic indexing. * Metadata can be displayed in the UI.
- Long-term memory outside user-controlled markdown/log files. * Tests cover serialization and loading.
Exit criteria: ## P3-003: Repository Summary Generation
- A user can deliberately attach context, switch stance, inspect what context is being sent, and act on suggestions through predictable terminal controls. Status: planned.
## Phase 3: Repo Awareness and Operational Notebooks Outcome: Exoshell can generate a high-level summary of a repository.
Goal: give Exoshell useful project awareness while keeping memory inspectable and scoped. Acceptance criteria:
Core features: * Summary includes major directories.
* Summary includes language breakdown.
* Summary identifies likely entry points.
* Summary avoids reading entire repositories by default.
* Tests cover large repositories.
- Repo detection and project-root selection. ## P3-004: Repository Ignore Rules
- File tree summaries with ignore rules.
- Structural parsing for common languages using tree-sitter where practical.
- Commands to add source files, symbols, recent git changes, and test output to context.
- Markdown notebooks scoped by global, repo, task, and session.
- Searchable operational notes and discoveries.
- Runbook generation from session notes.
- Diff-oriented patch suggestions that require user review.
- Better uncertainty reporting through explicit signal strength.
Non-goals: Status: planned.
- Hidden long-term behavioral profiling. Outcome: repository scanning remains efficient.
- Blind patch application.
- Autonomous multi-file rewrites.
Exit criteria: Acceptance criteria:
- A user can open a repo, ask informed questions about local code and recent command output, preserve useful notes, and generate reviewable runbooks or patch suggestions. * Honors .gitignore when practical.
* Supports Exoshell-specific ignore rules.
* Skips common build artifacts.
* Ignore behavior is configurable.
* Tests cover ignore matching.
## Phase 4: Follow Mode and Shell Integration ## P3-005: File Inventory Builder
Goal: let Exoshell observe explicit shell activity and provide contextual help without becoming a replacement shell. Status: planned.
Core features: Outcome: users can inspect repository contents.
- PTY or shell-hook integration strategy for supported environments. Acceptance criteria:
- Follow mode that can ingest selected command history, stdout/stderr snippets, current directory, and exit status.
- User-visible controls for what is observed and retained.
- Hotkey workflows for sending command output to Exoshell.
- Optional paste-to-shell and paste-and-run flows with clear confirmation boundaries.
- Command outcome interpretation and next-step suggestions.
- Per-shell integration docs for PowerShell, bash, zsh, and fish.
Non-goals: * Generates file inventories.
* Supports filtering by extension.
* Supports filtering by path.
* Supports size limits.
* Tests cover large inventories.
- Covert monitoring. ## P3-006: Symbol Discovery Framework
- Unreviewed execution.
- Replacing the user's shell configuration.
Exit criteria: Status: planned.
- A user can work in their normal shell while Exoshell follows explicitly permitted context and offers useful, nonintrusive operational guidance. Outcome: Exoshell can discover likely symbols without full semantic indexing.
## Phase 5: Tuning, Extensibility, and Local-First Depth Acceptance criteria:
Goal: make Exoshell configurable, hackable, and effective with weaker local models. * Extracts functions, structs, classes, interfaces, and modules where practical.
* Stores symbol metadata.
* Supports lookup by name.
* Supports language-specific extractors.
* Tests cover supported languages.
Core features: ## P3-007: Language Detector
- Robust local model support through OpenAI-compatible local servers and/or dedicated adapters. Status: planned.
- Prompt and stance customization.
- User-editable command policies and safety rules.
- Extension points for tools, context providers, and notebook processors.
- Model routing by task type.
- Context compression and retrieval tuned for smaller models.
- Export/import of user configuration and notebooks.
- Advanced visual themes that remain restrained and terminal-native.
Non-goals: Outcome: Exoshell understands repository language composition.
- Cloud lock-in. Acceptance criteria:
- Manager analytics.
- Personality behavior that compromises operational clarity.
Exit criteria: * Detects primary languages.
* Detects mixed-language repositories.
* Handles generated code separately.
* Displays language summary.
* Tests cover representative repositories.
- A technical user can tune Exoshell to their environment, run local-first workflows, and extend context/tool behavior without modifying core code. ## P3-008: Git Status Context Provider
## Phase 6: Mature Cognitive Shell Overlay Status: planned.
Goal: converge on the broader vision: a calm, inspectable, practitioner-oriented cognitive shell environment. Outcome: users can add current Git state as context.
Core features: Acceptance criteria:
- Mature cockpit-style TUI with dense but readable operational instrumentation. * Captures branch.
- Stable shell integrations across major platforms. * Captures modified files.
- High-quality repo, session, task, and notebook workflows. * Captures staged files.
- Reviewable command, patch, and runbook pipelines. * Captures untracked files.
- Strong safety posture around destructive commands and hidden state. * Tests cover detached HEAD and clean repositories.
- Comprehensive docs for installation, configuration, shell integration, model backends, stances, notebooks, and local-first operation.
- Packaging and release automation.
Exit criteria: ## P3-009: Recent Commit Context Provider
- Exoshell is a dependable daily-driver assistant for terminal-native practitioners who want AI augmentation while preserving control and understanding. Status: planned.
Outcome: users can add recent history to context.
Acceptance criteria:
* Supports configurable commit count.
* Includes author, timestamp, message, and changed files.
* Supports filtering by author.
* Supports filtering by path.
* Tests cover repositories with no commits.
## P3-010: Diff Context Provider
Status: planned.
Outcome: users can attach diffs to prompts.
Acceptance criteria:
* Supports staged diffs.
* Supports unstaged diffs.
* Supports specific files.
* Large diffs are truncated visibly.
* Tests cover truncation behavior.
## P3-011: Search Provider Framework
Status: planned.
Outcome: Exoshell can search repositories consistently.
Acceptance criteria:
* Search abstraction exists.
* Supports text search.
* Supports path search.
* Supports symbol search.
* Tests cover provider behavior.
## P3-012: Ripgrep Integration
Status: planned.
Outcome: repository search is fast and useful.
Acceptance criteria:
* Uses ripgrep when available.
* Provides fallback behavior.
* Search results include file and line information.
* Results are attachable as context.
* Tests cover common searches.
## P3-013: Source File Context Commands
Status: planned.
Outcome: users can add code directly to context.
Acceptance criteria:
* Supports adding files.
* Supports adding line ranges.
* Supports adding symbols.
* Supports adding search results.
* Tests cover invalid ranges.
## P3-014: Context Compression Pipeline
Status: planned.
Outcome: large repositories remain usable.
Acceptance criteria:
* Large context can be summarized.
* Compression preserves source references.
* Compression is visible to the user.
* Original context remains inspectable.
* Tests cover compression logic.
## P3-015: Global Notebook Support
Status: planned.
Outcome: users can maintain global notes.
Acceptance criteria:
* Global notebook exists outside repositories.
* Notes are markdown.
* Notes are searchable.
* Notes are user-editable.
* Tests cover creation and loading.
## P3-016: Repository Notebook Support
Status: planned.
Outcome: each repository can maintain its own notebook.
Acceptance criteria:
* Notebook is scoped to repository.
* Notebook persists between sessions.
* Notebook supports markdown.
* Notebook location is configurable.
* Tests cover notebook loading.
## P3-017: Task Notebook Support
Status: planned.
Outcome: users can organize work around tasks.
Acceptance criteria:
* Users can create tasks.
* Tasks have notes.
* Tasks can link context entries.
* Tasks can be completed or archived.
* Tests cover task lifecycle.
## P3-018: Notebook Search
Status: planned.
Outcome: stored discoveries remain useful.
Acceptance criteria:
* Supports keyword search.
* Supports filtering by notebook type.
* Results include source references.
* Results are attachable to prompts.
* Tests cover notebook search.
## P3-019: Discovery Linking
Status: planned.
Outcome: findings become navigable knowledge.
Acceptance criteria:
* Discoveries can link files.
* Discoveries can link symbols.
* Discoveries can link commits.
* Discoveries can link tasks.
* Tests cover link integrity.
## P3-020: Runbook Generation
Status: planned.
Outcome: sessions can produce operational documentation.
Acceptance criteria:
* Generates markdown runbooks.
* Includes commands, findings, and notes.
* Includes timestamps.
* Includes source references.
* Tests cover runbook generation.
## P3-021: Session Summarization
Status: planned.
Outcome: long sessions remain manageable.
Acceptance criteria:
* Session summaries are generated on demand.
* Summaries preserve important discoveries.
* Summaries include linked context.
* Summaries are written to notebooks.
* Tests cover summary generation.
## P3-022: Patch Suggestion Model
Status: planned.
Outcome: Exoshell can suggest code modifications.
Acceptance criteria:
* Suggestions are emitted as diffs.
* Suggestions never modify files automatically.
* Suggestions include rationale.
* Suggestions include uncertainty when appropriate.
* Tests cover patch formatting.
## P3-023: Diff Renderer
Status: planned.
Outcome: code changes are reviewable.
Acceptance criteria:
* Unified diff rendering.
* Syntax-aware formatting where practical.
* Clear additions and removals.
* Works without ANSI color.
* Tests cover rendering.
## P3-024: Patch Export
Status: planned.
Outcome: users can save suggested patches.
Acceptance criteria:
* Exports standard patch files.
* Exports markdown review files.
* Includes metadata.
* Does not modify repository state.
* Tests cover export behavior.
## P3-025: Evidence and Confidence Framework
Status: planned.
Outcome: Exoshell communicates uncertainty consistently.
Acceptance criteria:
* Responses distinguish evidence from inference.
* Confidence labels are documented.
* Confidence metadata can be rendered.
* Confidence survives transcript export.
* Tests cover confidence formatting.
## P3-026: Code Review Stance
Status: planned.
Outcome: Exoshell can behave like a skeptical reviewer.
Acceptance criteria:
* Focuses on correctness.
* Focuses on maintainability.
* Focuses on security.
* Prioritizes findings.
* Snapshot tests cover stance behavior.
## P3-027: Repository Dashboard
Status: planned.
Outcome: users can inspect project state quickly.
Acceptance criteria:
* Shows repository metadata.
* Shows notebook summary.
* Shows recent discoveries.
* Shows current task.
* Displays cleanly in terminal environments.
## P3-028: Tree-Sitter Foundation
Status: planned.
Outcome: semantic parsing foundation exists for supported languages.
Acceptance criteria:
* Tree-sitter integration is optional.
* Supports at least one language initially.
* Symbol extraction can use tree-sitter.
* Fallback path exists when unavailable.
* Tests cover parser loading.
## P3-029: Tree-Sitter Symbol Provider
Status: planned.
Outcome: symbol extraction becomes more accurate.
Acceptance criteria:
* Extracts functions.
* Extracts classes/structs.
* Extracts methods.
* Preserves source locations.
* Tests cover supported languages.
## P3-030: Phase 3 Documentation
Status: planned.
Outcome: users understand project-aware workflows.
Acceptance criteria:
* Documentation covers repositories.
* Documentation covers notebooks.
* Documentation covers patch suggestions.
* Documentation covers confidence reporting.
* Documentation covers limitations.
## P3-031: Phase 3 Test Coverage
Status: planned.
Outcome: repository-awareness features remain reliable.
Acceptance criteria:
* cargo test covers discovery, search, notebooks, patch generation, and confidence handling.
* Snapshot tests protect prompt assembly and patch formatting.
* Manual integration tests remain separate.
* Verification workflow is documented.
## P3-032: Phase 3 Manual Acceptance Test
Outcome: repository awareness milestone is validated.
Acceptance criteria:
* Open a repository.
* Generate repository summary.
* Add git diff context.
* Add recent commit context.
* Search for a symbol.
* Create a task notebook.
* Record discoveries.
* Generate a runbook.
* Request a patch suggestion.
* Export the patch.
* Verify no files are modified automatically.
* Verify all context remains inspectable.

View File

@ -100,3 +100,4 @@ Preserve existing codenames in project history.
Historical codenames should be tracked in docs/versioning.md below Historical codenames should be tracked in docs/versioning.md below
* 0.1.0 packet-kobold * 0.1.0 packet-kobold
* 0.2.0 context-relic

View File

@ -1,6 +1,9 @@
use std::path::PathBuf; use std::path::PathBuf;
use crate::config::Config; use crate::config::Config;
use crate::context::{
ContextProviderRegistry, SessionContextStore, register_default_context_providers,
};
use crate::prompts::phase1_system_prompt; use crate::prompts::phase1_system_prompt;
use crate::providers::{ChatMessage, ChatRequest, ChatResponse, ChatRole, Provider, ProviderError}; use crate::providers::{ChatMessage, ChatRequest, ChatResponse, ChatRole, Provider, ProviderError};
use crate::repl::ReplError; use crate::repl::ReplError;
@ -12,6 +15,8 @@ pub struct App {
provider: Box<dyn Provider>, provider: Box<dyn Provider>,
messages: Vec<ChatMessage>, messages: Vec<ChatMessage>,
transcript: Transcript, transcript: Transcript,
_context_store: SessionContextStore,
_context_registry: ContextProviderRegistry,
} }
impl App { impl App {
@ -25,12 +30,17 @@ impl App {
ChatRole::System, ChatRole::System,
phase1_system_prompt(config.shell.family), phase1_system_prompt(config.shell.family),
)]; )];
let mut context_registry = ContextProviderRegistry::new();
register_default_context_providers(&mut context_registry)
.expect("default context providers should register");
Self { Self {
config, config,
provider, provider,
messages, messages,
transcript, transcript,
_context_store: SessionContextStore::new(),
_context_registry: context_registry,
} }
} }
@ -151,6 +161,7 @@ impl CliOptions {
#[cfg(test)] #[cfg(test)]
mod tests { mod tests {
use super::*; use super::*;
use crate::config::{ProviderConfig, ShellConfig, TranscriptConfig};
#[test] #[test]
fn parses_config_path() { fn parses_config_path() {
@ -184,4 +195,54 @@ mod tests {
assert_eq!(options.transcript_directory, Some(PathBuf::from("out"))); assert_eq!(options.transcript_directory, Some(PathBuf::from("out")));
assert!(options.no_color); assert!(options.no_color);
} }
#[test]
fn app_registers_default_context_providers_on_startup() {
let app = App::new(test_config(), Box::new(NoopProvider));
let provider_names: Vec<String> = app
._context_registry
.list()
.into_iter()
.map(|metadata| metadata.name)
.collect();
assert_eq!(
provider_names,
vec![
"manual".to_string(),
"file".to_string(),
"command_output".to_string(),
"directory_summary".to_string()
]
);
assert_eq!(app._context_store.total_size().characters, 0);
}
struct NoopProvider;
#[async_trait::async_trait]
impl Provider for NoopProvider {
async fn chat(&self, _request: ChatRequest) -> Result<ChatResponse, ProviderError> {
Ok(ChatResponse::Complete("noop".into()))
}
}
fn test_config() -> Config {
Config {
provider: ProviderConfig {
base_url: "http://localhost:11434/v1".into(),
api_key: "test-key".into(),
api_key_env: "EXOSHELL_TEST_KEY".into(),
model: "test-model".into(),
},
shell: ShellConfig {
family: ShellFamily::PowerShell,
},
transcript: TranscriptConfig {
directory: PathBuf::from("transcripts"),
enabled: false,
},
}
}
} }

1217
src/context.rs Normal file

File diff suppressed because it is too large Load Diff

View File

@ -1,5 +1,6 @@
mod app; mod app;
mod config; mod config;
pub mod context;
mod formatting; mod formatting;
mod prompts; mod prompts;
mod providers; mod providers;