mirror of
https://github.com/khodges42/exoshell.git
synced 2026-06-14 18:08:37 +00:00
Context engine
This commit is contained in:
parent
37dca8954d
commit
8cfd968e72
|
|
@ -26,7 +26,7 @@ Phase 1 is closed.
|
|||
|
||||
The current implementation supports the first shell-adjacent model chat milestone: a Rust CLI, OpenAI-compatible provider abstraction, PowerShell/POSIX shell-family selection, command-suggestion formatting, markdown transcripts, and a basic interactive REPL.
|
||||
|
||||
The codebase also contains the initial Phase 1.5 context engine foundation: context entries, provenance metadata, priority and size estimates, a session context store, provider registry, default manual/file/command-output/directory-summary providers, deterministic pruning, and prompt-context rendering. REPL context commands are still planned.
|
||||
The codebase also contains the Phase 1.5 context engine foundation: context entries, provenance metadata, priority and size estimates, a session context store, provider registry, default manual/file/command-output/stdin/directory-summary providers, context REPL commands, deterministic pruning, budget checks, transcript events, startup context flags, piped stdin import, and prompt-context rendering.
|
||||
|
||||
The active milestone is Phase 1.5, the explicit context engine foundation tracked in [docs/tasks/phase15_context_tasks.md](docs/tasks/phase15_context_tasks.md). Phase 2 builds on that context engine with stances, safer command handling, hotkeys, and stronger operator controls.
|
||||
|
||||
|
|
@ -74,6 +74,7 @@ powershell -ExecutionPolicy Bypass -File scripts\manual_phase1_startup.ps1
|
|||
## Documentation
|
||||
|
||||
- [Design](docs/DESIGN.md): project philosophy, interaction model, non-goals, and technical direction.
|
||||
- [Context Engine](docs/context_engine.md): explicit context model, providers, commands, budgets, transcripts, serialization, and redaction limits.
|
||||
- [Phases](docs/PHASES.md): staged roadmap from Phase 1 through the mature cognitive shell overlay.
|
||||
- [Phase 1 Run Guide](docs/phase1_run.md): local setup, config examples, CLI flags, and current limitations.
|
||||
- [Versioning](docs/versioning.md): semantic versioning and release naming rules.
|
||||
|
|
|
|||
115
docs/context_engine.md
Normal file
115
docs/context_engine.md
Normal file
|
|
@ -0,0 +1,115 @@
|
|||
# Context Engine
|
||||
|
||||
The context engine stores explicit, user-controlled context for a session.
|
||||
|
||||
Context is not hidden memory. Entries are added through commands, startup flags, or piped stdin. Enabled entries are rendered into provider requests before the current user prompt. Disabled entries remain visible in the session but are not sent to the provider.
|
||||
|
||||
## Entry Model
|
||||
|
||||
Each context entry has:
|
||||
|
||||
* stable ID such as `ctx-001`
|
||||
* type
|
||||
* title
|
||||
* enabled flag
|
||||
* pinned flag
|
||||
* priority
|
||||
* creation timestamp
|
||||
* provenance
|
||||
* content
|
||||
* character count and approximate token estimate
|
||||
|
||||
Token estimates are deterministic and approximate. The current estimate is character count divided by four, rounded up.
|
||||
|
||||
## Providers
|
||||
|
||||
Built-in providers:
|
||||
|
||||
* `manual`: user-provided note text
|
||||
* `file`: UTF-8 text file loading
|
||||
* `command_output`: user-provided stdout/stderr
|
||||
* `stdin`: piped stdin
|
||||
* `directory_summary`: bounded directory listing without reading file contents
|
||||
|
||||
Providers return `ContextEntry` values and do not talk to model providers or terminal UI code.
|
||||
|
||||
## Commands
|
||||
|
||||
Inspection and mutation:
|
||||
|
||||
```text
|
||||
/context
|
||||
/context show <id>
|
||||
/context remove <id>
|
||||
/context enable <id>
|
||||
/context disable <id>
|
||||
/context pin <id>
|
||||
/context unpin <id>
|
||||
/context priority <id> <low|normal|high|critical>
|
||||
/context stats
|
||||
```
|
||||
|
||||
Adding context:
|
||||
|
||||
```text
|
||||
/add-note <text>
|
||||
/add-file <path>
|
||||
/add-dir <path>
|
||||
/add-output
|
||||
```
|
||||
|
||||
`/add-output` opens a multi-line paste flow and records the text as user-provided command output. Exoshell does not execute the command.
|
||||
|
||||
Startup flags:
|
||||
|
||||
```text
|
||||
--context-note <text>
|
||||
--context-file <path>
|
||||
```
|
||||
|
||||
Piped stdin is imported as explicit context with `stdin` provenance. Exoshell does not claim to know the upstream command unless it was explicitly provided by a caller.
|
||||
|
||||
## Budgets
|
||||
|
||||
Optional config:
|
||||
|
||||
```toml
|
||||
[context]
|
||||
max_characters = 12000
|
||||
max_estimated_tokens = 3000
|
||||
```
|
||||
|
||||
If these fields are absent, there is no explicit context budget limit.
|
||||
|
||||
Before a provider request, Exoshell checks enabled context against the configured budget. If the budget is exceeded, the request is stopped with a warning. Context is not silently dropped.
|
||||
|
||||
Pruning is deterministic and non-mutating. Low-priority unpinned entries are selected for removal first. Pinned entries and critical-priority entries are preserved as long as possible.
|
||||
|
||||
## Transcripts
|
||||
|
||||
Context lifecycle events are recorded as metadata:
|
||||
|
||||
* add
|
||||
* remove
|
||||
* enable
|
||||
* disable
|
||||
* pin
|
||||
* unpin
|
||||
* priority
|
||||
* budget warning
|
||||
|
||||
Transcript events do not include full context payloads by default.
|
||||
|
||||
## Serialization
|
||||
|
||||
Serialized context uses version `1`.
|
||||
|
||||
By default, serialized context excludes content payloads. Callers must explicitly request content inclusion. This keeps saved context inspectable without assuming it is safe to persist arbitrary file contents, command output, or pasted logs.
|
||||
|
||||
## Redaction
|
||||
|
||||
Exoshell cannot guarantee automatic secret detection.
|
||||
|
||||
Provider metadata can mark fields as sensitive. Transcript rendering also redacts obvious secret-shaped metadata keys such as `api_key`, `token`, `secret`, and `password`.
|
||||
|
||||
This is a guardrail, not a security boundary. Users should inspect context before sending or saving it.
|
||||
439
src/app.rs
439
src/app.rs
|
|
@ -2,7 +2,9 @@ use std::path::PathBuf;
|
|||
|
||||
use crate::config::Config;
|
||||
use crate::context::{
|
||||
ContextProviderRegistry, SessionContextStore, register_default_context_providers,
|
||||
ContextError, ContextPriority, ContextProviderRegistry, ContextProviderRequest,
|
||||
SessionContextStore, budget_warning, prune_context, register_default_context_providers,
|
||||
render_context_details, render_context_list, render_context_stats, render_prompt_context,
|
||||
};
|
||||
use crate::prompts::phase1_system_prompt;
|
||||
use crate::providers::{ChatMessage, ChatRequest, ChatResponse, ChatRole, Provider, ProviderError};
|
||||
|
|
@ -15,8 +17,8 @@ pub struct App {
|
|||
provider: Box<dyn Provider>,
|
||||
messages: Vec<ChatMessage>,
|
||||
transcript: Transcript,
|
||||
_context_store: SessionContextStore,
|
||||
_context_registry: ContextProviderRegistry,
|
||||
context_store: SessionContextStore,
|
||||
context_registry: ContextProviderRegistry,
|
||||
}
|
||||
|
||||
impl App {
|
||||
|
|
@ -39,8 +41,8 @@ impl App {
|
|||
provider,
|
||||
messages,
|
||||
transcript,
|
||||
_context_store: SessionContextStore::new(),
|
||||
_context_registry: context_registry,
|
||||
context_store: SessionContextStore::new(),
|
||||
context_registry,
|
||||
}
|
||||
}
|
||||
|
||||
|
|
@ -49,8 +51,9 @@ impl App {
|
|||
.push(ChatMessage::new(ChatRole::User, input.clone()));
|
||||
self.transcript.record_user(&input);
|
||||
|
||||
let request_messages = self.assembled_messages()?;
|
||||
let request = ChatRequest {
|
||||
messages: self.messages.clone(),
|
||||
messages: request_messages,
|
||||
stream: false,
|
||||
};
|
||||
|
||||
|
|
@ -69,6 +72,152 @@ impl App {
|
|||
Ok(response)
|
||||
}
|
||||
|
||||
pub fn handle_command(&mut self, input: &str) -> Result<String, AppError> {
|
||||
let trimmed = input.trim();
|
||||
if trimmed == "/context" {
|
||||
return Ok(render_context_list(self.context_store.entries()));
|
||||
}
|
||||
|
||||
if trimmed == "/context stats" {
|
||||
return Ok(render_context_stats(
|
||||
self.context_store.stats(),
|
||||
self.config.context.budget(),
|
||||
));
|
||||
}
|
||||
|
||||
if let Some(id) = trimmed.strip_prefix("/context show ") {
|
||||
let id = id.trim();
|
||||
let entry = self
|
||||
.context_store
|
||||
.get(id)
|
||||
.ok_or_else(|| ContextError::NotFound(id.to_string()))?;
|
||||
return Ok(render_context_details(entry));
|
||||
}
|
||||
|
||||
if let Some(id) = trimmed.strip_prefix("/context remove ") {
|
||||
let id = id.trim();
|
||||
let entry = self
|
||||
.context_store
|
||||
.remove(id)
|
||||
.ok_or_else(|| ContextError::NotFound(id.to_string()))?;
|
||||
self.transcript
|
||||
.record_context_event("remove", &entry, "removed from session context");
|
||||
return Ok(format!("removed {}", entry.id));
|
||||
}
|
||||
|
||||
if let Some(id) = trimmed.strip_prefix("/context enable ") {
|
||||
return self.set_enabled(id.trim(), true);
|
||||
}
|
||||
|
||||
if let Some(id) = trimmed.strip_prefix("/context disable ") {
|
||||
return self.set_enabled(id.trim(), false);
|
||||
}
|
||||
|
||||
if let Some(id) = trimmed.strip_prefix("/context pin ") {
|
||||
return self.set_pinned(id.trim(), true);
|
||||
}
|
||||
|
||||
if let Some(id) = trimmed.strip_prefix("/context unpin ") {
|
||||
return self.set_pinned(id.trim(), false);
|
||||
}
|
||||
|
||||
if let Some(rest) = trimmed.strip_prefix("/context priority ") {
|
||||
let (id, priority) = parse_context_priority_args(rest)?;
|
||||
self.context_store.set_priority(id, priority)?;
|
||||
let entry = self
|
||||
.context_store
|
||||
.get(id)
|
||||
.ok_or_else(|| ContextError::NotFound(id.to_string()))?;
|
||||
self.transcript.record_context_event(
|
||||
"priority",
|
||||
entry,
|
||||
&format!("priority set to {priority}"),
|
||||
);
|
||||
return Ok(format!("{} priority: {}", id, priority));
|
||||
}
|
||||
|
||||
if let Some(content) = trimmed.strip_prefix("/add-note ") {
|
||||
return self.add_context(
|
||||
"manual",
|
||||
ContextProviderRequest {
|
||||
content: Some(content.trim().to_string()),
|
||||
..ContextProviderRequest::default()
|
||||
},
|
||||
);
|
||||
}
|
||||
|
||||
if let Some(path) = trimmed.strip_prefix("/add-file ") {
|
||||
return self.add_context(
|
||||
"file",
|
||||
ContextProviderRequest {
|
||||
path: Some(PathBuf::from(path.trim())),
|
||||
..ContextProviderRequest::default()
|
||||
},
|
||||
);
|
||||
}
|
||||
|
||||
if let Some(path) = trimmed.strip_prefix("/add-dir ") {
|
||||
return self.add_context(
|
||||
"directory_summary",
|
||||
ContextProviderRequest {
|
||||
path: Some(PathBuf::from(path.trim())),
|
||||
..ContextProviderRequest::default()
|
||||
},
|
||||
);
|
||||
}
|
||||
|
||||
Err(ContextError::InvalidInput(format!("unknown context command: {trimmed}")).into())
|
||||
}
|
||||
|
||||
pub fn add_command_output(
|
||||
&mut self,
|
||||
stdout: String,
|
||||
command: Option<String>,
|
||||
exit_code: Option<i32>,
|
||||
) -> Result<String, AppError> {
|
||||
self.add_context(
|
||||
"command_output",
|
||||
ContextProviderRequest {
|
||||
stdout: Some(stdout),
|
||||
command,
|
||||
exit_code,
|
||||
cwd: std::env::current_dir().ok(),
|
||||
..ContextProviderRequest::default()
|
||||
},
|
||||
)
|
||||
}
|
||||
|
||||
pub fn add_stdin_context(&mut self, content: String) -> Result<String, AppError> {
|
||||
self.add_context(
|
||||
"stdin",
|
||||
ContextProviderRequest {
|
||||
content: Some(content),
|
||||
cwd: std::env::current_dir().ok(),
|
||||
..ContextProviderRequest::default()
|
||||
},
|
||||
)
|
||||
}
|
||||
|
||||
pub fn add_note_context(&mut self, content: String) -> Result<String, AppError> {
|
||||
self.add_context(
|
||||
"manual",
|
||||
ContextProviderRequest {
|
||||
content: Some(content),
|
||||
..ContextProviderRequest::default()
|
||||
},
|
||||
)
|
||||
}
|
||||
|
||||
pub fn add_file_context(&mut self, path: PathBuf) -> Result<String, AppError> {
|
||||
self.add_context(
|
||||
"file",
|
||||
ContextProviderRequest {
|
||||
path: Some(path),
|
||||
..ContextProviderRequest::default()
|
||||
},
|
||||
)
|
||||
}
|
||||
|
||||
pub fn save_transcript(&self) -> Result<Option<PathBuf>, AppError> {
|
||||
if !self.config.transcript.enabled {
|
||||
return Ok(None);
|
||||
|
|
@ -79,6 +228,100 @@ impl App {
|
|||
.write_to_dir(&self.config.transcript.directory)?;
|
||||
Ok(Some(path))
|
||||
}
|
||||
|
||||
fn assembled_messages(&mut self) -> Result<Vec<ChatMessage>, AppError> {
|
||||
let size = self.context_store.total_size();
|
||||
let budget = self.config.context.budget();
|
||||
if budget.is_over_budget(size) {
|
||||
let prune_result = prune_context(self.context_store.entries(), budget);
|
||||
let warning = budget_warning(size, budget, &prune_result);
|
||||
self.transcript.record_budget_warning(&warning);
|
||||
return Err(ContextError::TooLarge(warning).into());
|
||||
}
|
||||
|
||||
let context = render_prompt_context(self.context_store.entries());
|
||||
if context.is_empty() {
|
||||
return Ok(self.messages.clone());
|
||||
}
|
||||
|
||||
let mut messages = self.messages.clone();
|
||||
let insert_at = messages.len().saturating_sub(1);
|
||||
messages.insert(
|
||||
insert_at,
|
||||
ChatMessage::new(
|
||||
ChatRole::User,
|
||||
format!(
|
||||
"Explicit session context selected by the operator follows.\n\n{}",
|
||||
context
|
||||
),
|
||||
),
|
||||
);
|
||||
Ok(messages)
|
||||
}
|
||||
|
||||
fn add_context(
|
||||
&mut self,
|
||||
provider_name: &str,
|
||||
request: ContextProviderRequest,
|
||||
) -> Result<String, AppError> {
|
||||
let provider = self
|
||||
.context_registry
|
||||
.get(provider_name)
|
||||
.ok_or_else(|| ContextError::NotFound(format!("context provider '{provider_name}'")))?;
|
||||
let entry = provider.collect(request)?;
|
||||
let id = self.context_store.add(entry);
|
||||
let entry = self
|
||||
.context_store
|
||||
.get(&id)
|
||||
.ok_or_else(|| ContextError::NotFound(id.clone()))?;
|
||||
self.transcript
|
||||
.record_context_event("add", entry, "added to session context");
|
||||
Ok(format!("added {} ({})", entry.id, entry.title))
|
||||
}
|
||||
|
||||
fn set_enabled(&mut self, id: &str, enabled: bool) -> Result<String, AppError> {
|
||||
self.context_store.set_enabled(id, enabled)?;
|
||||
let entry = self
|
||||
.context_store
|
||||
.get(id)
|
||||
.ok_or_else(|| ContextError::NotFound(id.to_string()))?;
|
||||
self.transcript.record_context_event(
|
||||
if enabled { "enable" } else { "disable" },
|
||||
entry,
|
||||
if enabled {
|
||||
"enabled for model requests"
|
||||
} else {
|
||||
"disabled for model requests"
|
||||
},
|
||||
);
|
||||
Ok(format!(
|
||||
"{} {}",
|
||||
id,
|
||||
if enabled { "enabled" } else { "disabled" }
|
||||
))
|
||||
}
|
||||
|
||||
fn set_pinned(&mut self, id: &str, pinned: bool) -> Result<String, AppError> {
|
||||
self.context_store.set_pinned(id, pinned)?;
|
||||
let entry = self
|
||||
.context_store
|
||||
.get(id)
|
||||
.ok_or_else(|| ContextError::NotFound(id.to_string()))?;
|
||||
self.transcript.record_context_event(
|
||||
if pinned { "pin" } else { "unpin" },
|
||||
entry,
|
||||
if pinned {
|
||||
"pinned for pruning"
|
||||
} else {
|
||||
"unpinned for pruning"
|
||||
},
|
||||
);
|
||||
Ok(format!(
|
||||
"{} {}",
|
||||
id,
|
||||
if pinned { "pinned" } else { "unpinned" }
|
||||
))
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Debug, thiserror::Error)]
|
||||
|
|
@ -91,6 +334,25 @@ pub enum AppError {
|
|||
Repl(#[from] ReplError),
|
||||
#[error(transparent)]
|
||||
Transcript(#[from] TranscriptError),
|
||||
#[error(transparent)]
|
||||
Context(#[from] ContextError),
|
||||
}
|
||||
|
||||
fn parse_context_priority_args(input: &str) -> Result<(&str, ContextPriority), ContextError> {
|
||||
let mut parts = input.split_whitespace();
|
||||
let id = parts
|
||||
.next()
|
||||
.ok_or_else(|| ContextError::InvalidInput("context ID is required".into()))?;
|
||||
let priority = parts
|
||||
.next()
|
||||
.ok_or_else(|| ContextError::InvalidInput("priority is required".into()))?
|
||||
.parse::<ContextPriority>()?;
|
||||
if parts.next().is_some() {
|
||||
return Err(ContextError::InvalidInput(
|
||||
"too many arguments for context priority".into(),
|
||||
));
|
||||
}
|
||||
Ok((id, priority))
|
||||
}
|
||||
|
||||
#[derive(Debug, Default, PartialEq, Eq)]
|
||||
|
|
@ -99,6 +361,8 @@ pub struct CliOptions {
|
|||
pub shell_family: Option<ShellFamily>,
|
||||
pub transcript_enabled: Option<bool>,
|
||||
pub transcript_directory: Option<PathBuf>,
|
||||
pub context_notes: Vec<String>,
|
||||
pub context_files: Vec<PathBuf>,
|
||||
pub no_color: bool,
|
||||
pub show_help: bool,
|
||||
}
|
||||
|
|
@ -140,6 +404,18 @@ impl CliOptions {
|
|||
options.transcript_directory = Some(PathBuf::from(value));
|
||||
options.transcript_enabled = Some(true);
|
||||
}
|
||||
"--context-note" => {
|
||||
let value = args.next().ok_or_else(|| {
|
||||
crate::config::ConfigError::Invalid("--context-note requires text".into())
|
||||
})?;
|
||||
options.context_notes.push(value);
|
||||
}
|
||||
"--context-file" => {
|
||||
let value = args.next().ok_or_else(|| {
|
||||
crate::config::ConfigError::Invalid("--context-file requires a path".into())
|
||||
})?;
|
||||
options.context_files.push(PathBuf::from(value));
|
||||
}
|
||||
"--no-color" => options.no_color = true,
|
||||
other => {
|
||||
return Err(crate::config::ConfigError::Invalid(format!(
|
||||
|
|
@ -154,7 +430,7 @@ impl CliOptions {
|
|||
}
|
||||
|
||||
pub fn help() -> &'static str {
|
||||
"Usage: exoshell [--config <path>] [--shell powershell|posix] [--no-transcript] [--transcript-dir <path>] [--no-color]\n\nStarts the Exoshell Phase 1 interactive model chat. Exoshell suggests commands; it does not execute them."
|
||||
"Usage: exoshell [--config <path>] [--shell powershell|posix] [--context-note <text>] [--context-file <path>] [--no-transcript] [--transcript-dir <path>] [--no-color]\n\nStarts the Exoshell interactive model chat. Exoshell suggests commands; it does not execute them."
|
||||
}
|
||||
}
|
||||
|
||||
|
|
@ -162,6 +438,7 @@ impl CliOptions {
|
|||
mod tests {
|
||||
use super::*;
|
||||
use crate::config::{ProviderConfig, ShellConfig, TranscriptConfig};
|
||||
use std::sync::{Arc, Mutex};
|
||||
|
||||
#[test]
|
||||
fn parses_config_path() {
|
||||
|
|
@ -186,6 +463,10 @@ mod tests {
|
|||
"--no-transcript".to_string(),
|
||||
"--transcript-dir".to_string(),
|
||||
"out".to_string(),
|
||||
"--context-note".to_string(),
|
||||
"note".to_string(),
|
||||
"--context-file".to_string(),
|
||||
"Cargo.toml".to_string(),
|
||||
"--no-color".to_string(),
|
||||
])
|
||||
.expect("options parse");
|
||||
|
|
@ -193,6 +474,8 @@ mod tests {
|
|||
assert_eq!(options.shell_family, Some(ShellFamily::Posix));
|
||||
assert_eq!(options.transcript_enabled, Some(true));
|
||||
assert_eq!(options.transcript_directory, Some(PathBuf::from("out")));
|
||||
assert_eq!(options.context_notes, vec!["note".to_string()]);
|
||||
assert_eq!(options.context_files, vec![PathBuf::from("Cargo.toml")]);
|
||||
assert!(options.no_color);
|
||||
}
|
||||
|
||||
|
|
@ -201,7 +484,7 @@ mod tests {
|
|||
let app = App::new(test_config(), Box::new(NoopProvider));
|
||||
|
||||
let provider_names: Vec<String> = app
|
||||
._context_registry
|
||||
.context_registry
|
||||
.list()
|
||||
.into_iter()
|
||||
.map(|metadata| metadata.name)
|
||||
|
|
@ -213,10 +496,134 @@ mod tests {
|
|||
"manual".to_string(),
|
||||
"file".to_string(),
|
||||
"command_output".to_string(),
|
||||
"stdin".to_string(),
|
||||
"directory_summary".to_string()
|
||||
]
|
||||
);
|
||||
assert_eq!(app._context_store.total_size().characters, 0);
|
||||
assert_eq!(app.context_store.total_size().characters, 0);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn context_commands_add_and_mutate_entries() {
|
||||
let mut app = App::new(test_config(), Box::new(NoopProvider));
|
||||
|
||||
assert_eq!(
|
||||
app.handle_command("/add-note inspect Cargo.toml")
|
||||
.expect("add note"),
|
||||
"added ctx-001 (manual context)"
|
||||
);
|
||||
assert!(
|
||||
app.handle_command("/context")
|
||||
.expect("list")
|
||||
.contains("ctx-001")
|
||||
);
|
||||
assert!(
|
||||
app.handle_command("/context show ctx-001")
|
||||
.expect("show")
|
||||
.contains("inspect Cargo.toml")
|
||||
);
|
||||
assert_eq!(
|
||||
app.handle_command("/context priority ctx-001 high")
|
||||
.expect("priority"),
|
||||
"ctx-001 priority: high"
|
||||
);
|
||||
assert_eq!(
|
||||
app.handle_command("/context disable ctx-001")
|
||||
.expect("disable"),
|
||||
"ctx-001 disabled"
|
||||
);
|
||||
assert_eq!(
|
||||
app.handle_command("/context pin ctx-001").expect("pin"),
|
||||
"ctx-001 pinned"
|
||||
);
|
||||
assert!(
|
||||
app.handle_command("/context stats")
|
||||
.expect("stats")
|
||||
.contains("total_entries: 1")
|
||||
);
|
||||
assert_eq!(
|
||||
app.handle_command("/context remove ctx-001")
|
||||
.expect("remove"),
|
||||
"removed ctx-001"
|
||||
);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn enabled_context_is_inserted_before_current_user_prompt() {
|
||||
let seen = Arc::new(Mutex::new(Vec::new()));
|
||||
let provider = CapturingProvider { seen: seen.clone() };
|
||||
let mut app = App::new(test_config(), Box::new(provider));
|
||||
app.handle_command("/add-note repo uses cargo")
|
||||
.expect("add note");
|
||||
|
||||
app.send("what should I inspect?".into())
|
||||
.await
|
||||
.expect("send");
|
||||
|
||||
let messages = seen.lock().expect("seen lock").clone();
|
||||
assert_eq!(
|
||||
messages.last().expect("last").content,
|
||||
"what should I inspect?"
|
||||
);
|
||||
assert!(
|
||||
messages
|
||||
.iter()
|
||||
.any(|message| message.content.contains("[Context: ctx-001]")
|
||||
&& message.content.contains("repo uses cargo"))
|
||||
);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn disabled_context_is_omitted_from_provider_request() {
|
||||
let seen = Arc::new(Mutex::new(Vec::new()));
|
||||
let provider = CapturingProvider { seen: seen.clone() };
|
||||
let mut app = App::new(test_config(), Box::new(provider));
|
||||
app.handle_command("/add-note hidden context")
|
||||
.expect("add note");
|
||||
app.handle_command("/context disable ctx-001")
|
||||
.expect("disable");
|
||||
|
||||
app.send("hello".into()).await.expect("send");
|
||||
|
||||
let messages = seen.lock().expect("seen lock").clone();
|
||||
assert!(
|
||||
!messages
|
||||
.iter()
|
||||
.any(|message| message.content.contains("hidden context"))
|
||||
);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn over_budget_context_fails_before_provider_request() {
|
||||
let seen = Arc::new(Mutex::new(Vec::new()));
|
||||
let provider = CapturingProvider { seen: seen.clone() };
|
||||
let mut config = test_config();
|
||||
config.context.max_characters = Some(3);
|
||||
let mut app = App::new(config, Box::new(provider));
|
||||
app.handle_command("/add-note too much context")
|
||||
.expect("add note");
|
||||
|
||||
let error = app
|
||||
.send("hello".into())
|
||||
.await
|
||||
.expect_err("budget should stop request");
|
||||
|
||||
assert!(error.to_string().contains("context budget exceeded"));
|
||||
assert!(seen.lock().expect("seen lock").is_empty());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn stdin_context_uses_default_provider_path() {
|
||||
let mut app = App::new(test_config(), Box::new(NoopProvider));
|
||||
|
||||
let message = app
|
||||
.add_stdin_context("from pipe".into())
|
||||
.expect("stdin context");
|
||||
|
||||
assert_eq!(message, "added ctx-001 (piped stdin)");
|
||||
let entry = app.context_store.get("ctx-001").expect("entry");
|
||||
assert_eq!(entry.provenance.origin.to_string(), "stdin");
|
||||
assert_eq!(entry.content, "from pipe");
|
||||
}
|
||||
|
||||
struct NoopProvider;
|
||||
|
|
@ -228,6 +635,18 @@ mod tests {
|
|||
}
|
||||
}
|
||||
|
||||
struct CapturingProvider {
|
||||
seen: Arc<Mutex<Vec<ChatMessage>>>,
|
||||
}
|
||||
|
||||
#[async_trait::async_trait]
|
||||
impl Provider for CapturingProvider {
|
||||
async fn chat(&self, request: ChatRequest) -> Result<ChatResponse, ProviderError> {
|
||||
*self.seen.lock().expect("seen lock") = request.messages;
|
||||
Ok(ChatResponse::Complete("captured".into()))
|
||||
}
|
||||
}
|
||||
|
||||
fn test_config() -> Config {
|
||||
Config {
|
||||
provider: ProviderConfig {
|
||||
|
|
@ -235,6 +654,7 @@ mod tests {
|
|||
api_key: "test-key".into(),
|
||||
api_key_env: "EXOSHELL_TEST_KEY".into(),
|
||||
model: "test-model".into(),
|
||||
request_timeout_seconds: 120,
|
||||
},
|
||||
shell: ShellConfig {
|
||||
family: ShellFamily::PowerShell,
|
||||
|
|
@ -243,6 +663,7 @@ mod tests {
|
|||
directory: PathBuf::from("transcripts"),
|
||||
enabled: false,
|
||||
},
|
||||
context: crate::config::ContextConfig::default(),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -5,6 +5,7 @@ use std::path::{Path, PathBuf};
|
|||
use serde::Deserialize;
|
||||
|
||||
use crate::app::CliOptions;
|
||||
use crate::context::ContextBudget;
|
||||
use crate::shell::ShellFamily;
|
||||
|
||||
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||
|
|
@ -12,6 +13,7 @@ pub struct Config {
|
|||
pub provider: ProviderConfig,
|
||||
pub shell: ShellConfig,
|
||||
pub transcript: TranscriptConfig,
|
||||
pub context: ContextConfig,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||
|
|
@ -20,6 +22,7 @@ pub struct ProviderConfig {
|
|||
pub api_key: String,
|
||||
pub api_key_env: String,
|
||||
pub model: String,
|
||||
pub request_timeout_seconds: u64,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||
|
|
@ -33,11 +36,27 @@ pub struct TranscriptConfig {
|
|||
pub enabled: bool,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, PartialEq, Eq, Default)]
|
||||
pub struct ContextConfig {
|
||||
pub max_characters: Option<usize>,
|
||||
pub max_estimated_tokens: Option<usize>,
|
||||
}
|
||||
|
||||
impl ContextConfig {
|
||||
pub fn budget(&self) -> ContextBudget {
|
||||
ContextBudget {
|
||||
max_characters: self.max_characters,
|
||||
max_estimated_tokens: self.max_estimated_tokens,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Debug, Deserialize, Default)]
|
||||
struct RawConfig {
|
||||
provider: Option<RawProviderConfig>,
|
||||
shell: Option<RawShellConfig>,
|
||||
transcript: Option<RawTranscriptConfig>,
|
||||
context: Option<RawContextConfig>,
|
||||
}
|
||||
|
||||
#[derive(Debug, Deserialize, Default)]
|
||||
|
|
@ -45,6 +64,7 @@ struct RawProviderConfig {
|
|||
base_url: Option<String>,
|
||||
api_key_env: Option<String>,
|
||||
model: Option<String>,
|
||||
request_timeout_seconds: Option<u64>,
|
||||
}
|
||||
|
||||
#[derive(Debug, Deserialize, Default)]
|
||||
|
|
@ -58,6 +78,12 @@ struct RawTranscriptConfig {
|
|||
enabled: Option<bool>,
|
||||
}
|
||||
|
||||
#[derive(Debug, Deserialize, Default)]
|
||||
struct RawContextConfig {
|
||||
max_characters: Option<usize>,
|
||||
max_estimated_tokens: Option<usize>,
|
||||
}
|
||||
|
||||
impl Config {
|
||||
pub fn load(path: Option<&Path>) -> Result<Self, ConfigError> {
|
||||
let raw = match path {
|
||||
|
|
@ -72,6 +98,7 @@ impl Config {
|
|||
let provider = raw.provider.unwrap_or_default();
|
||||
let shell = raw.shell.unwrap_or_default();
|
||||
let transcript = raw.transcript.unwrap_or_default();
|
||||
let context = raw.context.unwrap_or_default();
|
||||
|
||||
let base_url = provider
|
||||
.base_url
|
||||
|
|
@ -92,12 +119,17 @@ impl Config {
|
|||
api_key,
|
||||
api_key_env,
|
||||
model: provider.model.unwrap_or_else(|| "gpt-4.1-mini".into()),
|
||||
request_timeout_seconds: provider.request_timeout_seconds.unwrap_or(120),
|
||||
},
|
||||
shell: ShellConfig { family },
|
||||
transcript: TranscriptConfig {
|
||||
directory: transcript.directory.unwrap_or_else(default_transcript_dir),
|
||||
enabled: transcript.enabled.unwrap_or(true),
|
||||
},
|
||||
context: ContextConfig {
|
||||
max_characters: context.max_characters,
|
||||
max_estimated_tokens: context.max_estimated_tokens,
|
||||
},
|
||||
})
|
||||
}
|
||||
|
||||
|
|
@ -272,6 +304,7 @@ mod tests {
|
|||
family: Some("cmd".into()),
|
||||
}),
|
||||
transcript: None,
|
||||
context: None,
|
||||
})
|
||||
.expect_err("shell family should be rejected");
|
||||
|
||||
|
|
@ -287,12 +320,17 @@ mod tests {
|
|||
[provider]
|
||||
base_url = "http://localhost:11434/v1"
|
||||
model = "local-model"
|
||||
request_timeout_seconds = 45
|
||||
|
||||
[shell]
|
||||
family = "posix"
|
||||
|
||||
[transcript]
|
||||
enabled = false
|
||||
|
||||
[context]
|
||||
max_characters = 12000
|
||||
max_estimated_tokens = 3000
|
||||
"#
|
||||
)
|
||||
.expect("write config");
|
||||
|
|
@ -302,8 +340,32 @@ enabled = false
|
|||
assert_eq!(config.provider.base_url, "http://localhost:11434/v1");
|
||||
assert_eq!(config.provider.api_key, "exoshell-local-provider");
|
||||
assert_eq!(config.provider.model, "local-model");
|
||||
assert_eq!(config.provider.request_timeout_seconds, 45);
|
||||
assert_eq!(config.shell.family, ShellFamily::Posix);
|
||||
assert!(!config.transcript.enabled);
|
||||
assert_eq!(config.context.max_characters, Some(12000));
|
||||
assert_eq!(config.context.max_estimated_tokens, Some(3000));
|
||||
assert_eq!(config.context.budget().max_characters, Some(12000));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn context_budget_defaults_to_unlimited() {
|
||||
unsafe {
|
||||
env::set_var("EXOSHELL_TEST_KEY", "secret");
|
||||
}
|
||||
|
||||
let config = Config::from_raw(RawConfig {
|
||||
provider: Some(RawProviderConfig {
|
||||
api_key_env: Some("EXOSHELL_TEST_KEY".into()),
|
||||
..RawProviderConfig::default()
|
||||
}),
|
||||
..RawConfig::default()
|
||||
})
|
||||
.expect("config loads");
|
||||
|
||||
assert_eq!(config.context.max_characters, None);
|
||||
assert_eq!(config.context.max_estimated_tokens, None);
|
||||
assert_eq!(config.provider.request_timeout_seconds, 120);
|
||||
}
|
||||
|
||||
#[test]
|
||||
|
|
|
|||
394
src/context.rs
394
src/context.rs
|
|
@ -2,6 +2,7 @@ use std::collections::HashMap;
|
|||
use std::fmt;
|
||||
use std::fs;
|
||||
use std::path::{Path, PathBuf};
|
||||
use std::str::FromStr;
|
||||
|
||||
#[derive(Debug, Clone, PartialEq, Eq, serde::Serialize, serde::Deserialize)]
|
||||
pub struct ContextEntry {
|
||||
|
|
@ -118,6 +119,22 @@ impl fmt::Display for ContextPriority {
|
|||
}
|
||||
}
|
||||
|
||||
impl FromStr for ContextPriority {
|
||||
type Err = ContextError;
|
||||
|
||||
fn from_str(value: &str) -> Result<Self, Self::Err> {
|
||||
match value {
|
||||
"low" => Ok(Self::Low),
|
||||
"normal" => Ok(Self::Normal),
|
||||
"high" => Ok(Self::High),
|
||||
"critical" => Ok(Self::Critical),
|
||||
other => Err(ContextError::InvalidInput(format!(
|
||||
"unsupported context priority '{other}', expected low, normal, high, or critical"
|
||||
))),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, PartialEq, Eq, serde::Serialize, serde::Deserialize)]
|
||||
pub struct ContextProvenance {
|
||||
pub origin: ContextOrigin,
|
||||
|
|
@ -125,6 +142,7 @@ pub struct ContextProvenance {
|
|||
pub command: Option<String>,
|
||||
pub cwd: Option<PathBuf>,
|
||||
pub provider_details: HashMap<String, String>,
|
||||
pub sensitive_provider_fields: Vec<String>,
|
||||
}
|
||||
|
||||
impl ContextProvenance {
|
||||
|
|
@ -135,6 +153,7 @@ impl ContextProvenance {
|
|||
command: None,
|
||||
cwd: None,
|
||||
provider_details: HashMap::new(),
|
||||
sensitive_provider_fields: Vec::new(),
|
||||
}
|
||||
}
|
||||
|
||||
|
|
@ -286,6 +305,7 @@ pub fn register_default_context_providers(
|
|||
registry.register(Box::new(ManualContextProvider))?;
|
||||
registry.register(Box::new(FileContextProvider::default()))?;
|
||||
registry.register(Box::new(CommandOutputContextProvider))?;
|
||||
registry.register(Box::new(StdinContextProvider))?;
|
||||
registry.register(Box::new(DirectorySummaryContextProvider::default()))?;
|
||||
Ok(())
|
||||
}
|
||||
|
|
@ -434,6 +454,40 @@ impl ContextProvider for CommandOutputContextProvider {
|
|||
}
|
||||
}
|
||||
|
||||
pub struct StdinContextProvider;
|
||||
|
||||
impl ContextProvider for StdinContextProvider {
|
||||
fn metadata(&self) -> ContextProviderMetadata {
|
||||
ContextProviderMetadata {
|
||||
name: "stdin".into(),
|
||||
kind: ContextKind::CommandOutput,
|
||||
description: "adds piped stdin as explicit context".into(),
|
||||
}
|
||||
}
|
||||
|
||||
fn collect(&self, request: ContextProviderRequest) -> Result<ContextEntry, ContextError> {
|
||||
let content = required_non_empty_content(request.content)?;
|
||||
let mut provenance = ContextProvenance::new(ContextOrigin::Stdin);
|
||||
provenance.cwd = request.cwd;
|
||||
provenance
|
||||
.provider_details
|
||||
.insert("provided_by_user".into(), "true".into());
|
||||
provenance.provider_details.insert(
|
||||
"upstream_command_known".into(),
|
||||
request.command.is_some().to_string(),
|
||||
);
|
||||
provenance.command = request.command;
|
||||
|
||||
Ok(ContextEntry::new(
|
||||
"",
|
||||
ContextKind::CommandOutput,
|
||||
request.title.unwrap_or_else(|| "piped stdin".into()),
|
||||
provenance,
|
||||
content,
|
||||
))
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct DirectorySummaryContextProvider {
|
||||
pub max_depth: usize,
|
||||
|
|
@ -581,6 +635,22 @@ impl SessionContextStore {
|
|||
)
|
||||
}
|
||||
|
||||
pub fn stats(&self) -> ContextStats {
|
||||
let total_entries = self.entries.len();
|
||||
let enabled_entries = self.entries.iter().filter(|entry| entry.enabled).count();
|
||||
let disabled_entries = total_entries.saturating_sub(enabled_entries);
|
||||
let pinned_entries = self.entries.iter().filter(|entry| entry.pinned).count();
|
||||
let size = self.total_size();
|
||||
|
||||
ContextStats {
|
||||
total_entries,
|
||||
enabled_entries,
|
||||
disabled_entries,
|
||||
pinned_entries,
|
||||
enabled_size: size,
|
||||
}
|
||||
}
|
||||
|
||||
fn next_context_id(&mut self) -> String {
|
||||
let id = format!("ctx-{:03}", self.next_id);
|
||||
self.next_id += 1;
|
||||
|
|
@ -588,6 +658,15 @@ impl SessionContextStore {
|
|||
}
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||
pub struct ContextStats {
|
||||
pub total_entries: usize,
|
||||
pub enabled_entries: usize,
|
||||
pub disabled_entries: usize,
|
||||
pub pinned_entries: usize,
|
||||
pub enabled_size: ContextSize,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq, Default)]
|
||||
pub struct ContextBudget {
|
||||
pub max_characters: Option<usize>,
|
||||
|
|
@ -677,6 +756,9 @@ pub fn render_prompt_context(entries: &[ContextEntry]) -> String {
|
|||
if let Some(cwd) = &entry.provenance.cwd {
|
||||
rendered.push_str(&format!("Cwd: {}\n", cwd.display()));
|
||||
}
|
||||
if let Some(truncated) = entry.provenance.provider_details.get("truncated") {
|
||||
rendered.push_str(&format!("Truncated: {truncated}\n"));
|
||||
}
|
||||
rendered.push_str(&format!("Priority: {}\n", entry.priority));
|
||||
rendered.push('\n');
|
||||
rendered.push_str(&entry.content);
|
||||
|
|
@ -686,6 +768,172 @@ pub fn render_prompt_context(entries: &[ContextEntry]) -> String {
|
|||
rendered.trim_end().to_string()
|
||||
}
|
||||
|
||||
pub fn render_context_list(entries: &[ContextEntry]) -> String {
|
||||
if entries.is_empty() {
|
||||
return "No context entries.".into();
|
||||
}
|
||||
|
||||
let mut rendered = String::from("ID Type State Pin Priority Size\n");
|
||||
for entry in entries {
|
||||
rendered.push_str(&format!(
|
||||
"{:<8} {:<18} {:<9} {:<4} {:<9} {} chars / ~{} tokens {}\n",
|
||||
entry.id,
|
||||
entry.kind,
|
||||
if entry.enabled { "enabled" } else { "disabled" },
|
||||
if entry.pinned { "yes" } else { "no" },
|
||||
entry.priority,
|
||||
entry.size.characters,
|
||||
entry.size.estimated_tokens,
|
||||
entry.title
|
||||
));
|
||||
}
|
||||
|
||||
rendered.trim_end().to_string()
|
||||
}
|
||||
|
||||
pub fn render_context_details(entry: &ContextEntry) -> String {
|
||||
let mut rendered = String::new();
|
||||
rendered.push_str(&format!("ID: {}\n", entry.id));
|
||||
rendered.push_str(&format!("Type: {}\n", entry.kind));
|
||||
rendered.push_str(&format!("Title: {}\n", entry.title));
|
||||
rendered.push_str(&format!("Enabled: {}\n", entry.enabled));
|
||||
rendered.push_str(&format!("Pinned: {}\n", entry.pinned));
|
||||
rendered.push_str(&format!("Priority: {}\n", entry.priority));
|
||||
rendered.push_str(&format!(
|
||||
"Size: {} chars / ~{} tokens\n",
|
||||
entry.size.characters, entry.size.estimated_tokens
|
||||
));
|
||||
rendered.push_str(&format!("Origin: {}\n", entry.provenance.origin));
|
||||
if let Some(path) = &entry.provenance.source_path {
|
||||
rendered.push_str(&format!("Path: {}\n", path.display()));
|
||||
}
|
||||
if let Some(command) = &entry.provenance.command {
|
||||
rendered.push_str(&format!("Command: {command}\n"));
|
||||
}
|
||||
if let Some(cwd) = &entry.provenance.cwd {
|
||||
rendered.push_str(&format!("Cwd: {}\n", cwd.display()));
|
||||
}
|
||||
for (key, value) in redacted_provider_details(&entry.provenance) {
|
||||
rendered.push_str(&format!("{key}: {value}\n"));
|
||||
}
|
||||
rendered.push('\n');
|
||||
rendered.push_str(&entry.content);
|
||||
rendered
|
||||
}
|
||||
|
||||
pub fn render_context_stats(stats: ContextStats, budget: ContextBudget) -> String {
|
||||
let mut rendered = String::new();
|
||||
rendered.push_str(&format!("total_entries: {}\n", stats.total_entries));
|
||||
rendered.push_str(&format!("enabled_entries: {}\n", stats.enabled_entries));
|
||||
rendered.push_str(&format!("disabled_entries: {}\n", stats.disabled_entries));
|
||||
rendered.push_str(&format!("pinned_entries: {}\n", stats.pinned_entries));
|
||||
rendered.push_str(&format!("characters: {}\n", stats.enabled_size.characters));
|
||||
rendered.push_str(&format!(
|
||||
"estimated_tokens: {}\n",
|
||||
stats.enabled_size.estimated_tokens
|
||||
));
|
||||
if let Some(max) = budget.max_characters {
|
||||
rendered.push_str(&format!(
|
||||
"character_budget: {}/{}\n",
|
||||
stats.enabled_size.characters, max
|
||||
));
|
||||
}
|
||||
if let Some(max) = budget.max_estimated_tokens {
|
||||
rendered.push_str(&format!(
|
||||
"token_budget: {}/{}\n",
|
||||
stats.enabled_size.estimated_tokens, max
|
||||
));
|
||||
}
|
||||
|
||||
rendered.trim_end().to_string()
|
||||
}
|
||||
|
||||
pub fn render_prune_result(result: &ContextPruneResult) -> String {
|
||||
format!(
|
||||
"included: {}\nremoved: {}\nfinal_size: {} chars / ~{} tokens\nover_budget: {}",
|
||||
comma_list(&result.included_ids),
|
||||
comma_list(&result.removed_ids),
|
||||
result.final_size.characters,
|
||||
result.final_size.estimated_tokens,
|
||||
result.over_budget
|
||||
)
|
||||
}
|
||||
|
||||
pub fn budget_warning(
|
||||
size: ContextSize,
|
||||
budget: ContextBudget,
|
||||
result: &ContextPruneResult,
|
||||
) -> String {
|
||||
let mut exceeded = Vec::new();
|
||||
if let Some(max) = budget.max_characters
|
||||
&& size.characters > max
|
||||
{
|
||||
exceeded.push(format!("characters {} > {}", size.characters, max));
|
||||
}
|
||||
if let Some(max) = budget.max_estimated_tokens
|
||||
&& size.estimated_tokens > max
|
||||
{
|
||||
exceeded.push(format!(
|
||||
"estimated_tokens {} > {}",
|
||||
size.estimated_tokens, max
|
||||
));
|
||||
}
|
||||
|
||||
format!(
|
||||
"context budget exceeded: {}. No context was silently dropped.\n{}",
|
||||
exceeded.join(", "),
|
||||
render_prune_result(result)
|
||||
)
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, PartialEq, Eq, serde::Serialize, serde::Deserialize)]
|
||||
pub struct SerializedContext {
|
||||
pub version: u32,
|
||||
pub entries: Vec<SerializedContextEntry>,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, PartialEq, Eq, serde::Serialize, serde::Deserialize)]
|
||||
pub struct SerializedContextEntry {
|
||||
pub entry: ContextEntry,
|
||||
pub content_included: bool,
|
||||
}
|
||||
|
||||
pub fn serialize_context(entries: &[ContextEntry], include_content: bool) -> SerializedContext {
|
||||
SerializedContext {
|
||||
version: 1,
|
||||
entries: entries
|
||||
.iter()
|
||||
.cloned()
|
||||
.map(|mut entry| {
|
||||
if !include_content {
|
||||
entry.content.clear();
|
||||
entry.size = ContextSize::from_content("");
|
||||
}
|
||||
SerializedContextEntry {
|
||||
entry,
|
||||
content_included: include_content,
|
||||
}
|
||||
})
|
||||
.collect(),
|
||||
}
|
||||
}
|
||||
|
||||
pub fn redacted_provider_details(provenance: &ContextProvenance) -> Vec<(String, String)> {
|
||||
let mut details: Vec<(String, String)> = provenance
|
||||
.provider_details
|
||||
.iter()
|
||||
.map(|(key, value)| {
|
||||
if provenance.sensitive_provider_fields.contains(key) || looks_sensitive(key) {
|
||||
(key.clone(), "[redacted]".into())
|
||||
} else {
|
||||
(key.clone(), value.clone())
|
||||
}
|
||||
})
|
||||
.collect();
|
||||
details.sort_by(|left, right| left.0.cmp(&right.0));
|
||||
details
|
||||
}
|
||||
|
||||
fn combined_size(entries: &[&ContextEntry]) -> ContextSize {
|
||||
entries.iter().fold(
|
||||
ContextSize {
|
||||
|
|
@ -699,6 +947,23 @@ fn combined_size(entries: &[&ContextEntry]) -> ContextSize {
|
|||
)
|
||||
}
|
||||
|
||||
fn comma_list(values: &[String]) -> String {
|
||||
if values.is_empty() {
|
||||
"none".into()
|
||||
} else {
|
||||
values.join(", ")
|
||||
}
|
||||
}
|
||||
|
||||
fn looks_sensitive(key: &str) -> bool {
|
||||
let key = key.to_ascii_lowercase();
|
||||
key.contains("secret")
|
||||
|| key.contains("token")
|
||||
|| key.contains("password")
|
||||
|| key.contains("api_key")
|
||||
|| key.contains("apikey")
|
||||
}
|
||||
|
||||
fn estimate_tokens(characters: usize) -> usize {
|
||||
characters.div_ceil(4)
|
||||
}
|
||||
|
|
@ -989,6 +1254,111 @@ mod tests {
|
|||
assert!(!rendered.contains("disabled"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn user_facing_renderers_show_list_details_stats_and_pruning() {
|
||||
let mut entry = sample_entry("visible", ContextPriority::High, true);
|
||||
entry.kind = ContextKind::Note;
|
||||
entry.title = "operator note".into();
|
||||
let entries = vec![entry.clone()];
|
||||
|
||||
let list = render_context_list(&entries);
|
||||
assert!(list.contains("ctx-visible"));
|
||||
assert!(list.contains("operator note"));
|
||||
assert!(list.contains("high"));
|
||||
|
||||
let details = render_context_details(&entry);
|
||||
assert!(details.contains("ID: ctx-visible"));
|
||||
assert!(details.contains("Pinned: true"));
|
||||
assert!(details.contains("visible"));
|
||||
|
||||
let stats = render_context_stats(
|
||||
ContextStats {
|
||||
total_entries: 1,
|
||||
enabled_entries: 1,
|
||||
disabled_entries: 0,
|
||||
pinned_entries: 1,
|
||||
enabled_size: entry.size,
|
||||
},
|
||||
ContextBudget {
|
||||
max_characters: Some(100),
|
||||
max_estimated_tokens: Some(25),
|
||||
},
|
||||
);
|
||||
assert!(stats.contains("total_entries: 1"));
|
||||
assert!(stats.contains("character_budget: 7/100"));
|
||||
|
||||
let prune = render_prune_result(&ContextPruneResult {
|
||||
included_ids: vec!["ctx-visible".into()],
|
||||
removed_ids: vec!["ctx-old".into()],
|
||||
final_size: entry.size,
|
||||
over_budget: false,
|
||||
});
|
||||
assert!(prune.contains("included: ctx-visible"));
|
||||
assert!(prune.contains("removed: ctx-old"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn context_serialization_excludes_content_by_default() {
|
||||
let entry = sample_entry("secret-ish body", ContextPriority::Normal, false);
|
||||
|
||||
let without_content = serialize_context(std::slice::from_ref(&entry), false);
|
||||
assert_eq!(without_content.version, 1);
|
||||
assert!(!without_content.entries[0].content_included);
|
||||
assert!(without_content.entries[0].entry.content.is_empty());
|
||||
|
||||
let with_content = serialize_context(std::slice::from_ref(&entry), true);
|
||||
assert!(with_content.entries[0].content_included);
|
||||
assert_eq!(with_content.entries[0].entry.content, "secret-ish body");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn provider_details_redact_sensitive_fields() {
|
||||
let mut provenance = ContextProvenance::manual();
|
||||
provenance
|
||||
.provider_details
|
||||
.insert("api_key".into(), "sk-test".into());
|
||||
provenance
|
||||
.provider_details
|
||||
.insert("plain".into(), "visible".into());
|
||||
provenance
|
||||
.provider_details
|
||||
.insert("custom".into(), "hidden".into());
|
||||
provenance.sensitive_provider_fields.push("custom".into());
|
||||
|
||||
let details = redacted_provider_details(&provenance);
|
||||
|
||||
assert!(details.contains(&("api_key".into(), "[redacted]".into())));
|
||||
assert!(details.contains(&("custom".into(), "[redacted]".into())));
|
||||
assert!(details.contains(&("plain".into(), "visible".into())));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn budget_warning_identifies_exceeded_limits() {
|
||||
let warning = budget_warning(
|
||||
ContextSize {
|
||||
characters: 10,
|
||||
estimated_tokens: 3,
|
||||
},
|
||||
ContextBudget {
|
||||
max_characters: Some(5),
|
||||
max_estimated_tokens: Some(2),
|
||||
},
|
||||
&ContextPruneResult {
|
||||
included_ids: vec!["ctx-002".into()],
|
||||
removed_ids: vec!["ctx-001".into()],
|
||||
final_size: ContextSize {
|
||||
characters: 4,
|
||||
estimated_tokens: 1,
|
||||
},
|
||||
over_budget: false,
|
||||
},
|
||||
);
|
||||
|
||||
assert!(warning.contains("characters 10 > 5"));
|
||||
assert!(warning.contains("estimated_tokens 3 > 2"));
|
||||
assert!(warning.contains("removed: ctx-001"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn context_errors_render_user_facing_messages() {
|
||||
assert_eq!(
|
||||
|
|
@ -1018,6 +1388,7 @@ mod tests {
|
|||
"manual".to_string(),
|
||||
"file".to_string(),
|
||||
"command_output".to_string(),
|
||||
"stdin".to_string(),
|
||||
"directory_summary".to_string()
|
||||
]
|
||||
);
|
||||
|
|
@ -1141,6 +1512,29 @@ mod tests {
|
|||
assert_eq!(stderr.content, "stderr:\nfailed");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn stdin_provider_records_stdin_provenance_without_guessing_command() {
|
||||
let entry = StdinContextProvider
|
||||
.collect(ContextProviderRequest {
|
||||
content: Some("piped text".into()),
|
||||
cwd: Some(PathBuf::from("/repo")),
|
||||
..ContextProviderRequest::default()
|
||||
})
|
||||
.expect("stdin context");
|
||||
|
||||
assert_eq!(entry.kind, ContextKind::CommandOutput);
|
||||
assert_eq!(entry.provenance.origin, ContextOrigin::Stdin);
|
||||
assert_eq!(entry.provenance.command, None);
|
||||
assert_eq!(entry.provenance.cwd, Some(PathBuf::from("/repo")));
|
||||
assert_eq!(
|
||||
entry
|
||||
.provenance
|
||||
.provider_details
|
||||
.get("upstream_command_known"),
|
||||
Some(&"false".to_string())
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn directory_summary_provider_skips_noisy_paths_and_truncates() {
|
||||
let tempdir = tempfile::tempdir().expect("tempdir");
|
||||
|
|
|
|||
22
src/main.rs
22
src/main.rs
|
|
@ -8,6 +8,8 @@ mod repl;
|
|||
mod shell;
|
||||
mod transcripts;
|
||||
|
||||
use std::io::{IsTerminal, Read};
|
||||
|
||||
use crate::app::{App, CliOptions};
|
||||
use crate::config::Config;
|
||||
use crate::providers::openai_compatible::OpenAiCompatibleProvider;
|
||||
|
|
@ -32,7 +34,25 @@ async fn run() -> Result<(), app::AppError> {
|
|||
let mut config = Config::load(options.config_path.as_deref())?;
|
||||
config.apply_cli_overrides(&options)?;
|
||||
let provider = OpenAiCompatibleProvider::from_config(&config)?;
|
||||
let app = App::new(config, Box::new(provider));
|
||||
let mut app = App::new(config, Box::new(provider));
|
||||
|
||||
for note in options.context_notes {
|
||||
println!("{}", app.add_note_context(note)?);
|
||||
}
|
||||
|
||||
for path in options.context_files {
|
||||
println!("{}", app.add_file_context(path)?);
|
||||
}
|
||||
|
||||
if !std::io::stdin().is_terminal() {
|
||||
let mut piped = String::new();
|
||||
std::io::stdin()
|
||||
.read_to_string(&mut piped)
|
||||
.map_err(|error| app::AppError::Repl(crate::repl::ReplError::Io(error)))?;
|
||||
if !piped.trim().is_empty() {
|
||||
println!("{}", app.add_stdin_context(piped)?);
|
||||
}
|
||||
}
|
||||
|
||||
Repl::new(app).run().await
|
||||
}
|
||||
|
|
|
|||
27
src/repl.rs
27
src/repl.rs
|
|
@ -35,6 +35,27 @@ impl Repl {
|
|||
break;
|
||||
}
|
||||
|
||||
if input == "/add-output" {
|
||||
let output = read_multiline_with_prompt("paste command output")?;
|
||||
match self.app.add_command_output(output, None, None) {
|
||||
Ok(message) => println!("{message}"),
|
||||
Err(error) => eprintln!("context command failed: {error}"),
|
||||
}
|
||||
continue;
|
||||
}
|
||||
|
||||
if input.starts_with("/context")
|
||||
|| input.starts_with("/add-note ")
|
||||
|| input.starts_with("/add-file ")
|
||||
|| input.starts_with("/add-dir ")
|
||||
{
|
||||
match self.app.handle_command(&input) {
|
||||
Ok(message) => println!("{message}"),
|
||||
Err(error) => eprintln!("context command failed: {error}"),
|
||||
}
|
||||
continue;
|
||||
}
|
||||
|
||||
let input = if input == "/multi" {
|
||||
read_multiline()?
|
||||
} else {
|
||||
|
|
@ -71,7 +92,11 @@ fn read_input() -> Result<Option<String>, ReplError> {
|
|||
}
|
||||
|
||||
fn read_multiline() -> Result<String, ReplError> {
|
||||
println!("multi-line input; finish with a single '.' line");
|
||||
read_multiline_with_prompt("multi-line input")
|
||||
}
|
||||
|
||||
fn read_multiline_with_prompt(prompt: &str) -> Result<String, ReplError> {
|
||||
println!("{prompt}; finish with a single '.' line");
|
||||
|
||||
let mut lines = Vec::new();
|
||||
loop {
|
||||
|
|
|
|||
|
|
@ -2,6 +2,7 @@ use std::fs;
|
|||
use std::path::{Path, PathBuf};
|
||||
use std::time::{SystemTime, UNIX_EPOCH};
|
||||
|
||||
use crate::context::{ContextEntry, redacted_provider_details};
|
||||
use crate::shell::ShellFamily;
|
||||
|
||||
#[derive(Debug, Clone)]
|
||||
|
|
@ -39,6 +40,28 @@ impl Transcript {
|
|||
.push(TranscriptEntry::Error(content.to_string()));
|
||||
}
|
||||
|
||||
pub fn record_context_event(&mut self, action: &str, entry: &ContextEntry, note: &str) {
|
||||
self.entries.push(TranscriptEntry::ContextEvent {
|
||||
action: action.to_string(),
|
||||
id: entry.id.clone(),
|
||||
kind: entry.kind.to_string(),
|
||||
title: entry.title.clone(),
|
||||
enabled: entry.enabled,
|
||||
pinned: entry.pinned,
|
||||
priority: entry.priority.to_string(),
|
||||
characters: entry.size.characters,
|
||||
estimated_tokens: entry.size.estimated_tokens,
|
||||
origin: entry.provenance.origin.to_string(),
|
||||
provider_details: redacted_provider_details(&entry.provenance),
|
||||
note: note.to_string(),
|
||||
});
|
||||
}
|
||||
|
||||
pub fn record_budget_warning(&mut self, warning: &str) {
|
||||
self.entries
|
||||
.push(TranscriptEntry::BudgetWarning(warning.to_string()));
|
||||
}
|
||||
|
||||
pub fn write_to_dir(&self, directory: &Path) -> Result<PathBuf, TranscriptError> {
|
||||
fs::create_dir_all(directory).map_err(|error| TranscriptError::CreateDir {
|
||||
path: directory.to_path_buf(),
|
||||
|
|
@ -85,6 +108,42 @@ impl Transcript {
|
|||
markdown.push_str(content);
|
||||
markdown.push_str("\n\n");
|
||||
}
|
||||
TranscriptEntry::ContextEvent {
|
||||
action,
|
||||
id,
|
||||
kind,
|
||||
title,
|
||||
enabled,
|
||||
pinned,
|
||||
priority,
|
||||
characters,
|
||||
estimated_tokens,
|
||||
origin,
|
||||
provider_details,
|
||||
note,
|
||||
} => {
|
||||
markdown.push_str("## Context Event\n\n");
|
||||
markdown.push_str(&format!("- action: `{action}`\n"));
|
||||
markdown.push_str(&format!("- id: `{id}`\n"));
|
||||
markdown.push_str(&format!("- type: `{kind}`\n"));
|
||||
markdown.push_str(&format!("- title: `{title}`\n"));
|
||||
markdown.push_str(&format!("- enabled: `{enabled}`\n"));
|
||||
markdown.push_str(&format!("- pinned: `{pinned}`\n"));
|
||||
markdown.push_str(&format!("- priority: `{priority}`\n"));
|
||||
markdown.push_str(&format!(
|
||||
"- size: `{characters}` chars / `~{estimated_tokens}` tokens\n"
|
||||
));
|
||||
markdown.push_str(&format!("- origin: `{origin}`\n"));
|
||||
for (key, value) in provider_details {
|
||||
markdown.push_str(&format!("- {key}: `{value}`\n"));
|
||||
}
|
||||
markdown.push_str(&format!("- note: `{note}`\n\n"));
|
||||
}
|
||||
TranscriptEntry::BudgetWarning(content) => {
|
||||
markdown.push_str("## Context Budget Warning\n\n");
|
||||
markdown.push_str(content);
|
||||
markdown.push_str("\n\n");
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
|
|
@ -97,6 +156,21 @@ enum TranscriptEntry {
|
|||
User(String),
|
||||
Assistant(String),
|
||||
Error(String),
|
||||
ContextEvent {
|
||||
action: String,
|
||||
id: String,
|
||||
kind: String,
|
||||
title: String,
|
||||
enabled: bool,
|
||||
pinned: bool,
|
||||
priority: String,
|
||||
characters: usize,
|
||||
estimated_tokens: usize,
|
||||
origin: String,
|
||||
provider_details: Vec<(String, String)>,
|
||||
note: String,
|
||||
},
|
||||
BudgetWarning(String),
|
||||
}
|
||||
|
||||
fn unix_millis() -> u128 {
|
||||
|
|
@ -125,6 +199,7 @@ pub enum TranscriptError {
|
|||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use crate::context::{ContextEntry, ContextKind, ContextProvenance};
|
||||
|
||||
#[test]
|
||||
fn writes_markdown_transcript() {
|
||||
|
|
@ -151,4 +226,40 @@ mod tests {
|
|||
assert!(contents.contains("## Assistant"));
|
||||
assert!(contents.contains("## Error"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn context_events_record_metadata_and_redact_provider_details() {
|
||||
let tempdir = tempfile::tempdir().expect("tempdir");
|
||||
let mut transcript = Transcript::new(
|
||||
"test-provider".into(),
|
||||
"test-model".into(),
|
||||
ShellFamily::Posix,
|
||||
);
|
||||
let mut provenance = ContextProvenance::manual();
|
||||
provenance
|
||||
.provider_details
|
||||
.insert("api_key".into(), "secret".into());
|
||||
let entry = ContextEntry::new(
|
||||
"ctx-001",
|
||||
ContextKind::Manual,
|
||||
"manual",
|
||||
provenance,
|
||||
"payload should not appear in context event",
|
||||
);
|
||||
|
||||
transcript.record_context_event("add", &entry, "added");
|
||||
transcript.record_budget_warning("context budget exceeded");
|
||||
|
||||
let path = transcript
|
||||
.write_to_dir(tempdir.path())
|
||||
.expect("transcript writes");
|
||||
let contents = fs::read_to_string(path).expect("read transcript");
|
||||
|
||||
assert!(contents.contains("## Context Event"));
|
||||
assert!(contents.contains("action: `add`"));
|
||||
assert!(contents.contains("id: `ctx-001`"));
|
||||
assert!(contents.contains("api_key: `[redacted]`"));
|
||||
assert!(!contents.contains("payload should not appear"));
|
||||
assert!(contents.contains("## Context Budget Warning"));
|
||||
}
|
||||
}
|
||||
|
|
|
|||
Loading…
Reference in New Issue
Block a user