super-man

Trilltechnician/super-man

Fork 0

Commit Graph

Author	SHA1	Message	Date
leetcrypt	220fc74eb6	test(ai): add 25-query benchmark harness + results for ai mode models Adds a repeatable AI-mode benchmark (ai-benchmark.sh) covering 25 queries across 8 categories and 3 difficulty tiers, plus captured latency results and manual correctness grades for qwen2.5-coder:1.5b, qwen2.5:3b, and westenfelder/NL2SH. Headline finding: on the larger suite NL2SH is most accurate (72% vs coder 40%, 3b 32%) and the only model with non-zero hard-task accuracy (50% vs coder 0%), while qwen2.5-coder is ~2x faster (median 3.0s vs 6.3s). This reverses the earlier 7-query result that favored qwen-coder overall. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>	2026-06-09 15:53:24 -07:00
leetcrypt	2fb20b0682	refactor: rebrand bash-helper to super-man + ESC returns straight to menu Rename bash-helper.sh -> super-man.sh and update all docs/tests to the super-man name and alias. In interactive mode, pressing Esc in the flag browser now returns directly to the home menu, removing the intermediary "Press Enter to search another command" prompt. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>	2026-06-05 20:52:18 -07:00
Trilltechnician	3a07f01b97	feat(testing): Add comprehensive testing suite with performance analysis ## Overview Add complete testing infrastructure for all Bash Buddy operating modes with automated testing, performance benchmarking, and detailed reporting. ## Features Added ### Test Framework - Modular testing utilities (test-framework.sh) - Performance timing and benchmarking - JSON/Markdown/HTML report generation - Color-coded output and statistics ### Test Suites 1. Non-Interactive Modes (test-all-modes.sh) - 45+ test cases covering all modes - Help, ask, task, category, explain, direct lookup, AI - Error handling and edge cases - Performance benchmarks (10 iterations) 2. Interactive Mode (test-interactive-mode.sh) - Automated testing with expect - Command selection and filtering - Startup performance analysis 3. Performance Analysis (analyze-performance.sh) - Historical comparison - Benchmark aggregation - Report generation (MD, HTML, JSON) ### Master Runner - run-all-tests.sh: One-command test execution - Pre-flight checks - CI/CD integration - Comprehensive reporting ## Test Coverage - ✅ All 10 operating modes - ✅ 45+ test cases - ✅ Performance benchmarks - ✅ Error conditions - ✅ Interactive automation ## Performance Results All targets met or exceeded: - Database search: ~50ms (target <100ms) - Category browse: ~30ms (target <100ms) - Direct lookup: ~100ms (target <200ms) - Interactive: ~50ms (target <100ms) ## Documentation - Comprehensive README.md in tests/ - TESTING-SUITE.md with full PR details - CI/CD integration examples - Troubleshooting guide ## Usage ```bash cd tests/ ./run-all-tests.sh ``` ## Impact - No breaking changes - Pure addition in tests/ directory - Opt-in testing infrastructure - CI/CD ready Closes: Testing infrastructure requirement Related: v2.1.0 release	2025-10-28 00:32:22 -07:00

Author

SHA1

Message

Date

leetcrypt

220fc74eb6

test(ai): add 25-query benchmark harness + results for ai mode models

Adds a repeatable AI-mode benchmark (ai-benchmark.sh) covering 25 queries across
8 categories and 3 difficulty tiers, plus captured latency results and manual
correctness grades for qwen2.5-coder:1.5b, qwen2.5:3b, and westenfelder/NL2SH.

Headline finding: on the larger suite NL2SH is most accurate (72% vs coder 40%,
3b 32%) and the only model with non-zero hard-task accuracy (50% vs coder 0%),
while qwen2.5-coder is ~2x faster (median 3.0s vs 6.3s). This reverses the
earlier 7-query result that favored qwen-coder overall.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

2026-06-09 15:53:24 -07:00

leetcrypt

2fb20b0682

refactor: rebrand bash-helper to super-man + ESC returns straight to menu

Rename bash-helper.sh -> super-man.sh and update all docs/tests to the
super-man name and alias. In interactive mode, pressing Esc in the flag
browser now returns directly to the home menu, removing the intermediary
"Press Enter to search another command" prompt.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

2026-06-05 20:52:18 -07:00

Trilltechnician

3a07f01b97

feat(testing): Add comprehensive testing suite with performance analysis

## Overview
Add complete testing infrastructure for all Bash Buddy operating modes
with automated testing, performance benchmarking, and detailed reporting.

## Features Added

### Test Framework
- Modular testing utilities (test-framework.sh)
- Performance timing and benchmarking
- JSON/Markdown/HTML report generation
- Color-coded output and statistics

### Test Suites
1. Non-Interactive Modes (test-all-modes.sh)
   - 45+ test cases covering all modes
   - Help, ask, task, category, explain, direct lookup, AI
   - Error handling and edge cases
   - Performance benchmarks (10 iterations)

2. Interactive Mode (test-interactive-mode.sh)
   - Automated testing with expect
   - Command selection and filtering
   - Startup performance analysis

3. Performance Analysis (analyze-performance.sh)
   - Historical comparison
   - Benchmark aggregation
   - Report generation (MD, HTML, JSON)

### Master Runner
- run-all-tests.sh: One-command test execution
- Pre-flight checks
- CI/CD integration
- Comprehensive reporting

## Test Coverage
- ✅ All 10 operating modes
- ✅ 45+ test cases
- ✅ Performance benchmarks
- ✅ Error conditions
- ✅ Interactive automation

## Performance Results
All targets met or exceeded:
- Database search: ~50ms (target <100ms)
- Category browse: ~30ms (target <100ms)
- Direct lookup: ~100ms (target <200ms)
- Interactive: ~50ms (target <100ms)

## Documentation
- Comprehensive README.md in tests/
- TESTING-SUITE.md with full PR details
- CI/CD integration examples
- Troubleshooting guide

## Usage
```bash
cd tests/
./run-all-tests.sh
```

## Impact
- No breaking changes
- Pure addition in tests/ directory
- Opt-in testing infrastructure
- CI/CD ready

Closes: Testing infrastructure requirement
Related: v2.1.0 release

2025-10-28 00:32:22 -07:00

3 Commits