Verified grades by running the actual generated commands in a sandbox. NL2SH's
q09 (sum 3rd CSV column) was marked wrong on paper but actually outputs the
correct value (60) despite an over-engineered pipeline; regraded wrong->partial.
NL2SH weighted accuracy 78%->80%, hard-tier 50%->58%; correct count unchanged.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Adds a repeatable AI-mode benchmark (ai-benchmark.sh) covering 25 queries across
8 categories and 3 difficulty tiers, plus captured latency results and manual
correctness grades for qwen2.5-coder:1.5b, qwen2.5:3b, and westenfelder/NL2SH.
Headline finding: on the larger suite NL2SH is most accurate (72% vs coder 40%,
3b 32%) and the only model with non-zero hard-task accuracy (50% vs coder 0%),
while qwen2.5-coder is ~2x faster (median 3.0s vs 6.3s). This reverses the
earlier 7-query result that favored qwen-coder overall.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Rename bash-helper.sh -> super-man.sh and update all docs/tests to the
super-man name and alias. In interactive mode, pressing Esc in the flag
browser now returns directly to the home menu, removing the intermediary
"Press Enter to search another command" prompt.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
leetcrypt
Trilltechnician