Adds a repeatable AI-mode benchmark (ai-benchmark.sh) covering 25 queries across
8 categories and 3 difficulty tiers, plus captured latency results and manual
correctness grades for qwen2.5-coder:1.5b, qwen2.5:3b, and westenfelder/NL2SH.
Headline finding: on the larger suite NL2SH is most accurate (72% vs coder 40%,
3b 32%) and the only model with non-zero hard-task accuracy (50% vs coder 0%),
while qwen2.5-coder is ~2x faster (median 3.0s vs 6.3s). This reverses the
earlier 7-query result that favored qwen-coder overall.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
leetcrypt
220fc74eb6