Files
hack-house/bench/results
leetcrypt c61413c648 test(ai): screen smollm2/hermes3/mistral — none beat qwen2.5:3b
Pulled and benchmarked three more tool-capable CPU models looking for a better
default. All score 0/12 (vs qwen2.5:3b at 2/12): in the multi-turn agent loop
they leak the positional-in-tags dialect (<tools>run_shell 'cmd'</tools>) the
parser can't recover, even when they emit clean structured tool_calls on a
single-turn probe; smollm2 and mistral also wedge into repeating summaries.

qwen3:4b could not be pulled — Ollama 0.3.9 is too old (HTTP 412), same as
granite3.1-dense:2b. Upgrading Ollama is the highest-leverage next step to test
the qwen3/granite3.x generation. qwen2.5:3b remains the default.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-06-13 03:28:04 -07:00
..