leetcrypt
f4060a25d3
Verified grades by running the actual generated commands in a sandbox. NL2SH's q09 (sum 3rd CSV column) was marked wrong on paper but actually outputs the correct value (60) despite an over-engineered pipeline; regraded wrong->partial. NL2SH weighted accuracy 78%->80%, hard-tier 50%->58%; correct count unchanged. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
77 lines
2.5 KiB
CSV
77 lines
2.5 KiB
CSV
model|id|difficulty|verdict
|
|
qwen2.5-coder:1.5b|q01|easy|wrong
|
|
qwen2.5-coder:1.5b|q02|easy|correct
|
|
qwen2.5-coder:1.5b|q03|medium|partial
|
|
qwen2.5-coder:1.5b|q04|medium|correct
|
|
qwen2.5-coder:1.5b|q05|hard|wrong
|
|
qwen2.5-coder:1.5b|q06|easy|correct
|
|
qwen2.5-coder:1.5b|q07|medium|correct
|
|
qwen2.5-coder:1.5b|q08|medium|wrong
|
|
qwen2.5-coder:1.5b|q09|hard|wrong
|
|
qwen2.5-coder:1.5b|q10|easy|correct
|
|
qwen2.5-coder:1.5b|q11|medium|correct
|
|
qwen2.5-coder:1.5b|q12|hard|wrong
|
|
qwen2.5-coder:1.5b|q13|easy|correct
|
|
qwen2.5-coder:1.5b|q14|medium|wrong
|
|
qwen2.5-coder:1.5b|q15|hard|wrong
|
|
qwen2.5-coder:1.5b|q16|easy|partial
|
|
qwen2.5-coder:1.5b|q17|medium|wrong
|
|
qwen2.5-coder:1.5b|q18|hard|wrong
|
|
qwen2.5-coder:1.5b|q19|easy|correct
|
|
qwen2.5-coder:1.5b|q20|medium|wrong
|
|
qwen2.5-coder:1.5b|q21|easy|correct
|
|
qwen2.5-coder:1.5b|q22|medium|wrong
|
|
qwen2.5-coder:1.5b|q23|easy|partial
|
|
qwen2.5-coder:1.5b|q24|medium|correct
|
|
qwen2.5-coder:1.5b|q25|hard|wrong
|
|
qwen2.5:3b|q01|easy|wrong
|
|
qwen2.5:3b|q02|easy|partial
|
|
qwen2.5:3b|q03|medium|correct
|
|
qwen2.5:3b|q04|medium|wrong
|
|
qwen2.5:3b|q05|hard|wrong
|
|
qwen2.5:3b|q06|easy|wrong
|
|
qwen2.5:3b|q07|medium|correct
|
|
qwen2.5:3b|q08|medium|correct
|
|
qwen2.5:3b|q09|hard|wrong
|
|
qwen2.5:3b|q10|easy|partial
|
|
qwen2.5:3b|q11|medium|correct
|
|
qwen2.5:3b|q12|hard|wrong
|
|
qwen2.5:3b|q13|easy|correct
|
|
qwen2.5:3b|q14|medium|correct
|
|
qwen2.5:3b|q15|hard|wrong
|
|
qwen2.5:3b|q16|easy|correct
|
|
qwen2.5:3b|q17|medium|wrong
|
|
qwen2.5:3b|q18|hard|correct
|
|
qwen2.5:3b|q19|easy|wrong
|
|
qwen2.5:3b|q20|medium|wrong
|
|
qwen2.5:3b|q21|easy|partial
|
|
qwen2.5:3b|q22|medium|wrong
|
|
qwen2.5:3b|q23|easy|partial
|
|
qwen2.5:3b|q24|medium|wrong
|
|
qwen2.5:3b|q25|hard|wrong
|
|
westenfelder/NL2SH|q01|easy|correct
|
|
westenfelder/NL2SH|q02|easy|partial
|
|
westenfelder/NL2SH|q03|medium|correct
|
|
westenfelder/NL2SH|q04|medium|correct
|
|
westenfelder/NL2SH|q05|hard|correct
|
|
westenfelder/NL2SH|q06|easy|correct
|
|
westenfelder/NL2SH|q07|medium|partial
|
|
westenfelder/NL2SH|q08|medium|wrong
|
|
westenfelder/NL2SH|q09|hard|partial
|
|
westenfelder/NL2SH|q10|easy|correct
|
|
westenfelder/NL2SH|q11|medium|correct
|
|
westenfelder/NL2SH|q12|hard|correct
|
|
westenfelder/NL2SH|q13|easy|correct
|
|
westenfelder/NL2SH|q14|medium|correct
|
|
westenfelder/NL2SH|q15|hard|wrong
|
|
westenfelder/NL2SH|q16|easy|correct
|
|
westenfelder/NL2SH|q17|medium|correct
|
|
westenfelder/NL2SH|q18|hard|correct
|
|
westenfelder/NL2SH|q19|easy|correct
|
|
westenfelder/NL2SH|q20|medium|correct
|
|
westenfelder/NL2SH|q21|easy|correct
|
|
westenfelder/NL2SH|q22|medium|partial
|
|
westenfelder/NL2SH|q23|easy|correct
|
|
westenfelder/NL2SH|q24|medium|correct
|
|
westenfelder/NL2SH|q25|hard|wrong
|