leetcrypt
f4060a25d3
Verified grades by running the actual generated commands in a sandbox. NL2SH's q09 (sum 3rd CSV column) was marked wrong on paper but actually outputs the correct value (60) despite an over-engineered pipeline; regraded wrong->partial. NL2SH weighted accuracy 78%->80%, hard-tier 50%->58%; correct count unchanged. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2.5 KiB
2.5 KiB
| 1 | model | id | difficulty | verdict |
|---|---|---|---|---|
| 2 | qwen2.5-coder:1.5b | q01 | easy | wrong |
| 3 | qwen2.5-coder:1.5b | q02 | easy | correct |
| 4 | qwen2.5-coder:1.5b | q03 | medium | partial |
| 5 | qwen2.5-coder:1.5b | q04 | medium | correct |
| 6 | qwen2.5-coder:1.5b | q05 | hard | wrong |
| 7 | qwen2.5-coder:1.5b | q06 | easy | correct |
| 8 | qwen2.5-coder:1.5b | q07 | medium | correct |
| 9 | qwen2.5-coder:1.5b | q08 | medium | wrong |
| 10 | qwen2.5-coder:1.5b | q09 | hard | wrong |
| 11 | qwen2.5-coder:1.5b | q10 | easy | correct |
| 12 | qwen2.5-coder:1.5b | q11 | medium | correct |
| 13 | qwen2.5-coder:1.5b | q12 | hard | wrong |
| 14 | qwen2.5-coder:1.5b | q13 | easy | correct |
| 15 | qwen2.5-coder:1.5b | q14 | medium | wrong |
| 16 | qwen2.5-coder:1.5b | q15 | hard | wrong |
| 17 | qwen2.5-coder:1.5b | q16 | easy | partial |
| 18 | qwen2.5-coder:1.5b | q17 | medium | wrong |
| 19 | qwen2.5-coder:1.5b | q18 | hard | wrong |
| 20 | qwen2.5-coder:1.5b | q19 | easy | correct |
| 21 | qwen2.5-coder:1.5b | q20 | medium | wrong |
| 22 | qwen2.5-coder:1.5b | q21 | easy | correct |
| 23 | qwen2.5-coder:1.5b | q22 | medium | wrong |
| 24 | qwen2.5-coder:1.5b | q23 | easy | partial |
| 25 | qwen2.5-coder:1.5b | q24 | medium | correct |
| 26 | qwen2.5-coder:1.5b | q25 | hard | wrong |
| 27 | qwen2.5:3b | q01 | easy | wrong |
| 28 | qwen2.5:3b | q02 | easy | partial |
| 29 | qwen2.5:3b | q03 | medium | correct |
| 30 | qwen2.5:3b | q04 | medium | wrong |
| 31 | qwen2.5:3b | q05 | hard | wrong |
| 32 | qwen2.5:3b | q06 | easy | wrong |
| 33 | qwen2.5:3b | q07 | medium | correct |
| 34 | qwen2.5:3b | q08 | medium | correct |
| 35 | qwen2.5:3b | q09 | hard | wrong |
| 36 | qwen2.5:3b | q10 | easy | partial |
| 37 | qwen2.5:3b | q11 | medium | correct |
| 38 | qwen2.5:3b | q12 | hard | wrong |
| 39 | qwen2.5:3b | q13 | easy | correct |
| 40 | qwen2.5:3b | q14 | medium | correct |
| 41 | qwen2.5:3b | q15 | hard | wrong |
| 42 | qwen2.5:3b | q16 | easy | correct |
| 43 | qwen2.5:3b | q17 | medium | wrong |
| 44 | qwen2.5:3b | q18 | hard | correct |
| 45 | qwen2.5:3b | q19 | easy | wrong |
| 46 | qwen2.5:3b | q20 | medium | wrong |
| 47 | qwen2.5:3b | q21 | easy | partial |
| 48 | qwen2.5:3b | q22 | medium | wrong |
| 49 | qwen2.5:3b | q23 | easy | partial |
| 50 | qwen2.5:3b | q24 | medium | wrong |
| 51 | qwen2.5:3b | q25 | hard | wrong |
| 52 | westenfelder/NL2SH | q01 | easy | correct |
| 53 | westenfelder/NL2SH | q02 | easy | partial |
| 54 | westenfelder/NL2SH | q03 | medium | correct |
| 55 | westenfelder/NL2SH | q04 | medium | correct |
| 56 | westenfelder/NL2SH | q05 | hard | correct |
| 57 | westenfelder/NL2SH | q06 | easy | correct |
| 58 | westenfelder/NL2SH | q07 | medium | partial |
| 59 | westenfelder/NL2SH | q08 | medium | wrong |
| 60 | westenfelder/NL2SH | q09 | hard | partial |
| 61 | westenfelder/NL2SH | q10 | easy | correct |
| 62 | westenfelder/NL2SH | q11 | medium | correct |
| 63 | westenfelder/NL2SH | q12 | hard | correct |
| 64 | westenfelder/NL2SH | q13 | easy | correct |
| 65 | westenfelder/NL2SH | q14 | medium | correct |
| 66 | westenfelder/NL2SH | q15 | hard | wrong |
| 67 | westenfelder/NL2SH | q16 | easy | correct |
| 68 | westenfelder/NL2SH | q17 | medium | correct |
| 69 | westenfelder/NL2SH | q18 | hard | correct |
| 70 | westenfelder/NL2SH | q19 | easy | correct |
| 71 | westenfelder/NL2SH | q20 | medium | correct |
| 72 | westenfelder/NL2SH | q21 | easy | correct |
| 73 | westenfelder/NL2SH | q22 | medium | partial |
| 74 | westenfelder/NL2SH | q23 | easy | correct |
| 75 | westenfelder/NL2SH | q24 | medium | correct |
| 76 | westenfelder/NL2SH | q25 | hard | wrong |