| benchmark_id | size | models | SE(A) | SE(A-B) | no_solve | tau- | details |
|---|---|---|---|---|---|---|---|
| agi_english_step02500 | 2546 | 10 | 0.81 | 0.96 | 35 | 30 | models | examples | data | more |
| agi_english_step27500 | 2546 | 10 | 0.83 | 1.1 | 26 | 36 | models | examples | data | more |
| agi_english_step50000 | 2546 | 10 | 0.84 | 1.1 | 26 | 33 | models | examples | data | more |
| arc_challenge_step02500 | 1165 | 10 | 1.3 | 0.91 | 61 | 10 | models | examples | data | more |
| arc_challenge_step27500 | 1165 | 10 | 1.4 | 1.1 | 44 | 14 | models | examples | data | more |
| arc_challenge_step50000 | 1165 | 10 | 1.4 | 1.1 | 41 | 16 | models | examples | data | more |
| gsm8k_step02500 | 1319 | 10 | 0.33 | 0.4 | 93 | 2.5 | models | examples | data | more |
| gsm8k_step27500 | 1319 | 10 | 0.45 | 0.6 | 82 | 6 | models | examples | data | more |
| gsm8k_step50000 | 1319 | 10 | 0.54 | 0.68 | 79 | 7.9 | models | examples | data | more |
| hellaswag_step02500 | 10042 | 10 | 0.48 | 0.3 | 50 | 11 | models | examples | data | more |
| hellaswag_step27500 | 10042 | 10 | 0.47 | 0.28 | 24 | 9.6 | models | examples | data | more |
| hellaswag_step50000 | 10042 | 10 | 0.46 | 0.26 | 21 | 8.8 | models | examples | data | more |
| humaneval_step02500 | 164 | 10 | 0.64 | 0.74 | 97 | 0 | models | examples | data | more |
| humaneval_step27500 | 164 | 10 | 2.1 | 1.6 | 84 | 3.7 | models | examples | data | more |
| humaneval_step50000 | 164 | 10 | 2.5 | 1.8 | 80 | 2.4 | models | examples | data | more |
| nq_step02500 | 3610 | 10 | 0.22 | 0.23 | 94 | 1.8 | models | examples | data | more |
| nq_step27500 | 3610 | 10 | 0.56 | 0.51 | 71 | 10 | models | examples | data | more |
| nq_step50000 | 3610 | 10 | 0.62 | 0.54 | 67 | 11 | models | examples | data | more |
| piqa_step02500 | 1838 | 10 | 1.1 | 0.76 | 22 | 13 | models | examples | data | more |
| piqa_step27500 | 1838 | 10 | 1 | 0.68 | 15 | 9.2 | models | examples | data | more |
| piqa_step50000 | 1838 | 10 | 0.98 | 0.65 | 15 | 9 | models | examples | data | more |
| siqa_step02500 | 1954 | 10 | 1.1 | 0.66 | 46 | 9.6 | models | examples | data | more |
| siqa_step27500 | 1954 | 10 | 1.1 | 0.68 | 42 | 9.5 | models | examples | data | more |
| siqa_step50000 | 1954 | 10 | 1.1 | 0.67 | 42 | 10 | models | examples | data | more |
| tqa_step02500 | 11313 | 10 | 0.19 | 0.19 | 88 | 4 | models | examples | data | more |
| tqa_step27500 | 11313 | 10 | 0.45 | 0.35 | 46 | 16 | models | examples | data | more |
| tqa_step50000 | 11313 | 10 | 0.47 | 0.35 | 38 | 16 | models | examples | data | more |
Generated on 2026-10-13 22:06:46