There are 2385 examples not solved by any model.
Solving some of these can be a good signal that your model is indeed better than leading models if these are good problems.
hellaswag/100, hellaswag/10001, hellaswag/10004, hellaswag/10009, hellaswag/10011, hellaswag/10016, hellaswag/10018, hellaswag/10020, hellaswag/10022, hellaswag/10026, hellaswag/10027, hellaswag/10029, hellaswag/1003, hellaswag/10030, hellaswag/10032, hellaswag/1005, hellaswag/1006, hellaswag/1008, hellaswag/101, hellaswag/1012, hellaswag/1015, hellaswag/1016, hellaswag/102, hellaswag/1022, hellaswag/1026, hellaswag/1029, hellaswag/1032, hellaswag/1038, hellaswag/1046, hellaswag/1047, hellaswag/1055, hellaswag/1057, hellaswag/1059, hellaswag/1063, hellaswag/1067, hellaswag/1071, hellaswag/1073, hellaswag/1074, hellaswag/1075, hellaswag/1078, hellaswag/1082, hellaswag/1085, hellaswag/1087, hellaswag/1088, hellaswag/109, hellaswag/1099, hellaswag/11, hellaswag/1100, hellaswag/1108, hellaswag/1109, hellaswag/1112, hellaswag/1115, hellaswag/1125, hellaswag/1126, hellaswag/1140, hellaswag/1146, hellaswag/115, hellaswag/1151, hellaswag/1163, hellaswag/1164, hellaswag/1165, hellaswag/1168, hellaswag/1176, hellaswag/1177, hellaswag/1178, hellaswag/118, hellaswag/1181, hellaswag/1184, hellaswag/1193, hellaswag/12, hellaswag/1200, hellaswag/1201, hellaswag/1202, hellaswag/1205, hellaswag/1210, hellaswag/1211, hellaswag/1212, hellaswag/1213, hellaswag/1215, hellaswag/1216, hellaswag/1217, hellaswag/1224, hellaswag/1227, hellaswag/123, hellaswag/1235, hellaswag/1237, hellaswag/1239, hellaswag/124, hellaswag/1244, hellaswag/1245, hellaswag/1247, hellaswag/1249, hellaswag/125, hellaswag/1252, hellaswag/1261, hellaswag/1265, hellaswag/1266, hellaswag/1271, hellaswag/1275, hellaswag/1282, hellaswag/1283, hellaswag/1285, hellaswag/1287, hellaswag/1291, hellaswag/1294, hellaswag/1304, hellaswag/131, hellaswag/1310, hellaswag/1315, hellaswag/1316, hellaswag/1321, hellaswag/1328, hellaswag/1337, hellaswag/1342, hellaswag/1345, hellaswag/1354, hellaswag/1356, hellaswag/1358, hellaswag/1364, hellaswag/1365, hellaswag/1375, hellaswag/1376, hellaswag/1379, hellaswag/1381, hellaswag/1383, hellaswag/1389, hellaswag/1390, hellaswag/1395, hellaswag/1397, hellaswag/1400, hellaswag/1401, hellaswag/1405, hellaswag/1407, hellaswag/1409, hellaswag/1412, hellaswag/1414, hellaswag/142, hellaswag/1420, hellaswag/1421, hellaswag/1424, hellaswag/1426, hellaswag/1434, hellaswag/1438, hellaswag/1440, hellaswag/1444, hellaswag/1446, hellaswag/1448, hellaswag/1449, hellaswag/145, hellaswag/1450, hellaswag/1452, hellaswag/1458, hellaswag/1460, hellaswag/1465, hellaswag/1467, hellaswag/1468, hellaswag/1469, hellaswag/1472, hellaswag/1474, hellaswag/1476, hellaswag/1477, hellaswag/148, hellaswag/1480, hellaswag/1482, hellaswag/1486, hellaswag/1487, hellaswag/1494, hellaswag/1497, hellaswag/1502, hellaswag/1505, hellaswag/1506, hellaswag/1508, hellaswag/1509, hellaswag/151, hellaswag/1510, hellaswag/1512, hellaswag/1515, hellaswag/1516, hellaswag/152, hellaswag/1525, hellaswag/1529, hellaswag/153, hellaswag/1532, hellaswag/1536, hellaswag/1538, hellaswag/1540, hellaswag/1541, hellaswag/1542, hellaswag/1543, hellaswag/1544, hellaswag/1545, hellaswag/1547, hellaswag/1549, hellaswag/1550, hellaswag/1553, hellaswag/1554, hellaswag/1555, hellaswag/1556, hellaswag/1565, hellaswag/1567, hellaswag/1569, hellaswag/1573, hellaswag/1575, hellaswag/1577, hellaswag/1579, hellaswag/1580, hellaswag/1584, hellaswag/1585, hellaswag/1603, hellaswag/1604, hellaswag/161, hellaswag/1610, hellaswag/1616, hellaswag/1617, hellaswag/1618, hellaswag/1626, hellaswag/163, hellaswag/1633, hellaswag/1640, hellaswag/1642, hellaswag/1648, hellaswag/1650, hellaswag/1652, hellaswag/1654, hellaswag/1655, hellaswag/166, hellaswag/1661, hellaswag/1662, hellaswag/1663, hellaswag/1665, hellaswag/1666, hellaswag/1670, hellaswag/1672, hellaswag/1677, hellaswag/1685, hellaswag/1687, hellaswag/1689, hellaswag/169, hellaswag/1697, hellaswag/1698, hellaswag/1699, hellaswag/1701, hellaswag/1703, hellaswag/1711, hellaswag/1712, hellaswag/1713, hellaswag/1720, hellaswag/1721, hellaswag/1723, hellaswag/1726, hellaswag/1736, hellaswag/1741, hellaswag/1743, hellaswag/1754, hellaswag/1755, hellaswag/1757, hellaswag/1761, hellaswag/1762, hellaswag/1769, hellaswag/1774, hellaswag/1775, hellaswag/1779, hellaswag/1781, hellaswag/1789, hellaswag/1791, hellaswag/1794, hellaswag/1797, hellaswag/1800, hellaswag/1806, hellaswag/1807, hellaswag/1812, hellaswag/1815, hellaswag/1819, hellaswag/1823, hellaswag/1825, hellaswag/1828, hellaswag/1829, hellaswag/183, hellaswag/1832, hellaswag/1835, hellaswag/1836, hellaswag/1838, hellaswag/1840, hellaswag/1844, hellaswag/1845, hellaswag/1853, hellaswag/1854, hellaswag/186, hellaswag/1862, hellaswag/1866, hellaswag/1867, hellaswag/1870, hellaswag/1871, hellaswag/1872, hellaswag/1874, hellaswag/1875, hellaswag/1877, hellaswag/1878, hellaswag/1881, hellaswag/1888, hellaswag/1891, hellaswag/1892, hellaswag/1893, hellaswag/1895, hellaswag/190, hellaswag/1906, hellaswag/191, hellaswag/1910, hellaswag/1912, hellaswag/1916, hellaswag/1917, hellaswag/192, hellaswag/1930, hellaswag/1941, hellaswag/1947, hellaswag/1948, hellaswag/195, hellaswag/1956, hellaswag/1959, hellaswag/1960, hellaswag/1961, hellaswag/1966, hellaswag/1970, hellaswag/1971, hellaswag/1976, hellaswag/1978, hellaswag/198, hellaswag/1980, hellaswag/1981, hellaswag/1984, hellaswag/1985, hellaswag/1987, hellaswag/1990, hellaswag/1997, hellaswag/20, hellaswag/2005, hellaswag/2011, hellaswag/2012, hellaswag/2016, hellaswag/202, hellaswag/2022, hellaswag/2027, hellaswag/2028, hellaswag/2031, hellaswag/2040, hellaswag/2046, hellaswag/2047, hellaswag/2055, hellaswag/2056, hellaswag/2058, hellaswag/2063, hellaswag/2066, hellaswag/2072, hellaswag/2074, hellaswag/2075, hellaswag/2083, hellaswag/2086, hellaswag/2087, hellaswag/2088, hellaswag/2093, hellaswag/2101, hellaswag/2104, hellaswag/2105, hellaswag/2106, hellaswag/2107, hellaswag/2108, hellaswag/2111, hellaswag/2112, hellaswag/2118, hellaswag/212, hellaswag/2126, hellaswag/2127, hellaswag/2128, hellaswag/2129, hellaswag/2134, hellaswag/2135, hellaswag/2138, hellaswag/214, hellaswag/2140, hellaswag/2143, hellaswag/2146, hellaswag/2149, hellaswag/2152, hellaswag/2162, hellaswag/2164, hellaswag/2165, hellaswag/2167, hellaswag/2173, hellaswag/2176, hellaswag/2178, hellaswag/218, hellaswag/2182, hellaswag/2183, hellaswag/2187, hellaswag/2189, hellaswag/2204, hellaswag/2206, hellaswag/2208, hellaswag/2209, hellaswag/221, hellaswag/2211, hellaswag/2216, hellaswag/2218, hellaswag/2231, hellaswag/2237, hellaswag/2238, hellaswag/224, hellaswag/2246, hellaswag/2249, hellaswag/2250, hellaswag/2251, hellaswag/2253, hellaswag/2255, hellaswag/2259, hellaswag/2262, hellaswag/2266, hellaswag/2271, hellaswag/2285, hellaswag/2287, hellaswag/2292, hellaswag/2295, hellaswag/2299, hellaswag/230, hellaswag/2305, hellaswag/2306, hellaswag/2307, hellaswag/2308, hellaswag/2312, hellaswag/2316, hellaswag/232, hellaswag/2321, hellaswag/2323, hellaswag/2325, hellaswag/233, hellaswag/234, hellaswag/2340, hellaswag/2346, hellaswag/2347, hellaswag/2348, hellaswag/2349, hellaswag/235, hellaswag/2351, hellaswag/2354, hellaswag/2355, hellaswag/2368, hellaswag/2371, hellaswag/2376, hellaswag/2379, hellaswag/239, hellaswag/2390, hellaswag/2391, hellaswag/2393, hellaswag/2396, hellaswag/2397, hellaswag/240, hellaswag/2404, hellaswag/2406, hellaswag/2409, hellaswag/2411, hellaswag/2419, hellaswag/242, hellaswag/2421, hellaswag/2424, hellaswag/2425, hellaswag/2427, hellaswag/2428, hellaswag/2429, hellaswag/2432, hellaswag/2435, hellaswag/2436, hellaswag/2437, hellaswag/2439, hellaswag/2445, hellaswag/245, hellaswag/2451, hellaswag/2454, hellaswag/2460, hellaswag/2466, hellaswag/2467, hellaswag/2470, hellaswag/248, hellaswag/2481, hellaswag/2483, hellaswag/2489, hellaswag/2495, hellaswag/2497, hellaswag/25, hellaswag/250, hellaswag/2505, hellaswag/251, hellaswag/2510, hellaswag/2511, hellaswag/2513, hellaswag/252, hellaswag/2521, hellaswag/2523, hellaswag/2526, hellaswag/2535, hellaswag/2539, hellaswag/2546, hellaswag/2549, hellaswag/2552, hellaswag/2559, hellaswag/2560, hellaswag/2561, hellaswag/2563, hellaswag/2565, hellaswag/2566, hellaswag/2575, hellaswag/2577, hellaswag/2580, hellaswag/2586, hellaswag/2589, hellaswag/2596, hellaswag/2598, hellaswag/2599, hellaswag/2600, hellaswag/2602, hellaswag/2608, hellaswag/2609, hellaswag/261, hellaswag/2617, hellaswag/2622, hellaswag/2623, hellaswag/2624, hellaswag/2632, hellaswag/2634, hellaswag/264, hellaswag/2641, hellaswag/2646, hellaswag/2656, hellaswag/2661, hellaswag/2662, hellaswag/2664, hellaswag/2665, hellaswag/2677, hellaswag/2681, hellaswag/2685, hellaswag/269, hellaswag/2690, hellaswag/2691, hellaswag/2699, hellaswag/270, hellaswag/2702, hellaswag/2703, hellaswag/2704, hellaswag/2712, hellaswag/2715, hellaswag/2719, hellaswag/2721, hellaswag/2722, hellaswag/2723, hellaswag/2733, hellaswag/2735, hellaswag/2736, hellaswag/2737, hellaswag/2738, hellaswag/2741, hellaswag/2742, hellaswag/2746, hellaswag/2749, hellaswag/2754, hellaswag/2759, hellaswag/2761, hellaswag/2762, hellaswag/2764, hellaswag/2765, hellaswag/2766, hellaswag/2767, hellaswag/2769, hellaswag/2770, hellaswag/2771, hellaswag/2774, hellaswag/2776, hellaswag/2777, hellaswag/278, hellaswag/2781, hellaswag/2783, hellaswag/2785, hellaswag/2787, hellaswag/2789, hellaswag/279, hellaswag/2792, hellaswag/2793, hellaswag/2794, hellaswag/2802, hellaswag/2805, hellaswag/2807, hellaswag/2808, hellaswag/2809, hellaswag/2813, hellaswag/2814, hellaswag/2817, hellaswag/2819, hellaswag/2821, hellaswag/2822, hellaswag/2824, hellaswag/2829, hellaswag/2833, hellaswag/2838, hellaswag/2844, hellaswag/2848, hellaswag/285, hellaswag/2851, hellaswag/2852, hellaswag/2855, hellaswag/2858, hellaswag/2859, hellaswag/286, hellaswag/2860, hellaswag/2865, hellaswag/2866, hellaswag/2878, hellaswag/2886, hellaswag/2888, hellaswag/2889, hellaswag/289, hellaswag/2891, hellaswag/2895, hellaswag/2898, hellaswag/2906, hellaswag/2908, hellaswag/2910, hellaswag/2912, hellaswag/2918, hellaswag/2921, hellaswag/2929, hellaswag/2933, hellaswag/2934, hellaswag/2938, hellaswag/2942, hellaswag/2944, hellaswag/2945, hellaswag/2950, hellaswag/2951, hellaswag/2954, hellaswag/2955, hellaswag/2956, hellaswag/296, hellaswag/2960, hellaswag/297, hellaswag/2976, hellaswag/2977, hellaswag/2981, hellaswag/2986, hellaswag/2990, hellaswag/30, hellaswag/300, hellaswag/3004, hellaswag/3005, hellaswag/3011, hellaswag/3015, hellaswag/3016, hellaswag/3019, hellaswag/302, hellaswag/3022, hellaswag/3023, hellaswag/3029, hellaswag/3032, hellaswag/3035, hellaswag/3037, hellaswag/3040, hellaswag/3041, hellaswag/3042, hellaswag/3047, hellaswag/3049, hellaswag/3059, hellaswag/306, hellaswag/3061, hellaswag/3062, hellaswag/3063, hellaswag/3065, hellaswag/3075, hellaswag/3083, hellaswag/3086, hellaswag/3095, hellaswag/3099, hellaswag/3101, hellaswag/3104, hellaswag/3106, hellaswag/3114, hellaswag/3117, hellaswag/3118, hellaswag/3122, hellaswag/3125, hellaswag/3126, hellaswag/3127, hellaswag/3133, hellaswag/3135, hellaswag/3136, hellaswag/3137, hellaswag/3138, hellaswag/3140, hellaswag/3142, hellaswag/3144, hellaswag/3145, hellaswag/3150, hellaswag/3155, hellaswag/3156, hellaswag/3160, hellaswag/3164, hellaswag/3168, hellaswag/317, hellaswag/3171, hellaswag/3176, hellaswag/3177, hellaswag/3179, hellaswag/3185, hellaswag/3189, hellaswag/3190, hellaswag/3191, hellaswag/3195, hellaswag/3202, hellaswag/3204, hellaswag/3207, hellaswag/3209, hellaswag/3212, hellaswag/322, hellaswag/3225, hellaswag/3227, hellaswag/323, hellaswag/3231, hellaswag/3233, hellaswag/3234, hellaswag/3239, hellaswag/324, hellaswag/3242, hellaswag/3244, hellaswag/3247, hellaswag/3248, hellaswag/3255, hellaswag/3266, hellaswag/327, hellaswag/3274, hellaswag/3275, hellaswag/3277, hellaswag/3279, hellaswag/3281, hellaswag/3284, hellaswag/3288, hellaswag/3291, hellaswag/3292, hellaswag/3295, hellaswag/3298, hellaswag/3299, hellaswag/33, hellaswag/3310, hellaswag/3313, hellaswag/3314, hellaswag/3315, hellaswag/3317, hellaswag/3319, hellaswag/3328, hellaswag/3331, hellaswag/3338, hellaswag/3343, hellaswag/3346, hellaswag/3348, hellaswag/3349, hellaswag/335, hellaswag/3350, hellaswag/3351, hellaswag/3352, hellaswag/3357, hellaswag/336, hellaswag/3362, hellaswag/3367, hellaswag/3377, hellaswag/338, hellaswag/3382, hellaswag/3387, hellaswag/3390, hellaswag/3392, hellaswag/3396, hellaswag/3398, hellaswag/3400, hellaswag/3401, hellaswag/3402, hellaswag/3413, hellaswag/3416, hellaswag/3417, hellaswag/3418, hellaswag/342, hellaswag/3434, hellaswag/3435, hellaswag/3439, hellaswag/344, hellaswag/3441, hellaswag/345, hellaswag/3453, hellaswag/3455, hellaswag/3457, hellaswag/3459, hellaswag/3461, hellaswag/3474, hellaswag/3477, hellaswag/3484, hellaswag/3487, hellaswag/349, hellaswag/3494, hellaswag/3498, hellaswag/3500, hellaswag/3504, hellaswag/3505, hellaswag/3508, hellaswag/351, hellaswag/3513, hellaswag/3522, hellaswag/353, hellaswag/3533, hellaswag/3537, hellaswag/354, hellaswag/3541, hellaswag/3546, hellaswag/3553, hellaswag/3554, hellaswag/3558, hellaswag/3560, hellaswag/3562, hellaswag/3567, hellaswag/357, hellaswag/3572, hellaswag/358, hellaswag/3592, hellaswag/3597, hellaswag/360, hellaswag/361, hellaswag/3611, hellaswag/3616, hellaswag/3617, hellaswag/3627, hellaswag/3633, hellaswag/3634, hellaswag/3636, hellaswag/365, hellaswag/3657, hellaswag/3659, hellaswag/3662, hellaswag/3666, hellaswag/3670, hellaswag/3675, hellaswag/368, hellaswag/3680, hellaswag/3682, hellaswag/3687, hellaswag/3689, hellaswag/3693, hellaswag/3694, hellaswag/3702, hellaswag/3703, hellaswag/3707, hellaswag/371, hellaswag/3714, hellaswag/3719, hellaswag/3721, hellaswag/3723, hellaswag/3725, hellaswag/3727, hellaswag/3740, hellaswag/3741, hellaswag/3742, hellaswag/3746, hellaswag/3747, hellaswag/3748, hellaswag/375, hellaswag/3752, hellaswag/3756, hellaswag/3758, hellaswag/3764, hellaswag/3766, hellaswag/3776, hellaswag/3780, hellaswag/3781, hellaswag/380, hellaswag/3801, hellaswag/3802, hellaswag/3810, hellaswag/3814, hellaswag/3818, hellaswag/3819, hellaswag/3826, hellaswag/3828, hellaswag/3830, hellaswag/3835, hellaswag/3836, hellaswag/384, hellaswag/3842, hellaswag/3843, hellaswag/3844, hellaswag/385, hellaswag/3861, hellaswag/3867, hellaswag/3875, hellaswag/3876, hellaswag/3877, hellaswag/389, hellaswag/3890, hellaswag/3893, hellaswag/3895, hellaswag/3897, hellaswag/3898, hellaswag/3901, hellaswag/3904, hellaswag/3908, hellaswag/3914, hellaswag/3915, hellaswag/392, hellaswag/3922, hellaswag/3926, hellaswag/393, hellaswag/3931, hellaswag/3934, hellaswag/3937, hellaswag/3939, hellaswag/3946, hellaswag/3949, hellaswag/395, hellaswag/3953, hellaswag/3954, hellaswag/3959, hellaswag/3960, hellaswag/3964, hellaswag/3965, hellaswag/3966, hellaswag/3969, hellaswag/3979, hellaswag/3980, hellaswag/3981, hellaswag/3984, hellaswag/3985, hellaswag/3989, hellaswag/3990, hellaswag/3994, hellaswag/3995, hellaswag/3997, hellaswag/3999, hellaswag/4, hellaswag/400, hellaswag/4000, hellaswag/4002, hellaswag/4007, hellaswag/401, hellaswag/4017, hellaswag/402, hellaswag/4021, hellaswag/4026, hellaswag/4027, hellaswag/4028, hellaswag/4038, hellaswag/4041, hellaswag/4043, hellaswag/4045, hellaswag/4046, hellaswag/4053, hellaswag/4054, hellaswag/4056, hellaswag/4077, hellaswag/4080, hellaswag/4082, hellaswag/410, hellaswag/4101, hellaswag/4112, hellaswag/4116, hellaswag/4119, hellaswag/4120, hellaswag/4138, hellaswag/415, hellaswag/4156, hellaswag/4157, hellaswag/4165, hellaswag/4167, hellaswag/4169, hellaswag/4173, hellaswag/4176, hellaswag/4178, hellaswag/4180, hellaswag/4182, hellaswag/4183, hellaswag/4184, hellaswag/4192, hellaswag/4193, hellaswag/4195, hellaswag/4202, hellaswag/4206, hellaswag/421, hellaswag/4211, hellaswag/4215, hellaswag/4218, hellaswag/4219, hellaswag/4222, hellaswag/4224, hellaswag/4234, hellaswag/4235, hellaswag/4236, hellaswag/4237, hellaswag/4239, hellaswag/4240, hellaswag/4250, hellaswag/427, hellaswag/4270, hellaswag/4276, hellaswag/4278, hellaswag/4285, hellaswag/4289, hellaswag/4291, hellaswag/4292, hellaswag/4305, hellaswag/4306, hellaswag/4308, hellaswag/4318, hellaswag/4323, hellaswag/434, hellaswag/4340, hellaswag/4345, hellaswag/436, hellaswag/4360, hellaswag/4361, hellaswag/4373, hellaswag/4374, hellaswag/4375, hellaswag/4381, hellaswag/4386, hellaswag/4388, hellaswag/440, hellaswag/4401, hellaswag/4413, hellaswag/4417, hellaswag/4420, hellaswag/4426, hellaswag/4436, hellaswag/4437, hellaswag/4441, hellaswag/445, hellaswag/4451, hellaswag/4459, hellaswag/4466, hellaswag/4467, hellaswag/4469, hellaswag/4482, hellaswag/4486, hellaswag/4488, hellaswag/4492, hellaswag/4493, hellaswag/4495, hellaswag/4497, hellaswag/4506, hellaswag/4509, hellaswag/451, hellaswag/4513, hellaswag/4516, hellaswag/452, hellaswag/4520, hellaswag/4524, hellaswag/454, hellaswag/4540, hellaswag/4542, hellaswag/4544, hellaswag/4549, hellaswag/455, hellaswag/4550, hellaswag/4551, hellaswag/4555, hellaswag/4557, hellaswag/4558, hellaswag/4563, hellaswag/4564, hellaswag/4569, hellaswag/4575, hellaswag/4579, hellaswag/4585, hellaswag/4586, hellaswag/4591, hellaswag/4592, hellaswag/4593, hellaswag/4603, hellaswag/4613, hellaswag/4626, hellaswag/4627, hellaswag/4631, hellaswag/4641, hellaswag/4642, hellaswag/4645, hellaswag/465, hellaswag/4654, hellaswag/4656, hellaswag/466, hellaswag/4660, hellaswag/4665, hellaswag/4675, hellaswag/468, hellaswag/4683, hellaswag/4688, hellaswag/4690, hellaswag/4693, hellaswag/4700, hellaswag/4706, hellaswag/471, hellaswag/4710, hellaswag/4712, hellaswag/4713, hellaswag/4714, hellaswag/4718, hellaswag/4727, hellaswag/4730, hellaswag/4734, hellaswag/4735, hellaswag/4736, hellaswag/4737, hellaswag/474, hellaswag/4744, hellaswag/4749, hellaswag/475, hellaswag/4750, hellaswag/4752, hellaswag/4753, hellaswag/4755, hellaswag/4756, hellaswag/4764, hellaswag/4767, hellaswag/4771, hellaswag/4773, hellaswag/4776, hellaswag/4777, hellaswag/4785, hellaswag/4788, hellaswag/4790, hellaswag/4797, hellaswag/48, hellaswag/4802, hellaswag/4804, hellaswag/482, hellaswag/4823, hellaswag/4827, hellaswag/4829, hellaswag/4836, hellaswag/4841, hellaswag/4847, hellaswag/4848, hellaswag/4852, hellaswag/4854, hellaswag/4869, hellaswag/487, hellaswag/4879, hellaswag/4884, hellaswag/4885, hellaswag/4887, hellaswag/4891, hellaswag/4892, hellaswag/4894, hellaswag/4897, hellaswag/4899, hellaswag/4905, hellaswag/4906, hellaswag/4913, hellaswag/4923, hellaswag/4930, hellaswag/4936, hellaswag/4937, hellaswag/4944, hellaswag/4949, hellaswag/4958, hellaswag/496, hellaswag/4963, hellaswag/497, hellaswag/4976, hellaswag/498, hellaswag/4982, hellaswag/50, hellaswag/5006, hellaswag/5008, hellaswag/501, hellaswag/5011, hellaswag/502, hellaswag/5021, hellaswag/5022, hellaswag/5023, hellaswag/5024, hellaswag/5034, hellaswag/5038, hellaswag/505, hellaswag/5055, hellaswag/5057, hellaswag/5065, hellaswag/5068, hellaswag/5070, hellaswag/5073, hellaswag/5078, hellaswag/5089, hellaswag/5090, hellaswag/5096, hellaswag/510, hellaswag/5100, hellaswag/5105, hellaswag/5109, hellaswag/5115, hellaswag/5117, hellaswag/5124, hellaswag/5125, hellaswag/5126, hellaswag/5127, hellaswag/513, hellaswag/5131, hellaswag/5132, hellaswag/514, hellaswag/5141, hellaswag/515, hellaswag/5151, hellaswag/5153, hellaswag/5154, hellaswag/5156, hellaswag/5169, hellaswag/517, hellaswag/5175, hellaswag/5177, hellaswag/5180, hellaswag/5197, hellaswag/5198, hellaswag/5200, hellaswag/5209, hellaswag/5210, hellaswag/5219, hellaswag/522, hellaswag/5221, hellaswag/5225, hellaswag/5226, hellaswag/5232, hellaswag/5237, hellaswag/5243, hellaswag/5247, hellaswag/5248, hellaswag/5250, hellaswag/5256, hellaswag/5259, hellaswag/5261, hellaswag/5264, hellaswag/5271, hellaswag/5273, hellaswag/5279, hellaswag/5280, hellaswag/5282, hellaswag/5283, hellaswag/529, hellaswag/5291, hellaswag/5292, hellaswag/5293, hellaswag/5294, hellaswag/53, hellaswag/530, hellaswag/5300, hellaswag/5301, hellaswag/5310, hellaswag/5317, hellaswag/5319, hellaswag/5323, hellaswag/5326, hellaswag/5330, hellaswag/5333, hellaswag/5334, hellaswag/5338, hellaswag/534, hellaswag/5345, hellaswag/5347, hellaswag/5352, hellaswag/536, hellaswag/5362, hellaswag/5375, hellaswag/5377, hellaswag/5383, hellaswag/5389, hellaswag/5397, hellaswag/54, hellaswag/5403, hellaswag/5404, hellaswag/5407, hellaswag/5412, hellaswag/5414, hellaswag/5420, hellaswag/5424, hellaswag/5427, hellaswag/543, hellaswag/5434, hellaswag/544, hellaswag/5446, hellaswag/5447, hellaswag/545, hellaswag/5456, hellaswag/546, hellaswag/5461, hellaswag/5462, hellaswag/5464, hellaswag/5469, hellaswag/5477, hellaswag/5478, hellaswag/5481, hellaswag/5482, hellaswag/5489, hellaswag/5493, hellaswag/5505, hellaswag/5513, hellaswag/5514, hellaswag/5517, hellaswag/5519, hellaswag/5524, hellaswag/5529, hellaswag/5544, hellaswag/5552, hellaswag/5554, hellaswag/5564, hellaswag/5568, hellaswag/5572, hellaswag/558, hellaswag/5580, hellaswag/5591, hellaswag/5592, hellaswag/5608, hellaswag/561, hellaswag/5611, hellaswag/5617, hellaswag/5618, hellaswag/5619, hellaswag/5622, hellaswag/5629, hellaswag/5635, hellaswag/564, hellaswag/5640, hellaswag/5646, hellaswag/5661, hellaswag/5670, hellaswag/5675, hellaswag/5683, hellaswag/5690, hellaswag/5692, hellaswag/5707, hellaswag/5708, hellaswag/5714, hellaswag/5724, hellaswag/5726, hellaswag/5727, hellaswag/5729, hellaswag/5739, hellaswag/574, hellaswag/5742, hellaswag/5743, hellaswag/5744, hellaswag/5749, hellaswag/576, hellaswag/5760, hellaswag/5762, hellaswag/5769, hellaswag/5773, hellaswag/578, hellaswag/5780, hellaswag/5784, hellaswag/5788, hellaswag/579, hellaswag/5790, hellaswag/5794, hellaswag/5797, hellaswag/58, hellaswag/5805, hellaswag/5808, hellaswag/5809, hellaswag/5825, hellaswag/583, hellaswag/5832, hellaswag/5833, hellaswag/5836, hellaswag/5855, hellaswag/5857, hellaswag/5863, hellaswag/5867, hellaswag/5871, hellaswag/5872, hellaswag/5874, hellaswag/5877, hellaswag/5880, hellaswag/5881, hellaswag/5883, hellaswag/5896, hellaswag/5897, hellaswag/5898, hellaswag/59, hellaswag/5900, hellaswag/5916, hellaswag/5925, hellaswag/5927, hellaswag/5935, hellaswag/5938, hellaswag/5944, hellaswag/5945, hellaswag/5954, hellaswag/5959, hellaswag/5961, hellaswag/5966, hellaswag/5970, hellaswag/5978, hellaswag/5979, hellaswag/5982, hellaswag/5990, hellaswag/5995, hellaswag/5996, hellaswag/6, hellaswag/60, hellaswag/6000, hellaswag/6006, hellaswag/6017, hellaswag/6019, hellaswag/6024, hellaswag/6025, hellaswag/6027, hellaswag/6029, hellaswag/6030, hellaswag/6031, hellaswag/6038, hellaswag/6039, hellaswag/6041, hellaswag/6042, hellaswag/605, hellaswag/6051, hellaswag/6053, hellaswag/6060, hellaswag/6062, hellaswag/6063, hellaswag/6071, hellaswag/6082, hellaswag/6083, hellaswag/6086, hellaswag/609, hellaswag/6092, hellaswag/610, hellaswag/6101, hellaswag/6123, hellaswag/6127, hellaswag/6128, hellaswag/6130, hellaswag/6135, hellaswag/614, hellaswag/6141, hellaswag/6150, hellaswag/6152, hellaswag/6155, hellaswag/6157, hellaswag/616, hellaswag/6161, hellaswag/6164, hellaswag/6168, hellaswag/617, hellaswag/6171, hellaswag/6172, hellaswag/6175, hellaswag/6177, hellaswag/6180, hellaswag/6197, hellaswag/6200, hellaswag/6204, hellaswag/6206, hellaswag/6208, hellaswag/6210, hellaswag/6218, hellaswag/6222, hellaswag/6223, hellaswag/6226, hellaswag/6228, hellaswag/6230, hellaswag/6235, hellaswag/624, hellaswag/6250, hellaswag/6251, hellaswag/6253, hellaswag/6262, hellaswag/6271, hellaswag/6273, hellaswag/6275, hellaswag/6283, hellaswag/6285, hellaswag/6286, hellaswag/629, hellaswag/6292, hellaswag/630, hellaswag/6304, hellaswag/6306, hellaswag/6307, hellaswag/6316, hellaswag/6317, hellaswag/6320, hellaswag/6323, hellaswag/6329, hellaswag/6330, hellaswag/6332, hellaswag/6336, hellaswag/6337, hellaswag/6341, hellaswag/6345, hellaswag/6358, hellaswag/6363, hellaswag/6364, hellaswag/6372, hellaswag/6375, hellaswag/6381, hellaswag/6383, hellaswag/6385, hellaswag/6387, hellaswag/639, hellaswag/6399, hellaswag/64, hellaswag/6400, hellaswag/6402, hellaswag/6406, hellaswag/6413, hellaswag/6415, hellaswag/642, hellaswag/6422, hellaswag/6427, hellaswag/6430, hellaswag/6432, hellaswag/6443, hellaswag/6444, hellaswag/6448, hellaswag/6449, hellaswag/645, hellaswag/6450, hellaswag/6456, hellaswag/6458, hellaswag/648, hellaswag/6480, hellaswag/6482, hellaswag/6483, hellaswag/649, hellaswag/6490, hellaswag/6493, hellaswag/6498, hellaswag/6501, hellaswag/651, hellaswag/6513, hellaswag/6514, hellaswag/6520, hellaswag/6521, hellaswag/6530, hellaswag/6533, hellaswag/6534, hellaswag/6539, hellaswag/6547, hellaswag/6565, hellaswag/6567, hellaswag/6568, hellaswag/657, hellaswag/6578, hellaswag/658, hellaswag/6588, hellaswag/6592, hellaswag/6593, hellaswag/6596, hellaswag/6600, hellaswag/6602, hellaswag/6605, hellaswag/661, hellaswag/6614, hellaswag/6623, hellaswag/6626, hellaswag/6634, hellaswag/6635, hellaswag/6637, hellaswag/6646, hellaswag/6649, hellaswag/665, hellaswag/6650, hellaswag/6654, hellaswag/6668, hellaswag/6675, hellaswag/6678, hellaswag/668, hellaswag/6688, hellaswag/669, hellaswag/6706, hellaswag/671, hellaswag/6710, hellaswag/6714, hellaswag/6718, hellaswag/6722, hellaswag/6725, hellaswag/6732, hellaswag/6737, hellaswag/6739, hellaswag/674, hellaswag/6741, hellaswag/6743, hellaswag/6750, hellaswag/6756, hellaswag/6757, hellaswag/6758, hellaswag/6761, hellaswag/6762, hellaswag/6763, hellaswag/6766, hellaswag/6767, hellaswag/6768, hellaswag/677, hellaswag/6771, hellaswag/6774, hellaswag/6777, hellaswag/6780, hellaswag/6784, hellaswag/6790, hellaswag/6798, hellaswag/6799, hellaswag/68, hellaswag/6802, hellaswag/6807, hellaswag/6811, hellaswag/6812, hellaswag/6815, hellaswag/6816, hellaswag/6819, hellaswag/6825, hellaswag/6827, hellaswag/6831, hellaswag/6833, hellaswag/6834, hellaswag/6839, hellaswag/684, hellaswag/6842, hellaswag/6843, hellaswag/6845, hellaswag/6848, hellaswag/6853, hellaswag/6874, hellaswag/6876, hellaswag/6877, hellaswag/6878, hellaswag/6883, hellaswag/6886, hellaswag/6887, hellaswag/6888, hellaswag/6891, hellaswag/6895, hellaswag/6900, hellaswag/6904, hellaswag/6905, hellaswag/6914, hellaswag/6918, hellaswag/6929, hellaswag/6936, hellaswag/6937, hellaswag/6939, hellaswag/6944, hellaswag/6945, hellaswag/6951, hellaswag/6963, hellaswag/6969, hellaswag/697, hellaswag/6970, hellaswag/698, hellaswag/6980, hellaswag/6982, hellaswag/6989, hellaswag/6991, hellaswag/6996, hellaswag/7, hellaswag/70, hellaswag/7000, hellaswag/7002, hellaswag/7004, hellaswag/7006, hellaswag/7009, hellaswag/702, hellaswag/7021, hellaswag/7022, hellaswag/7023, hellaswag/7028, hellaswag/7029, hellaswag/7031, hellaswag/7032, hellaswag/7044, hellaswag/7048, hellaswag/705, hellaswag/7057, hellaswag/706, hellaswag/7060, hellaswag/7070, hellaswag/7073, hellaswag/7076, hellaswag/7077, hellaswag/7079, hellaswag/708, hellaswag/7085, hellaswag/709, hellaswag/7092, hellaswag/7096, hellaswag/71, hellaswag/7104, hellaswag/7108, hellaswag/7114, hellaswag/712, hellaswag/7124, hellaswag/7129, hellaswag/7130, hellaswag/7134, hellaswag/7142, hellaswag/7148, hellaswag/7164, hellaswag/7167, hellaswag/7191, hellaswag/7193, hellaswag/7196, hellaswag/7199, hellaswag/7203, hellaswag/7205, hellaswag/7211, hellaswag/7214, hellaswag/7226, hellaswag/7227, hellaswag/7232, hellaswag/724, hellaswag/7241, hellaswag/7246, hellaswag/7253, hellaswag/7256, hellaswag/726, hellaswag/7260, hellaswag/7265, hellaswag/7268, hellaswag/7269, hellaswag/727, hellaswag/7271, hellaswag/7274, hellaswag/7280, hellaswag/7283, hellaswag/7285, hellaswag/7295, hellaswag/730, hellaswag/7301, hellaswag/7316, hellaswag/732, hellaswag/7330, hellaswag/7332, hellaswag/7336, hellaswag/7339, hellaswag/7344, hellaswag/7348, hellaswag/7354, hellaswag/7355, hellaswag/7359, hellaswag/7364, hellaswag/7366, hellaswag/7381, hellaswag/7384, hellaswag/7389, hellaswag/7395, hellaswag/740, hellaswag/7403, hellaswag/7405, hellaswag/7408, hellaswag/7411, hellaswag/7414, hellaswag/7415, hellaswag/7417, hellaswag/7418, hellaswag/7419, hellaswag/7421, hellaswag/7422, hellaswag/7424, hellaswag/7432, hellaswag/7436, hellaswag/7439, hellaswag/7443, hellaswag/7447, hellaswag/7451, hellaswag/7459, hellaswag/7460, hellaswag/7463, hellaswag/747, hellaswag/7474, hellaswag/7479, hellaswag/7481, hellaswag/7483, hellaswag/7497, hellaswag/7498, hellaswag/750, hellaswag/7501, hellaswag/7503, hellaswag/7506, hellaswag/7510, hellaswag/7517, hellaswag/7518, hellaswag/752, hellaswag/7521, hellaswag/7523, hellaswag/7527, hellaswag/7529, hellaswag/7533, hellaswag/7534, hellaswag/7535, hellaswag/754, hellaswag/7548, hellaswag/755, hellaswag/7554, hellaswag/7556, hellaswag/756, hellaswag/7560, hellaswag/7563, hellaswag/7564, hellaswag/7568, hellaswag/7574, hellaswag/7577, hellaswag/7583, hellaswag/7584, hellaswag/759, hellaswag/7590, hellaswag/7591, hellaswag/7594, hellaswag/7595, hellaswag/7598, hellaswag/7604, hellaswag/761, hellaswag/7612, hellaswag/7624, hellaswag/7627, hellaswag/7628, hellaswag/763, hellaswag/7632, hellaswag/7634, hellaswag/7638, hellaswag/7649, hellaswag/7651, hellaswag/7653, hellaswag/7654, hellaswag/7658, hellaswag/766, hellaswag/7660, hellaswag/7661, hellaswag/7662, hellaswag/7670, hellaswag/7674, hellaswag/768, hellaswag/7696, hellaswag/7702, hellaswag/7712, hellaswag/7721, hellaswag/7723, hellaswag/7731, hellaswag/7732, hellaswag/774, hellaswag/7758, hellaswag/7759, hellaswag/7760, hellaswag/7766, hellaswag/7767, hellaswag/7781, hellaswag/7785, hellaswag/780, hellaswag/7805, hellaswag/7807, hellaswag/7810, hellaswag/7813, hellaswag/7816, hellaswag/7818, hellaswag/7822, hellaswag/7823, hellaswag/7825, hellaswag/7827, hellaswag/783, hellaswag/7842, hellaswag/785, hellaswag/7860, hellaswag/7862, hellaswag/7863, hellaswag/7864, hellaswag/7868, hellaswag/7875, hellaswag/7876, hellaswag/788, hellaswag/7884, hellaswag/7887, hellaswag/789, hellaswag/7903, hellaswag/7910, hellaswag/7911, hellaswag/7912, hellaswag/7915, hellaswag/7916, hellaswag/7917, hellaswag/7923, hellaswag/7931, hellaswag/7936, hellaswag/7942, hellaswag/7943, hellaswag/7946, hellaswag/7947, hellaswag/7950, hellaswag/7953, hellaswag/7960, hellaswag/7962, hellaswag/7971, hellaswag/7976, hellaswag/798, hellaswag/7982, hellaswag/7983, hellaswag/7984, hellaswag/7989, hellaswag/80, hellaswag/800, hellaswag/8014, hellaswag/8020, hellaswag/8026, hellaswag/8027, hellaswag/803, hellaswag/8038, hellaswag/8040, hellaswag/8041, hellaswag/8046, hellaswag/8048, hellaswag/8050, hellaswag/8052, hellaswag/8059, hellaswag/806, hellaswag/8068, hellaswag/807, hellaswag/8075, hellaswag/8077, hellaswag/8090, hellaswag/8093, hellaswag/8094, hellaswag/8095, hellaswag/810, hellaswag/8111, hellaswag/8127, hellaswag/8132, hellaswag/8133, hellaswag/8134, hellaswag/8136, hellaswag/8137, hellaswag/814, hellaswag/8145, hellaswag/8149, hellaswag/8153, hellaswag/8155, hellaswag/8161, hellaswag/8166, hellaswag/8171, hellaswag/8172, hellaswag/8174, hellaswag/818, hellaswag/8181, hellaswag/8186, hellaswag/8190, hellaswag/8194, hellaswag/8195, hellaswag/820, hellaswag/8202, hellaswag/8205, hellaswag/8209, hellaswag/821, hellaswag/8218, hellaswag/822, hellaswag/8220, hellaswag/8221, hellaswag/8225, hellaswag/8226, hellaswag/8227, hellaswag/8228, hellaswag/8230, hellaswag/8241, hellaswag/8245, hellaswag/8246, hellaswag/8251, hellaswag/826, hellaswag/8268, hellaswag/8271, hellaswag/8272, hellaswag/828, hellaswag/8281, hellaswag/8290, hellaswag/8291, hellaswag/8293, hellaswag/8294, hellaswag/8298, hellaswag/8309, hellaswag/8312, hellaswag/8315, hellaswag/8329, hellaswag/833, hellaswag/8336, hellaswag/8353, hellaswag/8354, hellaswag/8357, hellaswag/8359, hellaswag/836, hellaswag/8360, hellaswag/8365, hellaswag/8370, hellaswag/8376, hellaswag/8379, hellaswag/8382, hellaswag/8389, hellaswag/839, hellaswag/8391, hellaswag/8393, hellaswag/8394, hellaswag/840, hellaswag/8408, hellaswag/8411, hellaswag/8419, hellaswag/8421, hellaswag/8437, hellaswag/844, hellaswag/8441, hellaswag/8443, hellaswag/845, hellaswag/8450, hellaswag/8451, hellaswag/8452, hellaswag/8454, hellaswag/8456, hellaswag/8462, hellaswag/8465, hellaswag/8466, hellaswag/8470, hellaswag/8473, hellaswag/8476, hellaswag/8477, hellaswag/8485, hellaswag/8487, hellaswag/8490, hellaswag/8491, hellaswag/8500, hellaswag/8501, hellaswag/8502, hellaswag/8510, hellaswag/8511, hellaswag/8512, hellaswag/8514, hellaswag/8517, hellaswag/8522, hellaswag/8533, hellaswag/854, hellaswag/8540, hellaswag/8551, hellaswag/8552, hellaswag/8561, hellaswag/8564, hellaswag/8568, hellaswag/8569, hellaswag/857, hellaswag/8575, hellaswag/8582, hellaswag/8587, hellaswag/859, hellaswag/8593, hellaswag/8594, hellaswag/86, hellaswag/8607, hellaswag/8611, hellaswag/8613, hellaswag/8619, hellaswag/862, hellaswag/8620, hellaswag/8621, hellaswag/8623, hellaswag/8631, hellaswag/8634, hellaswag/8636, hellaswag/8640, hellaswag/8642, hellaswag/8643, hellaswag/8645, hellaswag/8647, hellaswag/8663, hellaswag/8668, hellaswag/8670, hellaswag/8682, hellaswag/8689, hellaswag/869, hellaswag/8692, hellaswag/8693, hellaswag/8694, hellaswag/8698, hellaswag/8702, hellaswag/8703, hellaswag/8704, hellaswag/8713, hellaswag/8718, hellaswag/872, hellaswag/8727, hellaswag/8728, hellaswag/8734, hellaswag/8745, hellaswag/8749, hellaswag/875, hellaswag/8756, hellaswag/8762, hellaswag/8766, hellaswag/8767, hellaswag/8780, hellaswag/8789, hellaswag/8797, hellaswag/8808, hellaswag/8809, hellaswag/8817, hellaswag/8826, hellaswag/8829, hellaswag/8838, hellaswag/884, hellaswag/8840, hellaswag/8850, hellaswag/8860, hellaswag/8864, hellaswag/8869, hellaswag/8870, hellaswag/8873, hellaswag/8881, hellaswag/8888, hellaswag/8894, hellaswag/8895, hellaswag/8896, hellaswag/8898, hellaswag/8899, hellaswag/8901, hellaswag/8902, hellaswag/8905, hellaswag/8919, hellaswag/892, hellaswag/8925, hellaswag/8928, hellaswag/893, hellaswag/8930, hellaswag/8934, hellaswag/8942, hellaswag/8947, hellaswag/895, hellaswag/8950, hellaswag/8952, hellaswag/8957, hellaswag/8971, hellaswag/8978, hellaswag/8980, hellaswag/8981, hellaswag/899, hellaswag/8990, hellaswag/8995, hellaswag/9, hellaswag/90, hellaswag/900, hellaswag/9006, hellaswag/9022, hellaswag/9025, hellaswag/9026, hellaswag/9033, hellaswag/9034, hellaswag/9035, hellaswag/904, hellaswag/9040, hellaswag/9044, hellaswag/9046, hellaswag/9057, hellaswag/9058, hellaswag/9060, hellaswag/9066, hellaswag/9067, hellaswag/9069, hellaswag/9071, hellaswag/9072, hellaswag/9074, hellaswag/9075, hellaswag/9091, hellaswag/9112, hellaswag/9128, hellaswag/913, hellaswag/9130, hellaswag/9132, hellaswag/9134, hellaswag/9136, hellaswag/9144, hellaswag/9147, hellaswag/915, hellaswag/9151, hellaswag/9157, hellaswag/9159, hellaswag/9160, hellaswag/9166, hellaswag/9170, hellaswag/9172, hellaswag/9173, hellaswag/9185, hellaswag/9187, hellaswag/9198, hellaswag/9208, hellaswag/921, hellaswag/9214, hellaswag/9216, hellaswag/922, hellaswag/9220, hellaswag/9223, hellaswag/9226, hellaswag/923, hellaswag/9249, hellaswag/9252, hellaswag/927, hellaswag/9271, hellaswag/9278, hellaswag/9279, hellaswag/928, hellaswag/9280, hellaswag/9281, hellaswag/9284, hellaswag/9286, hellaswag/9319, hellaswag/932, hellaswag/9327, hellaswag/9329, hellaswag/9332, hellaswag/9334, hellaswag/9339, hellaswag/9340, hellaswag/9351, hellaswag/9352, hellaswag/9354, hellaswag/9359, hellaswag/9361, hellaswag/9365, hellaswag/9367, hellaswag/9368, hellaswag/9370, hellaswag/9371, hellaswag/9377, hellaswag/9378, hellaswag/938, hellaswag/939, hellaswag/9391, hellaswag/9395, hellaswag/9400, hellaswag/9401, hellaswag/9402, hellaswag/9405, hellaswag/9407, hellaswag/941, hellaswag/9410, hellaswag/9411, hellaswag/9412, hellaswag/9413, hellaswag/9416, hellaswag/942, hellaswag/9425, hellaswag/9432, hellaswag/9433, hellaswag/9437, hellaswag/9441, hellaswag/945, hellaswag/9451, hellaswag/9478, hellaswag/948, hellaswag/9481, hellaswag/9482, hellaswag/9493, hellaswag/9495, hellaswag/9498, hellaswag/9502, hellaswag/9503, hellaswag/9508, hellaswag/9512, hellaswag/9517, hellaswag/952, hellaswag/9520, hellaswag/9522, hellaswag/9524, hellaswag/9525, hellaswag/9540, hellaswag/9541, hellaswag/9544, hellaswag/9551, hellaswag/9554, hellaswag/9572, hellaswag/9576, hellaswag/9583, hellaswag/9587, hellaswag/9597, hellaswag/96, hellaswag/9607, hellaswag/9611, hellaswag/9615, hellaswag/962, hellaswag/9629, hellaswag/963, hellaswag/9630, hellaswag/9631, hellaswag/9637, hellaswag/9638, hellaswag/9642, hellaswag/9644, hellaswag/9646, hellaswag/9647, hellaswag/965, hellaswag/9653, hellaswag/9655, hellaswag/9657, hellaswag/9666, hellaswag/9668, hellaswag/9669, hellaswag/967, hellaswag/9673, hellaswag/9675, hellaswag/9679, hellaswag/968, hellaswag/9680, hellaswag/9684, hellaswag/9694, hellaswag/9695, hellaswag/9696, hellaswag/9702, hellaswag/9709, hellaswag/9716, hellaswag/972, hellaswag/9730, hellaswag/9740, hellaswag/9743, hellaswag/9744, hellaswag/9746, hellaswag/9747, hellaswag/9749, hellaswag/975, hellaswag/9754, hellaswag/9759, hellaswag/9762, hellaswag/9765, hellaswag/977, hellaswag/9770, hellaswag/9771, hellaswag/9778, hellaswag/9779, hellaswag/978, hellaswag/9785, hellaswag/9788, hellaswag/979, hellaswag/9797, hellaswag/9798, hellaswag/9799, hellaswag/98, hellaswag/980, hellaswag/9809, hellaswag/981, hellaswag/9823, hellaswag/9827, hellaswag/9833, hellaswag/9834, hellaswag/9837, hellaswag/9842, hellaswag/9844, hellaswag/9846, hellaswag/9847, hellaswag/9850, hellaswag/9853, hellaswag/986, hellaswag/9863, hellaswag/987, hellaswag/9873, hellaswag/9878, hellaswag/9879, hellaswag/9881, hellaswag/9886, hellaswag/9887, hellaswag/9889, hellaswag/9893, hellaswag/9898, hellaswag/9899, hellaswag/9902, hellaswag/9904, hellaswag/9911, hellaswag/9912, hellaswag/9913, hellaswag/9915, hellaswag/9921, hellaswag/9922, hellaswag/993, hellaswag/9931, hellaswag/9938, hellaswag/9945, hellaswag/9946, hellaswag/9956, hellaswag/9966, hellaswag/997, hellaswag/9973, hellaswag/9976, hellaswag/9985, hellaswag/9992, hellaswag/9996, hellaswag/9997
| example_link | model | min_pass1_of_model |
|---|---|---|
| hellaswag/10038 | seed17495 | 0.662 |
| hellaswag/8830 | seed17495 | 0.662 |
| hellaswag/8140 | seed17495 | 0.662 |
| hellaswag/2021 | seed17495 | 0.662 |
| hellaswag/8098 | seed17495 | 0.662 |
| hellaswag/7324 | seed17495 | 0.662 |
| hellaswag/7135 | seed17495 | 0.662 |
| hellaswag/6893 | seed17495 | 0.662 |
| hellaswag/2763 | seed17495 | 0.662 |
| hellaswag/3045 | seed17495 | 0.662 |
| hellaswag/2693 | seed17495 | 0.662 |
| hellaswag/2856 | seed17495 | 0.662 |
| hellaswag/3196 | seed17495 | 0.662 |
| hellaswag/4731 | seed17495 | 0.662 |
| hellaswag/3938 | seed17495 | 0.662 |
| hellaswag/3555 | seed17495 | 0.662 |
| hellaswag/3806 | seed17495 | 0.662 |
| hellaswag/9738 | seed17495 | 0.662 |
| hellaswag/1884 | seed17495 | 0.662 |
| hellaswag/4740 | seed17495 | 0.662 |
| hellaswag/5673 | seed17495 | 0.662 |
| hellaswag/5901 | seed17495 | 0.662 |
| hellaswag/5767 | seed17495 | 0.662 |
| hellaswag/5327 | seed17495 | 0.662 |
| hellaswag/9088 | seed17495 | 0.662 |
| hellaswag/1402 | seed17495 | 0.662 |
| hellaswag/177 | seed17495 | 0.662 |
| hellaswag/5229 | seed6115 | 0.662 |
| hellaswag/555 | seed6115 | 0.662 |
| hellaswag/7458 | seed6115 | 0.662 |
| hellaswag/6644 | seed6115 | 0.662 |
| hellaswag/7222 | seed6115 | 0.662 |
| hellaswag/7102 | seed6115 | 0.662 |
| hellaswag/7065 | seed6115 | 0.662 |
| hellaswag/852 | seed6115 | 0.662 |
| hellaswag/2062 | seed6115 | 0.662 |
| hellaswag/8372 | seed6115 | 0.662 |
| hellaswag/8164 | seed6115 | 0.662 |
| hellaswag/2583 | seed6115 | 0.662 |
| hellaswag/2463 | seed6115 | 0.662 |
| hellaswag/8324 | seed6115 | 0.662 |
| hellaswag/8345 | seed6115 | 0.662 |
| hellaswag/9968 | seed6115 | 0.662 |
| hellaswag/9882 | seed6115 | 0.662 |
| hellaswag/1238 | seed6115 | 0.662 |
| hellaswag/1507 | seed6115 | 0.662 |
| hellaswag/1725 | seed6115 | 0.662 |
| hellaswag/1967 | seed6115 | 0.662 |
| hellaswag/1846 | seed6115 | 0.662 |
| hellaswag/193 | seed6115 | 0.662 |
| hellaswag/9007 | seed6115 | 0.662 |
| hellaswag/1907 | seed6115 | 0.662 |
| hellaswag/9353 | seed6115 | 0.662 |
| hellaswag/3471 | seed6115 | 0.662 |
| hellaswag/424 | seed6115 | 0.662 |
| hellaswag/419 | seed6115 | 0.662 |
| hellaswag/4109 | seed6115 | 0.662 |
| hellaswag/4705 | seed6115 | 0.662 |
| hellaswag/3002 | seed6115 | 0.662 |
| hellaswag/3030 | seed6115 | 0.662 |
| hellaswag/2688 | seed6115 | 0.662 |
| hellaswag/5435 | seed6115 | 0.662 |
| hellaswag/509 | seed6115 | 0.662 |
| hellaswag/5415 | seed6115 | 0.662 |
| hellaswag/589 | seed6115 | 0.662 |
| hellaswag/608 | seed6115 | 0.662 |
| hellaswag/3987 | seed6115 | 0.662 |
| hellaswag/9711 | seed6115 | 0.662 |
| hellaswag/5599 | seed6115 | 0.662 |
| hellaswag/992 | seed14309 | 0.661 |
| hellaswag/9934 | seed14309 | 0.661 |
| hellaswag/8697 | seed14309 | 0.661 |
| hellaswag/10002 | seed14309 | 0.661 |
| hellaswag/8348 | seed14309 | 0.661 |
| hellaswag/793 | seed14309 | 0.661 |
| hellaswag/7599 | seed14309 | 0.661 |
| hellaswag/7605 | seed14309 | 0.661 |
| hellaswag/676 | seed14309 | 0.661 |
| hellaswag/6724 | seed14309 | 0.661 |
| hellaswag/6621 | seed14309 | 0.661 |
| hellaswag/6584 | seed14309 | 0.661 |
| hellaswag/7034 | seed14309 | 0.661 |
| hellaswag/6564 | seed14309 | 0.661 |
| hellaswag/7138 | seed14309 | 0.661 |
| hellaswag/719 | seed14309 | 0.661 |
| hellaswag/7763 | seed14309 | 0.661 |
| hellaswag/2687 | seed14309 | 0.661 |
| hellaswag/8126 | seed14309 | 0.661 |
| hellaswag/8235 | seed14309 | 0.661 |
| hellaswag/7815 | seed14309 | 0.661 |
| hellaswag/6395 | seed14309 | 0.661 |
| hellaswag/2971 | seed14309 | 0.661 |
| hellaswag/3188 | seed14309 | 0.661 |
| hellaswag/5000 | seed14309 | 0.661 |
| hellaswag/6035 | seed14309 | 0.661 |
| hellaswag/5981 | seed14309 | 0.661 |
| hellaswag/5697 | seed14309 | 0.661 |
| hellaswag/6047 | seed14309 | 0.661 |
| hellaswag/6366 | seed14309 | 0.661 |
| hellaswag/3421 | seed14309 | 0.661 |
| hellaswag/3084 | seed14309 | 0.661 |
| hellaswag/4081 | seed14309 | 0.661 |
| hellaswag/3715 | seed14309 | 0.661 |
| hellaswag/3879 | seed14309 | 0.661 |
| hellaswag/425 | seed14309 | 0.661 |
| hellaswag/9440 | seed14309 | 0.661 |
| hellaswag/9849 | seed14309 | 0.661 |
| hellaswag/1858 | seed14309 | 0.661 |
| hellaswag/1410 | seed14309 | 0.661 |
| hellaswag/1302 | seed16385 | 0.661 |
| hellaswag/1539 | seed16385 | 0.661 |
| hellaswag/8994 | seed16385 | 0.661 |
| hellaswag/9191 | seed16385 | 0.661 |
| hellaswag/8951 | seed16385 | 0.661 |
| hellaswag/5026 | seed16385 | 0.661 |
| hellaswag/449 | seed16385 | 0.661 |
| hellaswag/4881 | seed16385 | 0.661 |
| hellaswag/3536 | seed16385 | 0.661 |
| hellaswag/3706 | seed16385 | 0.661 |
| hellaswag/3790 | seed16385 | 0.661 |
| hellaswag/3668 | seed16385 | 0.661 |
| hellaswag/3753 | seed16385 | 0.661 |
| hellaswag/9707 | seed16385 | 0.661 |
| hellaswag/5142 | seed16385 | 0.661 |
| hellaswag/5118 | seed16385 | 0.661 |
| hellaswag/4772 | seed16385 | 0.661 |
| hellaswag/4653 | seed16385 | 0.661 |
| hellaswag/5084 | seed16385 | 0.661 |
| hellaswag/8373 | seed16385 | 0.661 |
| hellaswag/8300 | seed16385 | 0.661 |
| hellaswag/110 | seed16385 | 0.661 |
| hellaswag/8708 | seed16385 | 0.661 |
| hellaswag/8833 | seed16385 | 0.661 |
| hellaswag/8626 | seed16385 | 0.661 |
| hellaswag/7600 | seed16385 | 0.661 |
| hellaswag/3302 | seed16385 | 0.661 |
| hellaswag/3146 | seed16385 | 0.661 |
| hellaswag/2978 | seed16385 | 0.661 |
| hellaswag/5818 | seed16385 | 0.661 |
| hellaswag/786 | seed16385 | 0.661 |
| hellaswag/799 | seed16385 | 0.661 |
| hellaswag/8081 | seed16385 | 0.661 |
| hellaswag/2402 | seed16385 | 0.661 |
| hellaswag/8475 | seed16385 | 0.661 |
| hellaswag/8509 | seed16385 | 0.661 |
| hellaswag/83 | seed16385 | 0.661 |
| hellaswag/744 | seed24762 | 0.660 |
| hellaswag/581 | seed24762 | 0.660 |
| hellaswag/2791 | seed24762 | 0.660 |
| hellaswag/3218 | seed24762 | 0.660 |
| hellaswag/2590 | seed24762 | 0.660 |
| hellaswag/2353 | seed24762 | 0.660 |
| hellaswag/2076 | seed24762 | 0.660 |
| hellaswag/7302 | seed24762 | 0.660 |
| hellaswag/6961 | seed24762 | 0.660 |
| hellaswag/267 | seed24762 | 0.660 |
| hellaswag/84 | seed24762 | 0.660 |
| hellaswag/8560 | seed24762 | 0.660 |
| hellaswag/692 | seed24762 | 0.660 |
| hellaswag/7465 | seed24762 | 0.660 |
| hellaswag/9825 | seed24762 | 0.660 |
| hellaswag/9420 | seed24762 | 0.660 |
| hellaswag/8672 | seed24762 | 0.660 |
| hellaswag/8764 | seed24762 | 0.660 |
| hellaswag/8744 | seed24762 | 0.660 |
| hellaswag/1064 | seed24762 | 0.660 |
| hellaswag/879 | seed24762 | 0.660 |
| hellaswag/8788 | seed24762 | 0.660 |
| hellaswag/8409 | seed24762 | 0.660 |
| hellaswag/8597 | seed24762 | 0.660 |
| hellaswag/7954 | seed24762 | 0.660 |
| hellaswag/1559 | seed24762 | 0.660 |
| hellaswag/1740 | seed24762 | 0.660 |
| hellaswag/1280 | seed24762 | 0.660 |
| hellaswag/3891 | seed24762 | 0.660 |
| hellaswag/4335 | seed24762 | 0.660 |
| hellaswag/562 | seed24762 | 0.660 |
| hellaswag/953 | seed24762 | 0.660 |
| hellaswag/1501 | seed24762 | 0.660 |
| hellaswag/1582 | seed6529 | 0.660 |
| hellaswag/1286 | seed6529 | 0.660 |
| hellaswag/1528 | seed6529 | 0.660 |
| hellaswag/1728 | seed6529 | 0.660 |
| hellaswag/5396 | seed6529 | 0.660 |
| hellaswag/5324 | seed6529 | 0.660 |
| hellaswag/4806 | seed6529 | 0.660 |
| hellaswag/8985 | seed6529 | 0.660 |
| hellaswag/5471 | seed6529 | 0.660 |
| hellaswag/577 | seed6529 | 0.660 |
| hellaswag/3152 | seed6529 | 0.660 |
| hellaswag/2516 | seed6529 | 0.660 |
| hellaswag/2245 | seed6529 | 0.660 |
| hellaswag/215 | seed6529 | 0.660 |
| hellaswag/2352 | seed6529 | 0.660 |
| hellaswag/4880 | seed6529 | 0.660 |
| hellaswag/4337 | seed6529 | 0.660 |
| hellaswag/4447 | seed6529 | 0.660 |
| hellaswag/4162 | seed6529 | 0.660 |
| hellaswag/347 | seed6529 | 0.660 |
| hellaswag/5557 | seed6529 | 0.660 |
| hellaswag/1798 | seed6529 | 0.660 |
| hellaswag/9326 | seed6529 | 0.660 |
| hellaswag/173 | seed6529 | 0.660 |
| hellaswag/3339 | seed6529 | 0.660 |
| hellaswag/291 | seed6529 | 0.660 |
| hellaswag/6293 | seed6529 | 0.660 |
| hellaswag/3376 | seed6529 | 0.660 |
| hellaswag/7774 | seed6529 | 0.660 |
| hellaswag/713 | seed6529 | 0.660 |
| hellaswag/8805 | seed6529 | 0.660 |
| hellaswag/8782 | seed6529 | 0.660 |
| hellaswag/204 | seed6529 | 0.660 |
| hellaswag/8701 | seed6529 | 0.660 |
| hellaswag/8664 | seed6529 | 0.660 |
| hellaswag/2633 | seed6529 | 0.660 |
| hellaswag/2059 | seed6529 | 0.660 |
| hellaswag/6178 | seed6529 | 0.660 |
| hellaswag/3556 | seed24734 | 0.659 |
| hellaswag/5254 | seed24734 | 0.659 |
| hellaswag/2095 | seed24734 | 0.659 |
| hellaswag/8520 | seed24734 | 0.659 |
| hellaswag/7828 | seed24734 | 0.659 |
| hellaswag/2408 | seed24734 | 0.659 |
| hellaswag/3134 | seed24734 | 0.659 |
| hellaswag/2711 | seed24734 | 0.659 |
| hellaswag/5759 | seed24734 | 0.659 |
| hellaswag/2190 | seed24734 | 0.659 |
| hellaswag/2273 | seed24734 | 0.659 |
| hellaswag/2281 | seed24734 | 0.659 |
| hellaswag/9470 | seed24734 | 0.659 |
| hellaswag/5615 | seed24734 | 0.659 |
| hellaswag/549 | seed24734 | 0.659 |
| hellaswag/5885 | seed24734 | 0.659 |
| hellaswag/728 | seed24734 | 0.659 |
| hellaswag/7456 | seed24734 | 0.659 |
| hellaswag/3635 | seed24734 | 0.659 |
| hellaswag/3874 | seed24734 | 0.659 |
| hellaswag/5284 | seed24734 | 0.659 |
| hellaswag/9362 | seed24734 | 0.659 |
| hellaswag/914 | seed24734 | 0.659 |
| hellaswag/9536 | seed24734 | 0.659 |
| hellaswag/8999 | seed24734 | 0.659 |
| hellaswag/1676 | seed27712 | 0.658 |
| hellaswag/1628 | seed27712 | 0.658 |
| hellaswag/1788 | seed27712 | 0.658 |
| hellaswag/6279 | seed27712 | 0.658 |
| hellaswag/2911 | seed27712 | 0.658 |
| hellaswag/5699 | seed27712 | 0.658 |
| hellaswag/5974 | seed27712 | 0.658 |
| hellaswag/2594 | seed27712 | 0.658 |
| hellaswag/7968 | seed27712 | 0.658 |
| hellaswag/2587 | seed27712 | 0.658 |
| hellaswag/256 | seed27712 | 0.658 |
| hellaswag/1254 | seed27712 | 0.658 |
| hellaswag/6978 | seed27712 | 0.658 |
| hellaswag/6499 | seed27712 | 0.658 |
| hellaswag/4742 | seed27712 | 0.658 |
| hellaswag/9787 | seed27712 | 0.658 |
| hellaswag/4723 | seed27712 | 0.658 |
| hellaswag/9241 | seed27712 | 0.658 |
| hellaswag/1936 | seed27712 | 0.658 |
| hellaswag/2004 | seed27712 | 0.658 |
| hellaswag/3552 | seed27712 | 0.658 |
| hellaswag/3951 | seed27712 | 0.658 |
| hellaswag/3580 | seed27712 | 0.658 |
| hellaswag/506 | seed27712 | 0.658 |
| hellaswag/4600 | seed27712 | 0.658 |
| hellaswag/6327 | seed27712 | 0.658 |
| hellaswag/6404 | seed27712 | 0.658 |
| hellaswag/6005 | seed27712 | 0.658 |
| hellaswag/7056 | seed27712 | 0.658 |
| hellaswag/7516 | seed27712 | 0.658 |
| hellaswag/7402 | seed27712 | 0.658 |
| hellaswag/620 | seed27712 | 0.658 |
| hellaswag/8459 | seed149 | 0.658 |
| hellaswag/3123 | seed149 | 0.658 |
| hellaswag/2881 | seed149 | 0.658 |
| hellaswag/7187 | seed149 | 0.658 |
| hellaswag/2629 | seed149 | 0.658 |
| hellaswag/343 | seed149 | 0.658 |
| hellaswag/9337 | seed149 | 0.658 |
| hellaswag/3771 | seed149 | 0.658 |
| hellaswag/5605 | seed149 | 0.658 |
| hellaswag/4387 | seed149 | 0.658 |
| hellaswag/4545 | seed149 | 0.658 |
| hellaswag/9633 | seed149 | 0.658 |
| hellaswag/9336 | seed149 | 0.658 |
| hellaswag/9782 | seed149 | 0.658 |
| hellaswag/8274 | seed149 | 0.658 |
| hellaswag/2078 | seed149 | 0.658 |
| hellaswag/2607 | seed149 | 0.658 |
| hellaswag/8819 | seed149 | 0.658 |
| hellaswag/876 | seed149 | 0.658 |
| hellaswag/6984 | seed149 | 0.658 |
| hellaswag/7433 | seed149 | 0.658 |
| hellaswag/591 | seed149 | 0.658 |
| hellaswag/3151 | seed149 | 0.658 |
| hellaswag/2868 | seed149 | 0.658 |
| hellaswag/61 | seed149 | 0.658 |
| hellaswag/7420 | seed149 | 0.658 |
| hellaswag/4862 | seed30841 | 0.655 |
| hellaswag/4342 | seed30841 | 0.655 |
| hellaswag/982 | seed30841 | 0.655 |
| hellaswag/5664 | seed30841 | 0.655 |
| hellaswag/9650 | seed30841 | 0.655 |
| hellaswag/959 | seed30841 | 0.655 |
| hellaswag/9444 | seed30841 | 0.655 |
| hellaswag/1336 | seed30841 | 0.655 |
| hellaswag/1938 | seed30841 | 0.655 |
| hellaswag/1900 | seed30841 | 0.655 |
| hellaswag/1944 | seed30841 | 0.655 |
| hellaswag/9384 | seed30841 | 0.655 |
| hellaswag/3273 | seed30841 | 0.655 |
| hellaswag/6119 | seed30841 | 0.655 |
| hellaswag/6138 | seed30841 | 0.655 |
| hellaswag/6986 | seed30841 | 0.655 |
| hellaswag/6760 | seed30841 | 0.655 |
| hellaswag/6470 | seed30841 | 0.655 |
| hellaswag/3385 | seed30841 | 0.655 |
| hellaswag/6844 | seed30841 | 0.655 |
| hellaswag/6964 | seed30841 | 0.655 |
| hellaswag/6510 | seed30841 | 0.655 |
| hellaswag/3289 | seed30841 | 0.655 |
| hellaswag/6233 | seed30841 | 0.655 |
| hellaswag/2823 | seed30841 | 0.655 |
| hellaswag/2627 | seed30841 | 0.655 |
| hellaswag/2240 | seed30841 | 0.655 |
| hellaswag/8001 | seed30841 | 0.655 |
| hellaswag/2618 | seed30841 | 0.655 |
| hellaswag/2469 | seed30841 | 0.655 |
| hellaswag/103 | seed30841 | 0.655 |
These are 10 problems with the lowest correlation with the overall evaluation (i.e. better models tend to do worse on these. )
| example_link | pass1_of_ex | tau |
|---|---|---|
| hellaswag/3579 | 0.600 | -0.730 |
| hellaswag/5931 | 0.500 | -0.686 |
| hellaswag/4090 | 0.500 | -0.686 |
| hellaswag/1190 | 0.700 | -0.683 |
| hellaswag/8811 | 0.700 | -0.683 |
| hellaswag/3523 | 0.700 | -0.683 |
| hellaswag/6487 | 0.300 | -0.683 |
| hellaswag/3557 | 0.600 | -0.669 |
| hellaswag/3860 | 0.400 | -0.669 |
| hellaswag/7401 | 0.400 | -0.669 |
Histogram of problems by the accuracy on each problem.
Histogram of problems by the minimum win rate to solve each problem.