nq_step27500: by examples

Results Paper Code


Not solved by any model

There are 2579 examples not solved by any model. Solving some of these can be a good signal that your model is indeed better than leading models if these are good problems.
nq/0, nq/1, nq/10, nq/100, nq/1000, nq/1002, nq/1003, nq/1007, nq/1008, nq/1009, nq/1011, nq/1012, nq/1013, nq/1014, nq/1015, nq/1016, nq/1017, nq/1018, nq/1019, nq/102, nq/1021, nq/1022, nq/1024, nq/1025, nq/1026, nq/1027, nq/1028, nq/1029, nq/103, nq/1032, nq/1033, nq/1035, nq/1036, nq/1037, nq/1039, nq/104, nq/1041, nq/1042, nq/1043, nq/1044, nq/1047, nq/1048, nq/105, nq/1050, nq/1052, nq/1053, nq/1054, nq/1059, nq/106, nq/1060, nq/1062, nq/1063, nq/1064, nq/1068, nq/1070, nq/1071, nq/1074, nq/1075, nq/1076, nq/1077, nq/108, nq/1081, nq/1083, nq/1084, nq/1085, nq/1086, nq/1088, nq/1089, nq/109, nq/1090, nq/1094, nq/1095, nq/1096, nq/1097, nq/1098, nq/1099, nq/110, nq/1100, nq/1102, nq/1108, nq/111, nq/1111, nq/1112, nq/1113, nq/1114, nq/1115, nq/1116, nq/1117, nq/1119, nq/112, nq/1123, nq/1124, nq/1125, nq/1126, nq/1128, nq/1129, nq/113, nq/1130, nq/1132, nq/1133, nq/1136, nq/1137, nq/1139, nq/114, nq/1140, nq/1141, nq/1143, nq/1145, nq/1146, nq/1149, nq/1150, nq/1151, nq/1152, nq/1154, nq/1156, nq/1157, nq/1159, nq/116, nq/1160, nq/1161, nq/1162, nq/1163, nq/1166, nq/1167, nq/1169, nq/1170, nq/1172, nq/1173, nq/1174, nq/1175, nq/1176, nq/1177, nq/118, nq/1180, nq/1181, nq/1182, nq/1183, nq/1186, nq/1187, nq/1189, nq/119, nq/1190, nq/1191, nq/1192, nq/1195, nq/1196, nq/1197, nq/1198, nq/12, nq/120, nq/1200, nq/1201, nq/1202, nq/1203, nq/1205, nq/1206, nq/1208, nq/121, nq/1210, nq/1211, nq/1212, nq/1213, nq/1214, nq/1215, nq/1216, nq/1218, nq/122, nq/1220, nq/1221, nq/1222, nq/1223, nq/1224, nq/1226, nq/1227, nq/1228, nq/1229, nq/123, nq/1230, nq/1231, nq/1232, nq/1233, nq/1234, nq/1235, nq/1236, nq/1237, nq/1238, nq/1239, nq/124, nq/1240, nq/1241, nq/1243, nq/1244, nq/1245, nq/1246, nq/1247, nq/125, nq/1252, nq/1253, nq/1254, nq/1255, nq/1257, nq/1258, nq/1259, nq/126, nq/1260, nq/1261, nq/1262, nq/1263, nq/1265, nq/1266, nq/1267, nq/1269, nq/127, nq/1271, nq/1272, nq/1273, nq/1274, nq/1275, nq/1278, nq/1279, nq/1281, nq/1283, nq/1284, nq/1286, nq/1288, nq/1290, nq/1291, nq/1293, nq/1295, nq/1298, nq/1299, nq/130, nq/1300, nq/1301, nq/1302, nq/1303, nq/1304, nq/1306, nq/1307, nq/1308, nq/131, nq/1310, nq/1311, nq/1313, nq/1314, nq/1316, nq/1317, nq/1319, nq/132, nq/1320, nq/1321, nq/1322, nq/1324, nq/1325, nq/1326, nq/1328, nq/1329, nq/133, nq/1331, nq/1332, nq/1334, nq/1335, nq/1336, nq/1337, nq/1338, nq/134, nq/1340, nq/1342, nq/1343, nq/1345, nq/1347, nq/1348, nq/1349, nq/135, nq/1351, nq/1353, nq/1354, nq/1356, nq/1358, nq/1359, nq/1360, nq/1362, nq/1363, nq/1364, nq/1365, nq/1366, nq/1367, nq/1369, nq/1370, nq/1372, nq/1373, nq/1374, nq/1375, nq/1376, nq/1377, nq/1379, nq/138, nq/1380, nq/1381, nq/1382, nq/1383, nq/1384, nq/1386, nq/1387, nq/1390, nq/1391, nq/1392, nq/1393, nq/1396, nq/1397, nq/1398, nq/1399, nq/14, nq/140, nq/1400, nq/1401, nq/1402, nq/1403, nq/1404, nq/1405, nq/1406, nq/1407, nq/1411, nq/1412, nq/1414, nq/1415, nq/1417, nq/1418, nq/1419, nq/1420, nq/1421, nq/1422, nq/1423, nq/1425, nq/1426, nq/1427, nq/1428, nq/143, nq/1430, nq/1431, nq/1432, nq/1436, nq/1439, nq/1440, nq/1441, nq/1443, nq/1444, nq/1446, nq/1448, nq/1449, nq/1450, nq/1451, nq/1452, nq/1453, nq/1454, nq/1455, nq/1459, nq/146, nq/1460, nq/1461, nq/1462, nq/1463, nq/1464, nq/1465, nq/1466, nq/1467, nq/1468, nq/1469, nq/147, nq/1471, nq/1473, nq/1474, nq/1475, nq/1476, nq/1477, nq/1479, nq/148, nq/1480, nq/1481, nq/1482, nq/1483, nq/1484, nq/1485, nq/1487, nq/1488, nq/1489, nq/1491, nq/1492, nq/1493, nq/1494, nq/1495, nq/1496, nq/1498, nq/15, nq/150, nq/1500, nq/1502, nq/1503, nq/1504, nq/1505, nq/1506, nq/1507, nq/1509, nq/151, nq/1511, nq/1512, nq/1513, nq/1514, nq/1515, nq/1516, nq/1517, nq/152, nq/1520, nq/1521, nq/1522, nq/1525, nq/1526, nq/1528, nq/1529, nq/1530, nq/1532, nq/1533, nq/1534, nq/1536, nq/1537, nq/154, nq/1542, nq/1543, nq/1544, nq/1546, nq/1548, nq/1549, nq/155, nq/1550, nq/1553, nq/1554, nq/1556, nq/156, nq/1560, nq/1562, nq/1563, nq/1564, nq/1565, nq/1566, nq/1567, nq/1568, nq/157, nq/1572, nq/1574, nq/1575, nq/1577, nq/1578, nq/1579, nq/158, nq/1580, nq/1582, nq/1583, nq/1584, nq/1585, nq/1588, nq/1589, nq/1590, nq/1591, nq/1592, nq/1593, nq/1594, nq/1595, nq/1596, nq/1597, nq/160, nq/1601, nq/1602, nq/1603, nq/1604, nq/1605, nq/1606, nq/1607, nq/1608, nq/1609, nq/161, nq/1611, nq/1612, nq/1613, nq/1614, nq/1615, nq/1617, nq/162, nq/1621, nq/1624, nq/1626, nq/1627, nq/1629, nq/163, nq/1631, nq/1633, nq/1634, nq/1636, nq/1637, nq/1638, nq/1639, nq/164, nq/1640, nq/1641, nq/1642, nq/1643, nq/1645, nq/1646, nq/1647, nq/1648, nq/165, nq/1651, nq/1652, nq/1653, nq/1654, nq/1656, nq/1657, nq/1658, nq/1659, nq/166, nq/1660, nq/1661, nq/1662, nq/1663, nq/1664, nq/1666, nq/1667, nq/1669, nq/167, nq/1671, nq/1672, nq/1673, nq/1675, nq/1676, nq/1677, nq/1678, nq/1680, nq/1681, nq/1682, nq/1683, nq/1684, nq/1685, nq/1686, nq/1687, nq/1689, nq/169, nq/1691, nq/1693, nq/1694, nq/1695, nq/1697, nq/1698, nq/17, nq/170, nq/1700, nq/1701, nq/1702, nq/1705, nq/1706, nq/1707, nq/1708, nq/1709, nq/1710, nq/1712, nq/1714, nq/1715, nq/1716, nq/1717, nq/1719, nq/1722, nq/1723, nq/1724, nq/1725, nq/1726, nq/1727, nq/1728, nq/173, nq/1730, nq/1734, nq/1735, nq/1736, nq/1737, nq/1738, nq/1739, nq/174, nq/1741, nq/1742, nq/1743, nq/1745, nq/1746, nq/1749, nq/175, nq/1750, nq/1751, nq/1752, nq/1753, nq/1754, nq/1755, nq/1756, nq/1757, nq/1759, nq/176, nq/1761, nq/1762, nq/1763, nq/1764, nq/1765, nq/1767, nq/1768, nq/1770, nq/1773, nq/1774, nq/1776, nq/1778, nq/1779, nq/178, nq/1781, nq/1782, nq/1783, nq/1784, nq/1785, nq/1786, nq/1788, nq/1789, nq/179, nq/1793, nq/1794, nq/1795, nq/1796, nq/1798, nq/18, nq/1801, nq/1802, nq/1805, nq/1806, nq/1807, nq/1808, nq/1809, nq/181, nq/1812, nq/1813, nq/1814, nq/1815, nq/1816, nq/1817, nq/1818, nq/1819, nq/1821, nq/1823, nq/1824, nq/1825, nq/1827, nq/1828, nq/1829, nq/183, nq/1830, nq/1833, nq/1834, nq/1836, nq/1837, nq/1838, nq/1839, nq/184, nq/1841, nq/1842, nq/1843, nq/1844, nq/1845, nq/1846, nq/1848, nq/1849, nq/1850, nq/1851, nq/1853, nq/1854, nq/1856, nq/1857, nq/1858, nq/1859, nq/1860, nq/1862, nq/1865, nq/1866, nq/1867, nq/1868, nq/1869, nq/187, nq/1870, nq/1871, nq/1872, nq/1873, nq/1874, nq/1876, nq/1877, nq/1878, nq/1880, nq/1882, nq/1883, nq/1886, nq/1887, nq/1888, nq/1889, nq/189, nq/1890, nq/1892, nq/1893, nq/1894, nq/1896, nq/1897, nq/1899, nq/19, nq/1900, nq/1901, nq/1902, nq/1903, nq/1906, nq/1907, nq/1908, nq/1909, nq/1910, nq/1911, nq/1912, nq/1913, nq/1915, nq/1916, nq/1917, nq/1918, nq/1919, nq/192, nq/1920, nq/1923, nq/1924, nq/1925, nq/1926, nq/193, nq/1930, nq/1933, nq/1934, nq/1935, nq/1936, nq/1937, nq/1939, nq/194, nq/1940, nq/1941, nq/1942, nq/1943, nq/1944, nq/1945, nq/1946, nq/1947, nq/1948, nq/1949, nq/1950, nq/1951, nq/1952, nq/1953, nq/1954, nq/1955, nq/1956, nq/1959, nq/196, nq/1960, nq/1964, nq/1965, nq/1966, nq/1967, nq/1968, nq/1969, nq/197, nq/1971, nq/1972, nq/1973, nq/1977, nq/1979, nq/198, nq/1980, nq/1981, nq/1982, nq/1983, nq/1985, nq/1988, nq/1989, nq/199, nq/1992, nq/1993, nq/1996, nq/1997, nq/1998, nq/1999, nq/2, nq/200, nq/2000, nq/2001, nq/2002, nq/2003, nq/2004, nq/2005, nq/2008, nq/2009, nq/201, nq/2010, nq/2012, nq/2014, nq/2016, nq/2017, nq/2018, nq/202, nq/2020, nq/2022, nq/2023, nq/2024, nq/2029, nq/203, nq/2031, nq/2032, nq/2033, nq/2034, nq/2035, nq/2036, nq/2039, nq/204, nq/2042, nq/2043, nq/2044, nq/2045, nq/2046, nq/2047, nq/2049, nq/205, nq/2050, nq/2053, nq/2054, nq/2057, nq/2058, nq/2059, nq/206, nq/2060, nq/2061, nq/2062, nq/2063, nq/2065, nq/2066, nq/2067, nq/2068, nq/2070, nq/2072, nq/2073, nq/2074, nq/2076, nq/2077, nq/2079, nq/208, nq/2080, nq/2081, nq/2082, nq/2085, nq/2088, nq/2089, nq/2090, nq/2092, nq/2093, nq/2094, nq/2096, nq/2097, nq/21, nq/210, nq/2100, nq/2103, nq/2104, nq/2106, nq/2107, nq/2108, nq/2109, nq/211, nq/2112, nq/2113, nq/2114, nq/2116, nq/2117, nq/2118, nq/2119, nq/212, nq/2121, nq/2123, nq/2124, nq/2126, nq/2127, nq/2128, nq/2129, nq/213, nq/2133, nq/2134, nq/2135, nq/2136, nq/2137, nq/2139, nq/214, nq/2140, nq/2141, nq/2142, nq/2143, nq/2144, nq/2145, nq/2146, nq/2147, nq/2148, nq/2149, nq/215, nq/2150, nq/2151, nq/2153, nq/2154, nq/2155, nq/2156, nq/2157, nq/2158, nq/2159, nq/2160, nq/2162, nq/2166, nq/2168, nq/2169, nq/217, nq/2171, nq/2172, nq/2173, nq/2174, nq/2175, nq/2176, nq/2177, nq/2179, nq/218, nq/2183, nq/2184, nq/2187, nq/2188, nq/2189, nq/219, nq/2190, nq/2191, nq/2192, nq/2194, nq/2195, nq/2197, nq/2198, nq/220, nq/2200, nq/2201, nq/2202, nq/2203, nq/2204, nq/2205, nq/2206, nq/2207, nq/2208, nq/2209, nq/221, nq/2210, nq/2211, nq/2212, nq/2214, nq/2215, nq/2216, nq/2217, nq/2219, nq/222, nq/2222, nq/2223, nq/2224, nq/2226, nq/2227, nq/2228, nq/2229, nq/223, nq/2230, nq/2231, nq/2232, nq/2233, nq/2234, nq/2235, nq/2236, nq/2237, nq/2238, nq/2239, nq/224, nq/2241, nq/2242, nq/2243, nq/2244, nq/2245, nq/2246, nq/2248, nq/225, nq/2250, nq/2252, nq/2253, nq/2256, nq/2257, nq/2258, nq/2259, nq/226, nq/2261, nq/2262, nq/2263, nq/2264, nq/2265, nq/2267, nq/227, nq/2271, nq/2272, nq/2273, nq/2274, nq/2275, nq/2276, nq/2278, nq/2279, nq/2280, nq/2281, nq/2282, nq/2283, nq/2284, nq/2285, nq/2286, nq/2288, nq/2289, nq/229, nq/2290, nq/2292, nq/2293, nq/2294, nq/2295, nq/2298, nq/23, nq/230, nq/2300, nq/2302, nq/2303, nq/2304, nq/2305, nq/2307, nq/2308, nq/2309, nq/2310, nq/2311, nq/2312, nq/2313, nq/2314, nq/2315, nq/2317, nq/2318, nq/2319, nq/232, nq/2322, nq/2323, nq/2324, nq/2328, nq/2329, nq/233, nq/2330, nq/2331, nq/2333, nq/2334, nq/2338, nq/2339, nq/234, nq/2340, nq/2341, nq/2342, nq/2343, nq/2346, nq/2349, nq/2352, nq/2354, nq/2355, nq/2356, nq/2357, nq/2358, nq/2359, nq/2361, nq/2364, nq/2365, nq/2366, nq/2367, nq/237, nq/2370, nq/2371, nq/2372, nq/2373, nq/2374, nq/2375, nq/2376, nq/2378, nq/2379, nq/238, nq/2380, nq/2381, nq/2382, nq/2383, nq/2384, nq/2386, nq/2388, nq/2389, nq/2390, nq/2391, nq/2393, nq/2395, nq/2397, nq/24, nq/240, nq/2400, nq/2401, nq/2402, nq/2403, nq/2405, nq/2406, nq/2408, nq/2409, nq/2410, nq/2411, nq/2414, nq/2417, nq/2418, nq/242, nq/2421, nq/2422, nq/2423, nq/2424, nq/2426, nq/243, nq/2430, nq/2431, nq/2433, nq/2434, nq/2435, nq/2437, nq/244, nq/2440, nq/2441, nq/2443, nq/2444, nq/2445, nq/2446, nq/2447, nq/2448, nq/2449, nq/245, nq/2450, nq/2451, nq/2452, nq/2453, nq/2455, nq/2456, nq/2458, nq/2459, nq/246, nq/2460, nq/2462, nq/2464, nq/2465, nq/2466, nq/2467, nq/2468, nq/2469, nq/247, nq/2470, nq/2471, nq/2472, nq/2473, nq/2477, nq/2478, nq/248, nq/2480, nq/2481, nq/2482, nq/2483, nq/2485, nq/2486, nq/2487, nq/2489, nq/249, nq/2490, nq/2491, nq/2493, nq/2494, nq/2496, nq/2498, nq/2499, nq/25, nq/2500, nq/2501, nq/2503, nq/2506, nq/2507, nq/2508, nq/2509, nq/251, nq/2510, nq/2511, nq/2512, nq/2513, nq/2517, nq/2518, nq/2519, nq/252, nq/2520, nq/2523, nq/2524, nq/2525, nq/2526, nq/2528, nq/2529, nq/253, nq/2530, nq/2532, nq/2533, nq/2534, nq/2536, nq/2539, nq/2541, nq/2542, nq/2543, nq/2545, nq/2546, nq/2547, nq/2548, nq/2549, nq/255, nq/2550, nq/2552, nq/2553, nq/2554, nq/2555, nq/2557, nq/2559, nq/2560, nq/2562, nq/2563, nq/2565, nq/2566, nq/2567, nq/2568, nq/2569, nq/257, nq/2573, nq/2574, nq/2576, nq/2577, nq/2578, nq/2579, nq/258, nq/2580, nq/2581, nq/2582, nq/2585, nq/2586, nq/2587, nq/2588, nq/2589, nq/259, nq/2590, nq/2591, nq/2593, nq/2594, nq/2595, nq/2597, nq/2599, nq/260, nq/2601, nq/2602, nq/2603, nq/2604, nq/2605, nq/2606, nq/2607, nq/2608, nq/2611, nq/2613, nq/2614, nq/2615, nq/2617, nq/2618, nq/2619, nq/262, nq/2620, nq/2621, nq/2622, nq/2623, nq/2624, nq/2627, nq/2628, nq/2629, nq/263, nq/2631, nq/2633, nq/2635, nq/2636, nq/2637, nq/2638, nq/2644, nq/2645, nq/2648, nq/2649, nq/265, nq/2651, nq/2652, nq/2654, nq/2655, nq/2656, nq/2657, nq/2658, nq/2659, nq/266, nq/2660, nq/2661, nq/2662, nq/2664, nq/2666, nq/2668, nq/2669, nq/267, nq/2670, nq/2671, nq/2672, nq/2673, nq/2674, nq/2675, nq/2676, nq/2677, nq/2678, nq/2679, nq/268, nq/2680, nq/2681, nq/2684, nq/2685, nq/2686, nq/2687, nq/2688, nq/269, nq/2690, nq/2691, nq/2693, nq/2694, nq/2695, nq/2696, nq/2697, nq/2699, nq/270, nq/2701, nq/2702, nq/2703, nq/2706, nq/2707, nq/2708, nq/2709, nq/271, nq/2710, nq/2711, nq/2712, nq/2715, nq/2716, nq/2717, nq/2718, nq/2719, nq/272, nq/2720, nq/2722, nq/2724, nq/2725, nq/2726, nq/2727, nq/2728, nq/2729, nq/273, nq/2731, nq/2733, nq/2734, nq/2735, nq/2736, nq/2737, nq/2738, nq/2739, nq/2740, nq/2742, nq/2743, nq/2745, nq/2747, nq/2748, nq/2749, nq/275, nq/2750, nq/2751, nq/2752, nq/2754, nq/2756, nq/2757, nq/2758, nq/276, nq/2760, nq/2761, nq/2763, nq/2764, nq/2765, nq/2766, nq/2767, nq/2768, nq/2771, nq/2773, nq/2774, nq/2775, nq/2778, nq/278, nq/2781, nq/2782, nq/2783, nq/2785, nq/2786, nq/2787, nq/2788, nq/279, nq/2792, nq/2793, nq/2794, nq/2795, nq/2796, nq/2797, nq/2798, nq/28, nq/280, nq/2800, nq/2801, nq/2802, nq/2803, nq/2804, nq/2805, nq/2807, nq/2809, nq/2812, nq/2813, nq/2814, nq/2819, nq/282, nq/2821, nq/2822, nq/2823, nq/2824, nq/2827, nq/2828, nq/2829, nq/283, nq/2830, nq/2831, nq/2832, nq/2833, nq/2836, nq/2838, nq/2839, nq/2840, nq/2841, nq/2842, nq/2844, nq/2845, nq/2846, nq/2847, nq/2848, nq/2849, nq/2850, nq/2851, nq/2852, nq/2855, nq/2856, nq/2857, nq/2858, nq/2859, nq/286, nq/2860, nq/2861, nq/2862, nq/2863, nq/2864, nq/2865, nq/2866, nq/2867, nq/2868, nq/2869, nq/287, nq/2871, nq/2872, nq/2876, nq/2877, nq/2878, nq/2879, nq/288, nq/2881, nq/2883, nq/2884, nq/2885, nq/2886, nq/2887, nq/2889, nq/2890, nq/2893, nq/2894, nq/2895, nq/2897, nq/290, nq/2900, nq/2901, nq/2902, nq/2903, nq/2904, nq/2905, nq/2907, nq/2908, nq/291, nq/2911, nq/2912, nq/2913, nq/2914, nq/2916, nq/2918, nq/292, nq/2920, nq/2923, nq/2924, nq/2927, nq/2928, nq/2929, nq/2930, nq/2931, nq/2932, nq/2933, nq/2934, nq/2935, nq/2936, nq/2937, nq/2939, nq/294, nq/2940, nq/2941, nq/2942, nq/2944, nq/2945, nq/2950, nq/2951, nq/2952, nq/2954, nq/2955, nq/2957, nq/2958, nq/2961, nq/2962, nq/2965, nq/2966, nq/2968, nq/297, nq/2970, nq/2971, nq/2972, nq/2973, nq/2975, nq/2977, nq/2978, nq/2979, nq/2980, nq/2982, nq/2983, nq/2984, nq/2985, nq/2987, nq/2988, nq/299, nq/2990, nq/2991, nq/2992, nq/2993, nq/2994, nq/2996, nq/2998, nq/2999, nq/3, nq/300, nq/3002, nq/3003, nq/3004, nq/3005, nq/3006, nq/3007, nq/3008, nq/3009, nq/301, nq/3010, nq/3011, nq/3012, nq/3014, nq/3015, nq/3017, nq/3019, nq/302, nq/3022, nq/3025, nq/3028, nq/3029, nq/3030, nq/3031, nq/3032, nq/3033, nq/3034, nq/3035, nq/3036, nq/3037, nq/3038, nq/3039, nq/304, nq/3041, nq/3042, nq/3043, nq/3044, nq/3046, nq/3047, nq/3048, nq/3049, nq/3053, nq/3054, nq/3055, nq/3056, nq/3057, nq/3059, nq/3060, nq/3061, nq/3063, nq/3064, nq/3066, nq/3067, nq/3068, nq/3069, nq/307, nq/3070, nq/3071, nq/3072, nq/3074, nq/3075, nq/3076, nq/3077, nq/3078, nq/3079, nq/308, nq/3080, nq/3081, nq/3083, nq/3084, nq/3085, nq/3086, nq/3088, nq/3089, nq/309, nq/3090, nq/3093, nq/3094, nq/3095, nq/3096, nq/3097, nq/3098, nq/3099, nq/31, nq/310, nq/3100, nq/3104, nq/3106, nq/3107, nq/311, nq/3110, nq/3111, nq/3112, nq/3113, nq/3114, nq/3115, nq/3117, nq/3118, nq/3119, nq/312, nq/3120, nq/3121, nq/3122, nq/3124, nq/3125, nq/3126, nq/3127, nq/3128, nq/3129, nq/313, nq/3130, nq/3131, nq/3133, nq/3134, nq/3135, nq/3136, nq/3138, nq/314, nq/3141, nq/3142, nq/3143, nq/3144, nq/3145, nq/3146, nq/3147, nq/3148, nq/3149, nq/315, nq/3150, nq/3151, nq/3152, nq/3153, nq/3154, nq/3155, nq/3156, nq/3157, nq/3158, nq/3159, nq/316, nq/3160, nq/3161, nq/3162, nq/3163, nq/3165, nq/3166, nq/3169, nq/3170, nq/3171, nq/3172, nq/3173, nq/3174, nq/3175, nq/3176, nq/318, nq/3180, nq/3181, nq/3183, nq/3184, nq/3185, nq/3186, nq/3188, nq/3189, nq/319, nq/3190, nq/3191, nq/3192, nq/3193, nq/3195, nq/3196, nq/3197, nq/3198, nq/3199, nq/32, nq/3200, nq/3202, nq/3204, nq/3205, nq/3206, nq/3207, nq/3208, nq/3209, nq/3210, nq/3211, nq/3212, nq/3213, nq/3215, nq/3216, nq/3218, nq/322, nq/3220, nq/3221, nq/3222, nq/3224, nq/3225, nq/3227, nq/3230, nq/3231, nq/3232, nq/3234, nq/3235, nq/3236, nq/3237, nq/3238, nq/324, nq/3242, nq/3243, nq/3244, nq/3245, nq/3246, nq/3247, nq/3248, nq/325, nq/3252, nq/3254, nq/3255, nq/3256, nq/3259, nq/326, nq/3260, nq/3261, nq/3262, nq/3263, nq/3264, nq/3267, nq/3269, nq/3270, nq/3271, nq/3273, nq/3274, nq/3275, nq/3276, nq/3277, nq/3278, nq/328, nq/3281, nq/3283, nq/3284, nq/3285, nq/3286, nq/3287, nq/3288, nq/3289, nq/329, nq/3291, nq/3292, nq/3295, nq/3296, nq/3297, nq/3299, nq/33, nq/330, nq/3300, nq/3301, nq/3302, nq/3303, nq/3304, nq/3307, nq/3308, nq/3310, nq/3311, nq/3312, nq/3313, nq/3314, nq/3316, nq/3317, nq/3318, nq/332, nq/3320, nq/3321, nq/3322, nq/3324, nq/3325, nq/3326, nq/3328, nq/3329, nq/333, nq/3330, nq/3331, nq/3334, nq/3336, nq/3337, nq/3338, nq/3339, nq/3344, nq/3345, nq/3347, nq/3350, nq/3351, nq/3352, nq/3353, nq/3354, nq/3356, nq/3359, nq/3360, nq/3361, nq/3362, nq/3364, nq/3368, nq/3369, nq/337, nq/3370, nq/3371, nq/3374, nq/3375, nq/3376, nq/3377, nq/3378, nq/3379, nq/3380, nq/3381, nq/3382, nq/3383, nq/3388, nq/3389, nq/339, nq/3390, nq/3391, nq/3392, nq/3393, nq/3395, nq/3396, nq/3397, nq/3398, nq/3399, nq/340, nq/3400, nq/3401, nq/3402, nq/3403, nq/3404, nq/3405, nq/3406, nq/3408, nq/3409, nq/3410, nq/3411, nq/3412, nq/3415, nq/3416, nq/3417, nq/3418, nq/3419, nq/342, nq/3420, nq/3421, nq/3423, nq/3425, nq/3428, nq/3429, nq/343, nq/3430, nq/3431, nq/3432, nq/3433, nq/3434, nq/3435, nq/3437, nq/3438, nq/3439, nq/344, nq/3440, nq/3443, nq/3444, nq/3445, nq/3446, nq/3447, nq/3448, nq/3449, nq/345, nq/3450, nq/3453, nq/3455, nq/3456, nq/3457, nq/3459, nq/346, nq/3460, nq/3461, nq/3462, nq/3463, nq/3464, nq/3465, nq/3467, nq/3468, nq/347, nq/3470, nq/3471, nq/3472, nq/3475, nq/3476, nq/3477, nq/3478, nq/3479, nq/3480, nq/3482, nq/3484, nq/3486, nq/3489, nq/3490, nq/3491, nq/3494, nq/3495, nq/3496, nq/3498, nq/3499, nq/35, nq/3500, nq/3501, nq/3503, nq/3504, nq/3505, nq/3506, nq/3507, nq/3508, nq/3509, nq/351, nq/3510, nq/3511, nq/3513, nq/3514, nq/3515, nq/3516, nq/3518, nq/3519, nq/352, nq/3521, nq/3522, nq/3523, nq/3524, nq/3527, nq/353, nq/3532, nq/3533, nq/3534, nq/3535, nq/3536, nq/3538, nq/3539, nq/354, nq/3540, nq/3542, nq/3543, nq/3544, nq/3545, nq/3547, nq/3548, nq/3549, nq/3551, nq/3552, nq/3553, nq/3554, nq/3555, nq/3558, nq/3559, nq/356, nq/3561, nq/3562, nq/3563, nq/3564, nq/3565, nq/3566, nq/3567, nq/3568, nq/357, nq/3572, nq/3573, nq/3575, nq/3576, nq/3577, nq/3578, nq/3579, nq/358, nq/3580, nq/3581, nq/3582, nq/3583, nq/3584, nq/3585, nq/3586, nq/3587, nq/3589, nq/3590, nq/3594, nq/3595, nq/3596, nq/3597, nq/3598, nq/3599, nq/36, nq/3602, nq/3603, nq/3604, nq/3606, nq/3608, nq/363, nq/364, nq/366, nq/367, nq/368, nq/37, nq/370, nq/372, nq/373, nq/374, nq/376, nq/377, nq/378, nq/379, nq/38, nq/380, nq/381, nq/383, nq/384, nq/385, nq/386, nq/387, nq/388, nq/389, nq/39, nq/391, nq/392, nq/393, nq/394, nq/395, nq/399, nq/4, nq/40, nq/400, nq/401, nq/402, nq/404, nq/405, nq/407, nq/408, nq/409, nq/41, nq/411, nq/413, nq/414, nq/419, nq/42, nq/421, nq/422, nq/423, nq/424, nq/428, nq/429, nq/430, nq/432, nq/433, nq/436, nq/438, nq/439, nq/44, nq/440, nq/441, nq/442, nq/443, nq/444, nq/446, nq/447, nq/448, nq/449, nq/45, nq/452, nq/453, nq/454, nq/455, nq/456, nq/458, nq/459, nq/46, nq/460, nq/461, nq/462, nq/464, nq/465, nq/466, nq/467, nq/469, nq/470, nq/471, nq/473, nq/474, nq/476, nq/477, nq/478, nq/479, nq/48, nq/482, nq/483, nq/484, nq/485, nq/486, nq/488, nq/489, nq/49, nq/490, nq/491, nq/493, nq/494, nq/495, nq/496, nq/497, nq/498, nq/499, nq/5, nq/50, nq/500, nq/501, nq/503, nq/504, nq/505, nq/508, nq/509, nq/51, nq/510, nq/511, nq/512, nq/513, nq/514, nq/515, nq/517, nq/518, nq/519, nq/52, nq/520, nq/522, nq/524, nq/525, nq/528, nq/529, nq/53, nq/532, nq/534, nq/535, nq/536, nq/537, nq/538, nq/54, nq/540, nq/542, nq/545, nq/546, nq/547, nq/549, nq/556, nq/557, nq/558, nq/559, nq/56, nq/560, nq/563, nq/565, nq/567, nq/568, nq/569, nq/57, nq/570, nq/572, nq/573, nq/576, nq/577, nq/578, nq/579, nq/58, nq/580, nq/581, nq/583, nq/584, nq/585, nq/586, nq/587, nq/589, nq/590, nq/591, nq/592, nq/594, nq/596, nq/597, nq/598, nq/599, nq/6, nq/60, nq/600, nq/601, nq/602, nq/604, nq/605, nq/606, nq/607, nq/609, nq/61, nq/611, nq/612, nq/613, nq/614, nq/616, nq/618, nq/62, nq/620, nq/621, nq/623, nq/624, nq/625, nq/626, nq/627, nq/629, nq/631, nq/633, nq/634, nq/635, nq/636, nq/637, nq/639, nq/64, nq/640, nq/641, nq/644, nq/645, nq/646, nq/647, nq/648, nq/649, nq/652, nq/655, nq/657, nq/658, nq/66, nq/660, nq/662, nq/663, nq/664, nq/665, nq/666, nq/668, nq/669, nq/67, nq/670, nq/672, nq/673, nq/675, nq/676, nq/677, nq/678, nq/679, nq/68, nq/680, nq/682, nq/683, nq/684, nq/686, nq/69, nq/690, nq/691, nq/693, nq/695, nq/697, nq/698, nq/7, nq/70, nq/700, nq/701, nq/702, nq/703, nq/704, nq/706, nq/707, nq/709, nq/711, nq/714, nq/715, nq/717, nq/718, nq/719, nq/72, nq/720, nq/722, nq/723, nq/724, nq/726, nq/727, nq/728, nq/73, nq/732, nq/734, nq/735, nq/738, nq/739, nq/74, nq/742, nq/743, nq/745, nq/746, nq/747, nq/749, nq/75, nq/750, nq/751, nq/752, nq/753, nq/755, nq/756, nq/757, nq/758, nq/76, nq/760, nq/761, nq/762, nq/763, nq/766, nq/767, nq/768, nq/77, nq/771, nq/772, nq/775, nq/776, nq/777, nq/778, nq/779, nq/780, nq/781, nq/782, nq/784, nq/785, nq/786, nq/788, nq/79, nq/790, nq/791, nq/792, nq/793, nq/794, nq/795, nq/796, nq/8, nq/80, nq/800, nq/801, nq/802, nq/803, nq/804, nq/805, nq/806, nq/807, nq/808, nq/81, nq/811, nq/812, nq/813, nq/814, nq/816, nq/817, nq/818, nq/819, nq/82, nq/822, nq/823, nq/824, nq/827, nq/828, nq/829, nq/83, nq/830, nq/831, nq/832, nq/833, nq/834, nq/836, nq/837, nq/838, nq/839, nq/84, nq/840, nq/842, nq/844, nq/845, nq/846, nq/847, nq/848, nq/85, nq/853, nq/855, nq/858, nq/859, nq/86, nq/860, nq/862, nq/864, nq/865, nq/867, nq/868, nq/869, nq/87, nq/870, nq/871, nq/872, nq/873, nq/876, nq/877, nq/878, nq/879, nq/88, nq/880, nq/882, nq/884, nq/885, nq/886, nq/887, nq/888, nq/889, nq/89, nq/890, nq/891, nq/892, nq/893, nq/894, nq/896, nq/897, nq/899, nq/9, nq/90, nq/900, nq/901, nq/903, nq/904, nq/905, nq/907, nq/909, nq/91, nq/910, nq/913, nq/915, nq/916, nq/917, nq/919, nq/92, nq/920, nq/921, nq/922, nq/925, nq/926, nq/928, nq/930, nq/931, nq/932, nq/933, nq/934, nq/938, nq/939, nq/94, nq/941, nq/942, nq/943, nq/946, nq/947, nq/948, nq/949, nq/95, nq/950, nq/951, nq/953, nq/955, nq/957, nq/958, nq/959, nq/96, nq/960, nq/961, nq/962, nq/964, nq/965, nq/966, nq/968, nq/969, nq/97, nq/971, nq/972, nq/974, nq/975, nq/976, nq/977, nq/978, nq/98, nq/980, nq/981, nq/986, nq/988, nq/989, nq/990, nq/991, nq/993, nq/994, nq/995, nq/996, nq/997, nq/999

Problems solved by 1 model only

example_link model min_pass1_of_model
nq/1005 seed30841 0.147
nq/1155 seed30841 0.147
nq/1490 seed30841 0.147
nq/1555 seed30841 0.147
nq/1576 seed30841 0.147
nq/129 seed30841 0.147
nq/209 seed30841 0.147
nq/2461 seed30841 0.147
nq/2350 seed30841 0.147
nq/2268 seed30841 0.147
nq/2013 seed30841 0.147
nq/2048 seed30841 0.147
nq/1731 seed30841 0.147
nq/1810 seed30841 0.147
nq/3394 seed30841 0.147
nq/3319 seed30841 0.147
nq/2837 seed30841 0.147
nq/289 seed30841 0.147
nq/2744 seed30841 0.147
nq/2834 seed30841 0.147
nq/2632 seed30841 0.147
nq/2650 seed30841 0.147
nq/857 seed30841 0.147
nq/770 seed30841 0.147
nq/521 seed30841 0.147
nq/390 seed30841 0.147
nq/851 seed30841 0.147
nq/431 seed30841 0.147
nq/2348 seed17495 0.140
nq/2247 seed17495 0.140
nq/741 seed17495 0.140
nq/434 seed17495 0.140
nq/574 seed17495 0.140
nq/754 seed17495 0.140
nq/562 seed17495 0.140
nq/3298 seed17495 0.140
nq/3178 seed17495 0.140
nq/416 seed17495 0.140
nq/2531 seed17495 0.140
nq/2915 seed17495 0.140
nq/1861 seed17495 0.140
nq/1791 seed17495 0.140
nq/2432 seed17495 0.140
nq/2947 seed17495 0.140
nq/2870 seed17495 0.140
nq/3065 seed17495 0.140
nq/1540 seed17495 0.140
nq/1569 seed17495 0.140
nq/1395 seed17495 0.140
nq/1456 seed17495 0.140
nq/11 seed17495 0.140
nq/1056 seed17495 0.140
nq/1581 seed17495 0.140
nq/2019 seed17495 0.140
nq/3201 seed17495 0.140
nq/2906 seed16385 0.138
nq/2963 seed16385 0.138
nq/3358 seed16385 0.138
nq/2880 seed16385 0.138
nq/1668 seed16385 0.138
nq/1632 seed16385 0.138
nq/1518 seed16385 0.138
nq/2392 seed16385 0.138
nq/2360 seed16385 0.138
nq/2131 seed16385 0.138
nq/1628 seed16385 0.138
nq/1984 seed16385 0.138
nq/1822 seed16385 0.138
nq/1875 seed16385 0.138
nq/2196 seed16385 0.138
nq/1285 seed16385 0.138
nq/1294 seed16385 0.138
nq/696 seed16385 0.138
nq/3525 seed16385 0.138
nq/360 seed16385 0.138
nq/544 seed16385 0.138
nq/71 seed16385 0.138
nq/856 seed16385 0.138
nq/688 seed16385 0.138
nq/674 seed16385 0.138
nq/2646 seed16385 0.138
nq/2535 seed16385 0.138
nq/2497 seed16385 0.138
nq/406 seed16385 0.138
nq/3591 seed16385 0.138
nq/445 seed16385 0.138
nq/1144 seed16385 0.138
nq/1510 seed16385 0.138
nq/774 seed6529 0.136
nq/729 seed6529 0.136
nq/2570 seed6529 0.136
nq/2625 seed6529 0.136
nq/321 seed6529 0.136
nq/2704 seed6529 0.136
nq/3132 seed6529 0.136
nq/3203 seed6529 0.136
nq/671 seed6529 0.136
nq/764 seed6529 0.136
nq/284 seed6529 0.136
nq/2789 seed6529 0.136
nq/2873 seed6529 0.136
nq/3293 seed6529 0.136
nq/1552 seed6529 0.136
nq/1268 seed6529 0.136
nq/1339 seed6529 0.136
nq/2882 seed6529 0.136
nq/2122 seed6529 0.136
nq/2249 seed6529 0.136
nq/1852 seed6529 0.136
nq/2181 seed6529 0.136
nq/1207 seed6529 0.136
nq/1158 seed6529 0.136
nq/1855 seed6529 0.136
nq/1104 seed6115 0.132
nq/128 seed6115 0.132
nq/2291 seed6115 0.132
nq/1445 seed6115 0.132
nq/1478 seed6115 0.132
nq/593 seed6115 0.132
nq/656 seed6115 0.132
nq/564 seed6115 0.132
nq/3290 seed6115 0.132
nq/480 seed6115 0.132
nq/3454 seed6115 0.132
nq/2630 seed6115 0.132
nq/3102 seed6115 0.132
nq/1004 seed6115 0.132
nq/3103 seed6115 0.132
nq/185 seed6115 0.132
nq/2476 seed6115 0.132
nq/410 seed149 0.131
nq/2488 seed149 0.131
nq/710 seed149 0.131
nq/705 seed149 0.131
nq/1091 seed149 0.131
nq/16 seed149 0.131
nq/1990 seed149 0.131
nq/2182 seed149 0.131
nq/2225 seed149 0.131
nq/2269 seed149 0.131
nq/1368 seed149 0.131
nq/1330 seed149 0.131
nq/2399 seed149 0.131
nq/2069 seed149 0.131
nq/2502 seed149 0.131
nq/875 seed149 0.131
nq/3469 seed149 0.131
nq/550 seed149 0.131
nq/1046 seed149 0.131
nq/1109 seed149 0.131
nq/1352 seed149 0.131
nq/1020 seed149 0.131
nq/457 seed24734 0.126
nq/3050 seed24734 0.126
nq/2995 seed24734 0.126
nq/954 seed24734 0.126
nq/1720 seed24734 0.126
nq/1938 seed24734 0.126
nq/2484 seed24734 0.126
nq/1082 seed24734 0.126
nq/1079 seed24734 0.126
nq/1148 seed24734 0.126
nq/1105 seed24734 0.126
nq/1499 seed24734 0.126
nq/940 seed24734 0.126
nq/883 seed24734 0.126
nq/835 seed24734 0.126
nq/3600 seed24734 0.126
nq/2967 seed24734 0.126
nq/2917 seed24734 0.126
nq/3342 seed24734 0.126
nq/3194 seed24734 0.126
nq/1066 seed24762 0.122
nq/1058 seed24762 0.122
nq/1410 seed24762 0.122
nq/1573 seed24762 0.122
nq/3309 seed24762 0.122
nq/2180 seed24762 0.122
nq/1826 seed24762 0.122
nq/1804 seed24762 0.122
nq/1437 seed24762 0.122
nq/1296 seed24762 0.122
nq/99 seed24762 0.122
nq/982 seed24762 0.122
nq/3492 seed24762 0.122
nq/2826 seed24762 0.122
nq/2584 seed24762 0.122
nq/425 seed24762 0.122
nq/699 seed24762 0.122
nq/769 seed14309 0.121
nq/979 seed14309 0.121
nq/3116 seed14309 0.121
nq/2634 seed14309 0.121
nq/765 seed14309 0.121
nq/539 seed14309 0.121
nq/3592 seed14309 0.121
nq/362 seed14309 0.121
nq/3541 seed14309 0.121
nq/171 seed14309 0.121
nq/1650 seed14309 0.121
nq/1713 seed14309 0.121
nq/1344 seed14309 0.121
nq/1905 seed14309 0.121
nq/2218 seed14309 0.121
nq/2369 seed14309 0.121
nq/241 seed14309 0.121
nq/1545 seed14309 0.121
nq/1118 seed14309 0.121
nq/1539 seed14309 0.121
nq/1729 seed14309 0.121
nq/2091 seed14309 0.121
nq/1780 seed14309 0.121
nq/1718 seed14309 0.121
nq/2193 seed14309 0.121
nq/2948 seed14309 0.121
nq/3024 seed14309 0.121
nq/3280 seed14309 0.121
nq/2596 seed14309 0.121
nq/2974 seed14309 0.121
nq/2377 seed27712 0.121
nq/2161 seed27712 0.121
nq/235 seed27712 0.121
nq/1107 seed27712 0.121
nq/1416 seed27712 0.121
nq/13 seed27712 0.121
nq/2152 seed27712 0.121
nq/1957 seed27712 0.121
nq/1610 seed27712 0.121
nq/1914 seed27712 0.121
nq/1323 seed27712 0.121
nq/3182 seed27712 0.121
nq/3219 seed27712 0.121
nq/3052 seed27712 0.121
nq/3073 seed27712 0.121
nq/2949 seed27712 0.121
nq/2643 seed27712 0.121
nq/2799 seed27712 0.121
nq/29 seed27712 0.121
nq/2642 seed27712 0.121
nq/3413 seed27712 0.121
nq/3366 seed27712 0.121
nq/3601 seed27712 0.121
nq/3474 seed27712 0.121
nq/721 seed27712 0.121
nq/712 seed27712 0.121
nq/911 seed27712 0.121

Suspect problems

These are 10 problems with the lowest correlation with the overall evaluation (i.e. better models tend to do worse on these. )

example_link pass1_of_ex tau
nq/523 0.500 -0.745
nq/3607 0.700 -0.683
nq/1318 0.500 -0.626
nq/1010 0.300 -0.618
nq/1744 0.400 -0.609
nq/398 0.600 -0.609
nq/826 0.200 -0.596
nq/475 0.800 -0.596
nq/1127 0.200 -0.596
nq/1692 0.200 -0.596

Histogram of accuracies

Histogram of problems by the accuracy on each problem.

Histogram of difficulties

Histogram of problems by the minimum win rate to solve each problem.