AI Models Chase NanoGPT Optimizer Records
AI-generated image · Bay Street Wire
A series of autonomous runs across 18 frontier models tests the ability of AI agents to close the gap on human-set optimization benchmarks.
As Prime Intellect first reported, the company conducted a "speedrun" involving 153 autonomous runs across 18 frontier models to test their performance on the nanoGPT optimizer. According to the data provided by Prime Intellect, the exercise measured how effectively these models could close the gap toward a human record.
Among the participants, Fable 5 achieved the highest percentage of the human record gap closed at 81.7%, reaching a validated record of 2,726. Other top-performing models included Opus 5, which closed 53.6% of the gap, and Kimi K3, which closed 52.2% using the prime-agent harness. The results varied significantly across the cohort; for instance, Kimi K2.7 closed 7.2% of the gap, while GLM 5.3 produced no record.
The testing utilized various model harnesses, including claude-code, codex, grok-cli, and kimi-code. The data shows a wide range of resource consumption during these runs. GPT-5.6 Sol, using the codex harness, generated 2.9 billion total tokens over 6.1 days, while Grok 4.6 used the grok-cli harness to reach a record of 3,220 in just 0.6 days.
In an equal-budget comparison based on a 24-hour window, the models' best validated records ranged from Fable 5 at 3,010 to Kimi K2.7 and Muse Spark 1.1, both reaching 3,240.