FrontierMath Scores Reach 94%, Saturating Benchmark in Under Two Years

Frontier artificial intelligence models have seen their accuracy on the specialist FrontierMath benchmark jump from 2 per cent to 94 per cent in less than two years.

The trajectory mirrors a broader trend across automated reasoning benchmarks, where evaluation frameworks designed to withstand years of architectural iteration saturate within months. Developed by Epoch AI, FrontierMath was introduced to test frontier models on research-grade mathematical problems. The benchmark comprises hundreds of original problems spanning advanced mathematical disciplines, created in collaboration with professional mathematicians. Most problems require hours or days for human specialists to solve and are deliberately structured to prevent simple pattern-matching.

When evaluations began, the initial results underscored the difficulty of the task. Leading foundation models, including top-tier proprietary systems, solved fewer than 2 per cent of the problems. The failures highlighted persistent deficits in multi-step deductive reasoning, long-horizon planning, and intermediate self-correction.

The jump to 94 per cent demonstrates the impact of post-training paradigms focused on extended inference-time compute and reinforcement learning. By allocating compute dynamically during generation, contemporary models can search branch spaces, verify candidate proofs, and backtrack from erroneous deductions before producing a final answer. Combined with domain-specific synthetic data generation, these systems have largely bypassed the failure modes that crippled earlier iterations.

This compression of the evaluation timeline presents a practical challenge for researchers measuring model progress. Previous academic baselines, such as GSM8K and the Hendrycks MATH benchmark, took several years to reach functional saturation. FrontierMath was explicitly designed to provide a durable metric for expert-level performance that would resist rapid obsolescence.

With the benchmark now largely exhausted, evaluating the outer boundaries of mathematical reasoning will require novel testing methodologies. Generating uncontaminated, research-level mathematical problems demands significant contributions from human domain experts, making regular benchmark updates both expensive and labour-intensive. Alternative approaches, including autonomous theorem proving in interactive environments such as Lean, offer one potential avenue.

As static evaluations continue to fall faster than researchers can build them, how will frontier labs objectively verify whether a system is truly reasoning or merely optimising for existing benchmark topologies?