OpenAI and DeepMind Diverge on AI Math Verification

OpenAI and Google DeepMind are adopting divergent strategies for verifying artificial intelligence mathematical capabilities. OpenAI has focused on generating novel mathematical claims, some of which have proven incorrect upon expert review. Google DeepMind, conversely, is concentrating on achieving state-of-the-art performance on established mathematical benchmarks.

OpenAI's approach involved its models generating complex mathematical statements. One model, referred to as '722', produced a claim concerning an infinite series. This claim was initially deemed potentially revolutionary by some observers. However, upon review by Professor James Lipton of the Royal Institute of Mathematics, the claim was found to be erroneous. Specifically, the proof provided by the AI contained a critical flaw. Professor Lipton noted that the sum of the series diverges to infinity, contradicting the AI's assertion of a finite value.

The '722' model demonstrated the ability to produce highly technical and superficially convincing mathematical proofs. It articulated sophisticated concepts, including those related to the Riemann Hypothesis and algebraic geometry. The model's outputs were frequently verbose, generating proofs exceeding 20 pages in some instances. Professor Lipton highlighted that the language used by '722' was mathematically fluent, creating a challenge for rapid human verification.

This incident is not isolated. An earlier OpenAI model made claims about number theory that also failed to withstand expert scrutiny. These instances underscore a method where models generate potentially novel, yet unverified, mathematical assertions. This strategy contrasts with DeepMind's published work, which typically reports on models achieving new performance records on recognised mathematical problems, such as those found in the MATH dataset or competitive programming challenges.

The varying strategies suggest different priorities in advancing AI's role in mathematics. OpenAI appears to be exploring the boundary of AI-generated mathematical discovery, even at the risk of producing incorrect statements. DeepMind's work tends to validate model performance against consensus mathematical understanding. Both approaches contribute to understanding AI's potential in the field, yet they highlight ongoing challenges in verifying autonomous mathematical reasoning. Does the pursuit of novelty, even flawed, accelerate discovery more than rigorous benchmarking?