Physics
OpenAI's AI math dump: 372 results, some 'slop', scientists say
Mathematicians are parsing through a flood of AI-written papers on open problems, finding brilliance and nonsense, Scientific American reports.
It has been two days since OpenAI released what one mathematician called a "slop drop" of AI-generated results on 372 open math and computer science problems. Scientists are still sorting through the pile, Scientific American reports, and it will be months or years before the full impact is known.
Harvard mathematician Michael Douglas told Scientific American that Tuesday "will be the day when a new era of math and mathematical physics began." He called it even bigger than last month's solution of the Navier-Stokes problem, one of math's million-dollar challenges, by the same OpenAI model.
The quality varies wildly. Some papers are readable, others "AI slop," said Roland Bauerschmidt of New York University. Another mathematician said if he had received such a paper from a stranger, he would have deleted it. But 42% of the results are coded in the Lean programming language, meaning their logic is almost certainly sound even if the English is hard to follow.
Three papers have already been withdrawn after experts found a significant error. Andrew Sutherland of the Massachusetts Institute of Technology expects some proofs to contain mistakes, possibly serious ones. He also criticized OpenAI for not releasing the thousands of problems the model failed to solve, which he said "is how open science works" and that you need to report negative results along with positive ones.
Several mathematicians independently told Scientific American that the model itself may be coming up with problems to solve, perhaps through extended conversations among a swarm of AI agents, because some papers share strikingly similar ideas and methods. OpenAI did not reply to a request for comment.
The Association for Human Mathematics issued a statement calling the release "not a demonstration of scholarship, but a demonstration of power." It urged mathematicians to discontinue work with OpenAI and return to a vision of science centered on human understanding.
For now, mathematicians are left to sift through the heap, trying to separate genuine advances from the slop. As Douglas put it, it is the start of a new era, even if no one is sure yet what it looks like.
Related stories
Physics
OpenAI releases hundreds of math results, mathematicians skeptical
The company posted results on GitHub, but critics call for more transparency
Explainers
Super Mario levels proven mathematically impossible, MIT shows
A team of MIT researchers has shown that certain levels in newer Super Mario games are as hard as the halting problem—meaning no algorithm can always determine if they can be beaten.
Explainers
AI Safety Tests Face a Hard Problem: Defining Good
Why experts say current tests for misaligned AI agents are too weak, and what they propose instead.