Mathematicians may actually have a problem: AI is producing new mathematics faster than the mathematical community understands mathematics.
A few days ago, OpenAI unexpectedly released on GitHub a series of mathematical research results generated by front-end models that were not published internally, reaching a staggering 722 papers in various fields, including numerology, algebraic geometry, analysis, combinatorial mathematics, and some with Lean formalized proofs.
Address of the warehouse:
In the last few days, there have been real problems in the mathematical community.
The Verge interviewed more than 30 mathematicians who described the impact of OpenAI as "shocking, breath-taking, unprecedented, dream-like, just crazy".
Despite differing responses, researchers generally agree on the same thing: it may take years to understand the content of OpenAI's publication, and AI produces new mathematics at a rate that begins to exceed the speed at which the mathematical community understands, verifies and absorbs mathematics.
What is even more troubling is that OpenAI may not wait for the mathematical community to slowly digest these results and continue to move forward, even releasing more new results.
In an interview, New York University mathematics professor Tristan Buckmaster said that OpenAI's one-time publication of 722 manuscripts has destroyed the research paths of some early professional mathematicians and even made some complete research projects lose the meaning of continuing advancement.
After the outbreak, verification became a new bottleneck.
The number of achievements is so large that it is even difficult for the outside world to make a preliminary judgement about what OpenAI has released.
In the hours or even days after the results were published, most of the mathematicians interviewed by The Verge said they were still trying to digest the entire content. According to some researchers, it took almost an hour to read the nearly 40-page catalogue and summary.
According to Álvaro Lozano-Robledo, a professor of mathematics at the University of Connecticut, just browsing through a complete list of abstracts makes one feel overwhelmed.
In the hundreds of manuscripts, some formal proofs of writing using Lean are inserted.
But the extent to which these results have been verified varies greatly.
OpenAI acknowledges on GitHub that these results are currently in different verification stages, and that many manuscripts have been formalized, but not all.
Of the more than 700 manuscripts, only 300 core results have been formalised, accounting for about 42 percent, OpenAI said. The company said it would continue to update the warehouse as more formal proofs were obtained.
Many mathematicians complained to The Verge about the lack of formal verification coverage, especially in such a large number of results. They emphasized that even if a result is accompanied by Lean code, evaluation does not necessarily mean that it can be completed immediately.
The researchers also need to examine further: whether this formalized proof supports the conclusions claimed in the paper or merely supports a related but not entirely equivalent proposition.
This is also a time-consuming process.
Some researchers who are checking papers one at a time say that the quality is not consistent, even if proof has been provided that can be verified by computer; sometimes Lean's verified propositions are not completely consistent with the conclusions claimed in the paper's original text.
Kevin Buzzard, a professor of mathematics at Imperial College London, said that he found a number of related theorems in the area of algebraic number theory in his research, but only about six of these results were particularly prominent at first glance.
Buzzard said: "So I either have to read the trash that may or may not be correct, or wait for others to read it first, or wait for someone to formalize it, and then I can determine if the results are correct".
Several other researchers have expressed similar concerns.
Researchers Fan Nie said that when they used the self-contained verification system OSVerify to examine OpenAI's October 4 version of the Global quantum geometric Langlands at irrational level algorithm, they discovered two major problems, one of which the system proposed to fix.
This is the first time this has happened.
This further illustrates that after OpenAI mass-produced cutting-edge mathematical results, verification itself is turning into another task that needs to be scaled up. A new situation has even begun to emerge: AI is responsible for proof generation, and another set of AI systems are responsible for finding errors and fixing them.
The mathematical flood of OpenAI is also very sloppy.
Buzzard isn't the only one worried about AI slop.
The slop is often used to describe content produced by low-quality, often incorrect AI. In recent years, such content has become increasingly common on the Internet and even in the real world, including academic papers.
Mathematicians told The Verge that in recent years, there has been a significant increase in this type of content generated by tools such as ChatGPT, Claude and others. Many materials are logically confusing, difficult to read, and do not reflect a true understanding of the field; the problem is particularly apparent in terms of citations and credits to other researchers.
Some of OpenAI's previously published mathematical research texts have been widely criticized by experts for their rough writing, with one notable problem being insufficient citation, or even complete lack of attribution.
Therefore, prior to this mass release, some researchers have privately referred to the upcoming paper flood as slopocalypse, or the end of AI junk content.
As for whether the slopocalypse that is being feared has actually occurred, it is difficult to say at this time, mainly because too much material was actually released by OpenAI at one time.
Early indications suggested that OpenAI this time appeared to be more cautious in terms of paper quality, at least in part. Several mathematicians told The Verge that their first impressions were much better than expected.
However, they also acknowledge that this standard is not high because the quality of some of the mathematical materials that OpenAI has previously released would have been worse.
Better than before does not mean that it is already good enough, nor does it mean that it has reached the academic standards that such important mathematical claims should meet under normal circumstances.
The quality of supporting papers is especially important when the majority of OpenAI's conclusions have not yet been formally verified. These papers are the main basis for mathematicians to identify, understand, review and locate relevant results.
Writing a rigorous mathematical paper is not an easy task, even in the best of circumstances.
And like OpenAI, doing this work on such a large scale at the same time is a very difficult task.
OpenAI's models are producing new results in a large number of different branches of mathematics at an astonishing rate, far beyond the scrutiny of human researchers within the company.
The problem is that OpenAI does not have the extensive expertise and human resources to really rigorously review these outcomes at the forefront of mathematical research.
According to researchers interviewed by The Verge, this is already reflected in the paper.
Many people describe some of these papers as difficult to understand, and sometimes almost unreadable.
Professor of Mathematics at Brown University