The Future of Math Research in the Age of AI

A guest post by Tamara Kolda

Mathematicians are finally calling the industry’s bluff. After years of watching the results of performance on the International Mathematical Olympiad and other supposedly representative benchmark tests obtain wide and favorable publicity in support of the exaggerated predictions of what Eric Schmidt calls the San Francisco consensus,¹ the mathematical community, or more precisely a group of highly respected mathematicians, has developed a project, called First Proof, whose goal is

to develop an objective and realistic methodology for assessing the capabilities of AI systems to autonomously solve research-level math questions.

The first batch of ten questions, which team members had solved but not published in the course of their work, was released on February 5; their solutions were made public on February 13, after the industry had a week to work on the problems and the team had tested them on industry models.

For once the media was given the opportunity to report on an initiative involving math and AI that was not sponsored by the industry. Several articles covered the story with a tone markedly different from what we had been seeing. Siobhan Roberts interviewed several team members in a New York Times article entitled²

These Mathematicians Are Putting A.I. to the Test

Scientific American’s Joseph Howlett devoted three articles to the project.³ And my colleague Andrew Blumberg, a member of the First Proof team, was interviewed yesterday by Columbia News.

Also yesterday, OpenAI released the record of its own attempts to solve the First Proof problems. They explain

We ran the model with limited human supervision. When prompting versions of the model along training, we sometimes suggested retrying strategies that appeared fruitful in earlier attempts.

This human input is strictly against the rules of the project; as Howlett explains in the third of his articles,

Some submissions appeared to employ varying degrees of human input, with several seemingly being the result of weeklong dialogues that were checked by mathematicians. Importantly, the First Proof rules disallow human mathematical input or prodding.

Readers of Karen Hao’s book may already be inclined to be suspicious of anything OpenAI claims, in particular claims related to mathematics.

On March 14⁴ the First Proof team will be releasing details on its second batch of problems, including a timeline. This time, they

will implement a benchmarking phase prior to the community release. The benchmark phase will be designed to ensure the following features:

• Verification that the solutions are produced autonomously.

• A formal grading scheme and refereeing, modeled on the journal review system.

• An explicit description of the problem selection process, including advance internal testing on systems which have a zero data retention policy.

In the meantime, I am very pleased to be able to publish a guest post by Tamara Kolda, a member of the First Proof team, sharing in her own words her reaction to the first round results.

---

The Future of Math Research in the Age of AI

Can AI ever be a genuine collaborator?

by: Tamara Kolda

Results are in for the “first proof” experiment. First Proof is a project by a group of mathematicians, led by Mohammed Abouzaid (Stanford), Nikhil Srivastava (Cal), Rachel Ward (UT Austin), and Lauren Williams (Harvard), that tests whether AI systems can solve research-level math problems on their own. It presents ten original questions drawn from the authors’ own research across diverse areas of mathematics for which solutions exist but have not yet been published. (I am one of the problem contributors.) These are specialist problems which would normally require at least a graduate student with specialized knowledge to solve them. This is an attempt to develop a realistic way to measure AI’s ability to do genuine mathematical research, and the plan is to repeat this experiment with new problems in the future.

So how did AI systems do on this first batch of problems? Both major AI companies and individuals have shared their attempts to solve the problems, sometimes using a mixture of AI and human collaboration. Our own tests of Gemini Deep Research and ChatGPT 5.2 Pro solved 2 of 10 problems. My problem (#10) was one that both AI systems were able to solve. On the positive side, they identified a published technique that was not in my solution. On the negative side, they did not actually provide a citation for that technique. It was only the similarity of the two AI-generated solutions that led me to suspect that the AI systems had incorporated a known result, and I had to do my own detective work to find the source. Whatever we may learn from this experiment, it’s already clear to me that mathematical research is going to be forever changed by the advent of modern AI systems.

Before we dig in more deeply, I’d like to explain what a modern AI system is from my mathematical perspective: it is a system of equations with weights learned from training data. We have been using mathematical models for centuries, though usually with just a handful of parameters. In my early days as a researcher, I worked on circuit simulation models, which generally had only a dozen or so parameters. The AI models of today have billions and even trillions of parameters, arguably large enough to store the sum of human knowledge! There is much debate about whether AI works by memorizing its training data. Regardless, an AI system is ultimately a specific mathematical process: a set of equations generates its outputs, and this formalization means that it is vulnerable to mathematical attacks. While I respect that others may have different viewpoints, I personally harbor no illusions about AI consciousness or genuine reasoning capability; I am instead astounded by the creativity and breakthroughs in the design and training of AI systems.

Indeed, as I have experimented with AI systems in my own mathematical work, I have been increasingly impressed by their abilities. If a solution is accessible to an AI system as part of its training data or via web search, then it seems quite possible that an AI can solve the problem. Impressively, this tends to work even if the solution is couched in different nomenclature or has to be cobbled together from multiple sources. It seems to me that the likelihood that an AI can do this is proportional to how prevalent the general approach is in the literature. One issue — and it’s a killer — is that AI models cannot reliably provide sources for their knowledge. And the lack of fact-checking is a real problem. The AI may be confidently citing non-existent, untrue results or plagiarizing existing literature, as I observed with the solutions to my problem. On a good day, an AI system can blow your mind. On a bad day, I’ve seen it misrepresent what it’s actually done, feign remorse when called out, and do it all over again. All this means that it can be very hard to differentiate good results from mathematical slop: initially plausible answers that unravel as one digs into the details. The problem is that it is so tempting to accept an AI’s output at face value.

One of my worries for the future of academic publishing is the rising incidence of Human-AI scrapple, AI slop amalgamated by humans without careful, time-consuming validation. (Scrapple is a mixture of pork scraps that is a gristly cousin of spam.⁵) In my capacity as Vice President of Publications for SIAM, I deal with cases involving author integrity and can see the costs of taking shortcuts with AI. It requires more effort from editors and referees to detect the poor scholarship. Obvious cases involve fabricated citations. Less obvious cases consist of weak arguments, missing citations, and incoherent reasoning. We’ve already seen the negative ramifications of AI+human scrapple for conferences like NeurIPS, where hallucinated citations are endangering the credibility of this previously acclaimed venue.

So, what does the future hold for mathematics? Let us assume that all the present-day problems with AI systems (such as citations and hallucinations) can be fixed. What would be the role of a mathematician then? Well, first and foremost, the role of the mathematician is judgment: deciding what questions to ask, what theorems to prove, what algorithms to write. This requires someone who has experience, which is why the main job of an advisor is to help a beginning researcher choose what problem to work on. In my role as an applied mathematician, my major responsibility is translating vague stakeholder problems into specific mathematical questions. Once the problem is reduced to mathematics, I’ve often engaged with brilliant collaborators to find the answers. Looking back, could AI have fulfilled that role? Maybe an AI could solve a given math problem, but it has neither the desire to do so nor drive for creative insight, no opinion on whether or not the question makes sense, no stance on the right approach. In contrast, my collaborators have a point of view. They are able to debate whether we’re asking the right question, inspire a radically different approach, and sometimes change my entire mathematical viewpoint.

Mathematicians of the future will undoubtedly employ AI systems as powerful tools — just as they have employed computers and the Internet as those technologies came along — but they will not be replaced by them. My hope for the future of mathematical research is that it will be produced by people who care about the outcome of their work, who have a stake in the correctness of their results, and who are willing to put in the hard work (including carefully vetting the output of AI systems) to get there.

Acknowledgement: I am grateful to my colleagues Mohammed Abouzaid (Stanford), Andrew Blumberg (Columbia), Ernest Davis (NYU), Gary Marcus (NYU, Emeritus), Dan Spielman (Yale), Nikhil Srivastava (Cal) and Lauren Williams (Harvard) for their insights and feedback on this piece.

About the author: Tamara Kolda is an applied mathematician specializing in data science and artificial intelligence. Her consultancy is MathSci.ai. She is a Fellow of both SIAM and ACM as well as a member of the National Academy of Engineering. She currently serves as Vice President of Publications for SIAM.

1. "Picture explaining quantum mechanics to your golden retriever. Now imagine you are the retriever!” See this account for more details. Recently this publicity has taken the form of a four-author article in Nature claiming that Large Language Models have already acquired Artificial General Intelligence, and we’d better get used to it.

2. Although this was indeed the published title, my initial Google searches for the article turned up the very different title

These Mathematicians Are Trying to Educate A.I.,

obviously the work of some incorrigible editor who completely missed the point of the First Proof project. This title is still circulating on the web: type it into your search bar and see for yourself.

3. In reverse chronological order:

1. First Proof is AI’s toughest math test yet. The results are mixed

2. Is AI on the precipice of revolutionizing math? It depends

3. Mathematicians launch First Proof, a first-of-its-kind math exam for AI

In contrast to most recent coverage, and most coverage elsewhere, the mathematicians’ comments quoted in these articles appear to be an accurate reflection of current ambivalence to these techniques. Just as in the case of the Times article, though, another incorrigible editor gave the second of Howlett’s pieces a secret title much more aligned with the “San Francisco consensus”:

AI uncovers solutions to Erdős problems, moving closer to transforming math

and this alternative title actually floats above the article when you read it.

4. Guess why!

5. Note from the editor. As someone [M.H.] who grew up in Philadelphia, I feel compelled to defend this regional specialty, “also known” (as Wikipedia informs us) “by the Pennsylvania Dutch name Pannhaas.” I regularly saw it advertised on the local UHF TV channels and in pull-out sections of the Sunday papers, the very papers where I also learned how Li’l Abner foiled General Bullmoose’s plan to acquire all the money in the world.

Having done my duty in support of my regional culture, I should also confess that I have never actually tasted the stuff. At least one member of the First Proof team is apparently from Philadelphia, and is welcome to provide a first-hand report.