Hypothetical AI challenges and the construction of mathematical value
Hypothetical AI challenges and the construction of mathematical value
Venkatesh's essay and the upcoming Fields Medal Symposium, Part 1
A new orthodoxy has taken command, not so much by winning arguments, but by wielding systemic power on a global scale… People have learned to live with problems unresolved or unacknowledged… Conceptualization is weak and confused. Contextualization is thin and random.
(Helena Sheehan, Science for the People, May 2022)
The game of mathematics may not yet be quite over, but in 2020 a Science Magazine report on structural biology was already announcing that THE GAME HAS CHANGED. They were referring to a specific game called Critical Assessment of Protein Structure Prediction (CASP), but the title strongly implied that an “AI triumph” had changed the entire field of structural biology. CASP contestants predict the 3D structures of proteins on the basis of their amino acid sequences, with the winner chosen on the basis of closeness to the structure determined by experiment. 2020 was the year the DeepMind, the subsidiary of Google’s parent company Alphabet that specializes in massively publicized announcements of AI triumphs, not only won the CASP competition with its deep learning algorithm AlphaFold, but did so with a score “considered on par with experimental methods.” Earlier this year, the Guardian reported that DeepMind “finished the job” by publishing the predicted structures of 200 million proteins from “every organism that has had its genome sequenced.” For one researcher this amounts to
a complete paradigm shift. We can really accelerate where we go from here – and that helps us direct these precious resources to the stuff that matters.
while another anticipated “an avalanche of new and exciting discoveries.”
How is this like and unlike mathematics, whose DeepMind treatment has already been reported here? What is the “stuff that matters”? DeepMind promised in 2020 that AlphaFold “will have far-reaching effects, among them dramatically speeding the creation of new medications”; the 2022 Guardian article talked of “improved vaccines” and “digest[ing] and recycl[ing] plastics.”
DeepMind etched itself into the collective memory in 2017 when its AlphaZero attained literally superhuman status in chess and go in less than 24 hours, simply by playing against itself. Alluding to this unsettling development, Akshay Venkatesh, of the Institute for Advanced Study in Princeton, released a short essay earlier this year, inviting mathematicians to anticipate how AI, in the form of an algorithm that he calls1א(0), will alter the practice of “research in pure mathematics.”
Our starting point is to imagine that א(0) teaches itself high school and college mathematics and works its way through all of the exercises in the Springer-Verlag Graduate Texts in Mathematics series. The next morning, it is let loose upon the world – mathematicians download its children and run them with our own computing resources. What happens next – in the subsequent decade, say?
This year the Fields Institute in Toronto had scheduled its annual Fields Medal Symposium in honor of Venkatesh, who received the Fields Medal in Rio de Janeiro four years ago. Past Fields Medal Symposia have been devoted to “explor[ing] the current and potential impact of [the Medalist’s] work.” That’s a quote from the Fields Institute website, where it is explained that Venkatesh, whom they call “The Great Connector”
has asked to do things a little differently. His symposium will address "The Changing Face of Mathematical Research" where speakers from diverse fields will address the influx of new perspectives on the nature of research and of proof. Given the evolving technological landscape we are living in, it is a topic very much of the moment and one that acknowledges the inextricable relationship between research and its context.
I am one of the organizers of this symposium, along with Kevin Buzzard, Maia Fraser, and Alma Steingart. The Scientific Program is already online, and in a few days I will participate in the final Panel Discussion. To get in the mood, I am dedicating this week’s installment, in two parts, to challenges inspired by Venkatesh’s essay and by his reflections on value.
All but unrecognizable
"We should begin," Venkatesh writes, "by observing that human mathematical research is in no danger of being killed.” By the time this sentence appears he had already begun, however, by announcing his main point:
the mechanization of our cognitive processes will enhance our ability to do mathematics but also will alter our understanding of what mathematics is.
He immediately adds—
We cannot meaningfully assess the first point without taking into account the second.
— thus by implication agreeing with the rationale for my Substack project, that practically everything that has heretofore been written about the mechanization of mathematics is meaningless.2 It is my dearest hope that this year's Fields Medal Symposium will be the signal for a foray into the depths of the realm of meaning where philosophers are reluctant to venture. Perhaps there will also be the general acknowledgment that א(0) will not be able to teach itself the meaning of meaning overnight with the same hypothetical ease with which it will have taught itself to perform the training rituals of graduate mathematicians.
The mathematics that emerges from the mechanization process, Venkatesh suggests,
will be inestimably more powerful than ours, in the sense that its ability to solve any specific question will be vastly greater
but its meaning, at least as measured by "its central questions and values," will be so altered as to be "all but unrecognizable to us." Toward the end of the essay, seeking a metaphor for the magnitude of the alteration, he writes
The impact of א(0) on mathematical cognition may be much greater than the passage of a hundred years. To find a suitable parallel for this effect on our thought process, we might consider, for example, the introduction of algebraic notation in mathematics.
His final sentence insists that “it is important for us to consider seriously the possibility of such developments.” The “us” to whom he refers may initially have been just the attendees at the “ongoing interdisciplinary seminar examining some of the impacts of machine learning” at which he delivered the lecture on which this essay was based, or upcoming Fields Medal Symposium participants, or the mathematical community his essay collectively apostrophizes. I want to redirect his message: it is important for YOU who are reading this essay, or Venkatesh’s essay, to consider these matters seriously. And I submit that you cannot do that withou considering seriously how mathematicians come to a consensus on the “central questions and values” of the practice.
Constructing value
In the remainder of the essay, I will discuss how value and consensus is constructed and maintained in current research mathematics, and then consider how א(0) will affect some of these processes.
He clarifies that he is referring to
The valuation mechanism… [which] constrains with an iron, if invisible, hand the mathematics we can feasibly do. It is responsible for selecting what we are exposed to in talks, seminars and papers, and for incentivizing some questions over others. In a sense, it defines what mathematics is at any given time.
"processes by which our field decides which questions are interesting and fruitful." Although "as practitioners we rarely stop to think about” the processes by which value and consensus are constructed, he believes
there are many reasons not to leave the examination of such matters entirely to historians and sociologists of science.
Many, if not most, of my colleagues seem to believe that value flows from fountains on the campuses of selected institutions of higher learning; and in order to acquire value it’s enough to be admitted to one of these institutions and to drink from their fountain.3 Recognizing that value is constructed is a sign that Venkatesh is on the right track.
In addition to processes that mathematics has in common with other academic fields,4 Venkatesh agrees with social scientists5 who have studied mathematics that the high degree of consensus on values that is peculiar to mathematics is
eventually derived from our much higher level of consensus on the narrow issue of validity of proof.
He sees the construction of value as
a Bayesian process of updating our mental landscape of mathematics and mathematicians as we receive information about it.
Roughly half of Venkatesh’s essay is devoted to thinking through how the process works in practice. The following excerpt exemplifies his approach.
Suppose that we learn of a relationship between two conjectures in our field:
(1) conjectureX ⇒ conjectureY.
This could mean that (i) conjecture X is more important than we thought, or that (ii) conjecture Y is easier than we thought. In practice we decide (to some extent unconsciously) according to the prior uncertainty of our beliefs: if Y is a conjecture of long standing, option (i) is more likely, and if X is a conjecture of long standing, option (ii) is more likely. Nor does X need to imply Y for this conclusion - they need only be linked in some substantive way. A similar situation occurs if
(2) mathematician A proves conjecture Y ;
this is possible evidence that either A is a good mathematician, or that Y is an easy conjecture, and in practice we again choose in a fashion dependent on our prior information. In either of the situations (1) or (2), our views and uncertainty about both interacting parties are altered.
This makes so much sense that it takes some effort to realize how refreshingly different this is from mainstream philosophy of mathematics.6 Venkatesh concludes that
The parts of the mathematical process that can be speeded up the most by א(0) will have the greatest reductions in their perceived difficulty, and, according to our model above, will suffer the greatest reduction in status.
This sort of reduction in status is by no means unprecedented — the calculation of the area under a parabola by Archimedes, a triumph of ancient geometry, is now taught in the first week of integral calculus — but א(0) would accelerate this enormously.
Just positing the arrival of what is elsewhere called an “human-level” artificial mathematician has inspired Venkatesh to write the most concentrated meditation on mathematical value I’ve seen in a long time. But mathematical AI is not merely a fulcrum on which to rest our collective introspection on the meaning of our craft. In the remainder of this essay I’ll suggest some specific but concrete challenges that readers may want to contemplate while waiting for the Fields Medal Symposium to convene.
A common sense challenge for an artificial mathematician
Just about a decade ago international mathematics was agog at the news that a “little-known” mathematician named Yitang Zhang had proved the first result that could plausibly be included in a short New York Times article about the Twin Primes Conjecture. Of course what I just wrote is misleading; in mathematics there is no such thing as “the first result” about anything, or at least there hasn’t been since the invention of writing. Still, a story, whether by a journalist or a historian or a philosopher, has to begin somewhere. Part of our task, in evaluating how א(0) will make mathematics “all but unrecognizable,” is to decide how and where to draw the line between “before” and “after.”7
A more complete history of Zhang’s discovery would begin with the first recorded statement of the Twin Primes Conjecture. According to the Encyclopedia Britannica, this can be dated to 18468, when Alphonse de Polignac first formulated the conjecture, which we would state as follows:
Conjecture (de Polignac): For any integer k > 0, there are infinitely many prime numbers p such that p+2k is also prime.
The Twin Prime conjecture is the case k = 1. It has the advantage that it is easy to find examples (3 and 5; 5 and 7; 11 and 13; 29 and 31…) but it is also generally acknowledged that Yitang Zhang’s methods are too weak to prove it. What Zhang proved is a statement of an apparently different shape:
Bounded Gaps Conjecture: There exists an integer N > 0 such that the set of pairs of primes p < q such that q - p < N is infinite.
In 2013 Zhang announced a proof with N = 70000000. By the time his proof was published the very next year, in the prestigious Annals of Mathematics, James Maynard, who received a Fields Medal in 2022, had improved the estimate to N = 600; and the collaborative Polymath Project — which really did represent a new way for humans to do mathematics9 — had got N down to 246. As far as I can determine, this is still the best estimate. The experts have proved that one can get N = 6 if one assumes the Elliott-Halberstam Conjecture; but they insist that, even assuming such strong conjectures, the case N = 2, which is the Twin Primes Conjecture, would require new methods.
Whether or not you are a mathematician, you should be able to deduce the following theorem from the Bounded Gaps Conjecture (with N = 246):
Theorem: There is an integer k ≤ 123 for which Polignac’s conjecture is valid.
If you deduced the theorem before you saw it stated, merely upon reading that the Bounded Gaps Conjecture had been proved for N = 246, then you may be a mathematician. The argument, for those readers who are not mathematicians, is called the Pigeonhole Principle. The idea is that if you have to fit infinitely many pigeons (pairs of primes p < q) into finitely many holes (1 ≤ q - p ≤ N/2), then at least one of the holes will have to contain infinitely many of the pigeons.
Child psychologists will correct me if I’m wrong, but I guess that if you ask four five-year old children to share eight cookies, they will spontaneously agree that if would be fair for each of them to eat two cookies. Now if you ask them to be sure to eat all the cookies, eventually, even if they’re not hungry now, will they spontaneously conclude that at least one of them will have eaten two cookies? At what age does such reasoning become routine? Maybe only a mathematician would routinely think about cookies in that way?
In the spirit of the Winograd Schema Challenge mentioned in a previous essay, I propose two versions of a common sense challenge for mathematical AI:
Version 1. Given the statement of the Bounded Gaps theorem, deduce de Polignac's conjecture for some k ≤ 123 upon being prompted.
Version 2. Given the statement of the Bounded Gaps theorem, spontaneously deduce de Polignac’s conjecture for some k ≤ 123, without being prompted.
I would not be at all surprised or impressed by an א(0) meeting the challenge of Version 1, even if it surpasses the abilities of the average five-year old. Version 2, on the other hand, while not rising to anything I would recognize as “human-level” intelligence, would at least display some level of common sense.
A real (Turing?) challenge for an artificial mathematician
Pulling out of your pocket an AI able spontaneously to apply the Pigeonhole Principle would be a neat way to break the ice at parties but it would hardly make mathematical research “all but unrecognizable” or “inestimably more powerful.” To test whether א(0) qualifies for Venkatesh’s thought experiment, we need to set it a heftier challenge. My next proposal places א(0) in the situation that Yitang Zhang faced on the brink of his solution to the Bounded Gaps Conjecture. Readers are invited to submit their own challenges, in any branch of mathematics.
In 2005, Goldston, Pintz, and Yıldırım (GPY) had made what K. Soundararajan called “a spectacular breakthrough” on gaps between successive prime numbers. They had notably shown that, assuming the Elliott-Halberstam conjecture, their methods implied the Bounded Gap Conjecture with N = 16. Eight years would pass before Zhang, working at human level, realized how to combine GPY with a “stronger version of the Bombieri-Vinogradov method” to prove the conjecture unconditionally, albeit with a much larger N.
Version 1. א(0) is given the GPY paper and all references therein, and so on back through the mists of mathematical history, but without any special attention to the Bombieri-Vinogradov method, and is asked to deduce the Bounded Gaps Theorem (in Zhang's or Maynard's version).
Version 2. Same as 1, but now we wait for א(0) to deduce the Bounded Gaps Theorem unprompted. This and the next two challenges amount to Venkatesh’s “let[ting] loose” but only on a circumscribed part of “the world.” It would have to be handled carefully, since the GPY paper explicitly show N = 16 conditionally, as we already mentioned, and even mentions this fact in the abstract.
Version 2a. Same as 1 or 2, dropping GPY and its accompanying references, to be replaced by the analytic number theory literature of the 19th century (stopping with Hadamard and de la Vallée Poussin, or Riemann, or Dirichlet, or Gauss...).
Version 2b. Same as 2a, but given only the axioms of set theory as starting point.
It might be fun to estimate (on the back of an envelope) whether an unsupervised search in Version 2b would require more energy than is available in the visible universe before hitting on a proof of the Bounded Gaps Conjecture, and how many more universes would be needed to detect this proof amid all the consequences of the axioms of set theory computed up to that point. Both versions 2a and 2b are included merely to make the obvious point that Venkatesh’s scenario only makes sense if א(0) is launched into the world from the vantage point of contemporary human mathematics in all its complex ramifications.
On the other hand, I believe — though I haven’t worked it out on the back of an envelope — that human formalization of all the proposed background for Versions 1 or 2 would be impossible in the lifetime of anyone attending the Fields Medal Symposium, even with the undivided 24/7 attention of the entire staffs of several multimillion dollar Centers for Formal Mathematics. Venkatesh’s proposal to limit the formalization to the Springer-Verlag Graduate Texts in Mathematics is more tractable, and for all I know Springer-Verlag may already be negotiating with a formalization startup to prepare the ground for א(0)’s arrival.
Continued in the next post
Substack notices that aleph is a Hebrew letter and temporarily wants to write from right to left, thus mildly sabotaging Venkatesh’s typesetting. I will allow that to stand, including the placement of the footnote number, as a hint of unanticipated sabotage to come.
As Helena Sheehan put it in this week’s epigraph, “Conceptualization is weak and confused. Contextualization is thin and random.”
There actually is such a fountain in the middle of the Ecole Normale Supérieure in Paris, a stronghold of this way of thinking; but I’m told its water is not fit to drink.
Venkatesh’s helpful list includes (not in this order) “external validation,” “aesthetic considerations,” “infrastructure,” and the events and means of communication that punctuate the routine of the discipline, to which he refers, intriguingly, as “processes that direct our attention.”
Like Bettina Heintz and Christian Greiffenhagen, who was cited in a previous essay.
Jeremy Avigad could have had this passage in mind when he wrote recently that “mathematicians are among the most philosophically inclined people on the planet.” Avigad’s article suggests possible reasons for the sterility and irrelevance of most mainstream philosophy of mathematics and makes the radical but thoroughly justified suggestion that “philosophy of mathematical practice” — which overlaps with what Reuben Hersh called the “maverick” tradition in philosophy — be renamed “philosophy of mathematics.” Avigad is one of the invited speakers at the Fields Medal Symposium.
For א(0) itself, as conceived by Venkatesh, there will be no before or after or any time at all, except for the pulsing of its CPU clock. Like AlphaZero א(0) abolishes history by starting mathematics at the beginning.
However, the formulation was published in 1849 in the Comptes Rendus hébdomadaires des séances de l’Académie des Sciences. I haven’t found a source for Britannica’s 1846 date but I haven’t looked very hard either.
And which is hosted, perhaps presciently, on a .ai domain (asone.ai).
(commenting on part I because there's no comment box on part II)
The idea that virtual reality will lead to significant new results, or new proofs of old results, seems dubious to me. I am tempted to promise that if there is a VR proof of the Poincare conjecture I will quit mathematics, both because of how unlikely that is, and because I would be disappointed with the state of mathematics if literal hand-waving becomes an accepted measure of proof. (I know this is a bit at variance with my other comment.)
We can look at a series of different visual tools roughly ordered in increasing depth or fidelity:
Diagrams < Drawings or computer graphics with the illusion of 3d < actual 3d objects < videos < video games < VR experiences.
Certainly at least the first four have already been useful tools in understanding mathematics, though I think only diagrams and drawings have really been used in mathematical proofs.
The difference between VR experiences and videogames in terms of their applicability to mathematics seems less to me than the other differences on the list. A VR experience is basically a videogame that you control with your hands or other body movements, where the screen is pressed up against your face. Neither of these seems like a significant change compared to the ability to manipulation unphysical three-dimensional objects in real time that videogames offer.
But have videogames been used in mathematics in a significant way? As far as I know they have not. So I don't think VR will be useful either.
But writing this I fear that comment may have been more a joke than I understood at first...