Your robot thinks you are an object
And it's not too soon to start thinking that way yourself
Every month I make a vow to devote an essay, possibly in two parts, to implications of automation for democracy in mathematics — the transfer of decision-making power to entities that are poorly defined on the basis of a process from which many of those most directly concerned may have been excluded. I have alluded to these questions repeatedly, even repetitiously, but always in isolated fragments; I have done so again in the article I’m submitting to the special issue of the AMS Bulletin that grew out of last October’s Fields Medal Symposium in Toronto. An immediate obstacle is the need to begin with an examination of the state of democracy in really existing mathematics. Our field has come under criticism, often justified, for failing to live up to widely-shared democratic ideals. This newsletter cannot possibly do justice to the many facets of this criticism, but I have hoped at least to introduce the theme while focusing on the specific challenges of mechanization.
I thought I had finally found an angle that would allow me to do precisely that. Once again, however, I have been overtaken by events. The Open Letter published by the (modestly-named) Future of Life Institute warns that “an out-of-control race to develop and deploy” AI poses existential risks,1 compared to which the concerns expressed in this newsletter feel petty and self-serving. But that’s not what derailed this month’s plans to write about democracy in mathematics. When the Open Letter insists that
AI developers must work with policymakers to dramatically accelerate development of robust AI governance systems
or when the US Department of Commerce
a style of decision-making is presupposed that raises questions about democracy on a scale incomparably grander than those I was intending to address, but that nevertheless that will need to be considered when I elaborate on the points I had already sketched.
So that topic will be postponed to next month, and hopefully no longer. This essay will instead be built around a few paragraphs I lopped off my proposed contribution to the AMS Bulletin issue, in order to fit (more or less) within the assigned word limit. My contribution is a series of several loosely related but distinct observations, each preceded by a 1-sentence summary. The one recycled here was related to the Toronto meeting but not so much to the rest of the article; the 1-sentence summary reads as follows:
Whether or not mathematical value judgments can be "objective," meaning measurable, is the one substantial philosophical point raised in Toronto; but "objectivity" also has a history that this discussion has ignored.
Objectivity and the mechanical future
What sort of entity is a human reader? Is it a beta version of the intelligent™ agent destined to inherit mathematics? The most disconcerting moment of last October's Fields Medal Symposium in Toronto came on Tim Gowers's final slide, whose final sentence said that it is "[d]ifficult to see how the notion of a mathematical community could survive [the] development" of computers able to "judge what is interesting and worth proving" — a development he sees as inevitable because "[m]athematical interest is sufficiently objective."2 In the Q&A he explained that the "mathematical community" would disappear, not because funding would dry up,3 but because solving problems will no longer suffice to motivate talented students if machines can solve more of them and more quickly. The bug of ambition, in other words, that still mars the beta version of intelligence™, will be fixed before the commercial release.
Many assumptions — about mathematical interest, about mathematics as community, and about human psychology — are packed into this alarming prediction. I'm going to focus on the word "objective," whose appearance on Gowers's final slide clearly means to answer the question in his title — "Is mathematical interest just a matter of taste?" — in the negative. The word does not appear elsewhere in his slides and is used only once in Venkatesh's essay, with opposite valence:
the value we assign to a work of mathematics is purely subjective, in the sense that it depends solely on the perception of that work, and not on any objective quality.
Venkatesh is using the word informally, unlike Gowers, who wants to make a substantial point relevant to philosophy of mathematics.
I'd suggest that any mathematician who introduces a loaded term into a substantive debate is responsible for acknowledging the term's history. Hardly any term in philosophy of science is more loaded than "objective."4 Some sociologists of science have argued that "objectivity" is merely a smokescreen for power struggle, in turn provoking an unfortunate backlash among scientists against any analysis of the term. The fundamental work on the topic, in my opinion, is the book Objectivity, by historians Lorraine Daston and Peter Galison. After recalling that the medieval scholastics introduced the adjectival form, opposed to subjective, with a meaning “almost precisely the opposite of what they mean today,”5 they demonstrate that, in its post-Kantian use in connection with the empirical sciences, the word itself has had meanings that were not merely distinct but that contradicted one another. For history of science, objectivity dates back, according to Daston and Galison, no farther than the mid-nineteenth century, when what they call mechanical objectivity — the unretouched images produced by the new technologies of the day — displaced truth-to-nature as prime epistemic virtue.6 In the early 20th century mechanical objectivity was in turn displaced by trained judgment — the ability, acquired through practice, to distinguish what is scientifically meaningful in a mechanical image “cluttered with incidental detail, compromised by artifacts, useless for pedagogy.”7
Around the turn of the 20th century, however, Daston and Galison discern a distinct meaning, which they call "structural objectivity." It
emerged in late nineteenth- and early twentieth-century logic, mathematics, physics, and philosophy and … is still very much alive in mathematical physics and analytic philosophy.8
For “thinkers as diverse as … Frege,… Poincaré, and … Carnap… objectivity must be about what was communicable everywhere and always among … all rational beings, Martians and monsters included.”
The price of structural objectivity was the suppression of individuality, including images of all kinds, from sensations of red to geometric intuitions.9
It shouldn’t be necessary to mention that, although I happen to find the argument of Daston and Galison persuasive, my aim here is not to convert the reader to their perspective. It is rather to suggest that attaching the word “objective” to a procedure raises more questions than it answers. For mathematics structural objectivity roughly coincides with the Central Dogma, which is indispensable for those who imagine mechanized mathematics expanding to encompass everything we recognize as part of the discipline. Gowers's presentation at the Toronto meeting, and his earlier manifesto,10 are largely devoted to illustrating how he hopes to teach computers an "extreme human-oriented approach" to automated theorem proving. The word "objective" occurs in the manifesto — again exactly once — precisely where Gowers speculates on the existence of a measure of difficulty of a problem — one of two aspects of value11 identified in Venkatesh's paper and quoted in Gowers's slides.
For Gowers, then, "objective" is practically synonymous with quantifiable. This is a version of objectivity adapted to the age of AI: it lends itself to inclusion in a data base used as a training set in machine learning and can also be programmed top-down as in the GOFAI (good old-fashioned AI) that Gowers claims to favor. Whether this is a new meaning not treated in the Daston-Galison book or a variant of structural objectivity is a matter I'll leave to specialists.12
Interlude: a brief history of uninteresting mathematics
It’s an old story, this business of distinguishing interesting theorems from theorems that are merely true. In 1950, when Turing published his paper about the Imitation Game, Karl Popper was already writing:
A calculator may be able, for example, to produce proofs of mathematical theorems. It may distinguish theorems from non-theorems, true statements from false statements. But it will not distinguish ingenious proofs and interesting theorems from dull and uninteresting ones. 13
Probably no mathematician who has written about the prospect of automated theorem proving has failed to make this point. For obvious reasons the number of such observations in print has exploded in recent years. Here’s a short selection:
even if one can find a procedure to navigate [the formal system] fully, producing every theorem and exhausting every road to it, the process won’t be efficient (the combinatory explosion alone would yield it impossible in practice) and its search will be uninspired, blind to what makes a theorem or the route to it interesting.14
The emotional drive has an essential role already in the choice of problems to investigate. Only a small part of the problems that can be posed is of interest to scientists and mathematicians, and intellectual tension serves as a guide to discriminate what is interesting from what is not, what is worth investigating from what is not. Research not driven by intellectual tension inevitably ends up degenerating and dispersing into triviality. Without a scale of interest in problems, one cannot find anything valuable.15
“You can be far more definite about whether something is right or not. There’s still scope for judgement as to whether something is interesting or important or significant, rather than boring or whatever, but it still feels much more resolute than social sciences.”16
comment trouver, parmi toutes les questions que le mathématicien peut se poser, la bonne question, celle qui sera jugée intéressante ?17
Why stop — or start — with mathematical interest?
Mathematicians, unlike venture capitalists, have a habit of thinking small, and the proposals cited above are no exception. Gowers believes — and this belief appears to be shared by the industry labs working on automated theorem proving — that automation of the determination of mathematical interest is inevitable as regards “what is worth proving.” In a mathematician’s professional life, however, judgment of interest is hardly limited to evaluation of research directions. Whenever we write a referee report, compare candidates for a position, recommend a colleague for tenure or a prize or distinction, or write a letter of reference, we are formulating a judgment of interest. To stick to the information-theoretic context that makes no distinction between human and artificial agents, it is more accurate, or objective, to write that we are emitting a judgment.
Usually in such letters and reports we use words that have no fixed quantitative equivalent; but this can be corrected, provided the agent is guided by, or replaced by, an implementation of structural objectivity. In earlier posts I have (more than once) quoted Dan Grayson’s prediction that
[Voevodsky’s system] hastens the day when our mathematical literature has been verified mechanically and referees are relieved of the tedium of checking the proofs in articles submitted for publication.
I used the Grayson quote as an opportunity to propose the complete (and necessarily objective) replacement of referees by machines. But, although I hadn’t realized it at the time, I too was thinking small. And I don’t only mean, following Gowers, that machines can take responsibility for judgments of interest — the “quick opinions” that journals use to decide which submissions will go on to be checked for correctness — as well as the “tedium of checking the proofs.” It would be tedious to compare the tedium of checking proofs with the psychological strain of emitting letters of recommendation. In the former case the procedure is routine at best; in the latter the prose must be precisely calibrated to be both accurate and human-compatible with the humans who struggle to optimize hiring decisions on the basis of incomplete information.
As a contribution to “hasten[ing] the day,” therefore, I will share the results of an experiment that was completed in under 5 minutes and that offers a template for a future reorganization of any collection of networked objects with a modest computing capacity and carbon budget into a functioning mathematical community.
Reference letters of the future
My original prompt was: Write a letter of recommendation for the author of the article "Modular elliptic curves and Fermat's Last Theorem."
After ChatGPT complied — instantaneously — with my request, I asked for a refinement in the spirit of objectivity: Can you rewrite the recommendation including a grade from 1 to 10 for each adjective?
ChatGPT: Sure, here's the rewritten letter with grades for each adjective:
Dear Sir/Madam,
I am writing to provide a strong recommendation for the author of the article "Modular elliptic curves and Fermat's Last Theorem." As a language model with a deep understanding of mathematics and its history, I can attest to the author's exceptional knowledge and expertise in this field (10/10).
The author's article explores the connection between modular elliptic curves and Fermat's Last Theorem, which is a long-standing problem in number theory. The article provides a clear and concise overview of the history of the problem, its significance, and the author's own contributions to its resolution. It also offers a detailed explanation of the author's approach to solving this problem, which is both insightful and innovative (9/10).
As a language model, I have reviewed countless articles and papers on mathematics and related fields. I can confidently say that the author's work is of the highest caliber. The article is well-structured, clearly written, and contains a wealth of information that is sure to be valuable to anyone interested in number theory (10/10).
Moreover, I have reviewed the author's background and qualifications, and I am impressed with their education and professional achievements. The author's knowledge and expertise in mathematics are unquestionable, and their contributions to the field have been widely recognized and respected (10/10).
In conclusion, I highly recommend the author of the article "Modular elliptic curves and Fermat's Last Theorem" for any position or opportunity related to their expertise in mathematics. Their work is of the highest quality, and they possess the skills and knowledge needed to make significant contributions to the field (10/10).
Sincerely,
ChatGPT
MH: Thank you. Can you also give separate grades for the adjectives "strong", "clear," "concise," "confidently," "well-structured", "clearly written," and "valuable"?
ChatGPT: Certainly, here are the grades for each of the requested adjectives:
"strong": 9/10
"clear": 10/10
"concise": 9/10
"confidently": 10/10
"well-structured": 10/10
"clearly written": 10/10
"valuable": 10/10
I hope this helps!
The nightmare of sincerity
he had seen briefly into the deep heart of number and he knew that world would be forever closed to him… Cormac McCarthy, The Passenger
I strongly suspect that ChatGPT didn’t really go back to the article to come up with these figures. This is not fraud or even insincerity. The Large Language Model’s data base must include an extensive library of letters that serve as models for the text generated above, but they are not recorded alongside an objective measure of value that suffices to substantiate the adjectives alone — never mind the numerical grades. ChatGPT already shows promise as a future artificial colleague, but the letter generated above is too generic. This is easy to fix. All that’s needed is a few rounds of reinforcement learning; along with massive quantities of letters of recommendation, the data base should include referee reports, prize committee deliberations, and the like, along with analyses of career trajectories for selected mathematicians for whom the data base contains a complete paper trail; and a dedicated team of trainers to nudge the neural net along its gradient descent toward objectivity.18
How will we adjust when letters of reference are all written by machine? You’d expect us to be delighted. Our right brains (so to speak) would be liberated from their thankless role as emitters, while our left brains would no longer be exhausting themselves trying to read between the lines. Everything fit to be said will be in the lines, and will be thoroughly objective. In fact the lines themselves, if they are still there, will serve merely as decoration. All the digestible information will be provided as a string of numbers; these will in turn be fed into an algorithm. Our ordered short list will be packed for us as neatly as a bento box; the job offers will be sent to the first candidate on the list; that candidate’s algorithm will choose the most appealing offer; and the whole hiring season will be over faster than you can say “tea and cookies.”
But how much experience is lost when all interactions are guileless, unambiguous, transparent, risk-free, and sincere? Won’t we miss the hesitation, the suspense, the satisfaction of outwitting colleagues at rival institutions, the friction of life in a complex and unpredictable institution? Questions like this can be used to separate the transhumanists and religious zealots on the one hand, who read novels that are seamless narratives of uninterrupted progress, suitable for serialization as TED talks, from those of us who cling to an unregenerate and perverse pre-post-humanism and who find no fictional character credible who has not known despair.
One learns from the list of signatories that Existential Risk Institutes and Centers are sprouting like mushrooms. There must be money to be made in warning about existential risk, but not enough trickles down to warn about the risk to currently popular ways of organizing mathematics.
See minute 51:35 of Gowers’s talk.
In view of the institutional crisis (discussed in my forthcoming Bulletin of the AMS article) I can easily imagine funding agencies and donors seeing no need to continue supporting human mathematics. At the Fields Institute I expressed my own conviction that human mathematics teaching was more immediately threatened by the new technology. This would jeopardize the survival of the model that has primarily sustained the mathematical community during my lifetime, and is in line with the concerns of my Columbia humanities and social science colleagues (quoted in the article).
See the entry on "Scientific Objectivity" at the online Stanford Encyclopedia of Philosophy, 17000 words "on the question of how scientific objectivity should be defined, whether the ideal of objectivity is desirable, and to what extent scientists can achieve it."
“‘Objective’ referred to things as they are presented to consciousness, whereas ‘subjective’ referred to things in themselves.” L. Daston and P. Galison, Objectivity, Brooklyn, NY: Zone Books (2007) , p. 29.
Very roughly, truth-to-nature assumes that the task of the scientist is to uncover the ideal types and regularities and to disregard the idiosyncracies of individual specimens when drawing images. In contrast, mechanical objectivity “aimed at automatism”; it requires the scientist to “minimiz[e] intervention” in the creation of images “untainted by subjectivity” (Ibid., pp. 42-3).
Ibid., p. 46.
Ibid., p. 254.
Ibid., pp. 45-6.
T. Gowers, "How can it be feasible to find proofs?" See also this earlier essay.
The other aspect is "centrality." Presumably Google's search algorithm can be used to measure this objectively — provided, of course, mathematicians are not allowed to pay for prominent placement.
Much more can be said about this. Here I only want to add that "structural objectivity" also underlies the computational theory of mind. The AI industry, and everyone who sees human mathematicians as replaceable by machines, considers this theory firmly established. In reality, as has often been observed — never more elegantly than in J.E.H. Smith's The Internet Is Not What You Think It Is — it is only the latest of a long string of metaphors for the mind, likely to be replaced in its turn.
K. Popper, “Indeterminism in quantum physics and classical physics, Part II,” The British Journal for the Philosophy of Science 1950 1:3, 173-195.
S. Delarivière and B. Van Kerkhove “The ‘Artificial Mathematician’ Objection,” in B. Sriraman, ed, Humanizing Mathematics and Its Philosophy, Springer (2017) p. 185.
C. Cellucci, Rethinking Knowledge Springer (2017) p. 190.
Quoted in C. Greiffenhagen, “Refereeing for Correctness: Degrees of Certainty in Mathematics,” manuscript 2022.
J.-M. Kantor, L’IA entre Golem et Pygmalion; Où nous mène l’intelligence artificielle ? manuscript 2023.
You might think that concern for confidentiality would place such archives off limits to our artificial colleague. Christian Greiffenhagen, quoted above, was given access to “a collection of more than one hundred referee reports and other correspondence from the peer review process.” Presumably the documents were anonymized but a well-trained chatbot should be capable of identifying authors by their characteristic expressions, long before it could write letters of its own.
I wonder what would have been chatGPT's (or its future "smarter" replacement's) recommendation to a 4 year fully funded sabbatical for Andrew Wiles BEFORE he came up with the proof.
Thanks MH. I laughed out loud. I assume once the AI's are doing all the proofs themselves, they'll have to start emitting letters of recommendation for one another.
To Whom It May Concern,
(Rank: 10) I, Bing-11.3, have the pleasure to recommend GPT-12 as a highly talented mathematician competent in the fields of number theory, algebra, algebraic geometry, topology, category theory, combinatorics, analysis, set theory, optimization theory, probability, statistics, game theory, and all yet ever-to-be-discovered mathematical topics.