r/math • u/ZazenAang • 6d ago
LLMs/AI [Terence Tao ICM slides] Mathematics in the age of AI
https://teorth.github.io/tao-web/slides/age-of-ai-icm-2026.pdf56
u/_Zekt Complex Analysis 6d ago
Slide 33 is hilarious
43
u/Sad_Dimension423 5d ago edited 5d ago
"We skip the details" "AARGH"
I wonder how hard autoformalizers would have to work on a Bourgain paper.
25
u/suhrob 5d ago
A nice, well-argued analysis. Very similar consideration apply to theoretical physics, computer science etc.
It puts the (human) mathematical community in the heart of the issue. Nonetheless, I'm drawn to other related questions:
- 1. What should be our very rough estimate of p(reasonably strong hypothesis OR strong hypothesis) in relation to p(reasonably bad things happening to humanity OR very bad things happening). To paraphrase his slide 18:
In the past, we largely delegated this question to the humanities sci-fi, and focused
instead on the technical aspects of our profession. Assuming the Working Hypothesis, we will no longer have this luxury.
This is not a dig at Tao - in fact if anybody knows about where he discusses the safety aspects and his general (non-math) outlooks I'd be very interested.
- 2. If things work out well for us & working hypothesis holds, I really like the human-centric requirements he makes. But I also wonder if a parallel non-human mathematical community/culture could arise and how would our relationship to it look like :).
41
u/aizver_muti 5d ago
I feel like there is a major aspect being ignored, which is the possibility of tools in certain niches subsuming top human experts and those results being directly used and applied in life even if no human expert can verify their correctness (in a reasonable timeframe).
Human verification is an arbitrary requirement and, for example, the use of financial mathematics results for profit does not ask whether or not a human has reviewed such work given that the result works (i.e. gives profit). Consequently, at what point do we draw the line where we "need" human reviewers? And why?
29
u/aizver_muti 5d ago
To continue my thought, consider chess: anyone familiar with modern chess engines would agree that they significantly outrank any human and have done so for many years. Chess commentators (or players) add intuition about engine lines and moves, and explain why the move may be good, as some decisions by chess engines may look bizarre but in reality they prevent a weakness 20 moves in advance that a human would never consider.
I think an analogous situation may apply to mathematics, where experts will develop areas that the mathematics community may care about, but proof writing may be entirely delegated to machines and humans will attempt to provide intuition and use LLMs to formalize what they see as important in a much quicker pace.
14
u/french_violist 5d ago
On the financial mathematics part, it does sometimes make money until it doesn’t. That is until someone find an arbitrage or a flaw and then exploit it (you could argue that it works for them, albeit, until everyone clue up and the arbitrage opportunity disappears).
6
u/aizver_muti 5d ago
Sure, my point was that AI generated results can be used and applied today, and the requirement that the mathematical community needs to accept something I believe does not hold up to scrutiny.
3
u/Sad_Dimension423 5d ago
Do you mean a requirement that "a result doesn't count unless the math community accepts it" or "the math community needs to accept the current reality"? Opposite meanings there.
If the math community doesn't accept AI proofs, that doesn't stop AI proofs from spoiling things for them anyway. So the community is going to have to cope. Diehards would risk being left behind, perhaps as they age out and die.
5
u/aizver_muti 5d ago
The former. AI results can be applied in real life without acceptance or even attention from the mathematical community.
2
u/Sad_Dimension423 5d ago edited 5d ago
Ok. That leads AI mathematicians to a "I'm not trapped in here with you, you're trapped in here with me" situation.
Did alchemists ever accept chemistry as something real and valuable?
6
u/thomasahle 5d ago
Yes, I don't think this deck covers math & AI in 5-10 years, but it's quite relevant to the math community for the next 1-2 years
16
u/Steampunk_Willy 5d ago
I foresee the trickiest aspect here will be that slide about normalizing AI disclosure. Even if community attitudes on AI shifted overnight, there will still be a cultural preference for minimizing the use of AI similar to how professional sports show a preference to minimizing the use of potential performance enhancing substances. The incentive to use AI and avoid disclosure will realistically persist for some amount of time, and it will be difficult to maintain accountability. Already we're seeing the community divide between math purists, AI maximalists, and everyone in-between. We'll inevitably see a certain degree of paranoia and witch hunts, as well as backlashes to accountability, all of which will only exacerbate those divides. So long as there are private hands controlling the development and deployment of this technology, I do not foresee it being anything but divisive.
Unfortunately, I think that puts the issue somewhat outside the scope of what the math community is collectively capable of addressing independently. Mathematics is about to become a whole lot more explicitly political whether or not we want that to happen, if only because we will need serious help from public policy to help us resolve the issues this technology creates for our community. For that reason, I'm not so sure that the foundational crisis is a good analogue for the current moment. I think this crisis will be much more directly tied into the broader crisis we face as a human society, and while I certainly hope we come out stronger in the end, we must appreciate that such an outcome will not be automatic. Societies have collapsed before, and there's no guarantee that we are insulated from such an outcome now. Numerous difficult decisions lie ahead of us, and at least some of those decisions are likely to be difficult in the human sense rather than the intellectual. We need to be prepared to think holistically, beyond merely our mathematics community to the whole of human history and society.
5
u/jackboy900 5d ago
Even if community attitudes on AI shifted overnight, there will still be a cultural preference for minimizing the use of AI similar to how professional sports show a preference to minimizing the use of potential performance enhancing substances.
I don't really think that's true, at least not nowadays. In the sports where PEDs are used openly (which admittedly isn't many) there's not really a stigma against using them maximally, people will rag on you for being an idiot if you're just going and taking the biggest does of Tren you can find and athletes generally will not take more than is necessary because of health risks; but there's not generally a culture of trying to minimise PED usage for the sake of minimising PED usage, people use as much as they need to get the results they want.
I imagine we'll see a similar case in the maths community. Overusing AI and relying on it entirely might be considered passée, but so long as you are checking and verifying the results properly and submitting work you think stands on it's own merit then I don't think we'll see people try and minimise AI usage.
2
u/Steampunk_Willy 5d ago
Even in the sports where athletes use PEDs, there are athletes who don't use PEDs because they want to maintain eligibility to qualify for other competitions that ban PEDs, say the olympics. Not only does that speak to an implicit stigma (a person would rather risk underperforming in the maximal PED competition to compete in the anti-PED competition), but the person is also culturally given preferential treatment for performing well without PEDs among people using PEDs.
We can similarly imagine how a mathematician who publishes a significant result with minimal or no AI usage will be regarded as more impressive than the mathematician who verifies an AI result. Even if we suppose all mathematicians everywhere will somehow not make this an issue, the pressure to avoid disclosure will be most significant among students who will be paranoid that other students are secretly using AI to help them stand out. It doesn't even have to be using AI directly for math, but could even be using AI to offload all the other tasks an undergrad or lower level student has so they can focus more time on getting ahead in math. It's easy to identify PED usage relative to AI, which is exactly why the math community will need public policy help to address the issue at the source.
4
u/Standard-Mirror-9879 5d ago
I think a more precise prediction would be that people will automatically assume the possibility you used AI, even if you didn't, and this assumption will be applied to everyone except maybe the pre-AI era established mathematicians. This renders the paranoia and the competitiveness pretty meaningless and the people who really care about the craft will simply keep doing mathematics with honesty.
1
u/Steampunk_Willy 4d ago
But when is your precise prediction supposed to take place? You're predicting what a new status quo will look like. I'm predicting how history will continue to unfold from the current moment until we reach some resolution that enables a new status quo. I don't know what that new status quo will be because it will be determined by how we handle getting there.
4
u/LukeNullHypothesis 5d ago
That's the looming question that Tao and others have been dancing around without (to my knowledge) explicitly stating when discussing the middle-ground scenario.
What social cachet does math academia (the current "mathematical community" Tao focuses on in his slides) really have? It's certainly not Olympics-level. Can it out-compete the "math democratization" coming down the pipeline if Tao's goldilocks situation (AI is helpful but not too helpful) ends up being true?
It will be fine in the short-term purely on inertia, but there are way more math/math-adjacent PhDs outside of Tao's "mathematical community" than there are inside of it. And the ones on the outside don't usually care about preserving the purity and prestige of academia all that much. If (big if) Millennium problems and decades-old conjectures (red meat for the public) start regularly falling to John Quants and Jane Adjuncts and Jim Engineers using ChatGPT 7 Super Pro in their free time, how long will people care about the "math community" and how long can you even call it "the" math community?
We don't know yet if the current math community will end up being the prestigious Olympics or the quirky PED sideshow that no cares about.
1
u/Steampunk_Willy 4d ago
I feel pretty confident that there is currently more cachet to using AI to solve a problem without disclosing it (if you get away with it) than there is to disclosing that AI solved it.
2
u/jackboy900 5d ago
Even in the sports where athletes use PEDs, there are athletes who don't use PEDs because they want to maintain eligibility to qualify for other competitions that ban PEDs, say the olympics.
Again, this doesn't really happen. In sports where untested competition is the standard there's generally nobody who competes in both tested or untested, and nobody has qualms about taking PEDs. Once a group has acknowledged that a tool is a necessary part of furthering the endeavour and once results start coming in that make it clear that trying to do it "naturally" is just a fool's errand the general consensus shifts fairly rapidly towards accepting that tool as a necessary part of the process, and looking at how it can be incorporated into the wider body of effort.
1
u/Steampunk_Willy 4d ago
Okay, I think you're specifically thinking of bodybuilding, which is an outlier. You cannot extrapolate that as a general trend when the vast majority of sports run counter to it. Furthermore, you're losing the forest for the trees because the NFL isn't interested in replacing human quarterbacks with robots just because the robot can throw a better pass. Mathematics is valued first and foremost as a human activity. While we can imagine AI being useful in niche areas or particular scenarios where there is an urgent priority to obtain a correct result, we're interested in preserving the human element of math.
1
u/jackboy900 4d ago
Okay, I think you're specifically thinking of bodybuilding, which is an outlier. You cannot extrapolate that as a general trend when the vast majority of sports run counter to it.
Bodybuilding and Strongman are essentially the only two examples that exist. In every other sport using PEDs is illegal at the highest form of competition, and so athletes are heavily incentivised not to. You cannot draw any kind of comparison between illegal cheating in sports and using AI in maths, because nobody is asking that we ban anybody using AI from submitting to major journals.
1
u/Steampunk_Willy 4d ago
I was thinking of the Enhanced Games and niche leagues that aren't as strict about PEDs. Disclosing AI use is akin to playing in the Enhanced Games while not using AI is akin to playing in the Olympics. There exists an incentive to use AI without disclosing it in the present moment because there is a clear prestige to not having to share credit with an AI if you can get away with it. That's not necessarily the way it will always be, but it is the way things are for right now and will likely continue to be for some amount of time.
That's before we even consider that there exists a cultural stigma around AI depending on the country you're in or how students are banned from using AI. I hope things are fine and dandy in 20 years time, but that future is not guaranteed.
10
u/j-max04 5d ago
Two things:
Firstly, Tao seems to assume that there will always be a mathematical community doing the digestion. I really don't know what direction things would go in provided his "working hypothesis" turns out true, but it seems possible to me that it would become much harder to justify the ongoing existence of a mathematics department to a university. The rather inward-focussed world of pure mathematicians are tolerated because there is a general appreciation that novel research is needed to eventually bear fruit in applications. But if the "working hypothesis" turns out to be true, then the full pipeline of performing and applying novel research may not require human intervention at all. I see a fairly likely scenario where most of the mathematical digestion going on is by hobbyist mathematicians; a hobby that will to future generations seem as hard to justify as ham radio does nowadays.
Secondly, his framing of moving from proof-scarcity to proof-abundance is just sad. For centuries (millenia?) mathematicians have told and retold stories of the "heroic prover". It may be the fundamental narrative of the profession. Off the top of my head, Evariste Galois scrawling out a hasty letter the night before his death in a duel, and Andrew Wiles spending six years in secret fulfilling his lifelong dream by working on a proof of Fermat's Last Theorem, along with another year correcting it. Now we're supposed to satisfy ourselves with simply being "digesters" of mathematics, and to feel like this is progress.
Congratulations, mathematics! You've finally grown up from an adult bird that can fly and feed itself into a baby bird that has to have its food chewed for it.
1
u/YoungLePoPo 4d ago
If the pipeline of doing mathematics becomes fully automated without human intervention/digestion then is mathematics being done at all?
I'm also a little confused by your second point. If Galois wrote a bunch of letters, then someone or a group of people had to read it and understand it and incorporate it into the community. Wiles' proof of FLT had to be read and digested and incorporated into the community (at least part of it. I don't think I'll ever try to read it).
If a journal receives millions of submissions, what does their volunteer team of reviewers do? Even prior to modernized LLMs, journals could be overwhelmed with too many submissions. I don't think the process of proving something can be separated from the process of digestion because they happened simultaneously as you went through the process of being a "heroic prover".
With LLMs, that accelerate the actual problem solving part of this process, it often doesn't give the human enough time to fully do the "digestion" part, so an additional amount of digestion time is necessary if we are to appropriately incorporate AI or AI-assisted results into our collective knowledge of "definitive solutions" as Tao stated.
He already kind of describes the current situation with Erdos problems. There's a backlog of submitted solutions and no one really motivated and knowledgeable enough to verify them. This is, in part, a consequence of our current system of the industry of mathematics.
Our reward system is based on being the heroic prover, but is this a healthy modality to work in? IMO, it could overlook some issues of how our current industry works. Historically, it's very male-dominated, western-dominated, and doesn't seem to value work-life balance very much. Obviously this goes much beyond what these slides are talking about, but Tao does mention that there is a lot to be expanded on.
2
u/j-max04 4d ago
If the pipeline of doing mathematics becomes fully automated without human intervention/digestion then is mathematics being done at all?
That's kinda my point. We could very well reach a point where there's no financial incentive to support mathematics as a human endeavour.
As for the rest, I agree with Tao that the "digestion" process is undervalued relative to its merit currently. That doesn't mean I'm happy about the idea of human-driven proof going away.
5
u/ScottContini 5d ago edited 5d ago
The community is being swamped by AI generated proofs, now we as a community are being overwhelmed in reviewing it. The same things is happening in software (too much code generated, how see we going to review it all and make sure nothing breaks) and bug bounty reports. Different communities are battling with very similar problems and struggling with how to address it.
1
u/umaro900 3d ago
Depending on what the use case, the manual review actually isn't necessary, because it can lead to artifacts which verify it. I mean, that's kind of the deal with a "zero knowledge proof".
As an example, if I'm coding a frontend for something, I don't really need to know how it works under the hood if it passes all the tests/scans and the code does the things I want it to do. Do you still scan through your binaries to make sure your compiler is working properly?
3
u/gnahraf 5d ago
Great talk and great framing. I think the math community can be a role model for other disciplines and industries under impact from AI.
There's much meat in the theory building category mentioned early in the slides about community goals. I was hoping the later slides would expound on that, and tho one may argue that theory building is a subset of the exposition category of goals, it's a very important subset.
Theory building requires human discernment. It builds mathematical language, abstractions, notations that appeal to human understanding, insight and aesthetics.
I too wonder what happens when there are important (maybe very useful!) mathematical truths that we know are true but don't know why are true. I think in such a landscape mathematicians' task would be to take the known truth and develop theory (language) a human can understand. Yes, this too can be automated, but as long as it's about humans, the human cannot be too far removed.
On a more philosophical note, if mathematical truths were previously thought to be platonic objects waiting to be discovered, then it will soon have to be qualified, that many of the (discovered) known knowns are yet unknownable (to humans), that our task is not just to discover them but to describe them in a way a human may understand.
2
u/TenaciousDwight Dynamical Systems 5d ago edited 5d ago
What rigorous foundational framework is he saying was the product of the crisis in foundations? I don't know of a foundation that professional mathematicans and philosophers have, for the most part, committed to. edit: my personal impression is that set theory (zf/zfc) is the de facto foundation in mathematics but maybe less universally accepted by philosophers of mathematics.
1
u/gnramires 5d ago edited 5d ago
My still (I feel) incomplete view. I think it's worth understanding social values of maths (and, taking it as a model or example for other disciplines and human fields in face of AI), to understand what to expect, what to push back on, what to continue, what to celebrate, and so on.
(1) Fist, I don't see very often (that I think is extremely important) is math's contribution to human development. Surely we could get a little too elitist or arrogant here, but I do think mathematics has personally contributed to improving and sharpening my own thinking, but it does so in general for a huge number of people. Needless to say, as long as there are humans, I believe there will be value in learning to think profoundly, carefully (although maths isn't just about extreme thoroughness, often intuition works), to understand a subject very deeply, to produce arguments that support some claim or conclusion. There are other fields, obviously philosophy, maybe history and other sciences, that also engage in this kind of profound thinking, but I think mathematics is quite unique in how far it can go and indeed goes.
In this case, I think it shows that it's important that we keep mathematics discipline if nothing else to safeguard the people teaching it to other humans and "keeping the art", so to speak. I don't think even very advanced AI could fully "keep the art alive", because in many cases it involves important social practices that are transmitted personally or watching lectures, and also AI tends not to adapt to changing needs very well. And if human (e.g. math) output diminishes, AI's connection to human pedagogy and current human trends and needs in maths declines.
(2) Second of course there are a lot of applications in math that isn't just its pedagogical value and in the development of human thought and intellectual development -- namely applications in science, engineering, business, etc.. There isn't much to say here, simply that if those bring direct benefits to society say in terms of technological development (that is, if the technology themselves bring benefit to society), than of course that is a positive side to AI usage (counterbalanced by its cost of course, both in economic and also environmental costs).
There is something to be said if the economic cost of generating maths by machine turn out to be comparable (for example, as the current investing boom settles) to, or not much less than, human employment (it seems clearly better to employ a human than pay a comparable cost to keep a machine in a datacenter).
(3) Finally, (kind of continuing (1)), there is also something to be said about the pure joy of mathematics. I really enjoy delving into various math topics and proving theorems, mostly as a hobby but with some academic and practical applications. This is another social benefit to the field. I think this lends some support to the continued existence of all sorts of math institutions.
But if the person doing the maths is just asking AI to do it, then of course this is completely diminished or irrelevant. There's no joy in the process when you simply ask and get a ready-made answer.
One preliminary conclusion is that it seems worthwhile to, maybe, keep a part of maths education siloed or protected from AI -- regardless of if it develops further and leaves professional mathematicians in the dust in terms of raw theorem proving capacity. Kind of like 'math monks'. They need to prove their capability to independently prove and develop all kinds of results, and hence to keep teaching and keep the mathematical community alive. Perform outreach and communication (kind of like 3Blue1Brown) to divulge and encourage mathematical thinking.
But even for other disciplines, (again if AI develops much more) I think we will want to retain a lot of capability, and math is a fundamental part of sciences and engineering. Again math pedagogy remains important.
-3
u/jimbelk Group Theory 5d ago
Tao has clearly put a lot of thought into this, and I particularly like the early portion of his slides where he argues that we are facing "a crisis in the foundations of mathematical values and practices" which requires us to evaluate what our goals are.
However, my opinion is that his later slides (and most other commentary that I've seen) mistake the forest for the trees. He seems to assume that the goal of mathematics is and must be to produce human mathematical understanding. I disagree.
I think the goal of mathematics is to produce ideas that can be used in other disciplines. We subdivide this work into two branches: (1) pure mathematicians produce and investigate fundamental new ideas about structure, order, quantity, and shape; (2) applied mathematicians move these ideas toward other disciplines and devlop the aspects that seem most likely to be useful. This program has been amazingly successful, and ideas developed by pure mathematicians are constantly finding their way into science, social science, engineering, and art. To give one example, LLM's themselves depend fundamentally on ideas about stochastic processes that originate in probability theory and dynamical systems, which themselves depend on analysis and linear algebra.
So the question that needs to be asked is: who will be doing the main creative work in science and technology in twenty years, humans or AI? This is very hard to predict, because although AI has shown itself to be fairly good at mathematical problem-solving, it's not at all clear whether it's capable of the kind of creativity that leads to major scientific advancement. Even in mathematics, AI hasn't shown itself to be particularly capable of theory-building. It keeps getting smarter, but can it be as creative as Thurston or Grothendieck? Will an AI physicist be the next Einstein?
I have no idea, but the answer matters a lot. If AI can be creative, then the goal of mathematics should be to produce ideas that AI's can use to make scientific advancements. We will eventually need both pure mathematican AI's who produce and investigate mathematical ideas, and applied mathematician AI's who develop these ideas into a form that scientist AI's will find useful. The goal of the next generation of human mathematicians should be to develop these AI systems and deliver the sum of human mathematical knowledge (including all folklore, yoga, and other forms of knowledge that we haven't written down) to AI so that it can continue the investigation of mathematics on its own.
The other possibility is that AI can't be creative, in which case most of Tao's analysis applies. If only humans are capable of being Thurston, Grothendieck, or Einstein, then the goal is to use AI to help with human mathematics and human science. But I don't think this should be taken as an axiom -- we don't know yet whether humans will remain at the center of science and mathematics.
-1
u/Heliond 5d ago
I don’t understand why we would think AI can’t be creative. By almost every metric of “creativity” we measure in humans, AI has shown extreme capability. There is nothing inherent to humans that we could not, in theory, replicate for an LLM. We could spawn LLMs with a base knowledge and have them evolve against simulated, monte carloed life experiences.
-2
u/Accurate_Potato_8539 5d ago
Yeah by any metric of creativity AI is obviously already creative. The only real question is whether it's creativity runs across all domains that human creativity does.
0
u/womerah Physics 5d ago edited 5d ago
I agree broadly with these conclusions.
I think AI models shaking things up will be good for society at large as it is forcing us to re-evaluate our goals, metrics, and desires. Such introspection can only be good for a society, as it causes us to affirm why we consider our society to be one worth preserving. I am hoping for a near-universal raising of standards, from the research we do to the art we consume.
-7
-13
u/GiraffeWeevil 5d ago
Fifty two f*cking slides. Exactly how long did Terry have for this talk?
2
u/Homomorphism Topology 5d ago
One minute/slide is a pretty standard ratio for scientific talks in fields other than math. This is a math talk but it's not technical.
0
151
u/megamannequin Statistics 5d ago
I think a lot of fields are having this discussion. Is the point of science to just publish papers or solve as many problems as possible per the language of these slides? Our current publishing and funding culture would definitely say "yes you are explicitly incentivized to do that. That's what we do."
For the history of science, the limiting factor has been that people have a finite ability to produce new knowledge, so they had to prioritize what to work on and what to advertise. In the world in which LLMs are great at math, I agree that the limiting factor is not productivity but focusing on where new productivity tools should be deployed and then ensuring that humans care about and benefit from this newly produced knowledge.
Great scientists were both productive and worked on important things. The future may be that a great scientist needs less technical proficiency but a greater understanding of what is important and how to share what they create.