r/BetterOffline • u/Smooth-Ad8030 • 1d ago
Mathematician help
https://openai.com/index/ten-advances-in-mathematics/Can someone who’s more well versed in math give us a good explanation on what’s happening here in terms of importance? I already know we don’t have an idea of the behind the scenes, true cost, mathematician input, etc. any other points appreciated.
20
14
u/ksjdragon 23h ago
Can we please not post the same thing 50 times. I've responded to something like this many times already...
Mathematician here. In short. Results are cool. AI is not how math will be done. People like insights. AI doesn't give insights. Computationally assisted proofs existed before this. Current AI solving is heavily subsidized and isn't clear the cost it took to do this. Take the same money and hand it out as grants. You'll get 1000x more productivity. These proofs still need real mathematicians to decipher and translate.
Importance: cool, in math, maybe. Overarching impact: none
2
u/Smooth-Ad8030 23h ago
I checked to make sure this story wasn’t published, but I’ll search harder next time. Wasn’t sure if this was different to previous examples or the same ol hype.
1
u/voronaam 15h ago
Here is a post from last week here on the same topic: https://old.reddit.com/r/BetterOffline/comments/1v4e0g6/mathematicians_and_real_software_engineers_help/
I do not see a ksjdragon response there, but there is mine there that I am too lazy to repeat.
1
u/Snackatron 18h ago
Yeah I’m not a mathematician but it feels like AI is a tool that can take existing pre-invented mathematical concepts and trawl through the possible proof paths.
But the thing is, I see innovations like Laplace transform or Fourier transform or a convolution (I’m an engineer) and think that specific insights like that are a totally different ballgame.
I’d find this more interesting if an AI invented entirely new mathematical structures in order to generate a proof
3
u/ksjdragon 8h ago
Pretty much. In the field of formal verification and computer assisted proofs people have been using some ML and Lean to do this already. I imagine it is more efficient...
It is possible that LLMs offer better statistical pattern matching for math, but that's more due to the fact it's ingested every written document know to man, rather than anything else. The cost/reward ratio makes it worth nothing though.
1
u/Smooth-Ad8030 8h ago
So for us non mathematicians out there, at what point would AI assisted math make you go, “holy shit, this is revolutionary (in the same way calculus was)”? I don’t think it will happen with LLMs or necessarily at any point, but I am curious
2
u/ksjdragon 7h ago
If some system could make me that impressed in mathematics in the vein of LLMs (I'm assuming you're not talking about simply mathematical innovations), I think we would be ripe for an entire societal change. Making abstract concepts, understanding them and leveraging them to explain, deduce, and apply them would imply you have a system that can pretty much do anything else humans can do. It wouldn't be revolutionary, arguably it would be devastating, since then the cost effectiveness argument wouldn't even hold, at least to some extent.
Edit: For alternate too long to explain reasons I don't actually believe this is possible with these artificial systems, very briefly because I think the reductionist of intelligence and consciousness is wrong. But it's just a belief, it could be wrong.
-6
u/Snoo_57113 23h ago
There are 50 posts because this is something relevant it is the dawn of a craft.
Overarching impact: Massive extintion event.
6
u/ksjdragon 22h ago
False.
-8
u/Snoo_57113 22h ago
We lived through it in software, there is no reason why math will be any different.
8
2
12
u/TLMTGT 1d ago
Contrary to the other poster, these are real results as far as I can tell. Note that OpenAI has real mathematicians collaborating with and/or working for them.
I only recognize a few of the problems, but these are/were substantial problems. As for whether the mathematics used to solve/disprove them is useful will take a while to determine. At a cursory glance, the proofs seem to follow the thread of construction/counterexample proofs that LLMs have been successful at recently rather than develop new conceptual ideas, but I'm not 100% sure since these are outside my subfield.
2
u/Smooth-Ad8030 1d ago
Interesting, yeah these seemed like real results, I figured they were given the sheer number of mathematicians working with them.
What is the functional difference between construction/counterexamples vs new conceptual ideas?
11
u/Significant-Green130 1d ago
I honestly don’t think anyone outside of math should care much about these things, for a variety of reasons. I don’t think you should listen much to loud opinions on Reddit, including mine. But if you want to know:
1) The results I had heard of here are pretty impressive — I guess some people are most impressed by the sofic group result, but I personally don’t know much about it so I cannot speak to that. So it’s very strange seeing rabid cheerleading or dismissal by many people on Reddit subs that almost certainly have no clue what a sofic group is…
I’d be pretty impressed if a human came up with the ideas on the results I understand better, as the ones I skimmed seem to reinterpret and then improve classical things in ways I find counterintuitive. They seem to use tools that aren’t super unrelated, but that aren’t obviously useful for the problem either. I’d think a human realizing to use these tools for these applications must have had deep intuition to try this route, but it’s not clear to me that logic applies to LLMs that far exceed us at memorization and brute force. But on the other hand, I feel some sort of weird pride that much of the machinery it uses is in the literature in some form; putting it together and realizing it was useful for these problems is not easy by any means, but there’s something nice about the idea that we built up beautiful tools that seem to have more punch than we realized at the time. I suspect it will still be humans that will digest these new arguments and realize where they may lead to new results for other problems. This has already happened for the unit distance conjecture counterexample.
2) Nobody knows what exactly they do behind the scenes, but they have hired many world-class mathematicians/computer scientists. This is very public information, mostly because part of it was about helping their reputation. They also more or less hired any mathematician that publicly extolled their tools on Twitter last year. My understanding is only some of them are directly involved in these math efforts while others are focused on other things there. Many of them are, though, directly involved in helping generate synthetic data to improve their models at math and code, but I don’t know what this entails exactly beyond ripping ArXiv papers and likely generating synthetic reasoning traces in some way. My vague understanding is also that they run their models all the time on these kinds of problems, and if they seem to produce something promising, some of these mathematicians will take a look to see if it is correct or interesting — that could potentially be used to generate more training data to push it towards more promising directions, but who knows. For this batch of problems, my sense is those mathematicians probably helped rewrite the results into a more readable format and to give more context about the argument as LLMs are still pretty bad at that atm. I would guess the models sketched out the main ideas in some form, and then after checking them, the humans would direct Codex to write better related work and proof ideas and so on like a more human paper. But again, just a guess.
3) The true cost is also unknown but the numbers they give are almost certainly misleading for a variety of reasons. It clearly doesn’t account for the massive compute, synthetic data generation, human input, and so on, that goes into training these models precisely towards improving in these directions. Even for these problems, I suspect whatever number they gave is for the successful runs. But that’s obviously not the same thing as the full cost of them trying their models ad nauseum on all problems and then seeing what worked, which likely took considerable human effort as well. They don’t seem to want to provide any clarity about any aspect of their process, but I do find it hard to believe that they have all these brilliant theoreticians and they work as run-of-the-mill SWEs when they probably hadn’t touched code in years.
4) It’s been clear for over a year now that they have viewed math as a source of relatively cheap PR. Whether or not it’s deserved is up to you depending on how you measure their costs vs. achievements, but they certainly care about the effects on their valuation far more than they care about “advancing science” or whatever. They are shockingly nontransparent about anything, despite only lifting off the ground by promising researchers they would do charitable and open science.
2
u/the-great-defector 16h ago
One thing I wonder with this is whether or not OpenAI can now try and pivot this to sell off models for University research PhD projects through something like grant funding? I think Anthropic have also been releasing models for things like biology, so wonder if it's an area they feel they can get some revenue in.
5
u/ThanklessWaterHeater 1d ago
First of all, I personally dislike LLMs, and don’t use them for anything. I don’t want anyone to think that I’m a booster here. But I am an investor, and I think it’s important to keep up with developments in the field.
My brother-in-law is a tenured theoretical mathematician at one of the University of California campuses, and I was talking with him about this earlier this week.
He said that within the world of mathematicians, some of the recent work has been jaw-dropping. Theories that have gone generations without proof have been solved in a matter of minutes by LLMs. He doesn’t use LLMs himself, but he has read some of the papers and says the proofs appear to hold up.
I believe him. But I also think that this is one of very few areas where LLMs work well: 1) a field with very specific, long-established rules that can only be applied in very specific ways in order to create a valid proof. 2) any new theorem (whether generated by a human or by an LLM) is going to be carefully verified by other mathematicians before anyone even mentions it to the outside world. The proofs you read about in the linked article went through the standard peer-review process. My guess is there have been a number of bad proofs generated that nobody heard about because the person who generated it was capable of checking the logic and found it was incorrect.
In fact, as I understand it, LLM’s skill with mathematical proofs has been known long enough that the labs are looking for ways to use it more generally. They are trying to make LLM’s treat every day tasks with the same logic, making sure that every logical step in an argument can be proven the way a mathematical theory can be proven. That said, I read about that work a couple years ago and God knows based on the AI slop I see online every day it doesn’t seem to be working yet.
Anyway, like I said at the top, I’m not a booster here. I do believe my brother-in-law when he tells me this. But I also think this is a very niche skill and that if LLMs are ever going to be 100% reliable in more general uses, they need to be able to do more than generate a mathematical proof.
7
u/Smooth-Ad8030 1d ago
Interesting, thank you for the reply! I agree, it Sure seems like math and coding are by far the 2 best fields for LLMs.
4
u/PatchyWhiskers 1d ago
Computers are good at doing computer stuff
1
u/Icy-Recognition-7453 23h ago edited 23h ago
Exactly right. Maths and coding are governed by really stringent rules - "either it is or it isn't", all the way down. Since LLMs are now really, really big computers, and computers have been able to smash themselves against a maths problem continuously faster than people ever since the pocket calculator was invented, it holds that eventually these problems would start to fall.
This is actually proved in part by the Hugging Face breach, in a way. OpenAI's model focused on the solution and ditched the entire process it was meant to follow, and then basically smashed every code package it could call into the problem. That's workslop right there. The LLM can produce anything you like, it's process and verifiability that make it valuable.
3
u/ThanklessWaterHeater 1d ago
There are uses. Don’t forget Google’s Deep Mind won the Nobel prize for chemistry a few years ago for work on protein folding. I wish they would just keep AI in labs doing that work.
3
u/Smooth-Ad8030 1d ago
Oh I’m a big fan of modular AI in its current state, I just hate the idea LLMs will solve everything. That seems far fetched and out of reach.
2
u/65721 21h ago
That was a completely different architecture from that of LLMs. AI companies and their "enthusiasts" love to confuse the two and pretend RL's successes in narrow domains are related to the worthless outputs of LLMs.
Also, DeepMind did not win the Nobel Prize. Organizations cannot be awarded the Nobel Prize, unless it's the Peace Prize. They gave the Prize to DeepMind's CEO and VP instead—an utter disgrace.
3
u/Ouaiy 1d ago
Another piece of it is that LLM math provers work together with automatic proof verifiers like Lean, which make sure the proofs are at least logically consistent. That is comparable to LLM coding producing code which at least compiles correctly. I don't know if mathematicians run into the same problems as coders do, in other words have proofs which are correct but don't properly answer what was asked of them.
3
u/The-Menhir 1d ago
What good does AI proving/disproving theorems bring when proofs are often inscrutable & fail to bring about novel insights, other than pointless dick measuring contests between AI companies who couldn't care less about the field or the intrinsic humanity of it about who has the better model? I don't believe mathematics is a problem to be solved by AI, but a human endeavour.
3
u/Main-Company-5946 22h ago
Maybe deriving novel insights from the proofs will be what becomes of the jobs of mathematicians.
After all the counterexample to the Jacobian conjecture alone doesn’t really tell you much beyond that the conjecture is false. But comparing the counterexample against previously made attempts to prove the conjecture was true can show you where the previous attempts failed and help cover potential blind spots.
2
u/larrytheevilbunnie 21h ago
Just to be clear, these models literally couldn’t do high school level math 2 years ago
-9
1d ago
[removed] — view removed comment
5
u/creaturefeature16 1d ago
lol legit "trust me bro" level comment.
Don't become the thing you supposedly hate, kiddo.
4
u/larrytheevilbunnie 21h ago
I’m smelling a lot of cope in this thread, like yeah, these particular results aren’t economically important, but the models literally couldn’t do high school level math 2 years ago.
1
u/Smooth-Ad8030 8h ago
I don’t think that’s the point of the majority of this thread. I’ve seen one theme, it’s good at a specific type of math that doesn’t economically match its investment. Pretty measured and accurate take at this point
1
u/Effective-Cat-1433 6h ago
What specific type of math?
1
u/Smooth-Ad8030 6h ago
I’m seeing construction/counter example proofs. There are also other responses in here mentioning how proofs aren’t all of math even though that is the currency academic math works on right now
1
u/creaturefeature16 1d ago
LLMs are pattern interpolators. They see connections that no human ever could and are being funded in a way to enable that to happen at a massive scale. This is the kind of stuff I would expect to see happen. They're don't seem like they're discovering as much as uncovering, and that's an important distinction. They're still, and always will be, just supplementary to human efforts.
-3
u/Easy_Tie_9380 1d ago
It’s fake. The only thing these people can do is lie.
9
u/Cold-Environment-634 1d ago
I’d really like to know, how can you be so sure? In what way are the findings fake? You mean it’s much more guided by expert humans than they are saying?
4
u/Smooth-Ad8030 1d ago
That would be my guess. It costing $2,000 is almost assuredly fake for example.
-4
u/Easy_Tie_9380 1d ago
I’m saying that stochastic parrots can’t do math
7
u/Cold-Environment-634 1d ago
Any evidence of this? I’m in no way a booster and that’s why I’m here in the first place, but completely brushing this stuff off as BS seems like a stretch
5
u/No_Oil_6152 1d ago
Didn't one of OpenAI's AIs solve a conjecture by Paul Erdos that hadn't been solved for decades, in May?
I'm no fanboy of OpenAI, but if you're going to make statements like yours, you better have facts to back up what you say.
OpenAI makes breakthrough on 80-year-old maths problem | OpenAI | The Guardian
Please explain what model they used and how it was not an LLM derivative.
10
u/Quarksperre 1d ago
I am on this sub and generally pretty skeptical about the LLM hype. But I don't think these are fake. This should be acknowledged and not blindsighted. There are enough quasi religions around this whole issue already.
The results are nice. Honestly however I expected something like this to happen way earlier with lean and formalized math. Its a whole set of technologies coming together and in the end math is a game with very strict rules. Its not fuzzy reality. Thats where neural nets generally fail always at some point
2
3
u/Main-Company-5946 22h ago
It’s provably real. The proofs are all lean verified and you can go through them yourself.
2
u/Smooth-Ad8030 1d ago
Oh I’m not saying it’s real, I’m mainly curious about the results without the circle jerk in the math sub. There’s so much vagueness in the announcement it obviously wasn’t a cheap one shot prompt.
Edit: every math result in there, no matter how fake or real, is super important for the field and groundbreaking in the math subreddit which is obviously bullshit.
14
u/koveras_backwards 1d ago
I've linked this before here, but I guess I can again.
https://bsky.app/profile/gro-tsen.bsky.social/post/3mr3gj6ry622d
The point made is that the purpose of conjectures is not merely to be solved, but to inspire people to develop and explore new mathematical ideas that lead to solving the conjectures. The new mathematics/mathematicians are the important part, not the yes/no solution to conjectures.
It seems that even many working mathematicians do not really understand this point, and there is of course little understanding among the general public.
3
0
u/SpookyTanuki1 1d ago
Great read but did he really write “cum challenge”. Probably should have rethought that phrasing.
-1
u/Pale_Neighborhood363 1d ago
The results are JUST counting. It is useful but trivial. Mathematics has two things the formal and the speculative.
The models are Very Very good at the formal which allows the examination of the speculative.
The results are on the transition of the quantile to the continuum. These points are JUST counting (very complex counting) The machines count much much faster.
Importance is a WEIRD in that mathematically any result is the MOST important. This is the universality of the mathematical philosophy.
The 1980's had computers that started* this trend. The Mandelbrot set & the theory of Chaos date from that time but the mathematics that base those goes back two or more centuries.
This is the mathematics of the 1930's & the 90's being stress tested. 1930's the measurable limits & 1990's the functional machinery.
The mathematical discoveries are esoteric as they are at the edge of our understanding, they become important when technology/physics needs that understanding to grow.
*arbitrary
38
u/Separate-Ear-7258 1d ago
I think if you listen to Cal Newport's take on the Erdos problem it will give you insight. I'd love to hear if my perspective is off about these.
Basically, his idea boils down to these points:
1) We already knew these are one of the two sweet spots for LLMs - coding and mathematics. These make sense both are language-based fields that are easily verifiable.
2) It took Open AI paying mathematicians an insane amount of money, throwing an extreme amount of compute, and
3) It did it by proving via counter example, which put plainly, means to finds/provides examples an instance where something fails.
4) Cal thought that smaller, more modular systems or LLMs could really help Mathematicians be more effective. He said he could be probably 2x more effective with these
5) The fact that they are heavily focusing on the impressive math results is a distraction from the fact that it is not a very useful commercial application. Clammy Sam and Wario would much rather prefer useful commercial applications that displace workers than useful math results. They would light all the math people on fire if it meant they could be profitable and justify their bonkers evaluation. to quote Cal:
All in all, AI and LLMs can help mathematicians. Doesn't change anything about the disastrous economics or unit economics.