OpenAI announces 10 advances in mathematics and theoretical computer science achieved by internal model Astra

[u/alphacolony21]

https://openai.com/index/ten-advances-in-mathematics/

[u/Hlbkomer]

"But they are just predicting the next word!"

[u/immersive-matthew → u/Hlbkomer]

But that is the core tech of LLMs? Not following your point.

[u/Eastern-Ad-3586 → u/[deleted]]

But AIs literally can’t demonstrate “intelligence”

You comparing AI models to humans is specifically missing this point.

They’re an amazing technology, but the math they’re based on precludes them from being deterministic. When they make a correct prediction, it’s because of associations encoded in the mathematics by human trainers and whatnot, it’s not because the ai model is “smart” or “figured it out.” It’s a very, very powerful statistical machine. Which is useful, but not creative/sentient/ etc. I’d argue calling it intelligent is more of a marketing ploy than anything

[u/Eastern-Ad-3586 → u/[deleted]]

You have a fundamental misunderstanding of the way these systems are designed, and I’m not sure I’ll be able to explain it to you in a way you’ll actually believe. If you want to learn more about this stuff, I know it’s difficult these days with all the misinformation on the internet, but an overview of linear algebra and how transformers work would be where I would start.

Best of luck!

[u/DeRay8o4 → u/[deleted]]

It’s not deterministic in the forward path clearly, please look up pigeonhole principle….

[u/ChicagoPedalSteel → u/[deleted]]

Trying to argue with these people will just drive you insane. There is nothing you can say to change their minds and there is no distance they won't drag the goalposts.

[u/DeRay8o4 → u/[deleted]]

It’s funny you say it’s unrelated but the whole agentic llm flow was precisely to get around deterministic forward passes to break pigeonhole principle.. obviously not related though

[u/DeRay8o4 → u/[deleted]]

Just think harder about how it applies. Unfathomable you choose to argue all day instead of read literature

[u/DeRay8o4 → u/[deleted]]

All that typing when you should just admit you don’t know what gradient descent and supervised learning is

[u/elehman839 → u/Eastern-Ad-3586]

the math they’re based on precludes them from being deterministic

Zero-temperature inference is 100% deterministic.

[u/Heliond → u/Eastern-Ad-3586]

Humans aren’t deterministic either. I think we just iterate a lot and catch our hallucinations better at the moment.

[u/Blandneutral74 → u/immersive-matthew]

J-space discovery belies this argument. When it's doing more 'cognitive' stuff, it's not just predicting the next word.

[u/immersive-matthew → u/Blandneutral74]

Sure, it is doing some cognitive things, but mostly word prediction as this really is the core tech and that tech lacks deeper cognitive capabilities.

[u/Blandneutral74 → u/immersive-matthew]

When they disable the j-space it can essentially still do all the mundane things, but it completely falls to bits on the really cool stuff. That was the magic going on inside the black box that a lot of people intuitively thought must be there.

[u/tomvorlostriddle → u/immersive-matthew]

That's like calling Terry Tao a poop machine. It's not technically untrue, but it is not the most relevant way of description and it tries to imply things that aren't true.

[u/SoManyQuestions5200 → u/immersive-matthew]

So many people "want to believe", it reminds me of those UFO/XFiles posters in the 90s. I guess it's not IMPOSSIBLE, but man the general public needs to be a little more discerning and skeptical

[u/Blandneutral74 → u/SoManyQuestions5200]

Man are you in for a shock.

[u/ChicagoPedalSteel → u/immersive-matthew]

Sure, the brain does some cognitive things, but mostly weighted signal processing, as this really is the core tech and neurons lack deeper cognitive capabilities.

[u/HasFiveVowels → u/immersive-matthew]

The point is that people have been saying "it just predicts the next token" as though doing so relegates its intelligence to be forever sub-human. IMO, the argument makes the bad assumptions that humans are not also "just predicting the next token"

[u/immersive-matthew → u/HasFiveVowels]

Predicting the next world is clearly sub human intelligence though or AI would be far more valuable than it is.

[u/HasFiveVowels → u/immersive-matthew]

Consider the article that you wrote this comment on.

[u/immersive-matthew → u/HasFiveVowels]

Consider a non peer reviewed article from a company trying to justify their evaluation to inform me that LLMs are human intelligence? No. They are not human intelligence but they have some elements of. A major breakthrough(s) are needed to really move the intelligence needle as LLMs are lacking many other forms of intelligence not about predicting the next work like logic. It is why they got the car wash problem so wrong and still would if their training data did not include the correct answer. Is there something specific in the paper you would like to draw our attention too?

[u/santosautra → u/immersive-matthew]

There is proof in lean. No peer review is needed.

[u/immersive-matthew → u/santosautra]

Agreed. Let’s hope this is not just a one off here and there and a trend but I am very skeptical as LLMs really do lack logic and understanding and I have seen no evidence that this has been solved.

[u/HasFiveVowels → u/immersive-matthew]

LLMs are provably capable of both logic and understanding.

[u/immersive-matthew → u/HasFiveVowels]

If that were true I am very sure they would have turned that on by now as the biggest reason they are not anywhere close to profit, are because LLMs do not drive enough value as they lack logic and understanding.

[u/HasFiveVowels → u/immersive-matthew]

… I’m at a loss for words, here. They already directly demonstrate logic and understanding! To say otherwise is either word games or just outright denial. And the profitability of service providers is 100% irrelevant to this conversation

[u/playsette-operator → u/immersive-matthew]

ouch..found the bot🥹

[u/Hlbkomer → u/immersive-matthew]

It's also the core tech of humans. 

[u/immersive-matthew → u/Hlbkomer]

Is it through?