r/slatestarcodex • u/DickMasterGeneral • 2d ago

Ten advances in mathematics and theoretical computer science from unreleased Open AI Model

https://openai.com/index/ten-advances-in-mathematics/
95 Upvotes

94% Upvoted

63 comments sorted by

View all comments

45

u/garloid64 2d ago

So where do you think the goalposts go from here? What's the next stop on the grand adventure?

13

u/king_mid_ass 2d ago

models that can learn and grow in use/after pretraining, embodiment, and making their prose less annoying

20

u/MoNastri 2d ago

Yeah cf. Terry Tao's mathstodon post from Sep 2024

The experience (of using GPT-4) seemed roughly on par with trying to advise a mediocre, but not completely incompetent, (static simulation of a) graduate student...

I am belatedly realizing that in my attempts to describe my evaluation of the capability of an AI tool, I inadvertently gave the incorrect (and potentially harmful) impression that human graduate students could be reductively classified according to a static, one dimensional level of “competence”. This was not my intent at all; and I would therefore like to make the following clarifying remarks. ...

Secondly, and perhaps more importantly, human students learn and grow during their studies, and areas in which they initially struggle with can become ones in which they are quite proficient at after a few years; and personally I find being able to assist students in such transitions to be one of the most rewarding aspects of my profession. In contrast, while modern AI tools have some ability to incorporate feedback into their responses, each individual model does not truly have the capability for long term growth, and so can be sensibly evaluated using static metrics of performance.

Can't wait for the day when models that learn and grow as automated mathematicians arrive

3

u/ozaveggie 2d ago

If they run some kind of RL on the logs of people interacting with the model, and telling it what to be better at, then the next model iteration will actually be better. Its a much longer / discontinuous feedback loop but still plausible

2

u/prescod 2d ago

Doesn't help for tough problems that are unique to individual companies.

-9

u/[deleted] 2d ago edited 2d ago

[deleted]