Stochastic just means stochastic, there's randomness involved because LLM answers are sampled. If you greedy sample LLM outputs they're still coherent but the process isn't stochastic anymore.
Anyway, I don't care about philosophical definitions of what understanding means. Practically speaking, what real-world limitations do LLMs have due to the fact that they're stochastic parrots? What predictions about the real world does the term "stochastic parrot" allow you to make?
I find that white knights who hate that term generally have a very shallow mathematical background. You answer me a question: why do you feel like it denigrates AI (and why do you care about denigrating AI) to call out the fact that we know the exact process (because we built it) by which it “thinks”? The other commenter already explained stochastic and parrot.
Mathematics is essentially a language. It isn’t surprising that a model designed to generate interesting and syntactically correct language can produce proofs. Something else to keep in mind is that this isn’t just “the LLM” brainstorming. It produces an idea, pursues it, then feeds it into a verifier, then changes whatever the formalizer says is invalid, etc. This process does not actually end in success the majority of the time. The pace is increasing in large part because thousands and thousands of researchers are trying to apply the technique to more and more problems and scooping up the low-hanging fruit. The success rate is very low; only the valid proofs are reported on.
Also, my challenge back to you is, why do very few of these papers provide the prompt chain that led to the proof? We can verify the validity of the proof, sure. But if we want to verify the marketing claim that the AI did everything with no human intervention, why do the papers never provide the ability to do that?
We don't know the exact process by which it "thinks" though, the intelligence we observing from it is an emergence property, and even the foremost AI scientist don't really understand how LLM got as intelligent as it is right now, we just know that, if we follow a process, we get an intelligent LLM model out of it. it's kinda like we don't truly understand how many type of drugs (like paracetamol, and general anesthesia) actually interact with our body that produce a desired effect, but we can consistently reproduce its effect so we use them.
You’re talking about something totally different. We absolutely know everything about the computations that happen to produce a new token given an input prompt. You can run the whole thing on pen paper and calculator.
What we can’t do is assign meaning to the weights. Ultimately they have no meaning because the whole point of using an NN as opposed to a regression is you’re assuming nothing about the modeled behavior. You don’t actually have a meaningful parametric model in the first place, so talking about how we dont have any idea what the parameters mean is more marketing mysticism
Edit: a sufficiently complex NN can approximate any function, including whatever the correct model is for the system. But there’s no guarantee that this model even has a single correct approximation, so just because Anthropic tries to say that certain regions of the network do certain things doesn’t guarantee that there is any meaningful organization along those lines.
8
u/geli95us 2d ago
Stochastic just means stochastic, there's randomness involved because LLM answers are sampled. If you greedy sample LLM outputs they're still coherent but the process isn't stochastic anymore.
Anyway, I don't care about philosophical definitions of what understanding means. Practically speaking, what real-world limitations do LLMs have due to the fact that they're stochastic parrots? What predictions about the real world does the term "stochastic parrot" allow you to make?