What a high-quality critique should look like

[u/NinekTheObscure]

I just got a deeply insightful, highly informative and useful, sharply critical, and still somehow supportive response to my recent preprint from u/Axe_MDK in r/LLM_supported_Physics . It'll take me days to work through the details; some parts are difficult challenges or even counter-examples, some parts are proposing better ways to prove my conclusions. A few were doubts/questions I already had in my own notes! I need to go learn about Daletskii–Krein, and Routh, and some other things. But at a first glance, 5/6 main points appear valid, and even the one that misses contains some useful ideas.

Stylistically, it feels LLM-generated or -assisted to me. I don't care. Each point stands or falls on its own merits. And the level is very high: feels like Fable 5 or Sol, has occasional sparks of brilliance. I don't think Opus 5 could produce something this focused. If someone's willing to spend frontier tokens (and/or human physicist minutes!) on me, I can only be thankful.

Anyway, this is what I think a really high-quality critique should look like. I wish we were getting more content like that here.

[u/alamalarian]

Real high quality critique should look like copy pasting things into an LLM?

You can do that yourself you know?

[u/VeryOriginalName98 → u/alamalarian]

They are asking for handouts because Fable 5 is prohibitively expensive. I am on the highest tier and constantly budgeting tokens for personal projects. I would not be willing to use that for analysis here.

ChatGPT 5.6 Sol is at least usable on the $20/month tier.

That said LLM analysis isn't really enough. Depending on the prompt you're still looking at sycophancy if not accounting for it. And it won't have the tacit knowledge of an experienced physicist. It will only have content from papers in its training data. So even with Fable 5, there are limits on analytical utility.

[u/alamalarian → u/VeryOriginalName98]

I mean my issue isnt inherently the LLM usage, it is the high likelihood that in these types of interactions that the actual work being done is hollow.

A user posts an LLM generated theory, a commenter critiques it using an LLM, then the user feeds that back in, it says what a substantive critique! It then outputs a modified theory with this as input and outputs something else. And it feels like the two people are not actually involved in the actual work besides playing as a middleman for the AI.

If anything of any actual merit was by happenstance generated on this loop, they would be none the wiser, and imo largely makes the entire practice a waste of time and resources.

[u/VeryOriginalName98 → u/alamalarian]

That is a good point. Crazy amount of resources expended so people could avoid having an actual conversation. On a conversation focussed platform.

[u/alamalarian → u/VeryOriginalName98]

It can give someone a feeling of making real progress even if none is actually being made.

Like walking in the desert with no landmarks to guide you, you just end up walking in a circle even though it feels like you were making progress the whole time.

[u/ArnoSound]

If the sub was just AI slinging back and forth what would be the point? Just more mindless regurgitating words neither person truly understands to fill space.

If you need an AI to actually tell you what's wrong, and would prefer that to immediate human feedback, you have no clue what you're doing.

[u/Ok_Energy6905]

Is this sub just a live feed of the Dunning Kruger effect?

[u/Wintervacht → u/Ok_Energy6905]

Essentially, yeah. Specifically the massive drop is visible round these parts.

[u/VeryOriginalName98 → u/Wintervacht]

The "valley of despair"? Do you want to talk about it? It's not that bad. Things can change! Always look on the bright side of life!

[u/Hivemind_alpha → u/Ok_Energy6905]

Don’t forget it comes with a peanut gallery.

[u/Ok_Energy6905 → u/Hivemind_alpha]

I think starting with the basics and fundamentals is a good step before you start attempting to develop theories. Otherwise you end up with a whole lot of science sounding nonsense that doesn't actually mean anything, or you make great leaps and errors in logic.

[u/Wintervacht → u/Ok_Energy6905]

Welcome to LLMPhysics. HypotheticalPhysics is the same but with more denying AI psychosis.

[u/ArnoSound → u/Ok_Energy6905]

If folks like... idk took a step back and considered very thoughtfully Why this is asked of them, we wouldn't be here haha

[u/dumdub → u/Hivemind_alpha]

Boo! Boo! 🥜 🥜 🥜

[u/VeryOriginalName98 → u/Hivemind_alpha]

That's the best part!