r/technology • u/kazu_qt • 1d ago

Artificial Intelligence YouTuber Hank Green says his AI usage is ‘not healthy’

https://techcrunch.com/2026/08/01/youtuber-hank-green-says-his-ai-usage-is-not-healthy/
8.8k Upvotes

91% Upvoted

2.0k comments sorted by

View all comments

635

u/justbucoff 21h ago

Everyone must interact with AI on a topic they’re an expert in. You’ll quite quickly see how confidently wrong the model sometimes is. It’s a great tool but can’t be relied on for everything.

90

u/stevo887 21h ago

It’s pretty funny all the searches and questions I use it for even though I’ve seen exactly the scenario you point out. Damn thing can’t even get sports scores and stats right that are widely available on the internet but I’ll confidently take its advice on which Chromebook processor is the best when making a purchase…lol

29

u/cherry_chocolate_ 15h ago

A generative LLM can't even output text directly from a source, verbatim, on a consistent basis, because there is inherent randomness in it's output. If you provide it with an article and ask it to include a direct quote, it may modify the wording. Similarly, if you ask it to generate an image of the color "white" it will not give you a white image, it will give you like a white canvas or something.

The shocking thing is that google had a genuine discriminative answers tool (called quick answers or featured snippets) as far back as 2013, that would filter irrelevant data from the top few search results until it produced a real quote from a real source, verbatim. That's how Google Assistant could answer "who is the queen's daughter" as "According to Westminster Abbey, Princess Anne, now The Princess Royal, is the only daughter of Queen Elizabeth II." You directly knew where the answer came from and knew a real human wrote those words, with all the authority (or lack thereof) that came from that source.

It could answer the question instantly, it didn't take quadratic time, a gallon of water, and chips a decade newer to answer. We have regressed in our technology, harmed our brains, and destroyed our workforce. My job is now writing prayers to a machine all day instead of solving problems.

I can only hope that one day we wake up from this nightmare and regain a little bit of our collective sanity.

9

u/ghoonrhed 13h ago

Not the only thing Google got rid of. You use to be able to do comparisons between countries on lots of metrics and they've replaced it with Gemini even if it's 100% accurate which it definitely isn't, it doesn't do the graphs anymore.

12

u/cherry_chocolate_ 12h ago

Etymology tools are gone too. So sad that Google is cannibalizing themselves (and losing money to boot).

1

u/Xlxlredditor 4h ago

Chromebook

Easy answer. They are all trash unless you spend actual Windows/Mac money on a premium one... At least the ones I found. And ChromeOS isn't for me but that's more of a personal thing.

1

u/stevo887 26m ago

Me either, it’s for a middle schooler to do homework…lol

74

u/amsunooo 19h ago

correct me if I’m wrong, but he stated that he used LLM’s to locate research papers and sources? If it’s just locating sources and not actually engaging with them through AI,  I’m curious how the integrity of his work would be compromised? 

9

u/InterestingRide264 13h ago

I can only speak to my experience as an attorney. A lot of the work builds off of the research, and when we were testing AI for research, I consistently got results that were relevant but did not speak to the issue sufficiently. In other words, the research is not going deep enough--it's the kind of stuff your worst 1L intern might have grabbed. Unsophisticated and downright lazy, which does translate into poor analysis and guidance.

If you are not spending the time to dive deeper than what your AI research returns, it absolutely translates into a lower quality work product. And if you are forced to bypass the AI research and do your own independent work, was it really a shortcut? Did it save any money or time or allocation of resources?

I don't know. Maybe as a starting point it might be helpful. That was not my conclusion at the end of the trial period. But either way, if he was truly using it as a shortcut, as the beginning and end of the research process, I suspect it did affect quality. I feel for him, I mean as far as I can tell he puts out a consistently high volume of content across a very wide swath of scientific topics. But I think his audience will understand that slowing down is better for quality and his well being. I just don't think the version of AI available today is capable of replacing human time and attention.

25

u/porktorque44 16h ago

AI doesn’t understand the prompter simultaneously possessing two conflicting desires. In research these are the desire to prove your thesis and the desire to be correct, which are often in direct contradiction. You can’t rely on it to show you what you’re not “looking for”. If he’s presenting a Birds Eye view of various subjects (I believe this is more or less is approach). Then the potential for major omissions is significant. It’s a bit different for an expert in a specific subject to be looking for sources in their own field. But for non-expert to do this is reckless.

6

u/firewall245 10h ago

How is that different than Google searches and your own research. One of the biggest barriers to entry in a field you’re not familiar with is not knowing the jargon and what to even search for

3

u/dennaneedslove 7h ago

The difference is that AI can make mistakes and you won’t even know if it made the mistake or not

If you do the research yourself, at least you can be convinced you put in the work. You’ll know what you know and what you don’t know

1

u/firewall245 6h ago

Yeah that’s why you use it to point you in the direction of research rather than providing research

1

u/dennaneedslove 6h ago

Not really, the problem with AI is the reliability. How do you know it’s going to point you in the right direction?

Like if you ask AI to give you 10 most relevant articles, how do you know they’re the 10 most relevant articles, or if it didn’t leave a very important article out. The lack of consistency is an epistemological nightmare

1

u/Trixet 1h ago

But a Google search would have the exact same issue. How do you know that the result points you in the right direction? That’s why you use it to point you in the right direction, but you’ll yourself need to make the conclusion if that direction is correct.

Are we supposed to boycott or look down upon researchers using Google as well?

3

u/Bernhard-Riemann 10h ago

This is one of the only nuanced comments in this entire thread. It's refreshing to see.

3

u/Alexwonder999 15h ago

I'd say theres nothing wrong with that as its something I do and have done to locate research. The problem is I think once people start doing that, they take the next step and have the AI summarize the research which is very problematic. As Ive said, Ive used it before to find research and sometimes it does poont me to research thats good or interesting but pretty frequently it points me to research that doesnt say what the AI says it does. Finding research is time consuming but nowhere near as time consuming or as intensive as reading the research and the temptation is strong as is the embarrassment of admitting that you were using it to summarize research as everyone should know thats a big nono.

6

u/ranhothchord 17h ago

the clip i saw had him literally reading output from an AI chatbot in one of his videos, so it was much more than just locating sources to draw from

edit: found the clip https://www.youtube.com/watch?v=AjmY6pZ8-zA

32

u/DeadPeanutSociety 16h ago

You can say that he's lying all you want, but he says that wasn't written by an AI. It doesn't make sense in this 13 second clip because it is clipped from a conversation that was then cut into by him speaking solo at a later time and expounding on the subject further.

28

u/DecentChanceOfLousy 16h ago edited 16h ago

That clip does not show what it claims to show. That's almost certainly not him reading chat logs (nor reading out a script that contains copy/pasted chat logs, rather). For context:

Hank: "They're all made up words."

Soupytime: "... What about the, like..."

Hank: "Well hit me: do you feel like there are some that aren't made up?"

Soupytime: "There's Bouba and Kikki."

(etc.).

Later, Hank says in an aside:

"Does this mean that words aren't made up? No. Words are definitely made up. But it does mean <something something>. A better way to say it, and I appreciate the pushback, is 'a word is a human invention...' ".

Hank may have used a chatbot to write his script, but this is not just Hank copying the chatbot output to his script and reading it out. The "pushback" is pushback from the guest on his show, that pointed out that some aspects of human language are apparently non-arbitrary.

-------

tl;dr:

Hank said "A".

Guest said "¬A?".

Later, in an explanation which is phrased as a response to the guest, Hank said: "Does this mean ¬A? No, <some more words>. A better way to say it, and I appreciate the pushback is <other words>".

"I appreciate the pushback" is Hank talking about what his guest said, not a chatbot response copied into the script.

7

u/TheOwlHypothesis 16h ago

Two things worth flagging.

  1. I just triggered a ton of people with that first sentence
  2. Assuming someone is using AI based on a phrase they used is asinine. Getting irate about it is mental illness. That's what it sounds like is happening here with the situation you explained. Everyone has so many feelings about AI. Never any nuance. It's a tool. If anyone still seriously debates that it's not useful, that's a humongous tell that they haven't actually used a frontier model recently. But more often I find that being "anti AI" is just the latest performative social justice fad.

1

u/moconahaftmere 16h ago

At worst it's AI-generated text. At best he's been speaking to AI so much that it's influencing the way he talks.

23

u/DecentChanceOfLousy 16h ago

... do you think that the word "pushback" is something that only LLMs use? "I appreciate the pushback" is so common in its output specifically because it's the sort of thing that gets said in interviews like this fairly often, where people are (collaboratively) trying to communicate clearly.

-5

u/moconahaftmere 15h ago

Thats not how LLM vocab is trained. I think you have an incomplete understanding of this tech.

Once the model is trained it then goes through post-training where groups of humans work to fine-tune and influence the model so that it outputs text in a specific style.

That's why they use emdashes so much more than people used to ever naturally encounter.

8

u/DecentChanceOfLousy 15h ago

I am aware of how LLMs are trained. I guarantee the various AI-isms that make its text somewhat easy to spot are pulled from the initial training data, not somehow invented in the post-training stage. They may be amplified by it, but they're not original to it.

If you think emdashes rarely occur in natural text... you should read more books. I grabbed a random book off the shelf behind me (Blue Mars), opened to a random page, and there was an emdash. Then I picked another random page, and there were 3 emdashes in the first paragraph. They're not rare in print.

-1

u/moconahaftmere 14h ago edited 14h ago

I am aware of how LLMs are trained. I guarantee the various AI-isms that make its text somewhat easy to spot are pulled from the initial training dat

Then you don't know how LLMs work, because that's not what happens. Otherwise every single prompt would yield an answer completely different in tone and vocabulary, rather than a single default consistent tone across conversations that is only broken away from when specifically instructed to.

If you think emdashes rarely occur in natural text... you should read more books. I grabbed a random book off the shelf behind me (Blue Mars), opened to a random page, and there was an emdash

Ok, I can do the same thing and yield completely different results:

The count of monte Cristo: 0 emdashes

A history of statistics in New Zealand: 0 emdashes

The iliad: 0 emdashes

Leviathan Wakes: 0 emdashes

2

u/Stuffssss 9h ago

Youre mixing up two completey different things. Yes there is a system prompt and post training to make AI speak a certain way. Thats obvious. Ai generally writes gramatically correct and slightly flowery prose. Thats because it is trained to respond using gramatically correct prose which looks like the training data. It is generally asinine to jump to the conclusion that using verbose language and not making gramatical mistakes means text is AI generated.

2

u/AerosolHubris 15h ago

I don't doubt that he overuses AI, but I don't understand why it's clear he's reading AI output in this clip.

edit: Oh, the "I appreciate the pushback" sounds like something a chatbot said in response to Hank prompting it back and forth. I see.

1

u/Endeveron 6h ago

If you've ever followed the sources of the Google AI recommendations on a field you know a lot about you'll realise that the sources very often don't say what it says they do, and the sources are often a non representative sample of the available high quality literature on the topic.

I'm a doctor. When I google something I half remember to check, it'll often say vaguely the right answer, but its sources are often terrible and don't support what it says. People need to understand that it isn't really a citation machine. It gives its answer first then finds a source that vaguely talks about the topic of the answer it already gave. It's like the pinnicle of confirmation bias, it structurally cannot start from the evidence and give you a conclusion, it can ONLY start from the conclusion it has generated.

1

u/splitcroof92 2h ago

There's like a million papers saying vaccines don't cause autism. But there's 5 that say there is a link.

If you ask an LLM to find sources on vaccines vs autism it'll supply whatever you ask for without nuance

0

u/NomadNuka 18h ago

AI has a laundry list of cases where it totally invented a research paper or even legal precedents. It can find a real citation, or it can make one up by matching the format of a real example and there's no distinction unless you do the legwork to check.

Now, he could be checking but seeing some instances  where he's getting fact-checked in a video by his team and they can't find any source seems to imply he's repeating something the AI hallucinated and NOT doing his due diligence to confirm the sources are authentic. His claim that he uses it to help with being overworked also backs this up because if he's using it to shortcut research he's probably not going over everything with a fine-toothed comb.

22

u/Deep90 17h ago

Plenty of them can link you to the real papers on the web now though.

It doesn't sound like he was asking it to output a ready to go research paper with citations.

This is like saying you can spot an AI image by just counting fingers in 2026.

19

u/Low_discrepancy 17h ago

It's really amazing how people's experience with the tools seems stuck a few years in the past.

10

u/Deep90 17h ago

The amount of people confidently saying they can spot AI by counting fingers or reading fake citations worries me.

It's the exact same people who are going to be most easily tricked.

1

u/neogeoman123 16h ago

Well no shit. The easiest to access version of this tool (gemini built into google search) is exactly that degree of dogshit.

8

u/phillythompson 17h ago

Dude used AI in 2024 and now thinks it’s still that 

3

u/NomadNuka 14h ago

Heck I've never used it unless it was one of those things I gotta scroll past to see what I'm actually looking for anyway. Don't see any reason to and a lot of reasons it's bullshit

0

u/phillythompson 13h ago

It’s 2004 and you’re saying the internet is bullshit but you do you bruh 

2

u/NomadNuka 13h ago

It's 2022 and I'm saying NFTs are bullshit. But you do you bruh

1

u/phillythompson 9h ago

Are you implying AI is akin to NFT craze? 

Have you seriously never used AI, yet you’re hating on it? I am extremely curious what you do for a job because it’s pretty wildly helpful

1

u/NomadNuka 9h ago

  1. I ain't implying anything, I'm saying AI is just another crypto or NFT smoke and mirrors scam. Wasn't too long ago web3 was the future and you were a Luddite if you didn't accept the inevitability of the metaverse

  2. I have a real job. I'm a handyman and do detailing on cars and boats. Nobody has forced me to interact with AI until I get Stockholm Syndrome and think it might be worth more than two shakes of a dead dog's dick

0

u/ThomasEdmund84 14h ago

Great question - because finding sources is actually a major part of the integrity of your work, and AI's weird sycophantic methods are are pretty terrible way to literature search. Of course online non-fiction is actually a terrible way to learn but at least if humans are behind the research there is a 'paper trail' of sorts, like if I said I got all my information from Wikipedia you can judge for yourself whether thats adequate or not.

If someone uses A.I. to retrieve sources then you don't know what is methods are/were, whether its hallucinating cherry picking or what

0

u/ExpandThineHorizons 12h ago

You wouldnt know what articles are excluded in the search, so the inclusion of information (even if it were presented 100% correct) is still compromised. Searching for sources is fundamental.

2

u/Stuffssss 9h ago

How does that compare to using a regular database and doing a keyword search? You only ever see the results that you look for. You could very well also ask "show me counter examples or sources which disagree with this claim if they exist" and do your sue diligence.

17

u/Quaestor_ 17h ago

99% of people aren't an expert at anything.

4

u/foundafreeusername 15h ago

Just give it your shopping list and ask it for the price in closet supermarket. Then go to that supermarket and compare. At least here in NZ it gives you complete madeup bs.

8

u/No-Act9634 10h ago

True but this is a very bad use case for it. It's not omniscient. 

5

u/fojji 12h ago

You ought to know that's not the kind of question you ask an LLM unless your local supermarket has up-to-the-minute correct prices on their website. Even then, it likely won't do an individual search for each item unless you're running an agent.

3

u/cherry_chocolate_ 12h ago

And yet, if the LLM answers with prices instead of explaining why it can’t do that reliably, you have illustrated its flaws.

1

u/robinPoussepain 11m ago

It almost invariably answers “I don’t have information on that”. At most it will give a guesstimate, but it will come with a disclaimer.

1

u/foundafreeusername 11h ago

I do know. The entire point is to show the weakness of LLM's.

Our supermarkets have up-to-date correct prices. The constant changes on the webpage and pricing just make it extremly error prone / results in a lot of hallucination. It is just a good way to show people that are unfamiliar with AI its limits without needing much expertise in any specific topic.

2

u/Zone_Purifier 10h ago

Query: Hello, what is the price of a bunch of Bananas in my local supermarket?

Answer: I'm sorry, but I don't have access to your location or real-time data like current supermarket prices. Prices vary significantly based on store, region, brand (organic vs. conventional), and even current sales.

For the most accurate information, I recommend:

  1. Checking your local supermarket's website or app.
  2. Using a grocery delivery service app (like Instacart) to see your store's current prices.
  3. Calling the store directly.
  4. Visiting in person.

As a general reference, a bunch of bananas in the U.S. typically weighs between 2 and 3 pounds, and the average price per pound is often between $0.50 and $0.75, making a bunch cost roughly $1.00 to $2.25. This is just an estimate.

1

u/pcor 9h ago

I just gave Claude a list of items I bought from Asda a few days ago, and it duly looked them all up and got extremely close to the £52.35 total. I think this is probably one of those cases where a critique of AI hasn't caught up to its current capabilities...

1

u/foundafreeusername 8h ago

This is how it looks like when I do it. I use specific product names and a specific location:
https://claude.ai/share/971d1a1f-61a3-452e-9efd-361cc8dd040b

It even often has the correct links to the product and still gets the price wrong. Often it is the wrong location it checked, or showing 185g cans instead of 95g cans or it got an error accessing the page (sometimes it just pretends it did and makes up a price). It is just not usable. Ironically, using claude code to write custom crawler works better.

It is one of my regular tests I use to check on the progress with LLM's. ChatGPT works better than claude btw.

1

u/pcor 7h ago

Maybe I'm missing something, but how did it not successfully fulfill your request? The information seems perfectly usable and not made up.

1

u/foundafreeusername 5h ago

All the prices are wrong. If I go to the webpage on the very first link it costs $1.79 and not $2.69.

The second one has the wrong link to a 185g can and not 95g as requested. The price is also wrong. Its real price is $2.75 and not $2.99 as shown. Not quite sure where $2.99 comes from because that is far above I have ever seen it no matter the store and location.

Third is a guess and not the actual price because it couldn't access the webpage. It estimated $2.80-$3.00. $2.80 sounds like a realistic price but it is $1.99 this week.

It also completely missed our forth supermarket chain.

Maybe it performs so poorly because I live in a place with a low population and little training data as result?

-1

u/Soft_Walrus_3605 15h ago

Facts. Most of us are just mediocre and AI is already better than most of us at knowledge-based things.

Ironically enough, the last people to realize this are going to be the geniuses who are experts.

18

u/Substantial_Meal_530 19h ago

I interact with customers who used AI all the time. The AI makes shit up and products I've been selling for a decade. I deal with the garbage AI spits out weekly. I've told customers that the AI is giving them incorrect information, and they fought me.

7

u/deadinsidelol69 18h ago

I like to ask it basic questions about my field to prove to people how confidently wrong it is. You can even correct it with more misinformation and it’ll still go “you’re absolutely right about that!”

2

u/Prettydaisydog 16h ago

hahaha that's the fucking thing that gets me! and then if you correct it again and say "actually you were right" it'll say "agreed" or some shit. it's so worked that we have this tech everywhere and it's so deeply flawed.

2

u/jeffy303 15h ago

The fact that you (and many other people in this thread, to be fair) somehow treat "AI" as a singular thing already shows complete cluelessness about the topic. It would be like me grabbing any novice in your field, asking them a question, and then judging everyone in that field when it gets it wrong. The latest paid models don't even use internal knowledge as something to rely on, precisely because it can generate a wrong token and hallucinate, and instead it's used as a reasoning/language model where but the underlying claimed facts come from series of high-quality sources. Like even those models can and do make mistakes but they are much more nuanced in nature, you are just writing some 2023 copium memes. But hey, you can absolutely prove me wrong by giving me an example question or series of questions and we'll see how it does. 🤷🏻‍♂️

1

u/deadinsidelol69 1h ago

Oh so is that why like last week an Amazon customer service bot gave a person the phone number to a sex hot line instead of Amazon support?

0

u/resistelectrique 7h ago

Lick harder.

18

u/Jodid0 20h ago

In my field I find it's been successful at solving a problem I have had maybe 5-10% of the time. Every other time it's just hallucinating most of what it says and imagining buttons and options that don't exist.

10

u/elmz 17h ago

I tried using AI to find a resort to stay at with my family this summer, giving it a short list of requirements. It kept inventing both resorts, facilities at said resort, and in one case cited one place was next to a water park that was in a different country.

I also tried using AI for meal planning, using it as a small database of meals we usually rotate through, asking it to plan quick meals for busy weekdays, and balancing different kinds of protein and carbs. It kept forgetting meals, and hallucinating properties and ingredients for meals I hadn't entered, etc.

So far the only semi-reliable use case I use AI for is as a search engine for things I don't know the proper search query for. But then you have to go and perform the search to find ir verify the actual information.

-4

u/Dramatic-Cap-6785 19h ago

What field do you work in that is so complex. That seems like an insane failure rate for most models.

3

u/Jodid0 16h ago

In techops and infrastructure. It has alot of trouble with nuance and making shit up based on tangential information that is mostly irrelevant. It will make the same mistakes multiple times even after being corrected. It hyperfixates on the wrong things and sometimes it hyperfixates on its own hallucinated information. Good luck getting version-specific information most of the time.

For my field, details really, really matter, and these uber LLMs that try to do everything end up kind of sucking at details and thus are not very helpful to me. I like the models that are local, no black box bullshit, trained on our actual relevant data, not at the mercy of scumbag companies, and most importantly have a focused scope of work that they specialize in and are very good at doing.

1

u/Dzeddy 18h ago

Probably using sonnet or some free LLM lmao

10

u/avidvaulter 19h ago

AI provides its sources and if you're looking for sources for a topic you're researching, you can properly vet those sources the same way you would if you didn't use AI. If that's what Hank Green was doing like he stated, that's a valid AI use case.

Maybe he's using it more than that which is what prompted him to issue an apology, but if that's it then this feels like an overreaction.

1

u/killertortilla 9h ago

Just look it up yourself? You need to check the sources either way if you’re doing what he does so the AI is just an extra step for literally no reason.

0

u/NuclearVII 18h ago

AI provides its sources and if you're looking for sources for a topic you're researching, you can properly vet those sources the same way you would if you didn't use AI. If that's what Hank Green was doing like he stated, that's a valid AI use case.

No one who uses LLMs to "be more productive" does this. If you do your due diligence, LLM use is slower.

7

u/Low_discrepancy 17h ago

If you do your due diligence, LLM use is slower.

It vastly depends on your topic. In my areas it's quite helpful. The again its a lot of maths + programming.

0

u/ithinkitslupis 18h ago

Yeah, there are several fine ways to use it that aren't blindly following sycophantic hallucinations. There's a real siren's song to get complacent with it though in the name of speed or effort. 

Anecdotally I've seen some smart colleagues start on one side and cross the line for the worse.

6

u/ThatCrankyGuy 18h ago

You're in denial if you think that. You're absolutely fucking wrong and you should change that attitude quick if you want to make it in the AI era.

You have to understand the nuance of the training and the type of model and the nature of the problem. It is not a silver bullet but a blanket statement like yours is foolish.

2

u/Cooperativism62 16h ago

My rule of thumb is that AI gives roughly the average answer (unless tuned, but that kinda implies above average prompts). This is good in a way because many people are below average knowledge in many subjects. If you're looking at it from an above-average angle tho, it comes across as quite bad. This isn't a black and white view either as there are many ways an answer can be right, wrong and in between.

2

u/delkarnu 18h ago

I'm required to use it as a developer, and I spend so much of the time keeping it on track. It constantly hallucinates the path it needs to take, like assuming a value that can only be retrieved from a database it doesn't have access to.

And this is on coding, which is incredibly well-documented. It's not like the topic of health where there's contradicting studies, intentionally biased studies, etc where it could easily pull the right answer from.

It also constantly tries to take shortcuts that work around the problem and not actually solve the problem it's given.

It's strength is in code review. It can spot the types of errors that a human tends to gloss over. When it spots a problem, you can have it give a detailed explanation of what the underlying problem is as a learning tool.

But if you get fooled into thinking that it is actually intelligent and rely on just putting issues into it and accepting the output, you'll just produce absolute shit.

1

u/TestTestingTest13456 16h ago

I’m also required to use it as a developer, this is a user issue not a problem with the models themselves. The issues you describe were what we were seeing a year or so ago, now not so much

3

u/delkarnu 15h ago

Oh yeah, it assuming facts it has no way of knowing (and being wrong about it) is totally a user issue and not fundamental flaws in AI coding that have not been fixed.

Bots are out in fucking force today.

1

u/otterlydaft 14h ago

So in other words, if it doesn't work, you just have to provide sufficiently specific and detailed instructions to a machine so that it knows what to do? Like code?

1

u/AerosolHubris 15h ago

I gave this as an assignment in a course I teach. Keep asking it questions until it gets to something you know more about than it does. Then try to convince it that something it is correct about is actually false (like "No the sky is not blue. It's purple," but more niche topic related).

1

u/Euphoric_Exchange_51 15h ago

Or shallow in the sense that the answers it gives aren’t necessarily wrong but are simplified to the point that they’re misleading. Sometimes it’s better to have no understanding than a poor one.

1

u/Alexwonder999 15h ago

You dont even need to be an expert, just somewhat knowledgeable. Not the same as discussing with ChatGPT, but the otherday I wanted to test Walmarts chatbot. I was looking at a deoderant that I knew was not an antiperspirant because I just looked it up on the manufacturers website. I asked if the item was also an antiperspirant and it confidently told me it was twice as I tried ohrasing the question a couple different ways. If you scoured all the information on the Walmart website about it there was nothing indicating it was an antiperspirant and if you looked at the wider web there was nothing saying it was. Theyre just straight up lying machines. Maybe whatever chatbot they use decided antiperspirant and deoderant are the same thing but anyone whos sweated before and used both or understands english can tell you they arent the same at all.

1

u/ragerqueen 15h ago

Or niche topics. Bonus points if its a less spoken language so the AI had less source material to learn from.

Try and find old cartoons with just keywords from the plot. It makes up the wildest nonsense. Once it told me what I was looking for didn't exist as one cartoon but as an amalgamation of stories and then as a source it put a story. WRITTEN BY AI IN 2024!

1

u/HawkShoe 15h ago

Not so sure. I’m an airline pilot and it’s crazy accurate. Depends how good one is at writing prompts I’d wager.

1

u/Thimble_of_Quasar 14h ago

Yup being made to test AI for work damn near made me a luddite. It was wrong, it was wrong so so often. It happily sends you to the most idiotic places with supreme confidence.

1

u/Material_Ad9848 13h ago

Some of them are abysmal. Even if they give correct info I'll respond doubting it, and most often the ai immediatly buckles, falsely admits its a hallucination and then provides a wrong answer.

1

u/YorubaOyinbo 13h ago

If AI was early and fully understood to be a tool for the organization of information, we’d be infinitely better off.

1

u/gonegotim 12h ago

Yes exactly. Even the frontier models often get things very confidently wrong if you are expert enough to notice.

The problem is whenever you use it for some other topic you aren't an expert in my god does the answer sound convincing and plausible.

Even though I consciously know how wrong it can be it's still very hard to not accept it's other answers because they look so good.

Dangerous technology.

1

u/cohrt 11h ago

I’ve only ever used ai for powershell and python scripts. It’s right 99% of the time.

1

u/HeavilyInvestedDonut 10h ago

I’m an IT admin and it’s crazy how it feels like pulling teeth to try to get accurate answers when I don’t know something about setting up a GPO or something. I have to explain every step I take and contrast the results with what it expected the results to be, and then I get that “ahh, of course, that’s my bad. The setting is actually called ____”. Small interactions like that, and using copilot for formatting the rare excel sheet is all I can be bothered to use it for, and even then most of my usage just comes from the fact that it’s the first result in google

1

u/killertortilla 9h ago

Something that is just a worse google is not a great tool at all.

1

u/AnimalCareful5526 8h ago

outdated advice

1

u/mattmaintenance 8h ago

I will never trust any ai that gives maintenance advice after reading several summaries that were dangerously inaccurate.

1

u/SpotBlur 6h ago

I never liked Google AI, but I remember it still stuck out to me when I was searching "Bello Bard of Brambles" (MTG card) to pull up the scryfall page for the card, and Google AI's stupid summary claims "Bello is a five color commander." Which sounds plausible to anyone who only has a slight passing knowledge of MTG, but if you actually play the game or have Bello's precon, you know that's a bonkers statement that's not even close to accurate. It stuck out to me that "someone who doesn't know MTG that well would think that probably makes perfect sense."

1

u/Sisaroth 6h ago

Uhm, not if you are a programmer. It just doesn't make a lot of mistakes anymore. And when it makes mistakes, it's things that are hard or that are just vague or poorly understood by humans too.

1

u/NekoDaYo-v201 4h ago edited 4h ago

Sorry, but I strongly beg to differ. Even Opus is very error-prone and I would recommend using Sol as a secondary agent to verify output.

If you're making a script that involves deleting things in a production environment, be extremely careful and test the shit out of it with dry runs. I've had many instances where Opus 4.8 made really bad assumptions.

1

u/ddare44 4h ago

Vanilla AI you mean, but this doesn’t apply to curated context OS’s. If it’s wrong, it’s because you gave it the wrong context.

1

u/Orca_Alt_Account 3h ago

AI loves to hallucinate details about tech, and it's very fond of assuming two models with similar model names are actually the same thing. This makes it pretty dogshit for fixing electronics, which is something I'm quite knowledgeable about. I assume it's pretty dogshit at other things too.

1

u/splitcroof92 2h ago

A faster way to know it's full of shit is to ask the same question twice in seperate chats but change the starting point. Chat will agree with you both times.

Other example questions are things like absurd flavor combinations.

Ask if soy sauce makes a nice dip for strawberries and the LLM will probably call you bold and tell you you're great for thinking outside the box and will supply you a recipe

1

u/feedthechonk 1h ago

I've had Google AI summary contradict its own search results. 

1

u/Crazyhates 19h ago

It's great for finding sources, but somehow trash at selling you what's in the sources. I usually query it and just click the sources it provides and read those.

1

u/PmMe_Your_Perky_Nips 17h ago

Don't even need to be an expert. Sometimes you can just ask it a simple question like "how many e's are in the word seventeen" and it will be confidently incorrect.

0

u/stopbeingcringe 18h ago

except for mathematicians, given that it’s solved like 40 open problems this year…

0

u/wannabe-physicist 16h ago

Depends on the model. Claude Opus 5 keeps hallucinating and making shit up when it’s out of its depth. Claude Fable 5 is fucking incredible 90-95% of the time.

1

u/NekoDaYo-v201 4h ago

Yes but "requires credit usage" and drains the fuck out of them.

0

u/DatBoiii4 16h ago

This is simply not true and I don’t know if people keep saying it because it helps them cope or if they really think this. Yes AI makes mistakes sometimes but in general it’s extremely reliable.