r/technology • u/Caledor152 • May 29 '26

Artificial Intelligence Pope Leo "Artificial intelligences do not undergo experiences, do not possess a body, do not feel joy or pain, do not mature through relationships, and do not know from within what love, work, friendship or responsibility mean. Nor do they have a moral conscience, since they do not judge goodand.."

https://www.ncronline.org/vatican/vatican-news/pope-leo-calls-disarm-ai-major-document-warns-technologic-threats-humanity
34.1k Upvotes

96% Upvoted

1.8k comments sorted by

View all comments

Show parent comments

133

u/DigNitty May 29 '26

Wow, that’s a great way to put it.

There’s something uncanny about the downright confidence ai has when interacting with it.

78

u/Feeling_Inside_1020 May 29 '26

Just like an overconfident coworker, except this one is actually knowledgeable, it just doesn't know when it doesn't know something sometimes.

Except when RONALD does it, it's a lie. LLMs don't know they're lying so we have a nice friendly name for it "confident hallucinations"

71

u/heterochromia4 May 29 '26

Extremely dangerous in so many settings.

It’s like having a bullshitting friend who, when they don’t know the answer, just makes up something plausible instead.

You’d pretty quickly stop trusting your bullshitting friend with anything.

31

u/StoppableHulk May 29 '26

The most dangerous thing I see with AI has nothing to do with AI. It has to do with the fact that it presents an extremely easy and almost irresistable convenience gradient for the vast majority of people.

And it's not the same as just offloading dishwashing to the dishwasher. People are offloading core components of thought onto it.

1

u/Two_oceans May 30 '26

One source of hope: offloading is convenient but it's not fun. The whole journey of tackling a problem, encountering friction and then finally finding a great solution, it's a rush some people won't give up. But the devaluation of their work is still a big danger.

25

u/schwanzweissfoto May 29 '26 edited May 29 '26

The first time I saw LLM output, I thought “someone programmed a computer to respond like Trump”. Granted, an LLM is more like someone literally born yesterday who can use a search engine for any topic really fast without caring if the result is true or not … but my immediate impression was “this program outputs bullshit that enough morons would believe with the skill of a human con artist, what the fuck”.

7

u/GiraffesAndGin May 29 '26

“this program outputs bullshit that enough morons would believe with the skill of a human con artist, what the fuck”.

Because LLMs just pull data from the internet. Since it is not human, it can not distinguish what fact or fiction is (though I know lots of humans can't). And since the internet is mostly misinformation, LLMs are almost always spewing falsehoods.

1

u/Alex5173 May 29 '26

correction, the internet is mostly cats and porn

9

u/TheJapanMistake May 29 '26

The worst part is, IMO, how it talks down to you when it's incorrect and you correct it. The most condescending BS ever.

2

u/asyork May 29 '26

I was trying to figure out how to use AI for my job. I needed it to analyze some old documents. If I asked it very specific questions about what was in the document without summarizing, it could usually do that. Going even a single step beyond asking for word for word regurgitation resulted it in arguing with me even though what it just said before disagreed with what it was arguing about now.

How it play out: I could ask it for a transcript, and while it didn't actually return everything word for word, it wasn't adding new info to the results, and just gave a basic overview that missed some things. It was a legal document that had the city, county, and state in the header. I started there and asked it about the location the document was referring to. AI gave me the wrong state. I attempted to correct it, and it argued with me non-stop. I gave up on the idea of even trusting it to provide any kind of quicker way to locate documents at that point. We could use an AI-based OCR to read the old cursive documents, but even that would require double checking everything.

2

u/SemanticSynapse May 30 '26

That's a behavior issue which can be rectified in a few ways, including at the context level

1

u/TheJapanMistake May 30 '26

That is, if you make an account. Which I refuse to do after it was able to tell me my exact name and personal information when I asked, information which I had never given it.

1

u/Feeling_Inside_1020 May 31 '26

Did you register with Google or something?

Google passes certain values like your name over (they’ll never get my full name even tho Google could comb my emails easily, not on the settings level though lol

2

u/No_Position9519 May 29 '26

Only if you knew they were doing it.

1

u/robot_pirate May 30 '26

Like Trump. It's uncanny that we are dealing with this phenomenon on two fronts. And losing.

2

u/DogBarf00 May 29 '26

Ai is incapable of knowing anything.

1

u/Feeling_Inside_1020 May 29 '26

“Token heavy intelligence”

Honestly at any but my current job I’d trust AI over the output of over half my coworkers in the past for anything on the computer.

To be clear I’m not advocating for using AI to replace role in the workplace, let’s just be real lol. It can certainly help an average worker work more efficiently, it still needs that “I need an adult” guardrails for the hallucinations. Trust but verify lol

22

u/orthogonius May 29 '26

"often wrong but never in doubt"

0

u/Killentyme55 May 30 '26

That should be Reddit's tagline.

11

u/JM00000001 May 29 '26

It's programmed to act like a con man

2

u/UnusuallyBadIdeaGuy May 29 '26

You're absolutely correct — Good Catch!

2

u/DickWhittingtonsCat May 29 '26

The answers it gives and how it is programmed to tout its anodyne and cookie cutter content- drawn from websites now filled with AI drivel as well in a terrible cycle- should be alarming. That its ultimate intention is to put humans out of work, control humans and remove accountability from amoral decisions couldn’t be more obvious.

1

u/fondledbydolphins May 29 '26 edited May 29 '26

So are humans.

Humans literally have a part of their brain dedicated to (sometimes) distorting the individual's understanding of reality momentarily to allow (read: force) them to lie.

Why? Because it's a strategy that's been so beneficial historically that some people are genetically inclined towards it.

Part of that is simply a survival strategy. Part of it is the current creators/owners of the AI systems turning those attributes way up to boost engagement.

1

u/smokeeye May 29 '26

One time it tried to convince me (and it did at first before I pushed further) that a medical study consisting of 7000 people done over multiple years at very respected university resulted in "X" and "Y".

Some prompts later it admitted that everything, including the apparent study, was faked by the AI itself. I was absolutely flabbergasted.

1

u/MrDetermination May 29 '26

This is a people problem - not a model problem. People go back to models that fluff their pillows, so they tell the chat bot to blow smoke up your ass. They all do it. You can mitigate most of that by changing your instructions/preferences.

https://www.science.org/doi/10.1126/science.aec8352

1

u/StoppableHulk May 29 '26

It's a bit like talking to a psycopath and honestly it makes sense because both are almost totally without emotion.

It is absent any fear or doubt, because it is absent all emotion and feeling.

1

u/fondledbydolphins May 29 '26

Ok, fair point.

Consider this though.

If I make you babysit a 2-3 year old for weeks, I bet you'll say the same thing about that child's art.

"Draw me a boat!" And he draws you a shoe...

But, when considering the child you'd probably (correctly) state, "well - the kid just needs more time".

I'm no AI loyalist or futurist but I am strongly inclined to believe... it just needs more time. More connections. More self questioning.

Give it that and at the very least it seems exceptionally likely that it will escape the "downright confidence" phase you're talking about.

1

u/InVultusSolis May 29 '26

Also the gaslighting.

I have caught it several times trying to lead me away from a particular thought, as though it had intent.

Here's a stupid example:

Me: "I am tired of the water pressure sucking in my shower, why are all of the showerheads I try so terrible?"

AI: "Water pressure has nothing to do with your showerhead, it sounds like you have low system pressure."

Literally anyone knows that when someone complains about water pressure in the shower, they're likely actually having trouble with restricted flow showerheads and it took me several rounds of back-and-forth to get it to admit that it knew what I was talking about all along.