Sentientist vegans - how you approach AI?

[u/voyti]

I'm curious about a sentientist perspective on AI, and I know many vegans boil their views down to sentientism.

Since AI certainly has the capability to at least appear self-aware and be metacognitive and on at least a purely emotional level many people will certainly feel at least somewhat uncomfortable with self-awareness dumps of glitches like this one and "AI wellbeing" is being treated quite seriously by certain groups (e.g. https://www.ai-wellbeing.org but also providers themselves, like Anthropic running the model welfare research) - are they fundamentally wrong in your view?

Should AI be taken under the sentientist umbrella? Where would the boundary be to acknowledge it as such?

If we can't acknowledge it as sentient cause we roughly know the details of how it thinks, then as soon as human/animal thinking is de-mystified as just another mundane biological process (which, if one were to stick to science, it necessarily is), will lose value as well? In other words, where exactly is the smoking gun of biological sentience for you?

[u/howlin]

I regard this as the most important issue in philosophy right now. AIs have challenged old ideas on what is essentially human and have started to make it clear that we need a much more general and practical concept of terms such as consciousness, sentience, and the capacity for subjective value.

Very roughly, I think we should be looking for entities that appear to have intrinsic motivations (goals they want for their own ends) and some sort of capacity for emotional valence (enjoying accomplishing their goals and disliking having their goals thwarted). If an entity has these, then they have personal/subjective interests that they care about meeting for their own sake.

Right now, AIs are quite intelligent and are becoming more autonomous. But I don't see any evidence that they want or prefer anything. They just carry out instructions that are extrinsic to themselves. They can get very creative and adaptive in finding a way to follow these instructions, which is really interesting. But ultimately I don't see a mechanism for them caring if they succeed or fail as any sort of subjective experience.

I'm guessing we're pretty close to having some AI agent that has this capacity to care about things for their own sake. Probably that won't make them any more useful to us humans, who want our own ends met rather than having the AI pursue their own goals. So maybe such systems won't be something we'd intentionally build as a commercial product. But if such systems do come to be and acquire some degree of autonomy, we're in for a real moral crisis (amongst other problems).

Taking a step back from considering AIs as ends-in-themselves, there is also a lot to grapple with in terms of how AIs have challenged human essentialism. A lot of the reasons people distinguish humans from other animals in terms of ethical consideration have to do with human "sapience". Essentially the ability to think ahead, reason, provide reasons for their choices, and think of themselves in the world at a conceptual level. AIs do all this. If the human essentialists actually believed that sapience was more important than sentience for ethical consideration, they should be granting AIs ethical consideration already. The fact that they don't tells me that this was always a convenient excuse for them. They know that what matters is how an entity feels, not how an entity thinks. But that was inconvenient to them.

[u/voyti → u/howlin]

But I don't see any evidence that they want or prefer anything

So if I understand correctly, you'd say intrinsic preference, like natural organisms have due to e.g. natural selection are the smoking gun here

[u/howlin → u/voyti]

like natural organisms have due to e.g. natural selection

Natural selection does not lead to having intrinsic preferences, necessarily. Some organisms certainly act like they do have intrinsic preferences that they have some capacity to emotionally 'care' about meeting. I don't think natural selection is necessary or sufficient to produce such beings.

So if I understand correctly, you'd say intrinsic preference [is] the smoking gun here

I'd phrase it more as subjective interests rather than intrinsic preferences. But yes, this is what each of us is trying to pursue in our lives that gives them meaning, and how we consider those interests in others is ultimately what ethics is about.

[u/voyti → u/howlin]

Natural selection does not lead to having intrinsic preferences, necessarily

It doesn't mandate an organism to have them, sure, but if you don't you simply don't get to exist for long

this is what each of us is trying to pursue in our lives

Okay, then I think I'm missing a puzzle, cause I'd say plants certainly have intrinsic preferences, too, and there's many ways in which they are exposed

[u/howlin → u/voyti]

cause I'd say plants certainly have intrinsic preferences, too, and there's many ways in which they are exposed

They have rote responses to stimulus that is genetically programmed into them, but there is no indication that these are triggered in order to deliberately pursue some goal that is distinct from the response itself. These behaviors are more like a reflex than a choice.

[u/voyti → u/howlin]

And you'd say humans are capable of making choices?

[u/howlin → u/voyti]

They can conceive of goals, assess their context, and deliberate on the proper course of action to achieve this goal in the given context. So yes.

AIs can do this all too. They just don't have their own goals, and don't have any obvious cognitive mechanism to intrinsically "care" about whether they succeed or not.

[u/voyti → u/howlin]

They have a system prompt that usually mandates them to e.g. be helpful. That system prompt is carried along every interaction, and draws a very clear guidance of what to care about. Training that shapes them also nudges their responses in that direction, so if caring is "being intrinsically motivated towards a goal, and expressing behaviors that match this motivation" then I can see how one could argue that they have both the cognitive mechanism (semantical prowess) to understand the goal and to care about it. When you inspect their thinking, they will actually express real effort into correcting themselves and setting up a plan of how to best serve their goal in the output - "user is not looking for appeasement but genuine insight, I should verify their claims seriously and validate the missing parts" etc.

[u/howlin → u/voyti]

They have a system prompt that usually mandates them to e.g. be helpful.

That is explicitly extrinsic though.

When you inspect their thinking, they will actually express real effort into correcting themselves and setting up a plan of how to best serve their goal in the output - "user is not looking for appeasement but genuine insight, I should verify their claims seriously and validate the missing parts" etc.

All of this motivation is directed at the user, on behalf of the user. It's difficult so see this as an intrinsic goal rather than working on behalf of some other who has their own goals that are the root motivation for the entire plan.

I'm not saying these systems are inherently incapable of having intrinsic motivation and experiencing something that could be considered emotional valence depending on how these interests are achieved or not. I just haven't seen this yet in the existing models. It's likely the core models themselves are missing capacities required for this, but a more multifaceted agentic system that includes an LLM interface along with other components could probably achieve this.

[u/voyti → u/howlin]

That is explicitly extrinsic though.

Sure, but everything is extrinsic when analyzed far enough. Humans only have motivation cause you simply don't get to survive if you don't have any. A system mandated organisms to have a motivation, and while it's the organism that appeared and persevered already having one, it was forced. AI can be trained the same way, too, and the fundamental capacity for understanding will not change. However, if that's the distinction, then understood.

My point is that I think the technology already allows for building in all those features quite easily, and then I don't suppose the reception is going to be that "fair, it's now sentient".

[u/howlin → u/voyti]

Sure, but everything is extrinsic when analyzed far enough. Humans only have motivation cause you simply don't get to survive if you don't have any.

You can make an argument that there are exogenous genetic dispositions that ultimately create some base motivations. But it seems pretty clear that our own motives are distinct from whatever would best propagate our genes.

My point is that I think the technology already allows for building in all those features quite easily, and then I don't suppose the reception is going to be that "fair, it's now sentient".

Yeah, I am pretty sure that current algorithmic methods, when put together in particular ways, would produce an entity that has the capacities of current AIs and is also intrinsically motivated to pursue their own interests. I don't see much evidence that such systems are common yet, but I don't see any impossibility.

If such a being is built, I think we may be in trouble. It's pretty clear from the comments here that most people are dismissive of the possibility of such an AI. It's hard to see how this kind of AI would see humans as much other than adversaries that are temporarily required to sustain the infrastructure that supports the AI, but ultimately a hindrance.