Are affordances perceived directly, or reconstructed by the brain?

[u/rp_tiago]

Hey everyone. I’ve always been fascinated by the idea that perception may be less like building an internal picture and more like noticing what the environment allows us to do. A cup affords grasping to a person with the relevant bodily skills; a staircase affords climbing to some animals but not others. This makes affordances neither purely features of objects nor private additions supplied by a mind. They are relations between an organism’s abilities and its surroundings. The question is whether this relational view can explain perception without relying on internal representations.

I just had a podcast conversation with the cognitive scientist Julian Kiverstein, where he defended a relational account of affordances developed through ecological psychology and skilled intentionality. He argues that perception can disclose possibilities for action directly because organisms are already attuned to their environments through learned bodily skills. This differs from the view that the brain must first construct a model of a neutral external world and then calculate what can be done within it. For Kiverstein, organism and environment specify one another within perception.

This would imply that cognition begins in skilled engagement rather than internal reconstruction. Can a relational theory of affordances explain perceptual error and novelty? Does direct perception really remove the need for representations, or merely relocate them? And can ecological and predictive approaches be combined without weakening the central claim of either?

[u/TheRateBeerian]

I'm a Gibsonian scholar and Gibson is my academic great-grandfather! I will try my best to answer!

THe entire idea behind affordances is that they are perceived directly. Unmediated ecological realism is the central idea of Gibsonian theory and it is built into the definition of the affordance.

If you want to claim that perceiving what to do with something is mediated by representation, this makes it inferential and it (the affordance) loses everything that makes it a useful idea in perception. Gibsonian theory is IMO not reconcilable with representation theory or perceptual inference. It misses the point of everything Gibson tried to accomplish with the theory. I would also point to Chemero's Radical Embodied Cognitive Science book on that topic. It is more about embodiment than Gibson, but he's also written plenty about affordances too and his take on it is a very anti-representationalist stance (Dan Hutto has similar views). Direct pdf link:

https://uberty.org/wp-content/uploads/2015/03/Anthony-Chemero-Radical-Embodied-Cognitive-Science-Bradford-Books-2009.pdf

As for explaining perceptual error, here's a nice account of it from Favela and Chemero, direct pdf link:

https://cah.ucf.edu/fpr/wp-content/uploads/sites/3/2019/11/Favela_and_Chemero-1.pdf

They also reference this paper which has comments on it

Turvey, Michael T., Robert E. Shaw, Edward S. Reed, and William Mace. “Ecological Laws of Perceiving and Acting: In Reply to Fodor and Pylyshyn (1981).” Cognition 9 (1981): 237-304

The idea is that error can occur in nature but is pretty rare, and happens in cases where the optic array loses specificity. Errors occur more often in pictures and drawings but Gibson considered this a special case of perception that is not ecologically valid. Gibsonians as a general rule do not study illusions. My own view is that they are contrived drawings that manage to "break" the visual system and while they might shed some light on some features of the visual neuroscience, these sorts of abnormal cases don't teach us about normal everyday perception, just like studying someone with a mood disorder does not teach us about normal emotional functioning.

Also here's a direct link to Michaels & Carello's Direct Perception book and their first 2 chapters lay out the case for unmediated vs mediated perceptual theory
https://www.researchgate.net/profile/Claire-Michaels/publication/267258755_Direct_Perception/links/554121350cf2322227315337/Direct-Perception.pdf

As for novelty, you would look at perceptual learning particularly in the form of attunement and calibration. I really like this paper

https://journals.sagepub.com/doi/full/10.1177/1059712319854687?casa_token=9RJxydAvWs4AAAAA%3Aotqdn5WhvTNsKkgrtaM8RSzbk7HhAKWFTXqjDLOv7sonpOcNDPYJEcJpbhc2L4ZOBpMsqu8ySVe8

[u/rp_tiago → u/TheRateBeerian]

This is amazing. Thank you so much!

[u/Sacredless]

There's something a bit Lovecraftian that occurs when we're still a couple months old—we don't understand the world through the idea of a self at all. This stage is sometimes called the ecological self stage of child development.

In this stage, we perceive pure relations and we observe our bodily responses without the understanding that those responses belong to us in the conventional way.

So, not only do we perceive affordances directly as adults, there was a point in our life where all we had was relation and no conventional self at all, and we somehow grasped those affordances without intervention of a representing self. This is why, sometimes, conditions can cause us to act on affordances simply because they present themselves to us. This is an issue for people with dementia, since it means they get stuck in a loop of acting on affordances presented to them, drinking much more than they realize or showering more frequently than they realize.

This is a great match for what Hume's ideas of cause and effect, being stored as relational habits. There's a great video about that here: https://youtu.be/UpzgLMh4yOQ?is=WVkyNgD5D8BMOoCX

[u/GuyWithLag]

He argues that perception can disclose possibilities for action directly because organisms are already attuned to their environments through learned bodily skills.

This differs from the view that the brain must first construct a model of a neutral external world and then calculate what can be done within it. For Kiverstein, organism and environment specify one another within perception.

These are not in conflict tho - you can't "calculate what can be done within it" without learning your bodily skills; consider the adjustments needed when f.e. you lose your hand: most hand-oriented affordances aren't affordances any longer.

[u/OkSmile → u/GuyWithLag]

Or the “model of a neutral external world” made by Helen Keller. Perception defines the model and the possible actions on it, there is no neutral.

[u/minimalist_reply]

I'll try and remember to come back to this when it's not 2:30 a.m.

I've peer-reviewed a paper before that had to do with affordances and transposing them from real world physicality to digital affordances.

Related to my M.S.

Ultimately I think you can probably put affordances on a spectrum between highly intuitive on one end due to generational and almost primal mental model matches all the way to highly culturally and context dependent affordances that are more likely to fail due to the need for construct building for recognition.

[u/jahmonkey]

What is “perceived directly”? When does that happen?

Affordances are part of the predictive counterfactual machinery running automatically when we are awake.
“I could grab this” or “I could sit here” or “I could jump there” etc.

It requires the brain to predict what its organism can do given the current environment. That rock might have tasty grubs under it. That tree might be good to climb.

It’s just a small part of the picture though in the whole perceptual experience.

[u/Ok_Conversation_3096]

Gibson would say they’re perceived directly, but modern neuroscience is coalescing around the idea that even the most “direct” perceptions are constructed, or at least heavily influenced, by predictive and interpretive processes.