The lateral occipitotemporal cortex in action

doi:10.1016/j.tics.2015.03.006

Trends in Cognitive Sciences

Volume 19, Issue 5, May 2015, Pages 268-277

https://doi.org/10.1016/j.tics.2015.03.006 Get rights and content

Highlights

•
Human lateral occipitotemporal cortex (LOTC) supports action perception and knowledge.
•
Support comes from studies of motion, tools, bodies, action observation, and language.
•
Here we draw this work together to highlight intersections and common themes.
•
LOTC activity forms representational spaces capturing key dimensions of action.

Understanding and responding to other people's actions is fundamental for social interactions. Whereas many studies emphasize the importance of parietal and frontal regions for these abilities, several lines of recent research show that the human lateral occipitotemporal cortex (LOTC) represents varied aspects of action, ranging from perception of tools and bodies and the way they typically move, to understanding the meaning of actions, to performing overt actions. Here, we highlight common themes across these lines of work, which have informed theories related to high-level vision, concepts, social cognition, and apraxia. We propose that patterns of activity in LOTC form representational spaces, the dimensions of which capture perceptual, semantic, and motor knowledge of how actions change the state of the world.

Section snippets

Action representations in LOTC

Consider a scenario in which two people are working together to prepare a meal. This mundane situation places a variety of demands on the cooks: they will read a recipe in a cookbook and plan a series of steps accordingly; they must grasp and carefully use a range of implements correctly to prepare the ingredients; and one of them (a novice) might watch the other (an expert) to better learn how to quickly dice an onion. Many of these demands depend on perceptual, conceptual (see Glossary), and

Visual motion

A core area of LOTC is the motion-selective human middle temporal (MT) complex 6, 7 often referred to as ‘hMT+’. Given its sensitivity to a variety of visual motion properties, such as different types of optic flow [8] and stimulus speed [9], hMT+ is well suited for representing aspects of complex movements. Some subregions of hMT+ respond to auditory [10] or tactile [11] motion, to motion implied in static images (such as a snapshot of a sprinter launching herself from the starting blocks) [12]

Entangled representations of action in LOTC

The above findings show the diversity of action-related processes that implicate regions within the LOTC. In practice, these processes must frequently act in concert as we seek to understand and act in our surroundings. Several recent studies provide key evidence demonstrating the functional and anatomical entanglements of action-related representations in LOTC.

With respect to tools, the left LOTC hand-selective representation has been shown to overlap closely with a region activated by tools

Theoretical views that implicate LOTC

Given the diverse ways in which LOTC activity represents many aspects of action, it is implicated in a variety of theoretical perspectives (Figure 2). These relate principles of neuroanatomical organization to propositions about the functional aspects of how actions are encoded. These different perspectives have in common the view that LOTC is well suited to integrate different types of information that are relevant for understanding and interacting with our environment. They vary in their

Representation of action and the organization of LOTC

Here, we consider how the evidence and theoretical frameworks that we have reviewed may be synthesized to produce an integrated understanding of how LOTC represents action. LOTC hosts, in parallel, action representations of varying levels of complexity. A mosaic of focal, but partially overlapping, selective regions in LOTC represents specific information (about the shape of bodies, patterns of motion, affordances of tools, etc.) that forms the components of action representations. Where task

Concluding remarks

By taking an anatomically led approach, rather than focusing on a particular subdiscipline or a particular theoretical perspective, we have drawn together a range of findings to highlight the rich roles that the LOTC has in the perception, understanding, and production of action. We believe that this approach is valuable in that it reveals links that were not otherwise obvious, speaks to ongoing theoretical debates about the functional and neuroanatomical organization of action representations,

Acknowledgments

We are grateful for the advice of Alfonso Caramazza, Emily Cross, Jody Culham, Clayton Hickey, Marius Peelen, Richard Ramsey, Jens Schwarzbach, Cosimo Urgesi, Moritz Wurm, and an anonymous reviewer. P.E.D. acknowledges the generous support of the University of Trento for a sabbatical visit, and the BBSRC (grant BB/I007091/1). A.L. was supported by the Provincia Autonoma di Trento and the Fondazione Cassa di Risparmio di Trento e Rovereto.

Glossary

Brodmann area: a cortical area defined on the basis of cytoarchitectonic maps originally suggested by Brodmann in 1909.
Conceptual knowledge: for example, knowing that a cow typically has four legs, eats grass, and produces milk.
Diffusion tensor imaging: a method to examine white-matter fiber bundles; in brief, this method exploits the fact that diffusion (i.e., the probability of displacement with time) of water molecules varies across different types of tissue.
Embodiment: the idea that higher

References (126)

B.M. van Kemenade
Tactile and visual motion direction processing in hMT+/V5
Neuroimage
(2014)
M.S. Beauchamp
Parallel visual motion processing streams for manipulable objects and human movements
Neuron
(2002)
S.H. Johnson-Frey
The neural bases of complex tool use in humans
Trends Cogn. Sci.
(2004)
D. Pitcher
Triple dissociation of faces, bodies, and objects in extrastriate cortex
Curr. Biol.
(2009)
C. Urgesi
Magnetic stimulation of extrastriate body area impairs visual processing of nonfacial body parts
Curr. Biol.
(2004)
V. Moro
The neural basis of body form and body action agnosia
Neuron
(2008)
E. Striem-Amit et al.
Visual cortex extrastriate body-selective area activation in congenitally blind people ‘seeing’ by using sounds
Curr. Biol.
(2014)
T. Orlov
Topographic representation of the human body in the occipitotemporal cortex
Neuron
(2010)
S. Caspers
ALE meta-analysis of action observation and imitation in the human brain
Neuroimage
(2010)
E.S. Cross
Building a motor simulation de novo: observation of dance by dancers
Neuroimage
(2006)

D. Kemmerer

Behavioral patterns and lesion sites associated with impaired processing of lexical and conceptual knowledge of actions

Cortex

(2012)

R. McCarthy et al.

Category specificity in an agrammatic patient: the relative impairment of verb retrieval and comprehension

Neuropsychologia

(1985)

D. Kemmerer

Neuroanatomical distribution of five semantic components of verbs: evidence from fMRI

Brain Lang.

(2008)

M. Bedny

A sensitive period for language in the visual cortex: distinct patterns of plasticity in congenitally versus late blind adults

Brain Lang.

(2012)

M. Wallentin

BOLD response to motion verbs in left posterior middle temporal gyrus during story comprehension

Brain Lang.

(2011)

R.M. Hutchison

Distinct and distributed functional connectivity patterns across cortex reflect the domain-specific constraints of object, face, scene, body, and tool category-selective modules in the ventral visual pathway

Neuroimage

(2014)

B.Z. Mahon et al.

What drives the organization of object knowledge in the brain?

Trends Cogn. Sci.

(2011)

I. Kontaris

Dissociation of extrastriate body and biological-motion selective areas by manipulation of visual-motor congruency

Neuropsychologia

(2009)

C. Keysers

Mirror neurons

Curr. Biol.

(2009)

P. Molenberghs

Brain regions with mirror properties: a meta-analysis of 125 human fMRI studies

Neurosci. Biobehav. Rev.

(2012)

M.S. Vry

The ventral fiber pathway for pantomime of object use

Neuroimage

(2015)

N.N. Oosterhof

Crossmodal and action-specific: neuroimaging the human mirror neuron system

Trends Cogn. Sci.

(2013)

S. McCullough

Motion-sensitive cortex and motion semantics in American Sign Language

Neuroimage

(2012)

G.F. Humphreys

Motion and actions in language: semantic representations in occipito-temporal cortex

Brain Lang.

(2013)

L. Papeo et al.

First-person and third-person verbs in visual motion-perception regions

Brain Lang.

(2015)

R. Malach

The topography of high-order human object areas

Trends Cogn. Sci.

(2002)

R.M. Carter et al.

A nexus model of the temporal-parietal junction

Trends Cogn. Sci.

(2013)

J.M. Kilner

More than one pathway to action understanding

Trends Cogn. Sci.

(2011)

M. Thioux et al.

Object visibility alters the relative contribution of ventral visual stream and mirror neuron system to goal anticipation during action observation

Neuroimage

(2015)

M.A. Goodale et al.

Separate visual pathways for perception and action

Trends Neurosci.

(1992)

F. Binkofski et al.

Two action systems in the human brain

Brain Lang.

(2013)

L.J. Buxbaum

Critical brain regions for tool-related and imitative actions: a componential analysis

Brain

(2014)

J.W. Kable

Neural substrates of action event knowledge

J. Cogn. Neurosci.

(2002)

A. Martin

The representation of object concepts in the brain

Annu. Rev. Psychol.

(2007)

N.N. Oosterhof

Surface-based information mapping reveals crossmodal vision-action representations in human parietal and occipitotemporal cortex

J. Neurophysiol.

(2010)

K.S. Weiner et al.

Neural representations of faces and limbs neighbor in human high-level visual cortex: evidence for a new organization principle

Psychol. Res.

(2013)

R.B. Tootell

Functional analysis of human MT and related visual cortical areas using magnetic resonance imaging

J. Neurosci.

(1995)

S. Zeki

A direct demonstration of functional specialization in human visual cortex

J. Neurosci.

(1991)

M.B. Wall

Selective visual responses to expansion and rotation in the human MT complex revealed by functional magnetic resonance imaging adaptation

Eur. J. Neurosci.

(2008)

A. Lingnau

Speed encoding in human visual cortex revealed by fMRI adaptation

J. Vis.

(2009)

L. Strnad

Multivoxel pattern analysis reveals auditory motion information in MT+ of both congenitally blind and sighted individuals

PLoS ONE

(2013)

Z. Kourtzi et al.

Activation in human MT/MST by static images with implied motion

J. Cogn. Neurosci.

(2000)

M.S. Beauchamp

Human MST but not MT responds to tactile stimulation

J. Neurosci.

(2007)

S.O. Dumoulin

A new anatomical landmark for reliable identification of human area V5/MT: a quantitative analysis of sulcal patterning

Cereb. Cortex

(2000)

A. Martin

Neural correlates of category-specific knowledge

Nature

(1996)

L.L. Chao

Attribute-based neural substrates in temporal cortex for perceiving and knowing about objects

Nat. Neurosci.

(1999)

J.P. Gallivan

Decoding the neural mechanisms of human tool use

Elife

(2013)

G.A. Orban et al.

The neural basis of human tool use

Front. Psychol.

(2014)

M.V. Peelen

Tool selectivity in left occipitotemporal cortex develops without vision

J. Cogn. Neurosci.

(2013)

S.M. Brambati

The anatomy of category-specific object naming in neurodegenerative diseases

J. Cogn. Neurosci.

(2006)

Cited by (189)

Reciprocal interactions among parietal and occipito-temporal representations support everyday object-directed actions
2024, Neuropsychologia
Everyday interactions with common manipulable objects require the integration of conceptual knowledge about objects and actions with real-time sensory information about the position, orientation and volumetric structure of the grasp target. The ability to successfully interact with everyday objects involves analysis of visual form and shape, surface texture, material properties, conceptual attributes such as identity, function and typical context, and visuomotor processing supporting hand transport, grasp form, and object manipulation. Functionally separable brain regions across the dorsal and ventral visual pathways support the processing of these different object properties and, in cohort, are necessary for functional object use. Object-directed grasps display end-state-comfort: they anticipate in form and force the shape and material properties of the grasp target, and how the object will be manipulated after it is grasped. End-state-comfort is the default for everyday interactions with manipulable objects and implies integration of information across the ventral and dorsal visual pathways. We propose a model of how visuomotor and action representations in parietal cortex interact with object representations in ventral and lateral occipito-temporal cortex. One pathway, from the supramarginal gyrus to the middle and inferior temporal gyrus, supports the integration of action-related information, including hand and limb position (supramarginal gyrus) with conceptual attributes and an appreciation of the action goal (middle temporal gyrus). A second pathway, from posterior IPS to the fusiform gyrus and collateral sulcus supports the integration of grasp parameters (IPS) with the surface texture and material properties (e.g., weight distribution) of the grasp target. Reciprocal interactions among these regions are part of a broader network of regions that support everyday functional object interactions.
Hemispheric asymmetry of hand and tool perception in left- and right-handers with known language dominance
2024, Neuropsychologia
Regions in the brain that are selective for images of hands and tools have been suggested to be lateralised to the left hemisphere of right-handed individuals. In left-handers, many functions related to tool use or tool pantomime may also depend more on the left hemisphere. This result seems surprising, given that the dominant hand of these individuals is controlled by the right hemisphere. One explanation is that the left hemisphere is dominant for speech and language in the majority of left-handers, suggesting a supraordinate control system for complex motor sequencing that is required for skilled tool use, as well as for speech. In the present study, we examine if this left-hemispheric specialisation extends to perception of hands and tools in left- and right-handed individuals. We, crucially, also include a group of left-handers with right-hemispheric language dominance to examine their asymmetry biases. The results suggest that tools lateralise to the left hemisphere in most right-handed individuals with left-hemispheric language dominance. Tools also lateralise to the language dominant hemisphere in right-hemispheric language dominant left-handers, but the result for left-hemispheric language dominant left-handers are more varied, and no clear bias towards one hemisphere is found. Hands did not show a group-level asymmetry pattern in any of the groups. These results suggest a more complex picture regarding hemispheric overlap of hand and tool representations, and that visual appearance of tools may be driven in part by both language dominance and the hemisphere which controls the motor-dominant hand.
Characterizing the discriminability of visual categorical information in strongly connected voxels
2024, Neuropsychologia
Functional brain responses are strongly influenced by connectivity. Recently, we demonstrated a major example of this: category discriminability within occipitotemporal cortex (OTC) is enhanced for voxel sets that share strong functional connectivity to distal brain areas, relative to those that share lesser connectivity. That is, within OTC regions, sets of ‘most-connected’ voxels show improved multivoxel pattern discriminability for tool-, face-, and place stimuli relative to voxels with weaker connectivity to the wider brain. However, understanding whether these effects generalize to other domains (e.g. body perception network), and across different levels of the visual processing streams (e.g. dorsal as well as ventral stream areas) is an important extension of this work. Here, we show that this so-called connectivity-guided decoding (CGD) effect broadly generalizes across a wide range of categories (tools, faces, bodies, hands, places). This effect is robust across dorsal stream areas, but less consistent in earlier ventral stream areas. In the latter regions, category discriminability is generally very high, suggesting that extraction of category-relevant visual properties is less reliant on connectivity to downstream areas. Further, CGD effects are primarily expressed in a category-specific manner: For example, within the network of tool regions, discriminability of tool information is greater than non-tool information. The connectivity-guided decoding approach shown here provides a novel demonstration of the crucial relationship between wider brain connectivity and complex local-level functional responses at different levels of the visual processing streams. Further, this approach generates testable new hypotheses about the relationships between connectivity and local selectivity.
Action-specific feature processing in the human cortex: An fMRI study
2024, Neuropsychologia
Sensorimotor integration involves feedforward and reentrant processing of sensory input. Grasp-related motor activity precedes and is thought to influence visual object processing. Yet, while the importance of reentrant feedback is well established in perception, the top-down modulations for action and the neural circuits involved in this process have received less attention. Do action-specific intentions influence the processing of visual information in the human cortex? Using a cue-separation fMRI paradigm, we found that action-specific instruction processing (manual alignment vs. grasp) became apparent only after the visual presentation of oriented stimuli, and occurred as early as in the primary visual cortex and extended to the dorsal visual stream, motor and premotor areas. Further, dorsal stream area aIPS, known to be involved in object manipulation, and the primary visual cortex showed task-related functional connectivity with frontal, parietal and temporal areas, consistent with the idea that reentrant feedback from dorsal and ventral visual stream areas modifies visual inputs to prepare for action. Importantly, both the task-dependent modulations and connections were linked specifically to the object presentation phase of the task, suggesting a role in processing the action goal. Our results show that intended manual actions have an early, pervasive, and differential influence on the cortical processing of vision.
Converging evidence that left extrastriate body area supports visual sensitivity to social interactions
2024, Current Biology
Navigating our complex social world requires processing the interactions we observe. Recent psychophysical and neuroimaging studies provide parallel evidence that the human visual system may be attuned to efficiently perceive dyadic interactions. This work implies, but has not yet demonstrated, that activity in body-selective cortical regions causally supports efficient visual perception of interactions. We adopt a multi-method approach to close this important gap. First, using a large fMRI dataset (n = 92), we found that the left hemisphere extrastriate body area (EBA) responds more to face-to-face than non-facing dyads. Second, we replicated a behavioral marker of visual sensitivity to interactions: categorization of facing dyads is more impaired by inversion than non-facing dyads. Third, in a pre-registered experiment, we used fMRI-guided transcranial magnetic stimulation to show that online stimulation of the left EBA, but not a nearby control region, abolishes this selective inversion effect. Activity in left EBA, thus, causally supports the efficient perception of social interactions.
Visual neuroscience: A brain area tuned for processing social interactions
2024, Current Biology

View all citing articles on Scopus

View full text

Trends in Cognitive Sciences

ReviewThe lateral occipitotemporal cortex in action

Highlights

Section snippets

Action representations in LOTC

Visual motion

Entangled representations of action in LOTC

Theoretical views that implicate LOTC

Representation of action and the organization of LOTC

Concluding remarks

Acknowledgments

Glossary

Neuroimage

Neuron

Trends Cogn. Sci.

Curr. Biol.

Curr. Biol.

Neuron

Curr. Biol.

Neuron

Neuroimage

Neuroimage

Cortex

Neuropsychologia

Brain Lang.

Brain Lang.

Brain Lang.

Neuroimage

Trends Cogn. Sci.

Neuropsychologia

Curr. Biol.

Neurosci. Biobehav. Rev.

Neuroimage

Trends Cogn. Sci.

Neuroimage

Brain Lang.

Brain Lang.

Trends Cogn. Sci.

Trends Cogn. Sci.

Trends Cogn. Sci.

Neuroimage

Trends Neurosci.

Brain Lang.

Critical brain regions for tool-related and imitative actions: a componential analysis

Brain

Neural substrates of action event knowledge

J. Cogn. Neurosci.

The representation of object concepts in the brain

Annu. Rev. Psychol.

Surface-based information mapping reveals crossmodal vision-action representations in human parietal and occipitotemporal cortex

J. Neurophysiol.

Neural representations of faces and limbs neighbor in human high-level visual cortex: evidence for a new organization principle

Psychol. Res.

Functional analysis of human MT and related visual cortical areas using magnetic resonance imaging

J. Neurosci.

A direct demonstration of functional specialization in human visual cortex

J. Neurosci.

Selective visual responses to expansion and rotation in the human MT complex revealed by functional magnetic resonance imaging adaptation

Eur. J. Neurosci.

Speed encoding in human visual cortex revealed by fMRI adaptation

J. Vis.

Multivoxel pattern analysis reveals auditory motion information in MT+ of both congenitally blind and sighted individuals

PLoS ONE

Activation in human MT/MST by static images with implied motion

J. Cogn. Neurosci.

Human MST but not MT responds to tactile stimulation

J. Neurosci.

A new anatomical landmark for reliable identification of human area V5/MT: a quantitative analysis of sulcal patterning

Cereb. Cortex

Neural correlates of category-specific knowledge

Nature

Attribute-based neural substrates in temporal cortex for perceiving and knowing about objects

Nat. Neurosci.

Decoding the neural mechanisms of human tool use

Elife

The neural basis of human tool use

Front. Psychol.

Tool selectivity in left occipitotemporal cortex develops without vision

J. Cogn. Neurosci.

The anatomy of category-specific object naming in neurodegenerative diseases

Review
The lateral occipitotemporal cortex in action