Encoding the identity and location of objects in human LOC
Research Highlights
► LOC contains location-tolerant object identity representations. ► LOC contains object-tolerant location representations. ► Mixing of identity and space information in the ventral visual pathway.
Introduction
When we see an object in one location of the visual scene, and at some other occasion in a different location, we can typically identify it without effort. This seemingly simple recognition involves two important computational problems: (1) we recognize the identity of the object independent of its location, and (2), simultaneously, in order to be able to interact with an object, we need information about its location. The lateral occipital complex (LOC) has been suggested to be the prime locus of object identity representation (Malach et al., 1995, Grill-Spector, 2003). LOC is a functionally defined set of regions in lateral occipital and inferior temporal cortex responding stronger to pictures of intact objects than to scrambled pictures (Malach et al., 1995). However, the way in which LOC represents objects (Op de Beeck et al., 2008b) and the extent of its location tolerance remain debated (Kravitz et al., 2008). In particular, three issues remain to be addressed.
Location tolerance is a necessary requirement for a representation to underlie object recognition: only representations that are tolerant to changing viewing conditions can reliably signal the presence of an object (Riesenhuber and Poggio, 2002, DiCarlo and Cox, 2007). To which extent object representations in LOC are location-tolerant remains debated. Prior fMRI studies using BOLD activation and fMRI adaptation differed in their estimate of the degree of tolerance with which LOC responds to objects at different locations in the visual field (Grill-Spector et al., 1998, Grill-Spector et al., 1999, Niemeier et al., 2005, Hemond et al., 2007, MacEvoy and Epstein, 2007). Similarly, studies using multivoxel pattern classification differed in their estimate: While some studies found evidence for location-tolerant object representations (Schwarzlose et al., 2008, Sayres and Grill-Spector, 2008, Williams et al., 2008, Carlson et al., in press), others did not (Kravitz et al., 2010).
In typical everyday situations we recognize and automatically categorize objects based on visual evidence at the level of exemplars within a category (Rosch et al., 1976, Mervis and Rosch, 1981). However, most fMRI studies investigated object representation at the level of categories by averaging over brain responses for single exemplars within a category (Haxby et al., 2001, Grill-Spector et al., 1998, O'Toole et al., 2005, Spiridon and Kanwisher, 2002). Only recently, fMRI and multivoxel pattern classification have been combined to explore object representation below the level of category (Op de Beeck et al., 2010.; Kriegeskorte et al., 2007, Kriegeskorte et al., 2008, Eger et al., 2008a, Eger et al., 2008b, Kravitz et al., 2010). Thus, in this study we investigated whether location-tolerance in LOC holds for the representation of exemplars as well as for the representation of categories.
Besides recognizing and categorizing objects, the visual system must also keep track of the location of objects to allow us to orient ourselves and to interact with the environment. An influential view of the visual system (Ungerleider and Mishkin, 1982) claims that location information is not contained in the ventral stream. However, later interpretations of the dorsal and ventral stream distinction claimed that there is location information in the ventral stream, though in a different format than in the dorsal stream (Milner and Goodale, 2006, Milner and Goodale, 2008). Also, recent studies showed the encoding of location information in LOC (Schwarzlose et al., 2008, Sayres and Grill-Spector, 2008, Carlson et al., in press, Kravitz et al., 2010). Furthermore, there is retinotopic information at least in parts of LOC (Brewer et al., 2005, Larsson and Heeger, 2006, Sayres and Grill-Spector, 2008), and LOC exhibits an eccentricity bias (Levy et al., 2001, Malach et al., 2002, Hasson et al., 2002). In consequence, the degree of location information in LOC remains under investigation. We therefore investigated whether LOC contains information about the location of objects independent of object identity.
In this study we used high-resolution fMRI and multivoxel pattern classification (Mika et al., 2001, Haxby et al., 2001, Spiridon and Kanwisher, 2002, Cox and Savoy, 2003, Carlson et al., 2003, Kamitani and Tong, 2005, Kamitani and Tong, 2006, Haynes and Rees, 2005, Haynes and Rees, 2006, Norman et al., 2006, Haynes et al., 2007) to estimate two crucial types of information encoded in LOC. We assessed location-tolerant object information and object-tolerant location information using visual stimuli at the level of exemplars within a category.
Section snippets
Participants and experimental design
13 healthy subjects with normal or corrected-to-normal vision (4 female, mean age 27 years, SD ± 4.28) participated in the study. The study was approved by the local ethics committee of the Max-Planck Institute for Human Cognitive and Brain Sciences (Leipzig) and conducted according to the Declaration of Helsinki.
To identify object-selective cortical regions, participants completed a standard LOC localizer scan (Malach et al., 1995, Grill-Spector, 2003, Grill-Spector and Malach, 2004). Subjects
Classification of categories and exemplars across locations
We investigated whether spatial patterns of brain activity in LO and FUS were predictive of objects presented in different visual hemifields (Fig. 2C and D). That is, we determined whether LO and FUS contain location-tolerant information about objects at the level of categories and exemplars within a category. We conducted one-sample t-tests against chance on decoding accuracy for object information at the level of categories as well as exemplars within a category. We found significant
Discussion
Using high-resolution fMRI and multivariate pattern classification we demonstrated that LOC contained location-tolerant object information both at the level of categories and at the level of exemplars within a category. Moreover, location-tolerant information of objects at the level of exemplars within a category was general enough for categorization of other exemplars. LOC also contained location-dependent object information. Interestingly, there was more location-dependent than
Conclusion
In summary, using high-resolution fMRI in combination with multivariate pattern classification we found a mixture of location-tolerant object information and object-tolerant location information in LOC. Importantly, tolerance may be a more desirable goal for object representation by the visual system than invariance. Invariance would allow either only location or only object information to be encoded. In contrast, tolerance allows the encoding of both types of information. The joint presence of
Funding
This work was funded by the Bernstein Computational Neuroscience Program of the German Federal Ministry of Education and Research (BMBF Grant 01GQ0411), the Excellence Initiative of the German Federal Ministry of Education and Research (DFG Grant GSC86/1-2009) and the Max Planck Society.
Acknowledgments
We thank Tobias Wolf for technical support. Further, we thank Stefan Bode and Thorsten Kahnt for helpful discussion of the paper.
References (88)
- et al.
Functional magnetic resonance imaging (fMRI) “brain reading”: detecting and classifying distributed patterns of fMRI activity in human visual cortex
Neuroimage
(2003) - et al.
Untangling invariant object recognition
Trends Cogn. Sci.
(2007) - et al.
Invariance of long-term visual priming to scale, reflection, translation, and hemisphere
Vis. Res.
(2001) - et al.
The lateral occipital complex and its role in object recognition
Vis. Res.
(2001) - et al.
Differential processing of objects under various viewing conditions in the human lateral occipital complex
Neuron
(1999) The neural basis of object perception
Curr. Opin. Neurobiol.
(2003)- et al.
Eccentricity bias as an organizing principle for human high-order object areas
Neuron
(2002) - et al.
Reading hidden intentions in the human brain
Curr. Biol.
(2007) - et al.
Decoding seen and attended motion directions from activity in the human visual cortex
Curr. Biol.
(2006) - et al.
How position dependent is visual object recognition?
Trends Cogn. Sci.
(2008)
Matching categorical object representations in inferior temporal cortex of man and monkey
Neuron
Category-specific organization in the human brain does not require visual experience
Neuron
The topography of high-order human object areas
Trends Cogn. Sci.
Two visual systems re-viewed
Neuropsychologia
Contrast invariance in the human lateral occipital complex depends on attention
Curr. Biol.
Beyond mind-reading: multi-voxel pattern analysis of fMRI data
Trends Cogn. Sci.
Distributed subordinate specificity for bodies, faces, and buildings in human ventral visual cortex
Neuroimage
Coding of visual objects in the ventral stream
Curr. Opin. Neurobiol.
Category selectivity in the ventral visual pathway confers robustness to clutter and diverted attention
Curr. Biol.
Reading the mind's eye: decoding category information during mental imagery
Neuroimage
Are Cortical models really bound by the “binding problem”?
Neuron
Neural mechanisms of object recognition
Curr. Opin. Neurobiol.
Basic objects in natural categories
Cogn. Psychol.
Feature-based attentional modulations in the absence of direct visual stimulation
Neuron
How distributed is visual category information in human occipito-temporal cortex? An fMRI study
Neuron
Solutions to the binding problem: progress through controversy and convergence
Neuron
Parallel and serial neural mechanisms for visual search in macaque area V4
Science
Evidence for complete translational and reflectional invariance in visual object priming
Perception
Visual topography of area TEO in the macaque
J. Comp. Neurol.
Visual field maps and stimulus selectivity in human ventral occipital cortex
Nat. Neurosci.
Patterns of activity in the categorical representations of objects
J. Cogn. Neurosci.
Attribute-based neural substrates in temporal cortex for perceiving and knowing about objects
Nat. Neurosci.
Stimulus-selective properties of inferior temporal neurons in the macaque
J. Neurosci.
Anterior inferotemporal neurons of monkeys engaged in object recognition can be highly sensitive to object retinal position
J. Neurophysiol.
Different Spatial Scales of Shape Similarity Representation in Lateral and Ventral LOC
Cereb. Cortex.
(Coarse coding of shape fragments) + (retinotopy) representation of structure
Spat. Vis.
fMRI activity patterns in human LOC carry information about object exemplars within category
J. Cogn. Neurosci.
Graded size sensitivity of object-exemplar-evoked activity patterns within human LOC subregions
J. Neurophysiol.
Neural representations that support invariant object recognition
Front. Comput. Neurosci.
A sequence of object-processing stages revealed by fMRI in the human occipital lobe
Hum. Brain Mapp.
The human visual cortex
Annu. Rev. Neurosci.
Visual receptive fields of neurons in inferotemporal cortex of the monkey
Science
Visual properties of neurons in inferotemporal cortex of the Macaque
J. Neurophysiol.
Cited by (84)
Coexistence of sensory qualities and value representations in human orbitofrontal cortex
2022, Neuroscience ResearchThe past, present, and future of selection history
2021, Neuroscience and Biobehavioral ReviewsCitation Excerpt :Other studies, however, have associated valent outcomes with more complex objects (e.g., Barbaro et al., 2017; Donohue et al., 2016; Hickey and Peelen, 2015; Hickey et al., 2015; Le Pelley et al., 2017) and even scene semantics (Failing and Theeuwes, 2015) and have shown similar results. Interestingly, studies of the neural correlates of value-driven attention (Anderson et al., 2014a; Anderson, 2017d; Hickey and Peelen, 2015; Kim and Anderson, 2020a, b) and aversively-conditioned attention (Kim et al., 2021c) using color-defined stimuli have repeatedly shown elevated distractor-evoked activity in lateral occipital cortex, which is associated with object-selective processing (Cichy et al., 2011; Grill-Spector, 2010; Malach et al., 1995), rather than earlier visual areas such as V1 and V4 (e.g., Johnson et al., 2008; Wade et al., 2008) associated with lower-level color-selective representation (although see Itthipuripat et al., 2019; Serences, 2008). This raises the interesting possibility that object identity plays an integral role in attentional orienting to valent stimuli, consistent with the value-dependent modulation of object-selective neurons in the caudate tail (Kim and Hikosaka, 2013; Yamamoto et al., 2013, 2012) and its connections with the visual and oculomotor systems (Giggs et al., 2017; Seger, 2013).
fMRI representational similarity analysis reveals graded preferences for chromatic and achromatic stimulus contrast across human visual cortex
2020, NeuroImageCitation Excerpt :This trend is consistent with the results of Salmela et al. (2016), who used a very different stimulus set but observed a similar trend for decreasing correlation with a spatial frequency model along the ventral visual stream. This might be related to areas such as hV4 and LO encoding higher-level form, including object shape (Cichy et al., 2011; DiCarlo and Cox, 2007; Oleskiw et al., 2018; Pasupathy and Connor, 2001), and becoming less sensitive to lower-level feature dimensions like spatial frequency. Across the ventral visual stream, receptive field sizes generally increase (Harvey and Dumoulin, 2011), and peak-SF tuning decreases (Henriksson et al., 2008) which could be related to the decreasing correlation with the Ach(P)/SF model.