Category-selective patterns of neural response in the ventral visual pathway in the absence of categorical information

Highlights

• Object category is an important organizing principle in ventral stream.

• However, object categories differ systematically in their image properties.

• Scrambled images were used to preserve low-level but remove categorical properties.

• Intact and scrambled images evoked similar patterns of neural response.

• Neural representation in the ventral stream based on image properties

Abstract

Neuroimaging studies have revealed distinct patterns of response to different object categories in the ventral visual pathway. These findings imply that object category is an important organizing principle in this region of visual cortex. However, object categories also differ systematically in their image properties. So, it is possible that these patterns of neural response could reflect differences in image properties rather than object category. To differentiate between these alternative explanations, we used images of objects that had been phase-scrambled at a local or global level. Both scrambling processes preserved many of the lower-level image properties, but rendered the images unrecognizable. We then measured the effect of image scrambling on the patterns of neural response within the ventral pathway. We found that intact and scrambled images evoked distinct category-selective patterns of activity in the ventral stream. Moreover, intact and scrambled images of the same object category produced highly similar patterns of response. These results suggest that the neural representation in the ventral visual pathway is tightly linked to the statistical properties of the image.

Introduction

Visual areas involved in object perception form a ventral processing pathway that projects from the occipital toward the temporal lobe (Ungerleider and Mishkin, 1982, Milner and Goodale, 1995). Lesions to these regions of the brain often result in difficulties in the perception and recognition of different categories of objects (McNeil and Warrington, 1993, Moscovitch et al., 1997). Consistent with these neuropsychological reports, neuroimaging studies have shown that discrete regions of the ventral visual pathway are specialized for different categories of objects. For example, while some regions of the ventral visual pathway are more responsive to images of faces than to images of non-face objects (Kanwisher et al., 1997, McCarthy et al., 1997), other regions are selective for images of places (Epstein and Kanwisher, 1998), body parts (Downing et al., 2001) and visually presented words (Cohen et al., 2000). This selectivity has been regarded as characteristic of a modular organization in which distinct regions process specific object categories (Kanwisher, 2010).

Despite the evidence for category-selectivity in the ventral visual pathway, specialized regions have only been reported for a limited number of object categories (Downing et al., 2006, Op de Beeck et al., 2008). However, other studies using multivariate fMRI analysis methods have demonstrated that the spatial pattern of response across the entire ventral stream can distinguish a greater range of object categories (Ishai et al., 1999, Haxby et al., 2001, Kriegeskorte et al., 2008). The distributed nature of the fMRI response to different categories of objects within the ventral visual pathway has been interpreted as showing a topographic map of object form (Haxby et al., 2001).

The topographic organization of the ventral stream is thought to be analogous with the topographic organization found in early visual areas, in which responses are tightly linked to low-level properties of the image, such as spatial frequency, orientation and spatial position (Hubel and Wiesel, 1968, Bonhoeffer and Grinvald, 1991, Engel et al., 1994, Wandell et al., 2007). In contrast to early visual areas, a variety of evidence has suggested that patterns of response in the ventral visual pathway are linked to the categorical or semantic information that the images convey rather than to the image properties (Kriegeskorte et al., 2008, Naselaris et al., 2009, Connolly et al., 2012). Evidence for other organizing principles can be found in the large-scale patterns of response to animacy (Chao et al., 1999, Kriegeskorte et al., 2008) and the real-world size of objects (Konkle and Oliva, 2012). However, it remains unclear how the selectivity for high-level properties in ventral visual pathway might arise from image-based representations found in early visual regions (Op de Beeck et al., 2008). In recent studies, we showed that low-level properties of objects could predict patterns of neural response in the ventral visual pathway (Rice et al., 2014, Watson et al., 2016). However, images drawn from the same object category are likely to have similar low-level properties. So, the link between image properties and patterns of neural response is expected under both categorical and image-based accounts.

The aim of the present study was to directly determine whether the category-selective patterns of neural response across the ventral visual pathway can be explained by selectivity to more basic properties of the stimulus. To address this question, we measured the neural response across the ventral visual pathway to intact images of different object categories, as well as versions of these images that had been phase-scrambled on a global or local basis. Our rationale for using scrambled images is that they have many of the image properties found in intact images, but do not convey any categorical or semantic information, thus providing dissociation between higher-level and lower-level information. Our hypothesis was that, if neurons in the ventral stream are selective for the categorical or semantic properties conveyed by the image, there should be no correspondence between patterns of response evoked by intact and scrambled images. Conversely, if patterns of response in the ventral stream reflect selectivity to more basic dimensions of the stimulus, we would predict a significant correlation between patterns of response to intact and scrambled images.