Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Article
  • Published:

Underlying principles of visual shape selectivity in posterior inferotemporal cortex

Nature Neuroscience volume 7, pages 880–886 (2004)Cite this article

Abstract

Object perception depends on shape processing in the ventral visual pathway, which in monkeys culminates in inferotemporal cortex (IT). Here we provide a description of fundamental quantitative principles governing neural selectivity for complex shape in IT. By measuring responses to large, parametric sets of two-dimensional (2D) silhouette shapes, we found that neurons in posterior IT (Brodmann's areas TEO and posterior TE) integrate information about multiple contour elements (straight and curved edge fragments of the type represented in lower-level areas) using both linear and nonlinear mechanisms. This results in complex, distributed response patterns that cannot be characterized solely in terms of example stimuli. We explained these response patterns with tuning functions in multidimensional shape space and accurately predicted neural responses to the widely varying shapes in our stimulus set. Integration of contour element information in earlier stages of IT represents an important step in the transformation from low-level shape signals to complex object representation.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Figure 1: Quantitative analysis of shape selectivity for a typical IT neuron.
Figure 2: Quantitative analysis of shape selectivity for a highly selective IT neuron (conventions as in Fig. 1).
Figure 3: Population summary of IT shape selectivity.
Figure 4: Consistency of shape selectivity across position and size changes.

Similar content being viewed by others

References

  1. Gross, C.G., Rocha-Miranda, C.E. & Bender, D.B. Visual properties of neurons in inferotemporal cortex of the Macaque. J. Neurophysiol. 35, 96–111 (1972).

    Article  CAS  PubMed  Google Scholar 

  2. Kobatake, E. & Tanaka, K. Neuronal selectivities to complex object features in the ventral visual pathway of the macaque cerebral cortex. J. Neurophysiol. 71, 856–867 (1994).

    Article  CAS  PubMed  Google Scholar 

  3. Booth, M.C. & Rolls, E.T. View-invariant representations of familiar objects by neurons in the inferior temporal visual cortex. Cereb. Cortex 8, 510–523 (1998).

    Article  CAS  PubMed  Google Scholar 

  4. Fujita, I., Tanaka, K., Ito, M. & Cheng, K. Columns for visual features of objects in monkey inferotemporal cortex. Nature 360, 343–346 (1992).

    Article  CAS  PubMed  Google Scholar 

  5. Young, M.P. & Yamane, S. Sparse population coding of faces in the inferotemporal cortex. Science 256, 1327–1331 (1992).

    Article  CAS  PubMed  Google Scholar 

  6. Rolls, E.T., Treves, A. & Tovee, M.J. The representational capacity of the distributed encoding of information provided by populations of neurons in primate temporal visual cortex. Exp. Brain Res. 114, 149–162 (1997).

    Article  CAS  PubMed  Google Scholar 

  7. Op de Beeck, H., Wagemans, J. & Vogels, R. Inferotemporal neurons represent low-dimensional configurations of parameterized shapes. Nat. Neurosci. 4, 1244–1252 (2001).

    Article  CAS  PubMed  Google Scholar 

  8. Tsunoda, K., Yamane, Y., Nishizaki, M. & Tanifuji, M. Complex objects are represented in macaque inferotemporal cortex by the combination of feature columns. Nat. Neurosci. 4, 832–838 (2001).

    Article  CAS  PubMed  Google Scholar 

  9. Sakai, K. & Miyashita, Y. Neural organization for the long-term memory of paired associates. Nature 354, 152–155 (1991).

    Article  CAS  PubMed  Google Scholar 

  10. Miller, E.K. & Desimone, R. Parallel neuronal mechanisms for short-term memory. Science 263, 520–522 (1994).

    Article  CAS  PubMed  Google Scholar 

  11. Sigala, N. & Logothetis, N.K. Visual categorization shapes feature selectivity in the primate temporal cortex. Nature 415, 318–320 (2002).

    Article  CAS  PubMed  Google Scholar 

  12. Baker, C.I., Behrmann, M. & Olson, C.R. Impact of learning on representation of parts and wholes in monkey inferotemporal cortex. Nat. Neurosci. 5, 1210–1216 (2002).

    Article  CAS  PubMed  Google Scholar 

  13. Pasupathy, A. & Connor, C.E. Population coding of shape in area V4. Nat. Neurosci. 5, 1332–1338 (2002).

    Article  CAS  PubMed  Google Scholar 

  14. Wang, Y., Fujita, I. & Murayama, Y. Neuronal mechanisms of selectivity for object features revealed by blocking inhibition in inferotemporal cortex. Nat. Neurosci. 3, 807–813 (2000).

    Article  CAS  PubMed  Google Scholar 

  15. Carandini, M., Heeger, D.J. & Movshon, J.A. Linearity and normalization in simple cells of the macaque primary visual cortex. J. Neurosci. 17, 8621–8644 (1997).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  16. Mechler, F. & Ringach, D.L. On the classification of simple and complex cells. Vision Res. 42, 1017–1033 (2002).

    Article  PubMed  Google Scholar 

  17. Ito, M., Tamura, H., Fujita, I. & Tanaka, K. Size and position invariance of neuronal responses in monkey inferotemporal cortex. J. Neurophysiol. 73, 218–226 (1995).

    Article  CAS  PubMed  Google Scholar 

  18. DiCarlo, J.J. & Maunsell, J.H. Anterior inferotemporal neurons of monkeys engaged in object recognition can be highly sensitive to object retinal position. J. Neurophysiol. 89, 3264–3278 (2003).

    Article  PubMed  Google Scholar 

  19. Op de Beeck, H. & Vogels, R. Spatial sensitivity of macaque inferior temporal neurons. J. Comp Neurol. 426, 505–518 (2000).

    Article  CAS  PubMed  Google Scholar 

  20. Jones, J.P. & Palmer, L.A. The two-dimensional spatial structure of simple receptive fields in cat striate cortex. J. Neurophysiol. 58, 1187–1211 (1987).

    Article  CAS  PubMed  Google Scholar 

  21. Ringach, D.L., Sapiro, G. & Shapley, R. A subspace reverse-correlation technique for the study of visual neurons. Vision Res. 37, 2455–2464 (1997).

    Article  CAS  PubMed  Google Scholar 

  22. Theunissen, F.E. et al. Estimating spatio-temporal receptive fields of auditory and visual neurons from their responses to natural stimuli. Network 12, 289–316 (2001).

    Article  CAS  PubMed  Google Scholar 

  23. Marr, D. & Nishihara, H.K. Representation and recognition of the spatial organization of three-dimensional shapes. Proc. R. Soc. Lond B Biol. Sci. 200, 269–294 (1978).

    Article  CAS  PubMed  Google Scholar 

  24. Biederman, I. Recognition-by-components: a theory of human image understanding. Psychol. Rev. 94, 115–147 (1987).

    Article  PubMed  Google Scholar 

  25. Salinas, E. & Abbott, L.F. Invariant visual responses from attentional gain fields. J. Neurophysiol. 77, 3267–3272 (1997).

    Article  CAS  PubMed  Google Scholar 

  26. Wilson, H.R., Wilkinson, F. & Asaad, W. Concentric orientation summation in human form vision. Vision Res. 37, 2325–2330 (1997).

    Article  CAS  PubMed  Google Scholar 

  27. Riesenhuber, M. & Poggio, T. Hierarchical models of object recognition in cortex. Nat. Neurosci. 2, 1019–1025 (1999).

    Article  CAS  PubMed  Google Scholar 

  28. Mel, B.W. & Fiser, J. Minimizing binding errors using learned conjunctive features. Neural Comput. 12, 731–762 (2000).

    Article  CAS  PubMed  Google Scholar 

  29. Edelman, S. & Intrator, N. Towards structural systematicity in distributed, statically bound visual representations. Cognit. Sci. 27, 73–109 (2003).

    Article  Google Scholar 

  30. Janssen, P., Vogels, R. & Orban, G.A. Macaque inferior temporal neurons are selective for disparity-defined three-dimensional shapes. Proc. Natl. Acad. Sci. USA 96, 8217–8222 (1999).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  31. Kovacs, G. et al. Effects of surface cues on macaque inferior temporal cortical responses. Cereb. Cortex 13, 178–188 (2003).

    Article  CAS  PubMed  Google Scholar 

  32. Barlow, H. Redundancy reduction revisited. Network 12, 241–253 (2001).

    Article  CAS  PubMed  Google Scholar 

  33. Olshausen, B.A. & Field, D.J. Sparse coding with an overcomplete basis set: a strategy employed by V1? Vision Res. 37, 3311–3325 (1997).

    Article  CAS  PubMed  Google Scholar 

  34. Vinje, W.E. & Gallant, J.L. Sparse coding and decorrelation in primary visual cortex during natural vision. Science 287, 1273–1276 (2000).

    Article  CAS  PubMed  Google Scholar 

  35. Kapadia, M.K., Westheimer, G. & Gilbert, C.D. Spatial distribution of contextual interactions in primary visual cortex and in visual perception. J. Neurophysiol. 84, 2048–2062 (2000).

    Article  CAS  PubMed  Google Scholar 

  36. Polat, U., Mizobe, K., Pettet, M.W., Kasamatsu, T. & Norcia, A.M. Collinear stimuli regulate visual responses depending on cell's contrast threshold. Nature 391, 580–584 (1998).

    Article  CAS  PubMed  Google Scholar 

  37. Gallant, J.L., Braun, J. & Van Essen, D.C. Selectivity for polar, hyperbolic, and Cartesian gratings in macaque visual cortex. Science 259, 100–103 (1993).

    Article  CAS  PubMed  Google Scholar 

  38. Wolfe, J.M., Yee, A. & Friedman-Hill, S.R. Curvature is a basic feature for visual search tasks. Perception 21, 465–480 (1992).

    Article  CAS  PubMed  Google Scholar 

  39. Olson, C.R. & Gettner, S.N. Object-centered direction selectivity in the macaque supplementary eye field. Science 269, 985–988 (1995).

    Article  CAS  PubMed  Google Scholar 

  40. Connor, C.E., Preddie, D.C., Gallant, J.L. & Van Essen, D.C. Spatial attention effects in macaque area V4. J. Neurosci. 17, 3201–3214 (1997).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  41. Zhou, H., Friedman, H.S. & von der Heydt, R. Coding of border ownership in monkey visual cortex. J. Neurosci. 20, 6594–6611 (2000).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  42. Schwartz, E.L., Desimone, R., Albright, T.D. & Gross, C.G. Shape recognition and inferior temporal neurons. Proc. Natl. Acad. Sci. USA 80, 5776–5778 (1983).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  43. Albright, T.D. & Gross, C.G. Do inferior temporal neurons encode shape by acting as fourier descriptor filters? Proc. Int. Conf. on Fuzzy Logic & Neural Networks 2, 375–378 (1990).

    Google Scholar 

  44. Optican, L.M. & Richmond, B.J. Temporal encoding of two-dimensional patterns by single units in primate inferior temporal cortex. III. Information theoretic analysis. J. Neurophysiol. 57, 162–178 (1987).

    Article  CAS  PubMed  Google Scholar 

  45. Gochin, P.M., Colombo, M., Dorfman, G.A., Gerstein, G.L. & Gross, C.G. Neural ensemble coding in inferior temporal cortex. J. Neurophysiol. 71, 2325–2337 (1994).

    Article  CAS  PubMed  Google Scholar 

  46. Ullman, S. Three-dimensional object recognition based on the combination of views. Cognition 67, 21–44 (1998).

    Article  CAS  PubMed  Google Scholar 

  47. Tanaka, K. Inferotemporal cortex and object vision. Annu. Rev. Neurosci. 19, 109–139 (1996).

    Article  CAS  PubMed  Google Scholar 

  48. Hoffman, D.D. & Richards, W.A. Parts of recognition. Cognition 18, 65–96 (1984).

    Article  CAS  PubMed  Google Scholar 

  49. Leyton, M. A Generative Theory of Shape (Springer, Berlin, 1998).

    Google Scholar 

  50. Kimia, B.B. On the role of medial geometry in human vision. J. Physiol. (Paris) 97, 155–190 (2003).

    Article  Google Scholar 

Download references

Acknowledgements

We thank W. Nash, W. Quinlan, B. Sorensen and L. Ziehm for technical assistance, and A. Bastian, D. Hinkle, K. Johnson and R. von der Heydt for helpful comments. Our implementation of Hartigan's dip test is based on public domain code provided by F. Mechler and D. Ringach. This work was supported by the National Eye Institute and by the Pew Scholars Program in the Biomedical Sciences.

Author information

Authors and Affiliations

  1. Zanvyl Krieger Mind/Brain Institute and Department of Neuroscience, Johns Hopkins University, Baltimore, 21218, Maryland, USA

    Scott L Brincat & Charles E Connor

Authors
  1. Scott L Brincat
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Charles E Connor
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Charles E Connor.

Ethics declarations

Competing interests

The authors declare no competing financial interests.

Supplementary information

Supplementary Fig. 1

Further empirical and analytical results for the two example neurons described in the main text. a, Modified version of main stimulus set in which one end of each stimulus was centered in the RF. Responses of the Fig. 1 example neuron are shown here (all conventions as in Fig. 1). b, Full fitted model, including tuning for absolute position, for the Fig. 1 neuron. c, Responses of the Fig. 2 example neuron to the modified main stimulus set. d, Full fitted model for the Fig. 2 neuron. (PDF 2439 kb)

Supplementary Fig. 2

Position and size consistency of shape selectivity for Fig. 1 example neuron. a, Responses of the Fig. 1 example neuron to preferred and non-preferred shapes across a range of retinotopic positions. Gray level indicates actual responses; overlaid red contours show response predictions based on this neuron's optimal model. The icon above each gray level plot shows the shape used to generate it. Crosses represent center of gaze. b, Responses of the Fig. 1 neuron to preferred and non-preferred shapes across a range of sizes. The icons above the plot show stimulus sizes, relative to estimated RF (dashed circle), at three selected points within the tested range. The gray level of each plot line indicates the shape used to generate it (corresponding to the gray levels of the icons above the plot). Dotted lines show ± s.e.m. (PDF 79 kb)

Supplementary Fig. 3

Relative contributions of model tuning dimensions. a-d, Tuning width and tuning depth index values for orientation (a), curvature (b), relative position (c), and absolute position (d) dimensions for all sampled IT neurons. e, Mean tuning depth index value across all IT neurons for each dimension; lines denote upper and lower 99% confidence intervals (bootstrapped). f, Mean (± 99% confidence intervals) tuning width index value across all IT neurons for each dimension. (PDF 78 kb)

Supplementary Note (PDF 105 kb)

Rights and permissions

Reprints and permissions

About this article

Cite this article

Brincat, S., Connor, C. Underlying principles of visual shape selectivity in posterior inferotemporal cortex. Nat Neurosci 7, 880–886 (2004). https://doi.org/10.1038/nn1278

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/nn1278

This article is cited by

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing