Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Article
  • Published:

A neural code for three-dimensional object shape in macaque inferotemporal cortex

Nature Neuroscience volume 11, pages 1352–1360 (2008)Cite this article

This article has been updated

Abstract

Previous investigations of the neural code for complex object shape have focused on two-dimensional pattern representation. This may be the primary mode for object vision given its simplicity and direct relation to the retinal image. In contrast, three-dimensional shape representation requires higher-dimensional coding derived from extensive computation. We found evidence for an explicit neural code for complex three-dimensional object shape. We used an evolutionary stimulus strategy and linear/nonlinear response models to characterize three-dimensional shape responses in macaque monkey inferotemporal cortex (IT). We found widespread tuning for three-dimensional spatial configurations of surface fragments characterized by their three-dimensional orientations and joint principal curvatures. Configural representation of three-dimensional shape could provide specific knowledge of object structure to support guidance of complex physical interactions and evaluation of object functionality and utility.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Figure 1: Evolutionary three-dimensional shape experiment.
Figure 2: Neural tuning for three-dimensional configuration of surface fragments.
Figure 3: Prevalence of three-dimensional shape tuning in IT.
Figure 4: Three-dimensional surface configuration tuning patterns.
Figure 5: Distribution of three-dimensional shape tuning.
Figure 6: Configural coding of three-dimensional object structure.

Similar content being viewed by others

Change history

  • 12 October 2008

    In the version of this article initially published online, Figure 1 was too small to evaluate the data being presented. The figure has been enlarged for the print, PDF and HTML versions of this article.

References

  1. Ungerleider, L.G. & Mishkin, M. Analysis of Visual Behavior (eds. Ingle, D.G., Goodale, M.A. & Mansfield, R.J.Q.) 549–586 (MIT Press, Cambridge, Massachusetts, 1982).

  2. Felleman, D.J. & Van Essen, D.C. Distributed hierarchical processing in the primate cerebral cortex. Cereb. Cortex 1, 1–47 (1991).

    Article  CAS  PubMed  Google Scholar 

  3. Anzai, A., Peng, X. & Van Essen, D.C. Neurons in monkey visual area V2 encode combinations of orientations. Nat. Neurosci. 10, 1313–1321 (2007).

    Article  CAS  PubMed  Google Scholar 

  4. Ito, M. & Komatsu, H. Representation of angles embedded within contour stimuli in area V2 of macaque monkeys. J. Neurosci. 24, 3313–3324 (2004).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  5. Gallant, J.L., Braun, J. & Van Essen, D.C. Selectivity for polar, hyperbolic and Cartesian gratings in macaque visual cortex. Science 259, 100–103 (1993).

    Article  CAS  PubMed  Google Scholar 

  6. Pasupathy, A. & Connor, C.E. Shape representation in area V4: position-specific tuning for boundary conformation. J. Neurophysiol. 86, 2505–2519 (2001).

    Article  CAS  PubMed  Google Scholar 

  7. Pasupathy, A. & Connor, C.E. Responses to contour features in macaque area V4. J. Neurophysiol. 82, 2490–2502 (1999).

    Article  CAS  PubMed  Google Scholar 

  8. Pasupathy, A. & Connor, C.E. Population coding of shape in area V4. Nat. Neurosci. 5, 1332–1338 (2002).

    Article  CAS  PubMed  Google Scholar 

  9. Brincat, S.L. & Connor, C.E. Underlying principles of visual shape selectivity in posterior inferotemporal cortex. Nat. Neurosci. 7, 880–886 (2004).

    Article  CAS  PubMed  Google Scholar 

  10. Brincat, S.L. & Connor, C.E. Dynamic shape synthesis in posterior inferotemporal cortex. Neuron 49, 17–24 (2006).

    Article  CAS  PubMed  Google Scholar 

  11. Malach, R. et al. Object-related activity revealed by functional magnetic resonance imaging in human occipital cortex. Proc. Natl. Acad. Sci. USA 92, 8135–8139 (1995).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  12. Kourtzi, Z. & Kanwisher, N. Representation of perceived object shape by the human lateral occipital complex. Science 293, 1506–1509 (2001).

    Article  CAS  PubMed  Google Scholar 

  13. Tsao, D.Y., Freiwald, W.A., Knutsen, T.A., Mandeville, J.B. & Tootell, R.B.H. Faces and objects in macaque cerebral cortex. Nat. Neurosci. 6, 989–995 (2003).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  14. Gross, C.G., Rocha-Miranda, C.E. & Bender, D.B. Visual properties of neurons in inferotemporal cortex of the Macaque. J. Neurophysiol. 35, 96–111 (1972).

    Article  CAS  PubMed  Google Scholar 

  15. Schwartz, E.L., Desimone, R., Albright, T.D. & Gross, C.G. Shape recognition and inferior temporal neurons. Proc. Natl. Acad. Sci. USA 80, 5776–5778 (1983).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  16. Kobatake, E. & Tanaka, K. Neuronal selectivities to complex object features in the ventral visual pathway of the macaque cerebral cortex. J. Neurophysiol. 71, 856–867 (1994).

    Article  CAS  PubMed  Google Scholar 

  17. Fujita, I., Tanaka, K., Ito, M. & Cheng, K. Columns for visual features of objects in monkey inferotemporal cortex. Nature 360, 343–346 (1992).

    Article  CAS  PubMed  Google Scholar 

  18. Tsunoda, K., Yamane, Y., Nishizaki, M. & Tanifuji, M. Complex objects are represented in macaque inferotemporal cortex by the combination of feature columns. Nat. Neurosci. 4, 832–838 (2001).

    Article  CAS  PubMed  Google Scholar 

  19. Baker, C.I., Behrmann, M. & Olson, C.R. Impact of learning on representation of parts and wholes in monkey inferotemporal cortex. Nat. Neurosci. 5, 1210–1216 (2002).

    Article  CAS  PubMed  Google Scholar 

  20. Quiroga, R.Q., Reddy, L., Kreiman, G., Koch, C. & Fried, I. Invariant visual representation by single neurons in the human brain. Nature 435, 1102–1107 (2005).

    Article  CAS  PubMed  Google Scholar 

  21. Freedman, D.J., Riesenhuber, M., Poggio, T. & Miller, E.K. Categorical representation of visual stimuli in the primate prefrontal cortex. Science 291, 312–316 (2001).

    Article  CAS  PubMed  Google Scholar 

  22. Marr, D. & Nishihara, H.K. Representation and recognition of the spatial organization of three-dimensional shapes. Proc. R. Soc. Lond. B 200, 269–294 (1978).

    Article  CAS  PubMed  Google Scholar 

  23. Biederman, I. Recognition-by-components: a theory of human image understanding. Psychol. Rev. 94, 115–147 (1987).

    Article  PubMed  Google Scholar 

  24. Vetter, T., Hurlbert, A. & Poggio, T. View-based models of 3D object recognition: invariance to imaging transformations. Cereb. Cortex 5, 261–269 (1995).

    Article  CAS  PubMed  Google Scholar 

  25. Bulthoff, H.H., Edelman, S.Y. & Tarr, M.J. How are three-dimensional objects represented in the brain? Cereb. Cortex 5, 247–260 (1995).

    Article  CAS  PubMed  Google Scholar 

  26. Tarr, M.J. & Pinker, S. Mental rotation and orientation-dependence in shape recognition. Cognit. Psychol. 21, 233–282 (1989).

    Article  CAS  PubMed  Google Scholar 

  27. Riesenhuber, M. & Poggio, T. Hierarchical models of object recognition in cortex. Nat. Neurosci. 2, 1019–1025 (1999).

    Article  CAS  PubMed  Google Scholar 

  28. Serre, T., Oliva, A. & Poggio, T. A feedforward architecture accounts for rapid categorization. Proc. Natl. Acad. Sci. USA 104, 6424–6429 (2007).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  29. Tarr, M.J. & Barenholtz, E. Reconsidering the role of structure in vision. in The Psychology of Learning and Motivation (eds. Ross, B. & Markman, A.) 157–180 (Academic Press, London, 2007).

  30. Janssen, P., Vogels, R. & Orban, G.A. Macaque inferior temporal neurons are selective for disparity-defined three-dimensional shapes. Proc. Natl. Acad. Sci. USA 96, 8217–8222 (1999).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  31. Uka, T., Tanaka, H., Yoshiyama, K., Kato, M. & Fujita, I. Disparity selectivity of neurons in monkey inferior temporal cortex. J. Neurophysiol. 84, 120–132 (2000).

    Article  CAS  PubMed  Google Scholar 

  32. Watanabe, M., Tanaka, H., Uka, T. & Fujita, I. Disparity-selective neurons in area V4 of macaque monkeys. J. Neurophysiol. 87, 1960–1973 (2002).

    Article  PubMed  Google Scholar 

  33. Hinkle, D.A. & Connor, C.E. Three-dimensional orientation tuning in macaque area V4. Nat. Neurosci. 5, 665–670 (2002).

    Article  CAS  PubMed  Google Scholar 

  34. Janssen, P., Vogels, R. & Orban, G.A. Three-dimensional shape coding in inferior temporal cortex. Neuron 27, 385–397 (2000).

    Article  CAS  PubMed  Google Scholar 

  35. Sakata, H. et al. Neural coding of 3D features of objects for hand action in the parietal cortex of the monkey. Phil. Trans. R. Soc. Lond. B 353, 1363–1373 (1998).

    Article  CAS  Google Scholar 

  36. Rust, N.C., Mante, V., Simoncelli, E.P. & Movshon, A. How MT cells analyze the motion of visual patterns. Nat. Neurosci. 9, 1421–1431 (2006).

    Article  CAS  PubMed  Google Scholar 

  37. Ito, M., Tamura, H., Fujita, I. & Tanaka, K. Size and position invariance of neuronal responses in monkey inferotemporal cortex. J. Neurophysiol. 73, 218–226 (1995).

    Article  CAS  PubMed  Google Scholar 

  38. Connor, C.E., Brincat, S.L. & Pasupathy, A. Transformation of shape information in the ventral pathway. Curr. Opin. Neurobiol. 17, 140–147 (2007).

    Article  CAS  PubMed  Google Scholar 

  39. Hoffman, D.D. & Richards, W.A. Parts of recognition. Cognition 18, 65–96 (1984).

    Article  CAS  PubMed  Google Scholar 

  40. Koenderink, J.J. What does the occluding contour tell us about solid shape? Perception 13, 321–330 (1984).

    Article  CAS  PubMed  Google Scholar 

  41. Edelman, S. & Poggio, T. Models of object recognition. Curr. Opin. Neurobiol. 1, 270–273 (1991).

    Article  CAS  PubMed  Google Scholar 

  42. Wang, G., Obama, S., Yamashita, W., Sugihara, T. & Tanaka, K. Prior experience of rotation is not required for recognizing objects seen from different angles. Nat. Neurosci. 8, 1768–1775 (2005).

    Article  CAS  PubMed  Google Scholar 

  43. Roelfsema, P.R. Cortical algorithms for perceptual grouping. Annu. Rev. Neurosci. 29, 203–227 (2006).

    Article  CAS  PubMed  Google Scholar 

  44. Rolls, E.T. & Treves, A. The relative advantages of sparse versus distributed encoding for associative neuronal networks in the brain. Network 1, 407–421 (1990).

    Article  Google Scholar 

  45. Vinje, W.E. & Gallant, J.L. Sparse coding and decorrelation in primary visual cortex during natural vision. Science 287, 1273–1276 (2000).

    Article  CAS  PubMed  Google Scholar 

  46. Janssen, P., Vogels, R. & Orban, G.A. Selectivity for 3D shape that reveals distinct areas within macaque inferotemporal cortex. Science 288, 2054–2056 (2000).

    Article  CAS  PubMed  Google Scholar 

Download references

Acknowledgements

We thank B. Nash, B. Quinlan, C. Moses and L. Guruvadoo for technical support, and J. Bastian, A. Bastian, T. Poggio and M. Riesenhuber for comments on the manuscript. This work was supported by a grant from the US National Institutes of Health to C.E.C.

Author information

Authors and Affiliations

  1. Zanvyl Krieger Mind/Brain Institute, Johns Hopkins University, 3400 N. Charles Street, Baltimore, 21218, Maryland, USA

    Yukako Yamane, Eric T Carlson, Katherine C Bowman, Zhihong Wang & Charles E Connor

  2. Department of Biomedical Engineering, Johns Hopkins University, School of Medicine, 720 Rutland Avenue, Baltimore, 21205, Maryland, USA

    Eric T Carlson

  3. Department of Neuroscience, Johns Hopkins University, School of Medicine, 725 N. Wolfe Street, Baltimore, 21205, Maryland, USA

    Katherine C Bowman & Charles E Connor

Authors
  1. Yukako Yamane
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Eric T Carlson
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Katherine C Bowman
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zhihong Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Charles E Connor
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Charles E Connor.

Supplementary information

Supplementary Text and Figures

Supplementary Figures 1–17 (PDF 2453 kb)

Rights and permissions

Reprints and permissions

About this article

Cite this article

Yamane, Y., Carlson, E., Bowman, K. et al. A neural code for three-dimensional object shape in macaque inferotemporal cortex. Nat Neurosci 11, 1352–1360 (2008). https://doi.org/10.1038/nn.2202

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/nn.2202

This article is cited by

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing