Abstract
Learning is more than memory. It is not simply the building of a look-up table of labelled images, or a phone-directory-like list of motor acts and the corresponding sequences of muscle activation. Central to learning and intelligence is the ability to predict, that is, to generalize to new situations, beyond the memory of specific examples. The key to generalization, in turn, is the architecture of the system, more than the rules of synaptic plasticity. We propose a specific architecture for generalization for both the motor and the visual systems, and argue for a canonical microcircuit underlying visual and motor learning.
This is a preview of subscription content, access via your institution
Access options
Subscribe to this journal
Receive 51 print issues and online access
$199.00 per year
only $3.90 per issue
Buy this article
- Purchase on Springer Link
- Instant access to full article PDF
Prices may be subject to local taxes which are calculated during checkout
Similar content being viewed by others
References
Vapnik, V. N. Statistical Learning Theory (Wiley, New York, 1998).
Bülthoff, H. & Edelman, S. Psychophysical support for a two-dimensional view interpolation theory of object recognition. Proc. Natl Acad. Sci. USA 89, 60–64 (1992).
Riesenhuber, M. & Poggio, T. Hierarchical models of object recognition in cortex. Nature Neurosci. 2, 1019–1025 (1999).
Riesenhuber, M. & Poggio, T. The Visual Neurosciences Vol. 2 (eds Chalupa, L. M. & Werner, J. S.) 1640–1653 (MIT Press, Cambridge, MA, 2003).
Palmeri, T. & Gauthier, I. Visual object understanding. Nature Rev. Neurosci. 5, 291–303 (2004).
Hubel, D. H. & Wiesel, T. N. Receptive fields, binocular interaction and functional architecture in the cat's visual cortex. J. Physiol. 160, 106–154 (1962).
Hubel, D. & Wiesel, T. Receptive fields and functional architecture in two nonstriate visual areas (18 and 19) of the cat. J. Neurophysiol. 28, 229–289 (1965).
Mountcastle, V. B. Modality and topographic properties of single neurons of cat's somatic sensory cortex. J. Neurophysiol. 20, 408–434 (1957).
Gross, C. G. Handbook of Sensory Physiology Vol. VII/3B (eds Autrum, H., Jung, R., Lowenstein, W., Mckay, D. & Teuber, H.-L.) (Springer, Berlin, 1973).
Bruce, C., Desimone, R. & Gross, C. Visual properties of neurons in a polysensory area in the superior temporal sulcus of the macaque. J. Neurophysiol. 46, 369–384 (1981).
Perrett, D. et al. Viewer-centred and object-centred coding of heads in the macaque temporal cortex. Exp. Brain Res. 86, 159–173 (1991).
Perrett, D. & Oram, M. Neurophysiology of shape processing. Img. Vis. Comput. 11, 317–333 (1993).
Logothetis, N., Pauls, J. & Poggio, T. Shape representation in the inferior temporal cortex of monkeys. Curr. Biol. 5, 552–563 (1995).
Logothetis, N. & Sheinberg, D. Visual object recognition. Annu. Rev. Neurosci. 19, 577–621 (1996).
Kobatake, E. & Tanaka, K. Neuronal selectivities to complex object features in the ventral visual pathway of the macaque cerebral cortex. J. Neurophysiol. 71, 856–857 (1994).
Kobatake, E., Wang, G. & Tanaka, K. Effects of shape-discrimination training on the selectivity of inferotemporal cells in adult monkeys. J. Neurophysiol. 80, 324–330 (1998).
DiCarlo, J. & Maunsell, J. Form representation in monkey inferotemporal cortex is virtually unaltered by free viewing. Nature Neurosci. 3, 814–821 (2000).
Booth, M. & Rolls, E. View-invariant representations of familiar objects by neurons in the inferior temporal visual cortex. Cereb. Cortex 8, 510–523 (1998).
Tanaka, K. Neuronal mechanisms of object recognition. Science 262, 685–688 (1993).
Sato, T. Interactions of visual stimuli in the receptive fields of inferior temporal neurons in awake monkeys. Exp. Brain Res. 77, 23–30 (1989).
Hietanen, J., Perrett, D., Benson, P. & Dittrich, W. The effects of lighting conditions on responses of cells selective for face views in the macaque temporal cortex. Exp. Brain Res. 89, 157–171 (1992).
Missal, M., Vogels, R. & Orban, G. Responses of macaque inferior temporal neurons to overlapping shapes. Cereb. Cortex 7, 758–767 (1997).
Poggio, T. A. Theory of how the brain might work. Cold Spring Harb. Symp. Quant. Biol. 4, 899–910 (1990).
Olshausen, B. A. & Field, D. J. Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature 381, 607–609 (1996).
Pouget, A., Dayan, P. & Zemel, R. S. Computation and inference with population codes. Annu. Rev. Neurosci. 26, 381–410 (2003).
Pouget, A. & Sejnowski, T. J. Spatial transformations in the parietal cortex using basis functions. J. Cogn. Neurosci. 9, 222–237 (1997).
Georgopoulos, A. P., Kalaska, J. F., Caminiti, R. & Massey, J. T. On the relations between the direction of two-dimensional arm movements and cell discharge in primate motor cortex. J. Neurosci. 2, 1527–1537 (1982).
Amirikian, B. & Georgopoulos, A. P. Modular organization of directionally tuned cells in the motor cortex: is there a short-range order? Proc. Natl Acad. Sci. USA 100, 12474–12479 (2003).
Merzenich, M. M. & Brugge, J. F. Representation of the cochlear partition of the superior temporal plane of the macaque monkey. Brain Res. 50, 275–296 (1973).
Wise, S. P., Moody, S. L., Blomstrom, K. J. & Mitz, A. R. Changes in motor cortical activity during visuomotor adaptation. Exp. Brain Res. 121, 285–299 (1998).
Paz, R., Boraud, T., Natan, C., Bergman, H. & Vaadia, E. Preparatory activity in motor cortex reflects learning of local visuomotor skills. Nature Neurosci. 6, 882–890 (2003).
Gribble, P. L. & Scott, S. H. Overlap of internal models in motor cortex for mechanical loads during reaching. Nature 417, 938–941 (2002).
Gandolfo, F., Li, C. R., Benda, B., Padoa-Schioppa, C. & Bizzi, E. Cortical correlates of motor learning in monkeys adapting to a new dynamic environment. Proc. Natl Acad. Sci. USA 97, 2259–2263 (2000).
Li, C. R., Padoa-Schioppa, C. & Bizzi, E. Neuronal correlates of motor performance and motor learning in the primary motor cortex of monkeys adapting to an external force field. Neuron 30, 593–607 (2001).
Grillner, S. & Wallen, P. Central pattern generators for locomotion, with special reference to vertebrates. Annu. Rev. Neurosci. 8, 233–261 (1985).
Stein, P. S., Victor, J. C., Field, E. C. & Currie, S. N. Bilateral control of hindlimb scratching in the spinal turtle: contralateral spinal circuitry contributes to the normal ipsilateral motor pattern of fictive rostral scratching. J. Neurosci. 15, 4343–4355 (1995).
Loeb, G. E. Motoneurone task groups: coping with kinematic heterogeneity. J. Exp. Biol. 115, 137–146 (1985).
Bizzi, E., Giszter, S. & Mussa-Ivaldi, F. A. Computations underlying the execution of movement: a novel biological perspective. Science 253, 287–291 (1991).
Mussa-Ivaldi, F. A., Giszter, S. F. and Bizzi, E. Linear combinations of primitives in vertebrate motor control. Proc. Natl Acad. Sci. USA 91, 7534–7538 (1994).
Jing, J., Cropper, E. C., Hurwitz, I. & Weiss, K. R. The construction of movement with behavior-specific and behavior-independent modules. J. Neurosci. 24, 6315–6325 (2004).
Saltiel, P., Tresch, M. C. & Bizzi, E. Spinal cord modular organization and rhythm generation: an NMDA iontophoretic study in the frog. J. Neurophysiol. 80, 2323–2339 (1998).
Grillner, S. Handbook of Physiology — The Nervous System edn 4 (eds Brookhart, J. M. & Mountcastle, V. B.) 1179–1236 (American Physiological Society, Bethesda, MD, 1981).
d'Avella, A., Saltiel, P. & Bizzi, E. Combinations of muscle synergies in the construction of a natural motor behaviour. Nature Neurosci. 6, 300–308 (2003).
Lemay, M. A., Galagan, J. E., Hogan, N. & Bizzi, E. Modulation and vectorial summation of the spinalized frog's hindlimb end-point force produced by intraspinal electrical stimulation of the cord. IEEE Trans. Neural. Syst. Rehabil. Eng. 9, 12–23 (2001).
Aoyagi, Y., Stein, R. B., Mushahwar, V. K. & Prochazka, A. The role of neuromuscular properties in determining the end-point of a movement. IEEE Trans. Neural. Syst. Rehabil. Eng. 12, 12–23 (2004).
Tresch, M. C. & Bizzi, E. Responses from the spinal microstimulation in the chronically spinalized rats and their relationship to spinal systems activated by low threshold cutaneous stimulation. Exp. Brain Res. 129, 401–416 (1999).
Kargo, W. J. & Giszter, S. F. Rapid correction of aimed movements by summation of force field primitives. J. Neurosci. 20, 409–426 (2000).
Mussa-Ivaldi, F. A. in Proc. 1997 IEEE Int. Symp. Computational Intelligence in Robotics and Automation 84–90 (IEEE Computer Society, Los Alamitos, California, 1997).
Shadmehr, R. & Mussa-Ivaldi, F. A. Adaptive representation of dynamics during learning of a motor task. J. Neurosci. 14, 3208–3224 (1994).
Chance, F., Nelson, S. & Abbott, L. Complex cells as cortically amplified simple cells. Nature Neurosci. 2, 277–282 (1999).
Thoroughman, K. & Shadmer, R. Learning of action through adaptive combination of motor primitives. Nature 407, 740–746 (2000).
Douglas, R. & Martin, K. A functional microcircuit for cat visual cortex. J. Physiol. (Lond.) 440, 735–769 (1991).
Borg-Graham, L. J., Monier, C. & Frégnac Y. Visual input evokes transient and strong shunting inhibition in visual cortical neurons. Nature 393, 369–373 (1998).
Mussa-Ivaldi, F. A. & Bizzi, E. Motor learning through the combination of primitives. Phil. Trans. R. Soc. Lond. B 355, 1755–1769 (2000).
Giese, M. & Poggio, T. Neural mechanisms for the recognition of biological movements. Nature Rev. Neurosci. 4, 179–192 (2003).
Hastie, T., Tibshirani, R. & Friedman, J. The Elements of Statistical Learning (Springer, Basel, 2001).
Rodman, H. R., Scalaidhe, S. P. & Gross, C. G. Response properties of neurons in temporal cortical visual areas of infant monkeys. J. Neurophysiol. 70, 1115–1136 (1993).
Belkin, M., Niyogi, P. & Sindhwani, V. Technical Report TR–2004–05 (University of Chicago, Chicago, 2004).
Douglas, R. & Martin, K. Neural circuits of the neocortex. Annu. Rev. Neurosci. 27, 419–451 (2004).
Salinas, E. & Abbott, L. F. Transfer of coded information from sensory to motor networks. J. Neurosci. 15, 6461–6474 (1995).
Hogan, N. An organizing principle for a class of voluntary movements. J. Neusosci. 4, 2745–2754 (1984).
Poggio, T. & Smale, S. The mathematics of learning: dealing with data. Notices Am. Math. Soc. 50, 537–544 (2003).
Maruyama, M., Girosi, F. & T. Poggio, T. A. Connection Between GRBF and MLP. AI Memo 1291 (Massachusetts Institute of Technology, Cambridge, Massachusetts 1992).
Carandini, M. & Heeger, D. J. Summation and division by neurons in visual cortex. Science 264, 1333–1336 (1994).
Carandini, M., Heeger, D. J. & Movshon, J. A. Linearity and normalization in simple cells of the macaque primary visual cortex. J. Neurosci. 17, 8621–8644 (1997).
Reichardt, W., Poggio, T. & Hausen, K. Figure-ground discrimination by relative movement in the visual system of the fly II: towards the neural circuitry. Biol. Cybern. 46, 1–30 (1983).
Yu, A. J., Giese, M. A. & Poggio, T. Biophysiologically plausible implementations of the maximum operation. Neural Comput. 14, 2857–2881 (2002).
Acknowledgements
We thank S. Mussa-Ivaldi, R. Shadmeher, G. Kreiman, and M. Riesenhuber for insightful and helpful comments. This research was sponsored by grants from NIH, Office of Naval Research, DARPA and National Science Foundation. Additional support was provided by Eastman Kodak Company, Daimler Chrysler, Honda Research Institute, NEC Fund, Siemens Corporate Research, Toyota, Sony and the McDermott chair (T.P.).
Author information
Authors and Affiliations
Ethics declarations
Competing interests
The authors declare no competing financial interests.
Rights and permissions
About this article
Cite this article
Poggio, T., Bizzi, E. Generalization in vision and motor control. Nature 431, 768–774 (2004). https://doi.org/10.1038/nature03014
Published:
Issue Date:
DOI: https://doi.org/10.1038/nature03014
This article is cited by
-
Reinforcement learning establishes a minimal metacognitive process to monitor and control motor learning performance
Nature Communications (2023)
-
Manipulation Learning on Humanoid Robots
Current Robotics Reports (2022)
-
Improving performance of robots using human-inspired approaches: a survey
Science China Information Sciences (2022)
-
Rapid adaptation of brain–computer interfaces to new neuronal ensembles or participants via generative modelling
Nature Biomedical Engineering (2021)
-
Generalization of novel sensorimotor associations among pianists and non-pianists: more evidence that musical training effects are constrained
Psychological Research (2021)
Comments
By submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.