Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Letter
  • Published:

Real-time prediction of hand trajectory by ensembles of cortical neurons in primates

Abstract

Signals derived from the rat motor cortex can be used for controlling one-dimensional movements of a robot arm1. It remains unknown, however, whether real-time processing of cortical signals can be employed to reproduce, in a robotic device, the kind of complex arm movements used by primates to reach objects in space. Here we recorded the simultaneous activity of large populations of neurons, distributed in the premotor, primary motor and posterior parietal cortical areas, as non-human primates performed two distinct motor tasks. Accurate real-time predictions of one- and three-dimensional arm movement trajectories were obtained by applying both linear and nonlinear algorithms to cortical neuronal ensemble activity recorded from each animal. In addition, cortically derived signals were successfully used for real-time control of robotic devices, both locally and through the Internet. These results suggest that long-term control of complex prosthetic robot arm movements can be achieved by simple real-time transformations of neuronal population signals derived from multiple cortical areas in primates.

This is a preview of subscription content, access via your institution

Access options

Rent or buy this article

Prices vary by article type

from$1.95

to$39.95

Prices may be subject to local taxes which are calculated during checkout

Figure 1: Experimental design.
Figure 2: Real-time control of 1D hand movements.
Figure 3: Real-time prediction of 3D hand movements.
Figure 4: Neuron-dropping analysis.

Similar content being viewed by others

References

  1. Chapin, J. K., Moxon, K. A., Markowitz, R. S. & Nicolelis, M. A. L. Real-time control of a robot arm using simultaneously recorded neurons in the motor cortex. Nature Neurosci. 2, 664 –670 (1999).

    Article  CAS  Google Scholar 

  2. Evarts, E. V. Pyramidal tract activity associated with a conditioned hand movement in the monkey. J. Neurophysiol. 29, 1011– 1027 (1966).

    Article  CAS  Google Scholar 

  3. Mountcastle, V. B., Lynch, J. C., Georgopoulos, A., Sakata, H. & Acuna, C. Posterior parietal association cortex of the monkey: command functions for operation within extrapersonal space. J. Neurophysiol. 38, 871– 908 (1975).

    Article  CAS  Google Scholar 

  4. Fetz, E. E. & Cheney, P. D. Muscle fields of primate corticomotoneuronal cells. J. Physiol. (Paris) 74, 239– 245 (1978).

    CAS  Google Scholar 

  5. Georgopoulos, A. P., Kalaska, J. F., Caminiti, R. & Massey, J. T. On the relations between the direction of two-dimensional arm movements and cell discharge in primate motor cortex. J. Neurosci. 2, 1527–1537 (1982).

    Article  CAS  Google Scholar 

  6. Weinrich, M. & Wise, S. P. The premotor cortex of the monkey. J. Neurosci. 2, 1329–1345 (1982).

    Article  CAS  Google Scholar 

  7. Wise, S. P., Boussaoud, D., Johnson, P. B. & Caminiti, R. Premotor and parietal cortex: corticocortical connectivity and combinatorial computations. Annu. Rev. Neurosci. 20, 25 –42 (1997).

    Article  CAS  Google Scholar 

  8. Batista, A. P., Buneo, C. A., Snyder, L. H. & Andersen, R. A. Reach plans in eye-centered coordinates. Science 285 , 257–260 (1999).

    Article  CAS  Google Scholar 

  9. Mitz, A. R., Godschalk, M. & Wise, S. P. Learning-dependent neuronal activity in the premotor cortex: activity during the acquisition of conditional motor associations. J. Neurosci. 11, 1855– 1872 (1991).

    Article  CAS  Google Scholar 

  10. Humphrey, D. R., Schmidt, E. M. & Thompson, W. D. Predicting measures of motor performance from multiple cortical spike trains. Science 170, 758– 762 (1970).

    Article  ADS  CAS  Google Scholar 

  11. Georgopoulos, A. P., Schwartz, A. B. & Kettner, R. E. Neuronal population coding of movement direction. Science 233, 1416–1419 (1986).

    Article  ADS  CAS  Google Scholar 

  12. Schwartz, A. Direct cortical representation of drawing. Science 265, 540–542 (1994).

    Article  ADS  CAS  Google Scholar 

  13. Fetz, E. E. & Cheney, P. D. Postspike facilitation of forelimb muscle activity by primate corticomotoneuronal cells. J. Neurophysiol. 44, 751–772 ( 1980).

    Article  CAS  Google Scholar 

  14. Nicolelis, A. L. et al. Simultaneous representation of tactile information by distinct primate cortical areas rely on different encoding strategies. Nature Neurosci. 1, 621–630 (1998).

    Article  CAS  Google Scholar 

  15. Laubach, M., Wessberg, J. & Nicolelis, M. A. L. Cortical ensemble activity increasingly predicts behaviour outcomes during learning of a motor task. Nature 405, 567–571 (2000).

    Article  ADS  CAS  Google Scholar 

  16. Nicolelis, M. A. L., Ghazanfar, A. A., Faggin, B. M., Votaw, S. & Oliveira, L. M. Reconstructing the engram: simultaneous, multisite, many single neuron recordings. Neuron 18 , 529–537 (1997).

    Article  CAS  Google Scholar 

  17. Stepniewska, I., Preuss, T. M. & Kaas, J. H. Architectonics, somatotopic organization, and ipsilateral cortical connections of the primary motor area (M1) of owl monkeys. J. Comp. Neurol. 330, 238–271 (1993).

    Article  CAS  Google Scholar 

  18. Preuss, T. M., Stepniewska, I. & Kaas, J. H. Movement representation in the dorsal and ventral premotor areas of owl monkeys: a microstimulation study. J. Comp. Neurol. 371, 649–676 ( 1996).

    Article  CAS  Google Scholar 

  19. Brillinger, D. R. Time Series. Data Analysis and Theory (Holden-Day, San Fransisco, 1981).

    MATH  Google Scholar 

  20. Bendat, J. S. & Piersol, A. G. Random Data. Analysis and Measurement Procedures (Wiley, New York, 1986).

    MATH  Google Scholar 

  21. Halliday, D. M. et al. The Fourier approach to the analysis of mixed time series/point process data. Theory and application to the study of physiological tremor, single motor unit discharges and electromyograms. Prog. Biophys. Mol. Biol. 64, 237–278 ( 1995).

    Article  CAS  Google Scholar 

  22. Powell, M. J. D. Restart procedures for the conjugate gradient method. Math. Program. 12, 241–254 ( 1977).

    Article  MathSciNet  Google Scholar 

  23. Ghazanfar, A. A., Stambaugh, C. R. & Nicolelis, M. A. L. Encoding of tactile stimulus location by somatosensory thalamocortical ensembles. J. Neurosci. 20, 3761–3775 (2000).

    Article  CAS  Google Scholar 

  24. Salisbury, J. K. & Srinivasan, M. A. Phantom-based haptic interaction with virtual objects. IEEE Comput. Graph. Appl. 17, 6–10 (1997 ).

    Article  Google Scholar 

  25. Ferraina, S. et al. Combination of hand and gaze signals during reaching: Activity in parietal area 7m of the monkey. J. Neurophysiol. 77, 1034–1038 (1997).

    Article  CAS  Google Scholar 

  26. Mussa-Ivaldi, F. A. Do neurons in the motor cortex encode movement direction? An alternative hypothesis. Neurosci. Lett. 91, 106– 111 (1988).

    Article  CAS  Google Scholar 

  27. Scott, S. H., Sergio, L. E. & Kalaska, J. F. Reaching movements with similar hand paths but different arm orientations. II. Activity of individual cells in dorsal premotor cortex and parietal area 5. J. Neurophysiol. 78, 2413–2426 (1997).

    Article  CAS  Google Scholar 

  28. Schmidt, E. M. Single neuron recording from motor cortex as a possible source of signals for control of external devices. Ann. Biomed. Eng. 8, 339–349 (1980).

    Article  CAS  Google Scholar 

  29. Kennedy P. R. & Bakay, R. A. Restoration of neural output from a paralyzed patient by a direct brain connection. Neuroreport 9, 1707–1711 (1998).

    Article  Google Scholar 

  30. Nicolelis, M. A. L. Hybrid brain-machine interfaces for translating thoughts into action. Nature (in the press).

Download references

Acknowledgements

This work was supported by grants from the National Institutes of Health and DARPA-ONR to M.A.L.N., J.K.C. and M.A.S., and the National Science Foundation to M.A.L.N. J.W. was supported by The Swedish Foundation for International Cooperation in Research and Higher Education, and the Swedish Medical Research Council. J.K., P.B. and M.L. were supported by NIH postdoctoral fellowships.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Miguel A. L. Nicolelis.

Supplementary information

Rights and permissions

Reprints and permissions

About this article

Cite this article

Wessberg, J., Stambaugh, C., Kralik, J. et al. Real-time prediction of hand trajectory by ensembles of cortical neurons in primates. Nature 408, 361–365 (2000). https://doi.org/10.1038/35042582

Download citation

  • Received:

  • Accepted:

  • Issue Date:

  • DOI: https://doi.org/10.1038/35042582

This article is cited by

Comments

By submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.

Search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing