Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Brief Communication
  • Published:

Reinforcement learning in populations of spiking neurons

Abstract

Population coding is widely regarded as an important mechanism for achieving reliable behavioral responses despite neuronal variability. However, standard reinforcement learning slows down with increasing population size, as the global reward signal becomes less and less related to the performance of any single neuron. We found that learning speeds up with increasing population size if, in addition to global reward, feedback about the population response modulates synaptic plasticity.

This is a preview of subscription content, access via your institution

Access options

Rent or buy this article

Prices vary by article type

from$1.95

to$39.95

Prices may be subject to local taxes which are calculated during checkout

Figure 1: Scaling properties of reinforcement learning.
Figure 2: Mechanism and performance of the on-line rule.

Similar content being viewed by others

References

  1. Pouget, A., Dayan, P. & Zemel, R. Nat. Rev. Neurosci. 1, 125–132 (2000).

    Article  CAS  Google Scholar 

  2. Averbeck, B.B., Latham, P.E. & Pouget, A. Nat. Rev. Neurosci. 7, 358–366 (2006).

    Article  CAS  Google Scholar 

  3. Seung, H.S. Neuron 40, 1063–1073 (2003).

    Article  CAS  Google Scholar 

  4. Fiete, I.R. & Seung, H.S. Phys. Rev. Lett. 97, 048104 (2006).

    Article  Google Scholar 

  5. Pfister, J.P., Toyoizumi, T., Barber, D. & Gerstner, W. Neural Comput. 18, 1318–1348 (2006).

    Article  Google Scholar 

  6. Florian, R.V. Neural Comput. 19, 1468–1502 (2007).

    Article  Google Scholar 

  7. Izhikevich, E.M. Cereb. Cortex 17, 2443–2452 (2007).

    Article  Google Scholar 

  8. Wirth, S. et al. Science 300, 1578–1581 (2003).

    Article  CAS  Google Scholar 

  9. Rumelhart, D.E., Hinton, G.E. & Williams, R.J.D. Parallel Distributed Processing, Vol. 1 (eds. Rumelhart, E. & McClelland, J.L.) (MIT Press, Cambridge, Massachusetts, 1986).

    Google Scholar 

  10. Foehring, R.C. & Lorenzon, N.M. Can. J. Exp. Psychol. 53, 45–61 (1999).

    Article  CAS  Google Scholar 

  11. Centonze, D., Gubellini, P., Pisani, A., Bernardi, G. & Calabresi, P. Rev. Neurosci. 14, 207–216 (2003).

    Article  CAS  Google Scholar 

  12. Gütig, R. & Sompolinsky, H. Nat. Neurosci. 9, 420–428 (2006).

    Article  Google Scholar 

Download references

Acknowledgements

We thank W. Gerstner for helpful comments on the manuscript.

Author information

Authors and Affiliations

Authors

Contributions

R.U. designed and executed the research in close interaction with W.S. The manuscript was written jointly.

Corresponding author

Correspondence to Walter Senn.

Supplementary information

Supplementary Text and Figures

Supplementary Figure 1 and Supplementary Methods (PDF 553 kb)

Rights and permissions

Reprints and permissions

About this article

Cite this article

Urbanczik, R., Senn, W. Reinforcement learning in populations of spiking neurons. Nat Neurosci 12, 250–252 (2009). https://doi.org/10.1038/nn.2264

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/nn.2264

This article is cited by

Search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing