Abstract
Population coding is widely regarded as an important mechanism for achieving reliable behavioral responses despite neuronal variability. However, standard reinforcement learning slows down with increasing population size, as the global reward signal becomes less and less related to the performance of any single neuron. We found that learning speeds up with increasing population size if, in addition to global reward, feedback about the population response modulates synaptic plasticity.
This is a preview of subscription content, access via your institution
Access options
Subscribe to this journal
Receive 12 print issues and online access
$209.00 per year
only $17.42 per issue
Rent or buy this article
Prices vary by article type
from$1.95
to$39.95
Prices may be subject to local taxes which are calculated during checkout
Similar content being viewed by others
References
Pouget, A., Dayan, P. & Zemel, R. Nat. Rev. Neurosci. 1, 125–132 (2000).
Averbeck, B.B., Latham, P.E. & Pouget, A. Nat. Rev. Neurosci. 7, 358–366 (2006).
Seung, H.S. Neuron 40, 1063–1073 (2003).
Fiete, I.R. & Seung, H.S. Phys. Rev. Lett. 97, 048104 (2006).
Pfister, J.P., Toyoizumi, T., Barber, D. & Gerstner, W. Neural Comput. 18, 1318–1348 (2006).
Florian, R.V. Neural Comput. 19, 1468–1502 (2007).
Izhikevich, E.M. Cereb. Cortex 17, 2443–2452 (2007).
Wirth, S. et al. Science 300, 1578–1581 (2003).
Rumelhart, D.E., Hinton, G.E. & Williams, R.J.D. Parallel Distributed Processing, Vol. 1 (eds. Rumelhart, E. & McClelland, J.L.) (MIT Press, Cambridge, Massachusetts, 1986).
Foehring, R.C. & Lorenzon, N.M. Can. J. Exp. Psychol. 53, 45–61 (1999).
Centonze, D., Gubellini, P., Pisani, A., Bernardi, G. & Calabresi, P. Rev. Neurosci. 14, 207–216 (2003).
Gütig, R. & Sompolinsky, H. Nat. Neurosci. 9, 420–428 (2006).
Acknowledgements
We thank W. Gerstner for helpful comments on the manuscript.
Author information
Authors and Affiliations
Contributions
R.U. designed and executed the research in close interaction with W.S. The manuscript was written jointly.
Corresponding author
Supplementary information
Supplementary Text and Figures
Supplementary Figure 1 and Supplementary Methods (PDF 553 kb)
Rights and permissions
About this article
Cite this article
Urbanczik, R., Senn, W. Reinforcement learning in populations of spiking neurons. Nat Neurosci 12, 250–252 (2009). https://doi.org/10.1038/nn.2264
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1038/nn.2264
This article is cited by
-
Adaptive control of synaptic plasticity integrates micro- and macroscopic network function
Neuropsychopharmacology (2023)
-
Multi-context blind source separation by error-gated Hebbian rule
Scientific Reports (2019)
-
An unsupervised neuromorphic clustering algorithm
Biological Cybernetics (2019)
-
Control of synaptic plasticity in deep cortical networks
Nature Reviews Neuroscience (2018)
-
Random synaptic feedback weights support error backpropagation for deep learning
Nature Communications (2016)