Dynamics of networks of randomly connected excitatory and inhibitory spiking neurons
Introduction
The dynamics of networks of spiking neurones have received in the last decade a large amount of interest from theorists. Early studies focused on fully connected and noiseless systems [1], [28], [33], [34], [39], [48], [65], [69], or locally coupled systems [43], [64]. In these systems, the dynamics generally converges to states in which neurones behave like oscillators. The main question is to determine whether the network settles in a synchronous or asynchronous state. More recently, several groups have investigated network states in which neurones fire irregularly — a situation much closer to neuronal networks in vivo [5], [67], [70], [71]. Irregular firing tends to occur in networks with some kind of strong synaptic disorder, as in sparsely connected networks, generally in presence of strong inhibition. van Vreeswijk and Sompolinsky [70] studied the properties of ‘chaotic balanced states’ in a network of binary neurones. More recently, analytical studies have revealed the properties of the simplest spiking neuronal network models, i.e. networks of integrate-and-fire neurones [12], [15]. These networks have a very rich dynamics, even when no spatial structure and no synaptic dynamics are present. In particular, oscillatory states in which neurones fire irregularly at rates lower than the network frequency often occur. These oscillations resemble oscillatory phenoma observed in the hippocampus both in vivo [19], [25] and in vitro [18], [29].
Once the dynamics in systems with no connection structure is well understood, it is interesting to study the dynamics of networks with structure in the recurrent connections. Structured recurrent connections are known to endow neural networks with interesting computational properties. In particular, the computational role of recurrent connections has been extensively studied in recent years in the context of the visual system [9], [61], [67]. Recurrent connections are also useful to model working memory properties. Models of working memory are interesting because they can potentially bridge the gap between cellular neurophysiology and electrophysiological experiments in monkeys performing working memory tasks. In these tasks, persistent (significantly higher than spontaneous) activity is observed in cells in various areas in association cortices during the delay period during which monkeys have to maintain an item or a spatial location in working memory. This phenomenon (also called delay activity or mnemonic activity) is widely believed to subserve the cellular correlate of working memory. In the ventral stream, neurones exhibit delay activity for objects such as faces, fruits, or fractal patterns, in IT cortex [32], [47], [49], [59], perirhinal and entorhinal cortices [63], and in the ventral part of the prefrontal cortex [51], [52]. In the dorsal stream, neurones whose persistent activity is related to a memorized spatial location have been observed in parietal cortex as well as prefrontal cortex [21], [30]. Recently, implementations of old ideas about associative memory models in networks of excitatory and inhibitory spiking neurones have been proposed [5], [24]. These models, when endowed with specific synaptic structures, can function as working memory networks and show remarkable similarities with experimental data [3], [24], [76].
The first part of this paper will be concerned with results concerning the dynamics of neurones with either no recurrent connections, or random and structureless recurrent connections. Then, in the second part, examples in which structure in the recurrent connections endows the network with working memory properties will be discussed.
Section snippets
The cortical circuit model
The cortical circuit model is composed of a large number of pyramidal cells (80%) and interneurones (20%). Each cell receives a large number of recurrent connections, but connections are not all-to-all. Rather, considered networks have a connection probability of order ∼1–10%. All types of connections exist (i.e. pyramidal to pyramidal, pyramidal to interneurone, interneurone to pyramidal, interneurone to interneurone). Each cell receives a large number of long range connections from outside
White noise: f-I curve
The following results can be obtained when the input to a cell is constant, e.g. μ(t)=μ0, σ(t)=σ0:
The stationary probability density of membrane potentials is given by
The stationary firing probability is [57]
Plotting the curve ν as a function of the mean current μ, for a given noise level σ, corresponds to plotting the f-I curve of a neurone with a given noise level. Such a f-I curve is shown in
Discussion and conclusions
The present paper has reviewed systematic investigations of various aspects of recurrent network dynamics that might shed light on some issues in theoretical or experimental neuroscience.
In particular, a long standing issue in theoretical neuroscience is whether it is possible to describe the dynamics of a population of spiking neurones by simple firing rate formulations. In traditional ‘firing rate’ formulations (e.g. [42], [75]), the dynamics of the activity of a population of cells is
Acknowledgements
I would like to thank Larry Abbott, Daniel Amit, Frances Chance, Albert Compte, Paolo del Giudice, Boris Gutkin, Stefano Fusi, Vincent Hakim, Massimo Mascaro, Maurizio Mattia, Jesper Tegner and Xiao-Jing Wang for useful discussions related to dynamics of neurones and networks.
References (76)
- et al.
Firing frequency of integrate-and-fire neurons with finite synaptic time constants
J. Theor. Biol.
(1998) Phase-locking in weakly heterogeneous neuronal networks
Physica D
(1998)- et al.
Stability of the memory of eye position in a recurrent network of conductance-based model neurons
Neuron
(2000) - et al.
Salient features of synaptic organization in the cerebral cortex
Brain Res. Rev.
(1998) - et al.
Global competition and local cooperation in a network of neural oscillators
Physica D
(1995) - et al.
Saccades to remembered target locations: and analysis of systematic and variable errors
Vision Res.
(1994) - et al.
Excitatory and inhibitory interactions in localized populations of model neurons
Biophys. J.
(1972) - et al.
Asynchronous states in a network of pulse-coupled oscillators
Phys. Rev. E
(1993) Modeling Brain Function
(1989)- et al.
Correlations of cortical hebbian reverberations: experiment vs theory
J. Neurosci.
(1994)
Learning internal representations in an attractor neural network with analogue neurons
Network
Model of global spontaneous activity and local structured activity during delay periods in the cerebral cortex
Cerebral Cortex
Dynamic learning in neural networks with material sysnapses
Neural Comput.
Storing infinite numbers of patterns in a spin-glass model of neural networks
Phys. Rev. Lett.
Quantitative study of attractor neural network retrieving at low spike rates I: substrate-spikes, rates and neuronal gain
Network
Theory of orientation tuning in visual cortex
Proc. Natl. Acad. Sci. USA
Synaptic background activity determines spatio-temporal integration in single pyramidal cells
Proc. Natl Acad Sci USA
Dynamics of sparsely connected networks of excitatory and inhibitory spiking neurons
J. Comput. Neurosci.
Slow stochastic Hebbian learning of classes in recurrent neural networks
Network
Fast global oscillations in networks of integrate-and-fire neurons with low firing rates
Neural Comput.
Cholinergic activation and tonic excitation induce persistent gamma oscillations in mouse somatosensory cortex in vitro
J. Physiol. (Lond.)
High frequency network oscillation in the hippocampus
Science
A model of visuospatial short-term memory in prefrontal cortex: recurrent network and cellular bistability
J. Comput. Neurosci.
Neuronal activity in macaque prefrontal area 8a and posterior parietal area 7ip related to memory guided saccades
J. Neurophysiol.
Synaptic mechanisms and network dynamics underlying spatial working memory in a cortical network model
Cerebral Cortex
Oscillatory coupling of hippocampal pyramidal cells and interneurons in the behaving rat
J. Neurosci.
Kinetic models of synaptic transmission
Impact of network activity on the integrative properties of neocortical pyramidal neurons in vivo
J. Neurophysiol.
Synchronization induced by temporal delays in pulse-coupled oscillators
Phys. Rev. Lett.
Cholinergic induction of network oscillations at 40hz in the hippocampus in vitro
Nature
Mnemonic coding of visual space in the monkey's dorsolateral prefrontal cortex
J. Neurophysiol.
Neuronal activity related to saccadic eye movements in the monkey's dorsolateral prefrontal cortex
J. Neurophysiol.
Inferotemporal neurons distinguish and retain behaviourally relevant features of visual stimuli
Science
Time structure of the activity in neural network models
Phys. Rev. E
Cited by (88)
Intermittency properties in a temporal lobe epilepsy model
2023, Epilepsy and BehaviorToward the Next Generation of Retinal Neuroprosthesis: Visual Computation with Spikes
2020, EngineeringCitation Excerpt :Recent studies highlight the similarity between the structure of CNNs and retinal neural circuitry [41,42], which will be discussed in later sections. The dynamics of a recurrent network [43–45], together with the diversity of synaptic dynamics and plasticities [46,47], are important for understanding the brain’s function. Here, we hypothesize that recurrent connections are also important for the retina.
25 years of criticality in neuroscience — established results, open controversies, novel concepts
2019, Current Opinion in NeurobiologyScale-Change Symmetry in the Rules Governing Neural Systems
2019, iScienceCitation Excerpt :This hypothesis builds upon well-established physics of critical phase transitions; at criticality, multiple material properties are expected to be power law distributed according to universal scaling laws that are insensitive to many details of the physical system (Stanley, 1987, 1971; Wilson, 1979, 1975). From theory and computational modeling work, it is clear that neural systems can be tuned into a variety of different dynamical regimes or phases (e.g., asynchronous, oscillatory, bursting), often with distinct boundaries separating different regimes (Brunel, 2000; Gautam et al., 2015; Haldeman and Beggs, 2005; Poil et al., 2012; Wang et al., 2011). Similar to physical systems, near certain regime boundaries these neural models exhibit power-law-distributed collective dynamics with the same power law exponents, despite many detailed differences among models (Gautam et al., 2015; Haldeman and Beggs, 2005; Wang et al., 2011).
Complexity and 1/f slope jointly reflect brain states
2023, Scientific ReportsLogarithmically scaled, gamma distributed neuronal spiking
2023, Journal of Physiology