Dynamics of networks of randomly connected excitatory and inhibitory spiking neurons

https://doi.org/10.1016/S0928-4257(00)01084-6Get rights and content

Abstract

Recent advances in the understanding of the dynamics of populations of spiking neurones are reviewed. These studies shed light on how a population of neurones can follow arbitrary variations in input stimuli, how the dynamics of the population depends on the type of noise, and how recurrent connections influence the dynamics. The importance of inhibitory feedback for the generation of irregularity in single cell behaviour is emphasized. Examples of computation that recurrent networks with excitatory and inhibitory cells can perform are then discussed. Maintenance of a network state as an attractor of the system is discussed as a model for working memory function, in both object and spatial modalities. These models can be used to interpret and make predictions about electrophysiological data in the awake monkey.

Introduction

The dynamics of networks of spiking neurones have received in the last decade a large amount of interest from theorists. Early studies focused on fully connected and noiseless systems [1], [28], [33], [34], [39], [48], [65], [69], or locally coupled systems [43], [64]. In these systems, the dynamics generally converges to states in which neurones behave like oscillators. The main question is to determine whether the network settles in a synchronous or asynchronous state. More recently, several groups have investigated network states in which neurones fire irregularly — a situation much closer to neuronal networks in vivo [5], [67], [70], [71]. Irregular firing tends to occur in networks with some kind of strong synaptic disorder, as in sparsely connected networks, generally in presence of strong inhibition. van Vreeswijk and Sompolinsky [70] studied the properties of ‘chaotic balanced states’ in a network of binary neurones. More recently, analytical studies have revealed the properties of the simplest spiking neuronal network models, i.e. networks of integrate-and-fire neurones [12], [15]. These networks have a very rich dynamics, even when no spatial structure and no synaptic dynamics are present. In particular, oscillatory states in which neurones fire irregularly at rates lower than the network frequency often occur. These oscillations resemble oscillatory phenoma observed in the hippocampus both in vivo [19], [25] and in vitro [18], [29].

Once the dynamics in systems with no connection structure is well understood, it is interesting to study the dynamics of networks with structure in the recurrent connections. Structured recurrent connections are known to endow neural networks with interesting computational properties. In particular, the computational role of recurrent connections has been extensively studied in recent years in the context of the visual system [9], [61], [67]. Recurrent connections are also useful to model working memory properties. Models of working memory are interesting because they can potentially bridge the gap between cellular neurophysiology and electrophysiological experiments in monkeys performing working memory tasks. In these tasks, persistent (significantly higher than spontaneous) activity is observed in cells in various areas in association cortices during the delay period during which monkeys have to maintain an item or a spatial location in working memory. This phenomenon (also called delay activity or mnemonic activity) is widely believed to subserve the cellular correlate of working memory. In the ventral stream, neurones exhibit delay activity for objects such as faces, fruits, or fractal patterns, in IT cortex [32], [47], [49], [59], perirhinal and entorhinal cortices [63], and in the ventral part of the prefrontal cortex [51], [52]. In the dorsal stream, neurones whose persistent activity is related to a memorized spatial location have been observed in parietal cortex as well as prefrontal cortex [21], [30]. Recently, implementations of old ideas about associative memory models in networks of excitatory and inhibitory spiking neurones have been proposed [5], [24]. These models, when endowed with specific synaptic structures, can function as working memory networks and show remarkable similarities with experimental data [3], [24], [76].

The first part of this paper will be concerned with results concerning the dynamics of neurones with either no recurrent connections, or random and structureless recurrent connections. Then, in the second part, examples in which structure in the recurrent connections endows the network with working memory properties will be discussed.

Section snippets

The cortical circuit model

The cortical circuit model is composed of a large number of pyramidal cells (80%) and interneurones (20%). Each cell receives a large number of recurrent connections, but connections are not all-to-all. Rather, considered networks have a connection probability of order ∼1–10%. All types of connections exist (i.e. pyramidal to pyramidal, pyramidal to interneurone, interneurone to pyramidal, interneurone to interneurone). Each cell receives a large number of long range connections from outside

White noise: f-I curve

The following results can be obtained when the input to a cell is constant, e.g. μ(t)=μ0, σ(t)=σ0:

The stationary probability density of membrane potentials is given byP0(V)=2ν0Tσ0×exp(V−μ0)2σ02V−μ0σ0θ−μ0σ0θ(u−Vr)exp(u2)du

The stationary firing probability is [57]1ν0πVτ−μ0σ0θ−μ0σ0duexp(u2)(1+erf(u))

Plotting the curve ν as a function of the mean current μ, for a given noise level σ, corresponds to plotting the f-I curve of a neurone with a given noise level. Such a f-I curve is shown in

Discussion and conclusions

The present paper has reviewed systematic investigations of various aspects of recurrent network dynamics that might shed light on some issues in theoretical or experimental neuroscience.

In particular, a long standing issue in theoretical neuroscience is whether it is possible to describe the dynamics of a population of spiking neurones by simple firing rate formulations. In traditional ‘firing rate’ formulations (e.g. [42], [75]), the dynamics of the activity of a population of cells is

Acknowledgements

I would like to thank Larry Abbott, Daniel Amit, Frances Chance, Albert Compte, Paolo del Giudice, Boris Gutkin, Stefano Fusi, Vincent Hakim, Massimo Mascaro, Maurizio Mattia, Jesper Tegner and Xiao-Jing Wang for useful discussions related to dynamics of neurones and networks.

References (76)

  • D.J. Amit et al.

    Learning internal representations in an attractor neural network with analogue neurons

    Network

    (1995)
  • D.J. Amit et al.

    Model of global spontaneous activity and local structured activity during delay periods in the cerebral cortex

    Cerebral Cortex

    (1997)
  • D.J. Amit et al.

    Dynamic learning in neural networks with material sysnapses

    Neural Comput.

    (1994)
  • D.J. Amit et al.

    Storing infinite numbers of patterns in a spin-glass model of neural networks

    Phys. Rev. Lett.

    (1985)
  • D.J. Amit et al.

    Quantitative study of attractor neural network retrieving at low spike rates I: substrate-spikes, rates and neuronal gain

    Network

    (1991)
  • R. Ben-Yishai et al.

    Theory of orientation tuning in visual cortex

    Proc. Natl. Acad. Sci. USA

    (1995)
  • O. Bernander et al.

    Synaptic background activity determines spatio-temporal integration in single pyramidal cells

    Proc. Natl Acad Sci USA

    (1991)
  • Bethge M., Pawelzik K., Rothenstein R., Tsodyks M., Noise as signal for neuronal populations, Phys. Rev. Lett. (2000)...
  • N. Brunel

    Dynamics of sparsely connected networks of excitatory and inhibitory spiking neurons

    J. Comput. Neurosci.

    (2000)
  • N. Brunel et al.

    Slow stochastic Hebbian learning of classes in recurrent neural networks

    Network

    (1998)
  • Brunel N., Chance F., Fourcaud N., Abbott L., Effects of synaptic noise and filtering on the frequency response of...
  • N. Brunel et al.

    Fast global oscillations in networks of integrate-and-fire neurons with low firing rates

    Neural Comput.

    (1999)
  • Brunel N., Wang X.J., Fast network oscillations with intermittent cell firing in recurrent circuits, (2000) (in...
  • E.H. Buhl et al.

    Cholinergic activation and tonic excitation induce persistent gamma oscillations in mouse somatosensory cortex in vitro

    J. Physiol. (Lond.)

    (1998)
  • G. Buzsaki et al.

    High frequency network oscillation in the hippocampus

    Science

    (1992)
  • M. Camperi et al.

    A model of visuospatial short-term memory in prefrontal cortex: recurrent network and cellular bistability

    J. Comput. Neurosci.

    (1998)
  • M.V. Chafee et al.

    Neuronal activity in macaque prefrontal area 8a and posterior parietal area 7ip related to memory guided saccades

    J. Neurophysiol.

    (1998)
  • Chance F., Ph.D. thesis, Brandeis University,...
  • A. Compte et al.

    Synaptic mechanisms and network dynamics underlying spatial working memory in a cortical network model

    Cerebral Cortex

    (2000)
  • J. Csicsvari et al.

    Oscillatory coupling of hippocampal pyramidal cells and interneurons in the behaving rat

    J. Neurosci.

    (1999)
  • A. Destexhe et al.

    Kinetic models of synaptic transmission

  • A. Destexhe et al.

    Impact of network activity on the integrative properties of neocortical pyramidal neurons in vivo

    J. Neurophysiol.

    (1999)
  • U. Ernst et al.

    Synchronization induced by temporal delays in pulse-coupled oscillators

    Phys. Rev. Lett.

    (1995)
  • A. Fisahn et al.

    Cholinergic induction of network oscillations at 40hz in the hippocampus in vitro

    Nature

    (1998)
  • S. Funahashi et al.

    Mnemonic coding of visual space in the monkey's dorsolateral prefrontal cortex

    J. Neurophysiol.

    (1989)
  • S. Funahashi et al.

    Neuronal activity related to saccadic eye movements in the monkey's dorsolateral prefrontal cortex

    J. Neurophysiol.

    (1991)
  • J.M. Fuster et al.

    Inferotemporal neurons distinguish and retain behaviourally relevant features of visual stimuli

    Science

    (1981)
  • W. Gerstner

    Time structure of the activity in neural network models

    Phys. Rev. E

    (1995)
  • Cited by (88)

    • Toward the Next Generation of Retinal Neuroprosthesis: Visual Computation with Spikes

      2020, Engineering
      Citation Excerpt :

      Recent studies highlight the similarity between the structure of CNNs and retinal neural circuitry [41,42], which will be discussed in later sections. The dynamics of a recurrent network [43–45], together with the diversity of synaptic dynamics and plasticities [46,47], are important for understanding the brain’s function. Here, we hypothesize that recurrent connections are also important for the retina.

    • Scale-Change Symmetry in the Rules Governing Neural Systems

      2019, iScience
      Citation Excerpt :

      This hypothesis builds upon well-established physics of critical phase transitions; at criticality, multiple material properties are expected to be power law distributed according to universal scaling laws that are insensitive to many details of the physical system (Stanley, 1987, 1971; Wilson, 1979, 1975). From theory and computational modeling work, it is clear that neural systems can be tuned into a variety of different dynamical regimes or phases (e.g., asynchronous, oscillatory, bursting), often with distinct boundaries separating different regimes (Brunel, 2000; Gautam et al., 2015; Haldeman and Beggs, 2005; Poil et al., 2012; Wang et al., 2011). Similar to physical systems, near certain regime boundaries these neural models exhibit power-law-distributed collective dynamics with the same power law exponents, despite many detailed differences among models (Gautam et al., 2015; Haldeman and Beggs, 2005; Wang et al., 2011).

    View all citing articles on Scopus
    View full text