Elsevier

Neural Networks

Volume 20, Issue 3, April 2007, Pages 298-311
Neural Networks

2007 Special Issue
From memory-based decisions to decision-based movements: A model of interval discrimination followed by action selection

https://doi.org/10.1016/j.neunet.2007.04.015Get rights and content

Abstract

The interval discrimination task is a classical experimental paradigm that is employed to study working memory and decision making and typically involves four phases. First, the subject receives a stimulus, then holds it in the working memory, then makes a decision by comparing it with another stimulus and finally acts on this decision, usually by pressing one of the two buttons corresponding to the binary decision. This article demonstrates that simple linear readouts from generic neural microcircuits that send feedback of their activity to the circuit, can be trained using identical learning mechanisms to perform quite separate tasks of decision making and generation of subsequent motor commands. In this sense, the neurocomputational algorithm presented here is able to integrate the four computational stages into a single unified framework. The algorithm is tested using two-interval discrimination and delayed-match-to-sample experimental paradigms as benchmarks.

Introduction

‘Decision making followed by action selection’ is one of the most demanding and recurring event in our day to day lives (Latham & Dayan, 2005). For example, to choose a drink from a vending machine, you need to browse through the possible choices, hold the interesting ones in your working memory, decide which of these choices you want to buy, and finally press the button corresponding to your choice. The interval discrimination task (Machens et al., 2005, Rainer et al., 2004) is one of the classical experimental paradigms that is employed to study working memory and decision making. The experiment typically involves four phases, viz. the initial loading (L) of the first stimulus, maintaining (M) this stimulus in working memory till the subsequent stimulus is presented, making a binary decision (D), and finally acting (A) on this decision, usually by pressing one of the two buttons corresponding to the binary choice.

The precise computational and biophysical mechanism(s) through which the brain is able to execute this load-maintain-decide-act (LMDA) sequence is not understood. Several theoretical and modelling studies have tried to look at segregated phases of this sequence and give possible explanations for their working (Menshet al., 2004, Seung, Lee, Reis, and Tank, 2000b, Singh and Eliasmith, 2006). Typically neurocomputational models of working memory fail to address how the decisions made by neurons in the prefrontal cortex (PFC), are converted into motor commands, which are executed by the sensorimotor system (Durstewitz et al., 2000, Machens et al., 2005, Miller and Wang, 2006). Similarly, models for computational motor control tend to ignore the first three phases (LMD), that are responsible for generation of motor commands (Joshi and Maass, 2004, Joshi and Maass, 2005, Todorov, 2000).

Several interesting modelling approaches for tasks involving working memory and decision making have been proposed recently (Machens et al., 2005, Miller et al., 2003, Miller and Wang, 2006). These models propose different mechanisms e.g. precise tuning of mutual inhibition (Machens et al., 2005), fine-tuning of a heterogeneous recurrent network (Miller et al., 2003), using an integral feedback signal for inhibitory control (Miller & Wang, 2006), to obtain the persistent neural activity which in turn stores information in the working memory. Despite existing evidence that shows synaptic learning as a responsible mechanism for working memory related tasks (Rainer et al., 2004), all the models described above use static (no learning involved) neural circuits.

This article proposes a neurocomputational architecture that uses synaptic learning mechanisms (simply linear regression), and is able to integrate the four phases (LMDA) involved in the process of action selection in presence of a decision, into a unified computational framework. Essentially the neural model described here integrates two distinct cortical functions viz., working memory and decision making carried out by the neurons in PFC, and subsequent action selection executed by the sensorimotor system. More precisely, it is demonstrated that delayed-decision tasks that are followed by action selection can be solved, if feedback from trained linear readouts is provided to generic neural microcircuits whose internal dynamics have not been optimized for any particular computational task. Two classical experimental paradigms for interval-discrimination task are modeled using different mechanisms for encoding external sensory inputs. For comparison with earlier models of working memory, the unified framework is used to build a spiking neural network model of two-interval discrimination (Machens et al., 2005). Additionally, to demonstrate that this computational paradigm is task-independent, robust to how the external sensory inputs are encoded, and is capable of integrating the A phase, another spiking neural network model is presented for the delayed match-to-sample task (Rainer et al., 2004), followed by an arm movement to the decided goal position.

The core principles behind the working of this model make the assumption that the cortex can be thought of as an ultra-high dimensional dynamic system, where the afferent inputs arriving from the thalamus and the recurrent cortical feedbacks are churned in a nonlinear way to obtain a high-dimensional projection of the low-dimensional input space. Preceding work has demonstrated that such high dimensional transient dynamics endows the neural circuit with analog fading memory (see Appendix) that can provide the circuit with enough computational power for performing open-loop sensory processing tasks (Buonomano and Merzenich, 1995, Maass and Markram, 2004, Maass et al., 2002).

Analog fading memory by itself is not powerful enough to render the circuits the power to hold information in working memory. The obvious reason being that such memory has an upper limit in the order of tens of milliseconds, depending on the time constants of synapses and neurons in the neural circuit (Maass et al., 2002), whereas typically working memory holds information in the order of seconds. Recent results show that precisely tuned synaptic feedback can be a possible mechanism for maintaining persistent memory (Seung, 1996, Seung et al., 2000a), and that feedback from trained readout neurons can induce multiple coexisting “partial attractors” in the circuit dynamics (Maass et al., 2006, Maass et al., 2007). These results are further extended here to demonstrate that even in the presence of feedback noise, such “partial attractor” states can be maintained by generic neural circuits on the time scales of several seconds, that is obviously a requirement for tasks involving working memory. The results presented in this article indicate that simple linear readouts from generic neural microcircuit models, that send their output as a feedback signal to the circuit, can be a plausible model of how the interval discrimination task is executed and a subsequent action is chosen.

A preliminary version of some results from this article (a model for just working memory and decision making without subsequent action selection) was presented at a conference (Joshi, 2006).

Section snippets

Generic neural microcircuit models

In contrast to artificial neural networks, neural microcircuits in biological organisms are composed of diverse components such as different types of spiking neurons and dynamic synapses, each endowed with an inherently complex dynamics of its own. This poses a challenge to construct neural circuits out of biologically realistic computational units that solve specific computational problems, e.g. decision making or motor control. In fact, decision making and motor control are particularly

Results

The experiments for tasks described in this article consisted of two distinct phases. In the first phase linear readouts that received inputs from the circuit, were trained in an open-loop fashion to perform diverse computational tasks (e.g. to make decisions, to generate a motor command or to predict the joint angles during the arm movement). During open-loop training, the feedback from readouts performing diverse computational tasks were simulated by a noisy version of their target output

Discussion

This article describes a new neurocomputational paradigm that uses synaptic learning mechanisms to present a unified model for decision making followed by action selection. The model is unified in the sense that a single learning algorithm (simple linear regression) is used to train readouts that make decisions based on the activity of a model PFC circuit, and the readouts that generate motor commands or predict the joint angles based on the activity of the model M1 circuit. Biologically

Acknowledgments

Helpful comments from Wolfgang Maass, Herbert Jaeger, Carlos Brody and anonymous reviewers on the draft version of this manuscript are gratefully acknowledged. Written under partial support by the Austrian Science Fund FWF, project #P17229-N04 and project #FP6-015879 (FACETS) of the European Union.

References (48)

  • D. Durstewitz et al.

    Neurocomputational models of working memory

    Nature Neuroscience

    (2000)
  • T. Flash et al.

    The coordination of arm movements: An experimentally confirmed mathematical model

    Journal of Neuroscience

    (1965)
  • J.M. Fuster

    Unit activity in prefrontal cortex during delayed-response performance: Neuronal correlates of transient memory

    Journal of Neurophysiology

    (1973)
  • J.M. Fuster et al.

    Cross-modal and cross-temporal association in neurons of frontal cortex

    Nature

    (2000)
  • A.P. Georgopoulos et al.

    Spatial coding of movement: A hypothesis concerning the coding of movement direction by motor cortical populations

    Experimental Brain Research

    (1983)
  • A.P. Georgopoulos et al.

    Primate motor cortex and free arm movements to visual targets in three-dimensional space. II. Coding of the direction of movement by a neuronal population

    Journal of Neuroscience

    (1988)
  • A.P. Georgopoulos et al.

    Mental rotation of the neuronal population vector

    Science

    (1989)
  • A.P. Georgopoulos et al.

    Neuronal population coding of movement direction

    Science

    (1986)
  • A. Gupta et al.

    Organizing principles for a diversity of GABAergic interneurons and synapses in the neocortex

    Science

    (2000)
  • P. Joshi

    Modeling working memory and decision making using generic neural microcircuits

  • P. Joshi et al.

    Movement generation and control with generic neural microcircuits

  • P. Joshi et al.

    Movement generation with circuits of spiking neurons

    Neural Computation

    (2005)
  • R.E. Kettner et al.

    Primate motor cortex and free arm movements to visual targets in three-dimensional space. III. Positional gradients and population coding of movement direction from various movement origins

    Journal of Neuroscience

    (1988)
  • W. Kruse et al.

    Temporal relation of population activity in visual areas MT/MST and in primary motor cortex during visually guided tracking movements

    Cerebral Cortex

    (2002)
  • Cited by (0)

    View full text