2007 Special IssueFrom memory-based decisions to decision-based movements: A model of interval discrimination followed by action selection
Introduction
‘Decision making followed by action selection’ is one of the most demanding and recurring event in our day to day lives (Latham & Dayan, 2005). For example, to choose a drink from a vending machine, you need to browse through the possible choices, hold the interesting ones in your working memory, decide which of these choices you want to buy, and finally press the button corresponding to your choice. The interval discrimination task (Machens et al., 2005, Rainer et al., 2004) is one of the classical experimental paradigms that is employed to study working memory and decision making. The experiment typically involves four phases, viz. the initial loading (L) of the first stimulus, maintaining (M) this stimulus in working memory till the subsequent stimulus is presented, making a binary decision (D), and finally acting (A) on this decision, usually by pressing one of the two buttons corresponding to the binary choice.
The precise computational and biophysical mechanism(s) through which the brain is able to execute this load-maintain-decide-act (LMDA) sequence is not understood. Several theoretical and modelling studies have tried to look at segregated phases of this sequence and give possible explanations for their working (Menshet al., 2004, Seung, Lee, Reis, and Tank, 2000b, Singh and Eliasmith, 2006). Typically neurocomputational models of working memory fail to address how the decisions made by neurons in the prefrontal cortex (PFC), are converted into motor commands, which are executed by the sensorimotor system (Durstewitz et al., 2000, Machens et al., 2005, Miller and Wang, 2006). Similarly, models for computational motor control tend to ignore the first three phases (LMD), that are responsible for generation of motor commands (Joshi and Maass, 2004, Joshi and Maass, 2005, Todorov, 2000).
Several interesting modelling approaches for tasks involving working memory and decision making have been proposed recently (Machens et al., 2005, Miller et al., 2003, Miller and Wang, 2006). These models propose different mechanisms e.g. precise tuning of mutual inhibition (Machens et al., 2005), fine-tuning of a heterogeneous recurrent network (Miller et al., 2003), using an integral feedback signal for inhibitory control (Miller & Wang, 2006), to obtain the persistent neural activity which in turn stores information in the working memory. Despite existing evidence that shows synaptic learning as a responsible mechanism for working memory related tasks (Rainer et al., 2004), all the models described above use static (no learning involved) neural circuits.
This article proposes a neurocomputational architecture that uses synaptic learning mechanisms (simply linear regression), and is able to integrate the four phases (LMDA) involved in the process of action selection in presence of a decision, into a unified computational framework. Essentially the neural model described here integrates two distinct cortical functions viz., working memory and decision making carried out by the neurons in PFC, and subsequent action selection executed by the sensorimotor system. More precisely, it is demonstrated that delayed-decision tasks that are followed by action selection can be solved, if feedback from trained linear readouts is provided to generic neural microcircuits whose internal dynamics have not been optimized for any particular computational task. Two classical experimental paradigms for interval-discrimination task are modeled using different mechanisms for encoding external sensory inputs. For comparison with earlier models of working memory, the unified framework is used to build a spiking neural network model of two-interval discrimination (Machens et al., 2005). Additionally, to demonstrate that this computational paradigm is task-independent, robust to how the external sensory inputs are encoded, and is capable of integrating the A phase, another spiking neural network model is presented for the delayed match-to-sample task (Rainer et al., 2004), followed by an arm movement to the decided goal position.
The core principles behind the working of this model make the assumption that the cortex can be thought of as an ultra-high dimensional dynamic system, where the afferent inputs arriving from the thalamus and the recurrent cortical feedbacks are churned in a nonlinear way to obtain a high-dimensional projection of the low-dimensional input space. Preceding work has demonstrated that such high dimensional transient dynamics endows the neural circuit with analog fading memory (see Appendix) that can provide the circuit with enough computational power for performing open-loop sensory processing tasks (Buonomano and Merzenich, 1995, Maass and Markram, 2004, Maass et al., 2002).
Analog fading memory by itself is not powerful enough to render the circuits the power to hold information in working memory. The obvious reason being that such memory has an upper limit in the order of tens of milliseconds, depending on the time constants of synapses and neurons in the neural circuit (Maass et al., 2002), whereas typically working memory holds information in the order of seconds. Recent results show that precisely tuned synaptic feedback can be a possible mechanism for maintaining persistent memory (Seung, 1996, Seung et al., 2000a), and that feedback from trained readout neurons can induce multiple coexisting “partial attractors” in the circuit dynamics (Maass et al., 2006, Maass et al., 2007). These results are further extended here to demonstrate that even in the presence of feedback noise, such “partial attractor” states can be maintained by generic neural circuits on the time scales of several seconds, that is obviously a requirement for tasks involving working memory. The results presented in this article indicate that simple linear readouts from generic neural microcircuit models, that send their output as a feedback signal to the circuit, can be a plausible model of how the interval discrimination task is executed and a subsequent action is chosen.
A preliminary version of some results from this article (a model for just working memory and decision making without subsequent action selection) was presented at a conference (Joshi, 2006).
Section snippets
Generic neural microcircuit models
In contrast to artificial neural networks, neural microcircuits in biological organisms are composed of diverse components such as different types of spiking neurons and dynamic synapses, each endowed with an inherently complex dynamics of its own. This poses a challenge to construct neural circuits out of biologically realistic computational units that solve specific computational problems, e.g. decision making or motor control. In fact, decision making and motor control are particularly
Results
The experiments for tasks described in this article consisted of two distinct phases. In the first phase linear readouts that received inputs from the circuit, were trained in an open-loop fashion to perform diverse computational tasks (e.g. to make decisions, to generate a motor command or to predict the joint angles during the arm movement). During open-loop training, the feedback from readouts performing diverse computational tasks were simulated by a noisy version of their target output
Discussion
This article describes a new neurocomputational paradigm that uses synaptic learning mechanisms to present a unified model for decision making followed by action selection. The model is unified in the sense that a single learning algorithm (simple linear regression) is used to train readouts that make decisions based on the activity of a model PFC circuit, and the readouts that generate motor commands or predict the joint angles based on the activity of the model M1 circuit. Biologically
Acknowledgments
Helpful comments from Wolfgang Maass, Herbert Jaeger, Carlos Brody and anonymous reviewers on the draft version of this manuscript are gratefully acknowledged. Written under partial support by the Austrian Science Fund FWF, project #P17229-N04 and project #FP6-015879 (FACETS) of the European Union.
References (48)
- et al.
Neural basis of motor control and its cognitive implications
Trends in Cognitive Sciences
(1998) - et al.
Synapses as dynamic memory buffers
Neural Networks
(2002) - et al.
On the computational power of recurrent circuits of spiking neurons
Journal of Computer and System Sciences
(2004) - et al.
Spontaneous eye movements in goldfish: Oculomotor integrator performance, plasticity, and dependence on visual feedback
Vision Research
(2004) - et al.
Stability of the memory of eye position in a recurrent network of conductance-based model neurons
Neuron
(2000) - et al.
Motor cortical activity preceding a memorized movement trajectory with an orthogonal bend
Experimental Brain Research
(1993) - et al.
Timing and neural encoding of somatosensory parametric working memory in macaque prefrontal cortex
Cerebral Cortex
(2003) - et al.
Temporal information transformed into a spatial code by a neural network with realistic properties
Science
(1995) Introduction to robotics: Mechanics and control
(1989)The nature of explanation
(1943)