Trends in Neurosciences
ViewpointIs the short-latency dopamine response too short to signal reward error?
Section snippets
The short-latency dopamine response
The essential characteristics of the short-latency dopamine response have been considered in several recent reviews2, 5, 8, 10, 11 so we will provide only a brief summary of them here. Typically, dopaminergic neurones of several species exhibit a burst of impulses immediately following unexpected (salient) events, which include sudden novel stimuli, intense sensory stimuli, primary rewards and arbitrary stimuli that are classically conditioned by association with primary rewards (Fig. 1). The
The short-latency dopamine response and learning
A particularly arresting feature of the short-latency dopamine response is its propensity to change over time with repeated stimulus presentation and with changes in experimental context (for reviews see Refs 2, 8, 12). For example, the response elicited by a novel event habituates rapidly when the stimulus is repeated in the absence of behaviourally relevant consequences (reward or punishment). In this case, habituation of the neuronal response appears to correlate with the diminishing
The short-latency dopamine response to non-reward stimuli
The response of dopaminergic neurones to non-reward stimuli is related to precise experimental conditions. On the one hand, it has been shown in the monkey that the dopaminergic neurones react with a short burst of pulses when its hand touches a hidden morsel of food, but not when it touches similarly shaped non-food objects8, 12. It is also claimed that non-noxious, primary aversive stimuli, such as air puffs to the hand or drops of saline to the mouth, together with conditioned visual and
Dopamine as an ‘effective-reinforcement signal’
On the basis of the characteristics already mentioned, Schultz and his colleagues2, 5, 8, 12 have proposed that the short-latency dopamine response is related to the reinforcing function of rewards. It is important to note that only unexpected rewards (or punishments) lead to the acquisition of new conditioned responses14 – predicted reinforcement serves to maintain already established conditioned behaviour but is unable to promote the learning of new conditioned responses. The discovery that
An alternative hypothesis
We, however, would like to consider an alternative interpretation of the data summarized above. Our opinion is formed on the basis of a rather different assumption about the basic function of the short-latency dopamine response. Recently, we made a general proposal that vertebrate basal ganglia have evolved as a centralized selection device, specialized to resolve conflicts between multiple sub-systems that compete for access to limited motor or cognitive resources16, 17. Within this framework,
The response to novel stimuli
The discovery that unexpected novel or intense stimuli always elicit a robust short-latency dopamine response (Fig. 1A) is consistent with the idea that this signal could have a role in the processes of terminating current selections and opening new ones. In contrast, the response to novel stimuli causes some difficulty for the effective-reinforcement hypothesis5. Presumably, a sudden novel event could be ‘good’ (directly or indirectly linked with reward), ‘bad’ (directly or indirectly linked
Response transfer to conditioned stimuli
Why do the dopaminergic cells appear able to distinguish stimuli that predict reward? It has been known for many years that evoked responses in primary sensory areas of the brain are influenced by reinforcement outcome31. For example, Wurtz and Goldberg28 showed that non-reinforced presentation of light spots to a monkey led quickly to habituation of the neuronal responses within sensory receptive fields of collicular neurones. However, by associating a stimulus with reward, the previously
Response generalization
It has been reported that a monkey will reliably interrupt its current behaviour and orient to the opening of a never-baited box immediately adjacent to one that provides reward32. The dopaminergic neurones also respond consistently to the ‘never-rewarded’ stimulus. The generalization of classically conditioned activity, which is elicited in primary sensory networks that relay input to the dopaminergic neurones, could underlie these observations. However, if the function of dopaminergic
Suppression of dopaminergic-neurone activity by reward omission
The brief pause in dopaminergic-neurone activity when an anticipated reward is not delivered is one of the important pieces of evidence used to support the effective-reinforcement hypothesis5 (Fig. 1D). However, there are also problems both with the generality of this suggestion and with its likely mechanism. First, an action can have a negative outcome not only if an expected reward fails to materialize but also if it leads to an unexpected aversive or punishing stimulus. In both cases, the
Do dopaminergic neurones respond to aversive stimuli?
This is a crucial question that can separate the reinforcement-error hypothesis and the most general version of the switching hypotheses (that all stimuli with biological significance facilitate the re-allocation of limited processing resources by a mechanism that involves the short-latency dopamine response). As primary and conditioned aversive stimuli are particularly effective in terminating current behaviour and attracting attentional and behavioural resources, the general switching
Switching: a consistent theme in dopamine research
The current proposal links well to experimental literature suggesting, over several decades, that dopamine has an important role in behavioural switching. It has been shown that a range of treatments that alter levels of dopamine-mediated neurotransmission affect various aspects of selection and behavioural switching in a variety of experimental paradigms19, 24, 25, 26. Depending on the site and nature of the intervention, these effects include changes in the dominance relations between
The short-latency dopamine response and associative learning
The switching hypothesis outlined above raises the possibility that the short-latency dopamine response has a more general role in associative learning than that proposed by the effective-reinforcement model5. The disruption of processes that link salient events with resource selection could explain why experimental manipulations of dopamine-mediated neurotransmission can effect both positively and negatively reinforced associative learning33, and associative learning in the absence of
A general catecholamine response to salient stimuli?
The current proposal that short-latency dopamine responses contribute to the behavioural switching initiated by salient events has several features in common with suggestions that concern the function of the noradrenergic neurones of the locus coeruleus40. Dopaminergic neurones and noradrenergic neurones show strikingly similar responses to salient events (for reviews see 8, 40), both having a slow spontaneous rate of discharge (1–8 spikes/s) that is interrupted by a short-latency (∼50–100 ms),
Acknowledgements
Without any implication of assent to the ideas expressed in this article, the authors thank Wolfram Schultz, Jeff Wickens, Okihide Hikosaka, Gordon Arbuthnott and Ann Graybiel for their helpful, friendly and constructive discussion of the material presented in this paper.
References (40)
- et al.
Neurosci. Biobehav. Rev.
(1998) Curr. Opin. Neurobiol.
(1997)- et al.
Brain Res. Rev.
(1997) Curr. Opin. Neurobiol.
(1997)Neuroscience
(1999)Prog. Neurobiol.
(1996)Behav. Brain Res.
(1980)Neurosci. Biobehav. Rev.
(1985)Curr. Opin. Neurobiol.
(1996)Neurosci. Biobehav. Rev.
(1997)
Behav. Brain Res.
Brain Res.
Neuroscience
Prog. Brain Res.
J. Neurophysiol.
Comput. Neural Syst.
Science
Eur. Neurol.
Cited by (421)
The effect of target-related and target-irrelevant novel stimuli on response behaviour
2023, Acta Psychologica