Trends in Neurosciences
Volume 22, Issue 4, 1 April 1999, Pages 146-151
Journal home page for Trends in Neurosciences

Viewpoint
Is the short-latency dopamine response too short to signal reward error?

https://doi.org/10.1016/S0166-2236(98)01373-3Get rights and content

Abstract

Unexpected stimuli that are behaviourally significant have the capacity to elicit a short-latency, short-duration burst of firing in mesencephalic dopaminergic neurones. An influential interpretation of the experimental data that characterize this response proposes that dopaminergic neurones have a crucial role in reinforcement learning because they signal error in the prediction of future reward. In this article we propose a different functional role for this ‘short-latency dopamine response’ in the mechanisms that underlie associative learning. We suggest that the initial burst of dopaminergic-neurone firing could represent an essential component in the process of switching attentional and behavioural selections to unexpected, behaviourally important stimuli. This switching response could be a crucial prerequisite for associative learning and might be part of a general short-latency response that is mediated by catecholamines and prepares the organism for an appropriate reaction to biologically significant events.

Any act which in a given situation produces satisfaction becomes associated with that situation so that when the situation recurs the act is more likely than before to recur also. E.L. Thorndike (1911) 1 .

Section snippets

The short-latency dopamine response

The essential characteristics of the short-latency dopamine response have been considered in several recent reviews2, 5, 8, 10, 11 so we will provide only a brief summary of them here. Typically, dopaminergic neurones of several species exhibit a burst of impulses immediately following unexpected (salient) events, which include sudden novel stimuli, intense sensory stimuli, primary rewards and arbitrary stimuli that are classically conditioned by association with primary rewards (Fig. 1). The

The short-latency dopamine response and learning

A particularly arresting feature of the short-latency dopamine response is its propensity to change over time with repeated stimulus presentation and with changes in experimental context (for reviews see Refs 2, 8, 12). For example, the response elicited by a novel event habituates rapidly when the stimulus is repeated in the absence of behaviourally relevant consequences (reward or punishment). In this case, habituation of the neuronal response appears to correlate with the diminishing

The short-latency dopamine response to non-reward stimuli

The response of dopaminergic neurones to non-reward stimuli is related to precise experimental conditions. On the one hand, it has been shown in the monkey that the dopaminergic neurones react with a short burst of pulses when its hand touches a hidden morsel of food, but not when it touches similarly shaped non-food objects8, 12. It is also claimed that non-noxious, primary aversive stimuli, such as air puffs to the hand or drops of saline to the mouth, together with conditioned visual and

Dopamine as an ‘effective-reinforcement signal’

On the basis of the characteristics already mentioned, Schultz and his colleagues2, 5, 8, 12 have proposed that the short-latency dopamine response is related to the reinforcing function of rewards. It is important to note that only unexpected rewards (or punishments) lead to the acquisition of new conditioned responses14 – predicted reinforcement serves to maintain already established conditioned behaviour but is unable to promote the learning of new conditioned responses. The discovery that

An alternative hypothesis

We, however, would like to consider an alternative interpretation of the data summarized above. Our opinion is formed on the basis of a rather different assumption about the basic function of the short-latency dopamine response. Recently, we made a general proposal that vertebrate basal ganglia have evolved as a centralized selection device, specialized to resolve conflicts between multiple sub-systems that compete for access to limited motor or cognitive resources16, 17. Within this framework,

The response to novel stimuli

The discovery that unexpected novel or intense stimuli always elicit a robust short-latency dopamine response (Fig. 1A) is consistent with the idea that this signal could have a role in the processes of terminating current selections and opening new ones. In contrast, the response to novel stimuli causes some difficulty for the effective-reinforcement hypothesis5. Presumably, a sudden novel event could be ‘good’ (directly or indirectly linked with reward), ‘bad’ (directly or indirectly linked

Response transfer to conditioned stimuli

Why do the dopaminergic cells appear able to distinguish stimuli that predict reward? It has been known for many years that evoked responses in primary sensory areas of the brain are influenced by reinforcement outcome31. For example, Wurtz and Goldberg28 showed that non-reinforced presentation of light spots to a monkey led quickly to habituation of the neuronal responses within sensory receptive fields of collicular neurones. However, by associating a stimulus with reward, the previously

Response generalization

It has been reported that a monkey will reliably interrupt its current behaviour and orient to the opening of a never-baited box immediately adjacent to one that provides reward32. The dopaminergic neurones also respond consistently to the ‘never-rewarded’ stimulus. The generalization of classically conditioned activity, which is elicited in primary sensory networks that relay input to the dopaminergic neurones, could underlie these observations. However, if the function of dopaminergic

Suppression of dopaminergic-neurone activity by reward omission

The brief pause in dopaminergic-neurone activity when an anticipated reward is not delivered is one of the important pieces of evidence used to support the effective-reinforcement hypothesis5 (Fig. 1D). However, there are also problems both with the generality of this suggestion and with its likely mechanism. First, an action can have a negative outcome not only if an expected reward fails to materialize but also if it leads to an unexpected aversive or punishing stimulus. In both cases, the

Do dopaminergic neurones respond to aversive stimuli?

This is a crucial question that can separate the reinforcement-error hypothesis and the most general version of the switching hypotheses (that all stimuli with biological significance facilitate the re-allocation of limited processing resources by a mechanism that involves the short-latency dopamine response). As primary and conditioned aversive stimuli are particularly effective in terminating current behaviour and attracting attentional and behavioural resources, the general switching

Switching: a consistent theme in dopamine research

The current proposal links well to experimental literature suggesting, over several decades, that dopamine has an important role in behavioural switching. It has been shown that a range of treatments that alter levels of dopamine-mediated neurotransmission affect various aspects of selection and behavioural switching in a variety of experimental paradigms19, 24, 25, 26. Depending on the site and nature of the intervention, these effects include changes in the dominance relations between

The short-latency dopamine response and associative learning

The switching hypothesis outlined above raises the possibility that the short-latency dopamine response has a more general role in associative learning than that proposed by the effective-reinforcement model5. The disruption of processes that link salient events with resource selection could explain why experimental manipulations of dopamine-mediated neurotransmission can effect both positively and negatively reinforced associative learning33, and associative learning in the absence of

A general catecholamine response to salient stimuli?

The current proposal that short-latency dopamine responses contribute to the behavioural switching initiated by salient events has several features in common with suggestions that concern the function of the noradrenergic neurones of the locus coeruleus40. Dopaminergic neurones and noradrenergic neurones show strikingly similar responses to salient events (for reviews see 8, 40), both having a slow spontaneous rate of discharge (1–8 spikes/s) that is interrupted by a short-latency (∼50–100 ms),

Acknowledgements

Without any implication of assent to the ideas expressed in this article, the authors thank Wolfram Schultz, Jeff Wickens, Okihide Hikosaka, Gordon Arbuthnott and Ann Graybiel for their helpful, friendly and constructive discussion of the material presented in this paper.

References (40)

  • R.J. Beninger et al.

    Neurosci. Biobehav. Rev.

    (1998)
  • D.G. Beiser

    Curr. Opin. Neurobiol.

    (1997)
  • P.G. Overton et al.

    Brain Res. Rev.

    (1997)
  • W. Schultz

    Curr. Opin. Neurobiol.

    (1997)
  • P. Redgrave

    Neuroscience

    (1999)
  • J.W. Mink

    Prog. Neurobiol.

    (1996)
  • A.R. Cools

    Behav. Brain Res.

    (1980)
  • R.D. Oades

    Neurosci. Biobehav. Rev.

    (1985)
  • A.K. Moschovakis

    Curr. Opin. Neurobiol.

    (1996)
  • J.D. Salamone

    Neurosci. Biobehav. Rev.

    (1997)
  • J.D. Salamone

    Behav. Brain Res.

    (1994)
  • J.C. Horvitz

    Brain Res.

    (1997)
  • A.M.J. Young

    Neuroscience

    (1998)
  • G. Aston-Jones

    Prog. Brain Res.

    (1991)
  • W. Schultz

    J. Neurophysiol.

    (1998)
  • J. Wickens

    Comput. Neural Syst.

    (1997)
  • W. Schultz

    Science

    (1997)
  • M. Kimura et al.

    Eur. Neurol.

    (1997)
  • Cited by (421)

    View all citing articles on Scopus
    View full text