Elsevier

NeuroImage

Volume 50, Issue 3, 15 April 2010, Pages 1168-1176
NeuroImage

Retest reliability of reward-related BOLD signals

https://doi.org/10.1016/j.neuroimage.2010.01.036Get rights and content

Abstract

Reward processing is a central component of learning and decision making. Functional magnetic resonance imaging (fMRI) has contributed essentially to our understanding of reward processing in humans. The strength of reward-related brain responses might prove as a valuable marker for, or correlate of, individual preferences or personality traits. An essential prerequisite for this is a sufficient reliability of individual measures of reward-related brain signals. We therefore determined test–retest reliabilities of BOLD responses to reward prediction, reward receipt and reward prediction errors in the ventral striatum and the orbitofrontal cortex in 25 subjects undergoing three different simple reward paradigms (retest interval 7–13 days). Although on a group level the paradigms consistently led to significant activations of the relevant brain areas in two sessions, across-subject retest reliabilities were only poor to fair (with intraclass correlation coefficients (ICCs) of −0.15 to 0.44). ICCs for motor activations were considerably higher (ICCs 0.32 to 0.73). Our results reveal the methodological difficulties behind across-subject correlations in fMRI research on reward processing. These results demonstrate the need for studies that address methods to optimize the retest reliability of fMRI.

Introduction

Over the last 20 years, functional magnetic resonance imaging (fMRI) has made immense contributions to the understanding of the neural foundations of reward processing in humans. Reward processing plays a central role for basic cognitive functions like reinforcement learning and decision making, and is thus of fundamental importance for neuroscientific studies in the fields of learning theory, economic decision making (neuroeconomics) and social neurosciences (because social decisions appear to be grounded in the same reward processing brain structures as financial decisions; Fehr and Camerer, 2007).

The fundamentals of reward processing were first identified in animal studies, which showed that dopaminergic midbrain neurons increase their activity when a cue signals an upcoming reward or when an unexpected reward actually occurs (reward prediction error, RPE) (Schultz et al., 1997). Secondary reward signals occur in the primary projection sites of those midbrain neurons, i.e. the ventral striatum (VS) and the orbitofrontal cortex (OFC). FMRI studies have shown that BOLD signal responses in humans parallel those findings to a high extent: BOLD activity in the midbrain (D'Ardenne et al., 2008) and in its projection sites (Pagnoni et al., 2002, Rolls et al., 2008) increases with reward predictions and scales positively with the RPE. It shows temporal characteristics as expected from temporal difference models of reinforcement learning (O'Doherty et al., 2003). FMRI studies have demonstrated that a variety of different rewards can induce such reward-related brain activity. The strength of the brain's response to a reward correlates better with the subjective value of a reward for that individual (individual preference) than with objective reward magnitude (Kable and Glimcher, 2007, O'Doherty et al., 2006, Tobler et al., 2007). Together, these results suggest that activity in reward processing brain areas can be interpreted as a surrogate biomarker of individuals' preferences (Knutson et al., 2009). There are also promising results suggesting that inter-individual differences in reward-related BOLD signals are related to personality traits (Beaver et al., 2006) and to genetic polymorphisms that affect dopamine metabolism (Cohen et al., 2005, Jocham et al., 2009).

Further establishing these links requires better knowledge about the reliability of the respective measures. This subject has been the topic of a recent scientific debate that has evolved regarding the meaningfulness of across-subject correlations between BOLD signals and other individual measures such as personality traits, preferences or attitudes (Poldrack and Mumford, 2009, Vul et al., 2009). The expected correlation between two measures A and B is theoretically limited by measures' reliability, where the upper limit of correlation is given by the reliability index: √(relA ⁎ relB) (Nunnally, 1970). Therefore, when using the BOLD signal as an indicator for individual characteristics, it is essential to know its reliability, i.e. the stability of inter-individual differences in the magnitude of BOLD contrasts over time. Given the central role of reward processing for many fields of cognitive neuroscience and the potential use of reward-related BOLD signals as biomarkers for individuals' preferences, we therefore specifically analyzed the test–retest reliability of reward-related BOLD responses.

Section snippets

Subjects

Eight female and 17 male subjects, aged 19–35 (mean = 26.12, SD = 3.98), were scanned during two sessions separated by 7.7 days on average (range 7–13 days). The time of day of each session was kept fairly constant for each subject (median daytime differences between the sessions: 18 min, range 0–260 min). Subjects had no history of psychiatric or neurological disorders. Two female subjects had to be excluded from the entire analysis, one due to an incidental finding of a brain abnormality and the

Behavioral data

Behavioral data revealed no gross differences in subject's response behavior between both sessions: neither the percentage of missed responses, mean response times nor amount of rewards differed significantly between sessions 1 and 2 in any of the paradigms (p > 0.05, see Table 2), except that mean response times in Session 2 of Paradigm C were significantly lower than in session one (p = 0.015). Percentage of improbable guesses in Paradigm A and percentage of incorrect responses in Paradigm B also

Discussion

This study determined retest reliabilities of BOLD signals related to reward processing. This was based on the consideration that these signals might be used as surrogate markers for individuals' preferences, but that such an application requires sufficient reliability. Despite highly significant main effects of reward contrasts and good reliabilities of motor-related activity, we found that retest reliabilities of different reward contrasts in the ventral striatum and the orbitofrontal cortex

Acknowledgments

We would like to thank Courtney B. Phillipps, Florian Mormann and Jason Aimone for valuable comments on the manuscript.

References (36)

Cited by (46)

  • The longitudinal stability of fMRI activation during reward processing in adolescents and young adults

    2021, NeuroImage
    Citation Excerpt :

    Similarly, Wu et al. observed some evidence for improved stability when considering the raw average fMRI signal, relative to a contrast of an active and neutral condition (Wu et al., 2014). In contrast, other studies have examined the short-term reliability of gain vs. loss contrasts, observing lower reliabilities in the striatum (ICC = −0.13 – 0.45) (Elliott et al., 2020; Fliessbach et al., 2010; Li et al., 2020). Luking et al. compared the split-half reliability of gain>baseline and gain>loss activation during reward outcome, finding nearly double the reliability in gain>baseline in the ventral striatum (Luking et al., 2017).

View all citing articles on Scopus
1

These authors have equally contributed to the study.

View full text