Elsevier

Behavioural Brain Research

Volume 320, 1 March 2017, Pages 244-254
Behavioural Brain Research

Suboptimal choice in rats: Incentive salience attribution promotes maladaptive decision-making

https://doi.org/10.1016/j.bbr.2016.12.013Get rights and content

Highlights

  • Incentive salience was necessary to promote suboptimal choice.

  • Stimuli imbued with incentive salience overshadowed primary reinforcement value.

  • Rate of sign-tracking was not reflective of stimulus value.

Abstract

Stimuli that are more predictive of subsequent reward also function as better conditioned reinforcers. Moreover, stimuli attributed with incentive salience function as more robust conditioned reinforcers. Some theories have suggested that conditioned reinforcement plays an important role in promoting suboptimal choice behavior, like gambling. The present experiments examined how different stimuli, those attributed with incentive salience versus those without, can function in tandem with stimulus-reward predictive utility to promote maladaptive decision-making in rats. One group of rats had lights associated with goal-tracking as the reward-predictive stimuli and another had levers associated with sign-tracking as the reward-predictive stimuli. All rats were first trained on a choice procedure in which the expected value across both alternatives was equivalent but differed in their stimulus-reward predictive utility. Next, the expected value across both alternatives was systematically changed so that the alternative with greater stimulus-reward predictive utility was suboptimal in regard to primary reinforcement. The results demonstrate that in order to obtain suboptimal choice behavior, incentive salience alongside strong stimulus-reward predictive utility may be necessary; thus, maladaptive decision-making can be driven more by the value attributed to stimuli imbued with incentive salience that reliably predict a reward rather than the reward itself.

Introduction

Normative theories, such as optimal foraging theory [1] and rational choice theory [2], suggest that an individual should behave optimally or choose alternatives that maximize reinforcement and minimize effort. Although theoretically useful, the decisions an individual makes can deviate from the predictions provided by normative theories and result in less overall reinforcement than alternatives [3], [4], [5]. Indeed, such suboptimal behavior, also described as maladaptive decision-making [6], often appears in human pathologies such as gambling [7], substance abuse [8], and eating-disorders [9], all of which ultimately result in a net loss of resources [10], [11], [12]. Thus, maladaptive decision-making can be persistent and recurring despite the unfavorable outcomes that are associated with such behavior [13], [14], [15], [16], [17].

While maladaptive decision-making is present in a variety of pathologies, one disorder in which it is readily apparent is gambling and risk taking behavior (cf. [18]. Human gambling behavior, when surveyed, has been attributed to a multitude of subjective rationalizations that, from the individual’s perspective, account for the occurrence and recurrence of the phenomenon [19]. Interestingly, researchers have theorized that the basis of some subjective rationalizations is an effect of enhanced saliency of wins versus loses [20] or a misunderstanding of probabilities [21], [22]. Given that human gambling behavior is complex and influenced by individual experiences, researchers have often used animal models to better isolate the mechanisms mediating gambling-like behavior (e.g., [23], [24], [25], [26], [27]. One approach that has been fruitful in this endeavor consists of giving animals a choice between one of two options, each leading to a specific stimulus-reward outcome in which one option is suboptimal relative to the other, and examining the choices made. Using a similar approach [28], [27], researchers have consistently demonstrated pigeons show a large suboptimal preference for an option that offers less primary reinforcement over an option that offers greater primary reinforcement (e.g., [29], [24], [30], [31], [32]. Furthermore, the effects demonstrated by the suboptimal choice procedure used in pigeons have also been replicated in human subjects [33], suggesting comparable mechanisms that affect decision-making.

The suboptimal choice procedure operates as a concurrent-chain schedule [34], [35]. First, subjects are presented with a choice between two options, to which responding is required to make a choice; this choice phase is referred to as the initial-link. After making a choice, each option results in an event referred to as the terminal-link. Furthermore, by keeping the choice phase equivalent (i.e., initial-link response requirements), preference for one option over another option is assumed to be driven by the terminal-link event and the primary outcomes. More specifically, in the suboptimal choice procedure (e.g. [32] one alternative leads to a terminal-link in which a stimulus that perfectly predicts reinforcement is occasionally presented (e.g., 25% of the time) or leads directly to a non-signaled reward omission. The other alternative leads to a different terminal-link in which a different stimulus is always presented, but it predicts reinforcement probabilistically. Thus, in the suboptimal choice procedure, the effects of primary reinforcement (i.e., food) can be dissociated from those of conditioned reinforcement, (the different stimuli in the terminal-links that predict reward). Using concurrent-chain schedules, there have been findings suggesting that terminal-link stimuli associated with reward outcome function as conditioned reinforcers and can influence the relative allocation of choices [36], [37], [29], [38], [39], [40], [41], [42], [43], [44], [32], even if preference leads to significantly reduced primary reinforcement. Although not all stimuli associated with reward become conditioned reinforcers (cf. [45], conditions in which a reward-associated terminal-link stimulus is a conditioned reinforcer function by how well it serves as a predictor (e.g. probability) for reward and can promote suboptimal choice behavior [31], [46], [28], [44], [32].

Despite previous studies (e.g. [44], [32] demonstrating that a terminal-link stimulus with the greatest predictive probability of reinforcement can produce suboptimal choice in pigeons, similar procedures applied in rodents fail to produce a similar effect; instead, rats tend to behave optimally and choose an alternative that provides the greatest amount of reinforcement, regardless of how predictive a terminal-link stimulus might be [47], [48], [49]. While the above findings suggest species differences between pigeons and rats on the suboptimal choice procedure, it is also possible that the specific conditions used within the procedures could greatly influence choice behavior. Conceptually, the stimulus that is present in each terminal-link functions as a conditioned stimulus (CS), as it is predictive of the subsequent reinforcer (unconditioned stimulus; US). However, there are scenarios in which a CS can be attributed with incentive value that goes beyond its predictive function [50], [51], [52], [53]. Importantly, CSs attributed with incentive value serve as more robust conditioned reinforcers [51], [52], [53]. For example, when pitted against each other, rats have shown a preference for a CS attributed with incentive value over a CS without incentive value, even if the preference results in significantly reduced primary reinforcement (cf. [53], [54]. Collectively, these studies suggest that not all CSs function equivalently and, despite being equally predictive, CSs attributed with incentive value can significantly influence decision-making, leading to maladaptive decisions.

Although pigeons trained on a suboptimal choice procedure show preference for the terminal-link stimulus with the greatest predictive probability of reinforcement, it should be noted that the stimuli used in those experiments (e.g. [44], [32] consists of lights. For pigeons, light stimuli are known to elicit sign-tracking behavior (approach and contact with the stimulus; [55], in the form of key pecks [56], [57], which is often a key feature of stimuli attributed with incentive salience [53]. For example, [57] demonstrated that pigeons sign-track to a light-CS that predicted food and continued to do so as it was moved further away from the location of reward delivery, resulting in maladaptive sign-tracking to the light-CS that led to reduced eating time. Thus, for pigeons, it is possible that the use of a light stimulus, which elicits key pecking, could be coupled with incentive value attribution [58], [59] that in turn could drive maladaptive choice within the suboptimal choice procedure.

Parallel to suboptimal choice procedures with pigeons, studies using rats have also used lights [47], [48], [49]. Notably, lights are known to elicit goal-tracking behavior [60], [61], [53], described as approach to the location of reward delivery [62], when a food US is used and are not accompanied by the attribution of incentive value that has been shown to promote suboptimal choice behavior. Importantly, it has been shown that rats have a tendency to sign-track to a lever CS [63], [64], [65], and levers associated with sign-tracking behavior function as robust conditioned reinforcers [51]. Therefore, in the present study we examined how different terminal-link stimuli, with and without incentive salience (i.e., levers vs. lights), can influence decision-making in rats using a suboptimal choice procedure [32]. If incentive salience does not play a role, both groups should similarly prefer the optimal alternative, regardless of the type of terminal-link stimuli used. However, if using stimuli attributed with incentive salience does play a role, levers as terminal-link stimuli should direct decision making towards suboptimal choice behavior.

Section snippets

Animals

Twelve adult male Sprague-Dawley rats (Harlan Inc.; Indianapolis, IN, USA), weighing approximately 250–275 g at the beginning of experimentation, were used. Rats were individually housed in a temperature-controlled environment with a 12:12 h light:dark cycle, with lights on at 0600 h. All rats were first acclimated to the colony environment and handled daily for one week prior to experimentation, had ad libitum access to water in their home cage throughout experimentation, and were maintained at

Experiment 1: obtaining suboptimal choice behavior via incentive salience

Fig. 2 illustrates the percent choice of the predictive alternative during Phase 1 (A), Phase 2 (B), Phase 3 (C), and the average during the last four sessions of each phase (D), in which the frequency that the predictive stimulus was presented decreased as a function of phase. Fig. 2A illustrates initial acquisition on the suboptimal choice procedure, in which the predictive alternative and non-predictive alternative were equivalent in expected value at 50% probability of reinforcement. Linear

Discussion

The present experiment examined how different terminal-link stimuli (i.e., levers vs. lights) differentially associated with incentive salience attribution can function in tandem with varying terminal-link stimulus predictive utility (100% vs. 50%) to promote suboptimal choice behavior. The results reported here reveal a number of factors that can promote suboptimal choice behavior. First, consistent with the current literature (e.g., [44], [32], reward-associated stimuli that have greater

Disclosure

The authors report no conflicts of interest.

Acknowledgements

We would like to thank Joshua N. Lavy and Andrew Edel for their technical support. This work was supported by the National Institute on Drug Abuse (NIDA), DA033373 and DA016176.

References (95)

  • T.E. Robinson et al.

    Dissociating the predictive and incentive motivational properties of reward-related cues through the study of individual differences

    Biol. Psychiatry

    (2009)
  • J.J. Chow et al.

    Toward isolating the role of dopamine in the acquisition of incentive salience attribution

    Neuropharmacology

    (2016)
  • J.E. Mazur

    Species differences between rats and pigeons in choices with probabilistic and delayed reinforcers?

    Behav. Process.

    (2007)
  • S.E. Chang

    Effects of orbitofrontal cortex lesions on autoshaped lever pressing and reversal learning

    Behav. Brain Res.

    (2014)
  • P. Anselme

    Incentive salience attribution under reward uncertainty: a Pavlovian model

    Behav. Process.

    (2015)
  • P. Anselme

    Motivational control of sign-tracking behaviour: a theoretical framework

    Neurosci. Biobehav. Rev.

    (2016)
  • P. Anselme et al.

    Reward uncertainty enhances incentive salience attribution as sign-tracking

    Behav. Brain Res.

    (2013)
  • M.J. Robinson et al.

    Initial uncertainty in Pavlovian reward prediction persistently elevates incentive salience and extends sign-tracking to normally unattractive cues

    Behav. Brain Res.

    (2014)
  • A. Poling et al.

    Self-control revisited: some factors that affect autoshaped responding?

    Behav. Process.

    (1985)
  • J.J. Clark et al.

    Pavlovian valuation systems in learning and decision making?

    Curr. Opin. Neurobiol.

    (2012)
  • P. Dayan et al.

    The misbehavior of value and the discipline of the will

    Neural Netw.

    (2006)
  • M.J. Robinson et al.

    Instant transformation of learned repulsion into motivational wanting

    Curr. Biol.

    (2013)
  • P. Cavedini et al.

    Frontal lobe dysfunction in pathological gambling patients

    Biol. Psychiatry

    (2002)
  • T.R. Zentall

    Suboptimal choice by pigeons: an analog of human gambling behavior

    Behav. Process.

    (2014)
  • D.W. Stephens et al.

    Foraging Theory

    (1986)
  • J. Scott

    Rational choice theory

    Understand. Contemp. Soc. Theori. Present

    (2000)
  • G.H. Pyke

    Optimal foraging theory: a critical review

    Annu. Rev. Ecol. Syst.

    (1984)
  • R.J. Herrnstein

    Rational choice theory: necessary but not sufficient

    Am. Psychol.

    (1990)
  • E. Shafir et al.

    Rationality

    Annu. Rev. Psychol.

    (2002)
  • T.R. Zentall et al.

    Maladaptive choice behaviour by pigeons: an animal analogue and possible mechanism for gambling (sub-optimal human decision-making behaviour)?

    Proc. R. Soc. London B Biol. Sci.

    (2011)
  • A. Brogan et al.

    Anorexia bulimia, and obesity: shared decision making deficits on the Iowa Gambling Task (IGT)

    J. Int. Neuropsychol. Soc.

    (2010)
  • H.R. Lesieur et al.

    Alcoholism, drug abuse, and gambling

    Alcohol. Clin. Exp. Res.

    (1986)
  • H.R. Lesieur

    Compulsive gambling

    Society

    (1992)
  • H.W. Hoek et al.

    Review of the prevalence and incidence of eating disorders

    Int. J. Eat. Disord.

    (2003)
  • M. Dickerson

    Internal and external determinants of persistent gambling: problems in generalising from one form of gambling to another?

    J. Gambl. Stud.

    (1993)
  • A. Blaszczynski et al.

    Cognitive and behavioral therapies for pathological gambling

    J. Gambl. Stud.

    (1995)
  • K. Chen et al.

    The natural history of drug use from adolescence to the mid-thirties in a general population sample

    Am. J. Public Health

    (1995)
  • S.E. Hyman et al.

    Addiction and the brain: the neurobiology of compulsion and its persistence

    Nat. Rev. Neurosci.

    (2001)
  • C. Neighbors et al.

    Exploring college student gambling motivation?

    J. Gambl. Stud.

    (2002)
  • A. Tversky et al.

    Judgment under uncertainty: heuristics and biases

    Utility, Probability, and Human Decision Making

    (1975)
  • K.H. Teigen

    Studies in subjective probability IV: probabilities, confidence, and luck

    Scand. J. Psychol.

    (1983)
  • E. Fantino et al.

    Percentage reinforcement and choice

    J. Exp. Anal. Behav.

    (1979)
  • M.L. Spetch et al.

    Suboptimal choice in a percentage‐reinforcement procedure: effects of signal condition and terminal‐link length

    J. Exp. Anal. Behav.

    (1990)
  • G.J. Madden et al.

    Toward an animal model of gambling: delay discounting and the allure of unpredictable outcomes

    J. Gambl. Stud.

    (2007)
  • F.D. Zeeb et al.

    Serotonergic and dopaminergic modulation of gambling behavior as assessed using a novel rat gambling task

    Neuropsychopharmacology

    (2009)
  • T.R. Zentall

    Resolving the paradox of suboptimal choice

    J. Exp. Psychol. Anim. Learn. Cognit.

    (2016)
  • M.A. McDevitt et al.

    When good news leads to bad choices

    J. Exp. Anal. Behav.

    (2016)
  • Cited by (0)

    View full text