Suboptimal choice in rats: Incentive salience attribution promotes maladaptive decision-making
Introduction
Normative theories, such as optimal foraging theory [1] and rational choice theory [2], suggest that an individual should behave optimally or choose alternatives that maximize reinforcement and minimize effort. Although theoretically useful, the decisions an individual makes can deviate from the predictions provided by normative theories and result in less overall reinforcement than alternatives [3], [4], [5]. Indeed, such suboptimal behavior, also described as maladaptive decision-making [6], often appears in human pathologies such as gambling [7], substance abuse [8], and eating-disorders [9], all of which ultimately result in a net loss of resources [10], [11], [12]. Thus, maladaptive decision-making can be persistent and recurring despite the unfavorable outcomes that are associated with such behavior [13], [14], [15], [16], [17].
While maladaptive decision-making is present in a variety of pathologies, one disorder in which it is readily apparent is gambling and risk taking behavior (cf. [18]. Human gambling behavior, when surveyed, has been attributed to a multitude of subjective rationalizations that, from the individual’s perspective, account for the occurrence and recurrence of the phenomenon [19]. Interestingly, researchers have theorized that the basis of some subjective rationalizations is an effect of enhanced saliency of wins versus loses [20] or a misunderstanding of probabilities [21], [22]. Given that human gambling behavior is complex and influenced by individual experiences, researchers have often used animal models to better isolate the mechanisms mediating gambling-like behavior (e.g., [23], [24], [25], [26], [27]. One approach that has been fruitful in this endeavor consists of giving animals a choice between one of two options, each leading to a specific stimulus-reward outcome in which one option is suboptimal relative to the other, and examining the choices made. Using a similar approach [28], [27], researchers have consistently demonstrated pigeons show a large suboptimal preference for an option that offers less primary reinforcement over an option that offers greater primary reinforcement (e.g., [29], [24], [30], [31], [32]. Furthermore, the effects demonstrated by the suboptimal choice procedure used in pigeons have also been replicated in human subjects [33], suggesting comparable mechanisms that affect decision-making.
The suboptimal choice procedure operates as a concurrent-chain schedule [34], [35]. First, subjects are presented with a choice between two options, to which responding is required to make a choice; this choice phase is referred to as the initial-link. After making a choice, each option results in an event referred to as the terminal-link. Furthermore, by keeping the choice phase equivalent (i.e., initial-link response requirements), preference for one option over another option is assumed to be driven by the terminal-link event and the primary outcomes. More specifically, in the suboptimal choice procedure (e.g. [32] one alternative leads to a terminal-link in which a stimulus that perfectly predicts reinforcement is occasionally presented (e.g., 25% of the time) or leads directly to a non-signaled reward omission. The other alternative leads to a different terminal-link in which a different stimulus is always presented, but it predicts reinforcement probabilistically. Thus, in the suboptimal choice procedure, the effects of primary reinforcement (i.e., food) can be dissociated from those of conditioned reinforcement, (the different stimuli in the terminal-links that predict reward). Using concurrent-chain schedules, there have been findings suggesting that terminal-link stimuli associated with reward outcome function as conditioned reinforcers and can influence the relative allocation of choices [36], [37], [29], [38], [39], [40], [41], [42], [43], [44], [32], even if preference leads to significantly reduced primary reinforcement. Although not all stimuli associated with reward become conditioned reinforcers (cf. [45], conditions in which a reward-associated terminal-link stimulus is a conditioned reinforcer function by how well it serves as a predictor (e.g. probability) for reward and can promote suboptimal choice behavior [31], [46], [28], [44], [32].
Despite previous studies (e.g. [44], [32] demonstrating that a terminal-link stimulus with the greatest predictive probability of reinforcement can produce suboptimal choice in pigeons, similar procedures applied in rodents fail to produce a similar effect; instead, rats tend to behave optimally and choose an alternative that provides the greatest amount of reinforcement, regardless of how predictive a terminal-link stimulus might be [47], [48], [49]. While the above findings suggest species differences between pigeons and rats on the suboptimal choice procedure, it is also possible that the specific conditions used within the procedures could greatly influence choice behavior. Conceptually, the stimulus that is present in each terminal-link functions as a conditioned stimulus (CS), as it is predictive of the subsequent reinforcer (unconditioned stimulus; US). However, there are scenarios in which a CS can be attributed with incentive value that goes beyond its predictive function [50], [51], [52], [53]. Importantly, CSs attributed with incentive value serve as more robust conditioned reinforcers [51], [52], [53]. For example, when pitted against each other, rats have shown a preference for a CS attributed with incentive value over a CS without incentive value, even if the preference results in significantly reduced primary reinforcement (cf. [53], [54]. Collectively, these studies suggest that not all CSs function equivalently and, despite being equally predictive, CSs attributed with incentive value can significantly influence decision-making, leading to maladaptive decisions.
Although pigeons trained on a suboptimal choice procedure show preference for the terminal-link stimulus with the greatest predictive probability of reinforcement, it should be noted that the stimuli used in those experiments (e.g. [44], [32] consists of lights. For pigeons, light stimuli are known to elicit sign-tracking behavior (approach and contact with the stimulus; [55], in the form of key pecks [56], [57], which is often a key feature of stimuli attributed with incentive salience [53]. For example, [57] demonstrated that pigeons sign-track to a light-CS that predicted food and continued to do so as it was moved further away from the location of reward delivery, resulting in maladaptive sign-tracking to the light-CS that led to reduced eating time. Thus, for pigeons, it is possible that the use of a light stimulus, which elicits key pecking, could be coupled with incentive value attribution [58], [59] that in turn could drive maladaptive choice within the suboptimal choice procedure.
Parallel to suboptimal choice procedures with pigeons, studies using rats have also used lights [47], [48], [49]. Notably, lights are known to elicit goal-tracking behavior [60], [61], [53], described as approach to the location of reward delivery [62], when a food US is used and are not accompanied by the attribution of incentive value that has been shown to promote suboptimal choice behavior. Importantly, it has been shown that rats have a tendency to sign-track to a lever CS [63], [64], [65], and levers associated with sign-tracking behavior function as robust conditioned reinforcers [51]. Therefore, in the present study we examined how different terminal-link stimuli, with and without incentive salience (i.e., levers vs. lights), can influence decision-making in rats using a suboptimal choice procedure [32]. If incentive salience does not play a role, both groups should similarly prefer the optimal alternative, regardless of the type of terminal-link stimuli used. However, if using stimuli attributed with incentive salience does play a role, levers as terminal-link stimuli should direct decision making towards suboptimal choice behavior.
Section snippets
Animals
Twelve adult male Sprague-Dawley rats (Harlan Inc.; Indianapolis, IN, USA), weighing approximately 250–275 g at the beginning of experimentation, were used. Rats were individually housed in a temperature-controlled environment with a 12:12 h light:dark cycle, with lights on at 0600 h. All rats were first acclimated to the colony environment and handled daily for one week prior to experimentation, had ad libitum access to water in their home cage throughout experimentation, and were maintained at
Experiment 1: obtaining suboptimal choice behavior via incentive salience
Fig. 2 illustrates the percent choice of the predictive alternative during Phase 1 (A), Phase 2 (B), Phase 3 (C), and the average during the last four sessions of each phase (D), in which the frequency that the predictive stimulus was presented decreased as a function of phase. Fig. 2A illustrates initial acquisition on the suboptimal choice procedure, in which the predictive alternative and non-predictive alternative were equivalent in expected value at 50% probability of reinforcement. Linear
Discussion
The present experiment examined how different terminal-link stimuli (i.e., levers vs. lights) differentially associated with incentive salience attribution can function in tandem with varying terminal-link stimulus predictive utility (100% vs. 50%) to promote suboptimal choice behavior. The results reported here reveal a number of factors that can promote suboptimal choice behavior. First, consistent with the current literature (e.g., [44], [32], reward-associated stimuli that have greater
Disclosure
The authors report no conflicts of interest.
Acknowledgements
We would like to thank Joshua N. Lavy and Andrew Edel for their technical support. This work was supported by the National Institute on Drug Abuse (NIDA), DA033373 and DA016176.
References (95)
- et al.
Decision-making impairments in patients with pathological gambling
Psychiatry Res.
(2005) - et al.
Decision-making and addiction (part I): impaired activation of somatic states in substance dependent individuals when pondering decisions with negative future consequences
Neuropsychologia
(2002) - et al.
Drug addiction: the neurobiology of disrupted self-control
Trends Mol. Med.
(2006) When animals misbehave: analogs of human biases and suboptimal choice
Behav. Process.
(2015)- et al.
The subjective beliefs underlying probability overestimation
J. Exp. Soc. Psychol.
(1997) Conditioned reinforcement: experimental and theoretical issues
Behav. Anal.
(1994)- et al.
Observing behavior in pigeons: the effect of reinforcement probability and response cost using a symmetrical choice procedure
Learn. Motiv.
(1999) - et al.
The two-alternative observing response procedure in rats: preference for nondiscriminative stimuli and the effect of delay
Learn. Motiv.
(2004) - et al.
Rats are optimal in a choice task in which pigeons are not
Behav. Process.
(2015) - et al.
Optimal behavior by rats in a choice task is associated to a persistent conditioned inhibition effect
Behav. Process.
(2016)