Discounting of Future Rewards and Punishments in Rats

Overview of the shock and reward timings for each experiment. After an animal entered a decision arm (0 s), the doors were closed. Afterwards, the rewards and shocks were delivered. In general, an early reward had a delay of 2 s and a late reward of 21 s. The early shock was administered after 1 s (relative to entering) and the late shock after 20 s.

Overview of the shock and reward contingencies in the two arms of the T-maze for each experiment and the predictions for each theory. For the shocks and rewards, each column represents one arm of the T-maze (early reward + late reward [EE]; late shock + late reward [LL]; early shock + early reward [EE]). In the first experiment, the entry-to-reward contingencies are identical in both arms, but the entry-to-shock delays differ between both arms. The second experiment tested constant shock-to-reward delays with different entry-to-shock delays. The third experiment used constant entry-to-shock delays, but the shock-to-reward contingencies differ between arms. In the predictions, “∼” represents no predictions. Note that the assignment of shock/reward contingencies to the left or right arm of the T-maze will be pseudo-randomized across sessions for all experiments. The left side is always the condition for which the percentage of decisions was calculated.

Depicted is the used Bayesian hierarchy model. As the likelihood function for the decisions, a Bernoulli distribution was used. Each estimation of θ followed a β function with μθ(κ−2)+1,(1−μθ)(κ−2)+1 . For μθ , an uninformed β prior was used, and κ used a gamma(0.01,0.01) function (for prior predictions, see Extended Data Figs. 4-1, 4-2, 4-3).

Mean parameter estimation with simulated data for all predictions according to the Bayesian hierarchy model. The predictions of both models are color coded, and on the top left corner are the specific experiments (EL: early shock + late rerwad; LL: late shock + late reward; EE: early shock + early reward). The y-axis shows the posterior distribution of μθ . The vertical gray lines represent the upper and lower bound for the 95% highest density interval. Experiment 1 is in the top row, and the utility from anticipation model is on the left side (upper bound = 0.42, lower bound = 0.38). On the other side is the prediction for the aversive discounting model (upper bound = 0.62, lower bound = 0.589). In the second row is experiment 2, and the prediction for the utility from anticipation model is displayed (upper bound = 0.42, lower bound = 0.38). Finally, in the bottom row, experiment 3 can be seen, and both models have the same predictions (upper bound = 0.42, lower bound = 0.38).

A, Mean percentage of decisions for each experiment. In each experiment, animals performed 10 sessions and up to 21 trials per session (6 forced trials and 16 free trials). The timings of the reward and shocks are displayed on the x-axis as well as the experiment. Specifically, for experiment 1, the percentage of decisions are calculated for early shock + late reward (EL) versus late shock + late reward (LL). For experiments 2 and 3, the percentage is calculated for early shock + early reward (EE) versus late shock + late reward (LL) and early shock + early reward (EE) versus early shock + early shock + late reward (EL), respectively. The vertical lines represent the SEM, and each dot represents a single animal. For all experiments, we calculated one-sample t tests (two-tailed). In the first experiment, animals showed a significant preference above chance level. The second experiment failed to yield any significant results; and in the third experiment, animals revealed a significant preference below chance level. The black horizontal line represents chance level. B, Mean parameter estimation for the Bayesian hierarchy model. On the top left corner are the specific experiments. On the y-axis is the posterior distribution of μθ . Additionally, the conditions are displayed again. The vertical gray lines show the upper and lower bound for the 95% highest density interval. From top to bottom is experiment 1 (upper bound = 0.57, lower bound = 0.52), experiment 2 (upper bound = 0,55, lower bound = 0.49), and experiment 3 (upper bound = 0.48, lower bound = 0.42). See the extended data for the posterior θ distribution for all experiments (Extended Data Fig. 6-1); ***p < 0.00.