Habitual Behavior Is Mediated by a Shift in Response-Outcome Encoding by Infralimbic Cortex

Experimental and recording session time line. A, Illustration of recording session time line. Unreinforced presses (indicated by vertical red lines) were those that did not result in reinforcer delivery. Only isolated unreinforced presses were included in analyses. After reinforced presses (indicated by vertical green line), reinforcer delivery began immediately and lasted 1.7 s. The mean time of consummatory behavior onset was 2.0 s (±0.203 s, SEM) following a reinforced lever press. While the mean time of magazine entry occurred after the termination of the reinforcer delivery epoch, the initiation of consummatory behavior was self-paced and overlap between the reinforcer delivery epochs and consummatory behavior epochs was variable. For analysis of reinforced presses, the pre-press interval consisted of 0.5 s before the initiation of press. The post-press epoch was defined as 0.5 s following the lever press. The reinforcer delivery epoch was defined as the remainder of the “pump on” interval. B, Mice were implanted with multielectrode array probes and, following recovery from surgery, were trained to self-administer sucrose in the RR/RI protocol (Fig. 1). Shown is an example of electrolytic lesions of the recording site. C, Placement of multielectrode arrays within IfL-C for all mice in the study. Blue shaded region represents the maximal size of the lesioned area within each of the anatomic sections shown, while red represents the smallest lesion area within each section.

IfL-C firing rate was mediated by response strategy. A, Firing rate surrounding a reinforced lever press at the early training time point (i.e., goal directed on both the RR and RI levers). Time 0 represents the time of lever press. Reinforcer delivery lasted for 1.7 s following lever press. B, Firing rate surrounding a reinforced lever press at the extended training time point. At this time point, responding is habitual on the RI lever but remains goal directed on the RR lever. Data in A, B represent the mean ± SEM, indicated by shaded area for each line. C, In early training, the mean firing rate during the reinforcer delivery epoch was higher than firing rate during the pre-press interval. This effect was not mediated by reinforcement schedule. D, During responding on the RR schedule (i.e., in a goal-directed manner), firing rate was significantly higher during the reinforcer delivery epoch than during the pre-press interval. This relationship was lost during habitual responding. E, At an early training time point, IfL-C firing rate is modulated by outcome delivery in both action-promoting (RR) and habit-promoting schedules (RI). F, After extended training, IfL-C firing rate is only modulated by outcome delivery in the action-promoting (RR) schedule, and this effect is attenuated in the habit-promoting condition (RI).

Task-epoch specific modulation of cells. A, At the early training time point, no significant differences in the proportion of cells modulated by lever press were present (left panel). In contrast, during the reinforcer delivery epoch, a greater proportion of cells showed a reduction in firing rate during the RI condition than during the RR condition (p < 0.05). B, At the extended training time point, no significant differences in the number or proportion of cells responding were present between RR and RI responding at the post-press (left panel) or reinforcer delivery (right panel) epochs. In contrast, for the RI schedule, the distribution of significantly modulated cells was distinct between the post-press and reinforcer delivery intervals. In the RI condition, a greater proportion of cells showed a decrease in firing rate during the reinforcer delivery epoch (p < 0.05). Dec, decreased firing rate, Inc, increased firing rate; NC, no change in firing rate; * indicates significant χ² result; ★ indicates significant differences in z score comparison of population proportions.

Activity patterns of cells with increased firing rates during reinforcer delivery. A, At the early training time point, of cells that showed an increase in firing rate during either the RI or RR schedule, 30.0% exhibited elevated firing rates in both conditions. B, At the extended training time point, 20.0% of cells that had increased firing rates during either the RI or RR schedule showed elevated firing rates in both conditions. C, D, Firing rate of cells showing an increase in firing surrounding a reinforced lever press epoch (i.e., goal directed on both the RR and RI levers) at the early training (C) or extended training (D) time points. Time 0 represents the time of lever press. Reinforcer delivery lasted for 1.7 s following lever press. E, F, Mean firing rates of cells showing an increase in firing rate during the RI schedule. E, At the early training time point, RI-modulated cells showed similar activity patterns when responding on the RR schedule. F, Similarly, at the late time point, RI-modulated cells showed similar activity patterns when responding on the RR schedule. G, H, Mean firing rates of cells showing an increase in firing rate during the RR schedule. G, At the early training time point, RR-modulated cells showed similar activity patterns when responding on the RI schedule. H, After extended training, cells that exhibited increased firing rates during reinforcer delivery on the RR schedule showed a similar pattern when responding on the RI schedule; however, they exhibited lower firing rates during the reinforcer delivery epoch on the RI (habit-promoting) schedule than on the RR schedule; **p < 0.01, ***p < 0.001.

Activity patterns of cells with decreased firing rates during reinforcer delivery. A, At the early training time point, 16.3% of cells that reduced firing rate during either the RI or RR schedule showed decreased firing rates in both conditions. B, After extended training, 24.1% of cells that exhibited a decrease in firing rate during either the RI or RR schedule showed reductions in firing rates in both conditions. C, D, Firing rate of cells showing a reduction in firing rate surrounding a reinforced lever press epoch (i.e., goal directed on both the RR and RI levers) at the early training (C) or extended training (D) time points. Time 0 represents the time of lever press. Reinforcer delivery lasted for 1.7 s following lever press. E, F, Mean firing rates of cells showing reduced firing rates on the RI schedule. E, At the early training time point, RI-modulated cells were not significantly modulated by epoch or by schedule. F, In contrast, after extended training, RI-modulated cells showed a significant reduction in firing rate when responding on the RI schedule, exhibiting lower mean firing rates during the reinforcer delivery epoch than the pre- or post-press periods. However, cells modulated during the RI condition were not significantly modulated during responding on the RR condition. G, H, Mean firing rates of cells showing reduced firing rates on the RR schedule. G, At the early training time point, RR-modulated cells were not significantly modulated by epoch or by schedule, similar to findings from RI-modulated cells. H, Unlike cells reduced on the RI schedule, cells identified as modulated during responding on the RR schedule after extended training exhibited similar response patterns on both the RR and RI schedule; *p > 0.05, **p < 0.01, ***p < 0.001.

Firing rate following isolated, unreinforced lever presses. A, Firing rate was differentially modulated based on reinforcement schedule following an isolated, unreinforced press at the extended training time point. Data represent mean ± SEM (indicated by shaded area for each line). Time 0 represents lever press. B, After extended training, firing rate in the IfL-C during the post-press interval was significantly lower following unreinforced presses during the RR schedule than during the RI schedule. Firing post-press on the RR schedule was also significantly lower than firing during the epoch where reinforcer would be delivered during a reinforced press. C, When mice are responding on an action-promoting (RR) schedule, unreinforced lever presses are followed by a reduction in IfL-C firing rate. In contrast, this modulation is attenuated when mice are responding on the habit-promoting RI schedule. D, At the extended training time point, the distribution of significantly modulated cells was distinct between the RI and RR schedule during the post-press interval; *p < 0.05; Dec, decreased firing rate; Inc, increased firing rate; NC, no change in firing rate; ★ indicates significant differences in z score comparison of population proportions.

Habitual reward seeking was associated with loss of modulation in the post-press interval. A, During an outcome devaluation test session, performed in extinction, IfL-C firing rate during the intervals surrounding a lever press was mediated by reinforcement schedule. Data represent mean ± SEM (indicated by shaded area for each line). Time 0 represents lever press. B, During the post-press interval, neural activity was significantly lower during responding on the RR lever than the RI lever. IfL-C activity during the post-press epoch was significantly lower than the pre-press or the epoch where the reward delivery pump was on during responding on the RR schedule. C, In an outcome devaluation test session, performed in extinction, IfL-C activity surrounding lever press is schedule dependent. When mice are responding on an action-promoting (RR) schedule, IfL-C activity is reduced immediately following lever press. In contrast, when mice are responding on a habit-promoting (RI) schedule, IfL-C activity is not modulated. D, Significantly modulated cells were identified by d' analysis. At the extended training time point, the distribution of significantly modulated cells in the post-press interval was distinct between the RR and RI schedules. A greater number of cells showed a significant increase in firing rate in the post-press epoch during the RI schedule compared to the RR schedule, while a smaller proportion showed reductions in firing rate during this epoch; *p < 0.05, **p < 0.01; ★ indicates significant differences in z score comparison of population proportions.

Optogenetic inhibition of IfL-C selectively impacted expression of habitual reward seeking. A, Schematic showing placement of optic fibers and virus within the IfL-C. B, Optogenetic inhibition of IfL-C selectively in the post-press epoch restored goal-directed behavior during a contingency degradation test. C, Inhibition of IfL-C during the post-press epoch (at the same epoch shown to restore goal-directed actions) does not impact extinction learning; *p < 0.05. Data represent mean ± SEM.

Consummatory behavior mediates firing rate in IfL-C independent of response strategy or reinforcement schedule. At the early training time point (A) and at the extended training time point (B), IfL-C activity is modulated across the onset of consummatory behavior. Time 0 represents time of entry (indicated by a beam break) into the magazine where the reinforcer was delivered and provides a putative indicator of initiation of consumption. The 0- to 4-s time period represents the putative consumption period. The one second interval before and after this period were considered the pre- and postconsumption intervals, respectively. C, Binned means of firing rate at the early training period indicated that IfL-C firing rate during the consummatory period is significantly lower than mean firing rate pre- or postconsumption. D, At the extended training time, firing rate in IfL-C during both the consumption and postconsumption interval was lower than the preconsumption interval. E, During the outcome devaluation probe test, which was performed in extinction, IfL-C activity was not modulated during magazine entries, suggesting that magazine entry in the absence of consummatory behavior did not contribute to IfL-C activity. F, No significant differences in IfL-C activity were observed during magazine entries during extinction. Data in represent the mean ± SEM indicated by shaded area for each line (A, B, E) or error bars (C, D, F); **p < 0.01.

Firing rate during consummatory behavior at the early training time point. Cells that were significantly modulated during consumption were identified by d' analysis. A, Firing rate of cells the exhibited a significant reduction in firing rate during consumption at the early training time point. B, Firing rate of cells that increased their firing rate during consumption at the early training time point. Data presented in C, D represent the mean ± SEM indicated by the shaded area for each line. C, For cells that exhibited a reduction in firing rate, the rates were significantly lower during the consumption epoch than during the pre- or postconsumption epochs, and firing rates remained lower during the postconsumption period than during the preconsumption epoch. D, In contrast, no significant differences were observed in the preconsumption, consumption, or postconsumption epochs in the cell that were identified as showing an increase in firing rate during the consummatory epoch. Data represent the mean ± SEM indicated by shaded area for each line (A, B) or error bars (C, D); **p < 0.01.