Abstract
The primate superior colliculus (SC) is a midbrain structure that participates in the control of spatial attention. Previous studies examining the role of the SC in attention have mostly used luminance-based visual features (e.g., motion, contrast) as the stimuli and saccadic eye movements as the behavioral response, both of which are known to modulate the activity of SC neurons. To explore the limits of the SC’s involvement in the control of spatial attention, we recorded SC neuronal activity during a task using color, a visual feature dimension not traditionally associated with the SC, and required monkeys to detect threshold-level changes in the saturation of a cued stimulus by releasing a joystick during maintained fixation. Using this color-based spatial attention task, we found substantial cue-related modulation in all categories of visually responsive neurons in the intermediate layers of the SC. Notably, near-threshold changes in color saturation, both increases and decreases, evoked phasic bursts of activity with magnitudes as large as those evoked by stimulus onset. This change-detection activity had two distinctive features: activity for hits was larger than for misses, and the timing of change-detection activity accounted for 67% of joystick release latency, even though it preceded the release by at least 200 ms. We conclude that during attention tasks, SC activity denotes the behavioral relevance of the stimulus regardless of feature dimension and that phasic event-related SC activity is suitable to guide the selection of manual responses as well as saccadic eye movements.
Significance Statement
Control of spatial attention by primate superior colliculus (SC) is traditionally viewed as supporting orienting or targeting behaviors, and has been tested with a limited set of visual features. Additionally, despite SC’s characterization as a novel event detector, analysis of SC neuronal activity has been limited to stimulus onset or delay periods. Here, we document change-detection activity in SC: large phasic bursts shortly after threshold-level isoluminant color saturation changes during a task requiring maintained fixation and manual joystick response. Change-detection activity is modulated by spatial cueing, but is also predictive of response choice and accounts for a substantial proportion of response latency. Activity in SC may thus guide the selection of motor responses to visual events regardless of effector or feature dimension.
Introduction
There is ample evidence that the primate superior colliculus (SC) is important for the control of spatial attention. Intermediate-layer SC neurons exhibit cue-related modulation during attention tasks (Kustov and Robinson, 1996; Ignashchenkova et al., 2004). Microstimulation produces a spatially specific improvement in detection performance at retinotopic locations corresponding to the stimulation site (Cavanaugh and Wurtz, 2004; Müller et al., 2005). Conversely, reversible inactivation causes deficits in attention for stimuli inside the inactivation-affected area of the visual field, as well as difficulty in suppressing responses to distractor stimuli outside that area (Lovejoy and Krauzlis, 2010, Zénon and Krauzlis, 2012).
Several crucial aspects of the SC’s role in spatial attention are unclear. First, studies establishing a causal role for the SC in controlling spatial attention have all used motion (Cavanaugh and Wurtz, 2004; Müller et al., 2005; Lovejoy and Krauzlis, 2010; Zénon and Krauzlis, 2012). This is problematic because motion may have a privileged status in the SC. Several studies have identified motion-sensitive neurons in the SC, although direction selectivity is uncommon (Marrocco and Li, 1977; Moors and Vendrik, 1979; Davidson and Bender, 1991; Krauzlis, 2004). Because the SC is an evolutionarily conserved structure for orienting, the SC may be particularly sensitive to motion because of its salience in predatory-prey interactions.
Second, recording studies have not characterized SC neuronal activity during stimulus events like those used in causal attention experiments (Cavanaugh and Wurtz, 2004; Lovejoy and Krauzlis, 2010; Zénon and Krauzlis, 2012) but, instead, have focused on cue-related modulation. One of the ways that SC activity might contribute to the performance of attention tasks is through the detection of behaviorally relevant events. Indeed, there is evidence that the SC plays a functional role in event-detection in frogs (Lettvin et al., 1961; Gaillard, 1990) and rats (Comoli et al., 2003; Dommett et al., 2005). If similar event-detection activity were present in the primate SC, it could play a central role in the performance of attention tasks.
Third, in most attention studies, monkeys indicate their choice with saccades, introducing possible confounds between attention-related modulation and the effects of saccade planning in the SC. Previous studies have addressed this problem by focusing on epochs of the trial after attention has been cued, but before the choice stimulus has been presented, minimizing the contribution of neuronal activity related to the planning and execution of saccades (Ignashchenkova et al., 2004). However, this approach precludes analysis of possible attention-related effects during the stimulus event that guides the behavioral choice.
In the current study, we addressed these limitations by recording from SC neurons during the performance of an attention task using a novel color stimulus rather than motion, and had monkeys indicate their choice with their hands rather than their eyes. We used a dynamic color stimulus similar to that used in a study of attention in human subjects (Herman et al., 2015); monkeys were required to detect near-threshold changes in the color saturation of a peripheral stimulus and respond with a joystick while maintaining central fixation. Because the processing of color signals involves circuits very different from those involved in motion processing, in particular, detecting changes in color saturation is unlikely to be accomplished without cortical processing, a task with color stimuli provides a good test of whether SC’s role in attention extends to other visual features.
Our results demonstrate substantial attention-related modulation with color stimuli, as well as modest color preferences, both of which varied in magnitude during the trial. Most striking, however, was the phasic activity evoked by threshold-level changes in color saturation, which was as large as the activity evoked by stimulus onset. This activity was predictive of the monkey’s choice to release or hold the joystick, and accounted for a substantial proportion of joystick reaction time (jRT). Thus, beyond its specific roles in processing motion and saccades, the primate SC may signal behaviorally relevant events for the selection of actions in general.
Materials and Methods
General
Data were collected from two adult male rhesus monkeys (Macaca mulatta) weighing 9–12 kg. All experimental protocols were approved by the National Eye Institute Animal Care and Use Committee and all procedures were performed in accordance with the United States Public Health Service policy on the humane care and use of laboratory animals.
Tasks
Monkeys were seated in primate chairs (Crist Instrument) with their heads fixed in a darkened booth. Primate chairs were modified so that a joystick extended toward the animal from the center of the front face of the chair. Gaze was measured with an Eyelink 1000 infrared video tracking system (SR Research), and experiments were orchestrated with a modified version of PLDAPS (Eastman and Huk, 2012).
Attention task trials began when the monkey pressed down on the joystick, triggering the appearance of a 0.25° wide white fixation square (48 cd/m2) against a gray background (28.5 cd/m2), which the monkey was required to fixate (1° window) within 1 s of its appearance (Fig. 1A); fixation was required until reward delivery, and joystick press was required until a stimulus change or reward delivery. After 500 ms of fixation, a white cue-ring (48 cd/m2), was presented [8–11° eccentric depending on receptive field (RF) eccentricity] for 133 ms (inner radius 3.75°, outer radius 4°). Two stimulus patches (3.25° radius) were then presented 500 ms after cue offset (Fig. 1C); the “cued” patch was centered on the former location of the cue-ring, and the “foil” patch was presented at an equally eccentric opposing location: 180° away. After 68 trials with this configuration, the location of the cue-ring was switched to the opposing, 180° away location. A saturation change was possible 1–4 s after stimulus onset (uniform distribution), and stimuli were extinguished 1 s after the change for a total presentation time of 2–5 s. Because of the dynamic nature of the stimulus (see below), the change itself had a duration of 80 ms. If the cued stimulus changed, the monkey was required to release the joystick within a time window 175–750 ms after the change; if the foil or if neither stimulus changed, the monkey was required to maintain joystick press for 1000 ms after the change (or after an identically distributed “sham change time” if neither stimulus changed). If the monkey correctly released the joystick for a cued change or correctly maintained his hold for a foil/neither change, he was given a liquid reward 1000 ms after the stimulus change (or after the sham change), ending the trial. For monkey 1, only one of the two stimuli (cue or foil) could change in each trial, not both; we modified the task structure for monkey 2 to correct an inappropriate behavioral strategy that was not a problem with monkey 1. The results focus on the trial conditions that were identical in both monkeys.
In addition to the main attention task, a visually guided saccade task was used to map RFs and a memory-guided task was used to classify neurons. In these tasks, saccades were detected using a 25°/s threshold on an online estimate of gaze velocity. In visually guided saccade trials a 0.25° wide green fixation square (22 cd/m2) appeared in the center of the screen against a gray background (28.5 cd/m2). The monkey was required to maintain his gaze within a 1–1.5° window around this point until it was extinguished. A 0.25° wide white target square appeared after 500–750 ms, and the fixation was extinguished 1000–1500 ms later, indicating that the monkey should saccade to the target. The monkey was required to start his saccade 100–400 ms after target extinction, had to land within 2–5° of the target (adjusted online depending on target eccentricity), and had to maintain fixation on the target for 375–750 ms to obtain his liquid reward. In memory-guided saccade trials, the fixation square was instead red (23.2 cd/m2). After 1000–1500 ms, the target square was flashed for 100 ms, and the fixation remained on for a further 500–1000 ms before extinguishing. The monkey was required to saccade to the remembered location of the target 100–600 ms after fixation extinction and had to land within 3–6° of the target location to trigger the reappearance of the target stimulus before maintaining target fixation to obtain reward.
Stimuli
Stimuli were displayed at 1920 × 1200 resolution (∼60 × 38°), 100-Hz frame-rate on a VIEWPixx display (VPixx Technologies) controlled by a mid-2010 Mac Pro (Apple) running MATLAB R2012b (The Mathworks) with the Psychophysics Toolbox extensions (Brainard, 1997; Pelli, 1997; Kleiner et al., 2007). Display luminances and chromaticities were measured with a Tektronix J18 LumaColor II Photometer with J1803 Luminance Head and J1810 Chromaticity Head (Tektronix).
Stimulus colors were defined on an isoluminant plane of the DKL color space (Derrington et al., 1984). Conversion between DKL and RGB was achieved by a method described previously (Herman et al., 2015). DKL space was defined using cone contrasts computed from monitor CIE xyY coordinates and the cone fundamentals of Smith and Pokorny (1975). The xyY coordinates for our display were: R (0.68, 0.32, 23.0), G (0.13, 0.71, 40.0), B (0.14, 0.08, 6.4). We scaled each axis of DKL space between −1 and +1, with the ends of this interval corresponding to the maximum possible color contrast on the display used (Hansen et al., 2008).
Our color stimuli were inspired by “random dot kinematogram” motion stimuli (Britten et al., 1992), in that they were dynamic, stochastic, and used fixed-lifetime elements. They consisted of circularly windowed 11 × 11 “checkerboards” of 18-pixel checks (Fig. 1E); windowing kept 109 unique checks. Each check’s color was drawn from a 1D Gaussian (SD = 0.05) on an isoluminant plane of DKL color space ([27.8 cd/m2, 28.2 cd/m2]; Fig. 1D), so that they varied in saturation but not hue; check luminance was then independently set to a value in the interval: [26.5 cd/m2, 29.5 cd/m2]. Thus, regardless of hue or saturation, stimuli were (on average) physically isoluminant with the background, but also had nonzero contrast due to check-to-check variation in luminance. Each check had a lifetime of eight frames (80 ms), during which its color and luminance were static. At the end of a check’s lifetime, new color and luminance values were drawn, and its frame counter was reset. In the first frame of stimulus presentation, check counters were initialized with random “age” values between 1 and 8, so on each subsequent frame only a subset (∼1/8) of checks was reborn. An important consequence of check lifetime is that saturation changes were not complete until all of the checks had been “reborn” eight frames (80 ms) after change onset.
Saturation changes were increases or decreases in the mean of the Gaussian distribution from which color values were drawn (Fig. 1F). In saturation increase trials the mean of the underlying Gaussian distribution was initially 0.1 from the origin (white point) along one of the two axes; at change-onset, the mean was shifted radially outward by 0.03–0.05 so the color of checks born after change-onset would be drawn from a Gaussian with a larger mean (Fig. 1D–F). The magnitude of the change in mean saturation was chosen to keep the monkeys performance at ∼80% (Fig. 1B); saturation decrease magnitudes had to be two to four times larger than increase magnitudes to maintain comparable performance. In sessions with both saturation increases and decreases, the pre-change mean saturation in decrease trials was chosen so that the post-change mean saturation would be identical for increase and decrease trials (Fig. 1D, numbered arrows). Saturation decreases were used in a minority of sessions (n = 22) and were presented in blocks of trials that alternated with saturation increases.
Electrophysiological recordings
Sharp epoxy-coated tungsten electrodes (FHC) were introduced into the SC with a motorized microdrive (Narishige), and putative action potential waveforms were digitized and saved with a Plexon MAP system (Plexon). Once a neuron was stably isolated, the visual saccade task was used to map its RF: a 2D Gaussian was fit to the responses in real-time using an extension of PLDAPS running in MATLAB. Next, 15–30 trials of the memory-guided saccade task were used to characterize the neuron’s baseline, visual, delay, and saccadic activity with the saccade target placed at the center of the neuron’s RF (or movement field if it was not visually responsive). Next, attention task data were gathered, initially placing either the cued or the foil patch such that it overlapped with the RF center. As mentioned above, the location of the cue-ring was alternated every 68 trials so that cue and foil were presented in the neuron’s RF in alternate blocks. After the attention task, the memory guided saccade task was repeated to confirm the neuron’s identity. All spike data were sorted offline.
Neuron type categorization
We used spike rates from four time windows in the memory guided saccade task to categorize neurons (McPeek and Keller, 2002): baseline (−75 to +25 ms from target onset), visual (+50 to +200 ms from target onset), delay (−150 to +50 ms from fixation offset), and saccade (−25 ms from saccade onset up to saccade offset). Saccade onset and offset times were calculated offline using combined velocity and acceleration criteria and verified by inspection. We then computed a one-way Kruskal-Wallis nonparametric ANOVA using spike rates in each window for each trial and used post hoc testing to determine the presence of significant visual, delay, or saccade activity compared with baseline. Neurons with significant visual, delay, and saccade activity were labeled “visual-movement prelude” (VMp) (n = 26), those with significant visual and delay activity were “visual-delay” (n = 9), significant delay and saccade activity were “movement-delay” (n = 8), significant visual and saccade were “visual-movement” (VM) (n = 45), significant visual only were “visual” (V) (n = 42), and significant saccade only were “movement” (n = 9).
Data analysis
Analyses of covariance (ANCOVAs) were performed in R (R Core Team, 2016), all other analyses were performed in MATLAB. A summary of statistical tests can be found in Table 1.
Average firing rate traces were computed by convolving individual spike trains with a fast rising (τ = 1 ms), slow falling (τ = 20 ms) combination of exponentials resembling a postsynaptic potential (Thompson et al., 1996). In all figures and for all analyses, stimulus-onset-aligned data were truncated at change-onset.
Because we used a limited number of hues (Fig. 1D), more traditional tuning curves (e.g., White et al., 2009, their Fig. 1) would have been difficult to interpret. Instead, we characterized individual neuron color preferences by comparing evoked activity between the color pairs “yellow”/“blue,” and “red”/“green” (Fig. 1E), which lie along two of the axes of the DKL colorspace (Derrington et al., 1984). A neuron’s preference in each pair was computed from spike counts in nonoverlapping 20 ms bins (aligned either on stimulus onset or change). For each unit, receiver operating characteristic (ROC) area was calculated between spike counts in each bin for yellow stimulus presentations and counts for blue stimulus presentations (Fig. 1E). A bootstrapping procedure was used to compute the confidence interval on ROC area in each bin, and if that confidence interval was completely above or below 0.5, the preference in that bin was considered significant; the position (above or below) dictated whether the preference was for one or the other member of a pair. The confidence interval was the values marking the 2.5th and the 97.5th percentiles of a 10,000 sample bootstrapped distribution of ROC area. Each sample of the distribution was an ROC area computed on spike counts drawn with replacement from the originals without combining them. Population preferences were the proportion of units with a significant preference for a color in each bin (Fig. 2B,D). A comparable method, bin-by-bin estimation of bootstrapped ROC area confidence interval, was used to compare activity for the cued stimulus to that evoked by the foil (Fig. 3C), activity evoked by saturation increases to decreases (Fig. 5B), and of hits to that of misses (Fig. 6B). These analyses follow common practice in the analysis of neuronal data by choosing time-windows of interest, dividing each window into bins, and interrogating each bin for statistical significance. Because the number of tests is not determined by the structure of the data itself but arbitrarily by the choice of time window and bin duration, we chose not to correct these tests. However, it is important to point out that the risk of Type I errors (false positives) is increased by conducting multiple tests.
A χ2-proportion test was used to compare population preferences for one or the other member of a color pair, as well as to compare cue/foil activity and hits/misses activity at the population level. This test evaluates the hypothesis that two or more proportions are different with the null hypothesis that the proportions are the same (Fleiss et al., 2013).
An attention-related modulation index (AMI) was computed for individual units from spike counts in 20 ms nonoverlapping bins for cue and foil stimuli as: cue – foil/cue + foil. Individual AMIs were averaged to obtain the population AMI (Fig. 3B).
To compare peak onset-evoked and change-evoked firing rates across neurons, we extracted each neuron’s maximum firing rate 50–200 ms after stimulus onset and 100–300 ms after stimulus change from average firing rate traces; different time windows were used to account for the slower time course of change-evoked compared with onset-evoked activity.
To compare change-detection activity for hits to change-detection activity for misses, we chose a time window comprising the top 90% of the peak of change-detection activity. We first found the maximum of the change-aligned firing rate, averaged across all 139 units. Then, we located the first point before and the first point after the peak where the firing rate trace crossed 90% of the maximum, these being 190 and 290 ms. This window was also used to assess the significance of change-detection activity in individual units: we compared the spike counts in the peak window to the spike counts in a baseline window extending from −100 to 0 ms relative to the change using a Wilcoxon rank sum test.
To quantify the contribution of several factors to differences in the timing of change-detection activity observed in different reaction time quantiles, we used a “hinge” model that has been used previously to determine the onset latency of smooth pursuit eye movements (Krauzlis and Miles, 1996; Adler et al., 2002). This model had three parameters: baseline offset, “start-of-rise” time, and “rate-of-rise”; baseline offset was fixed to the average firing rate in the 100 ms immediately preceding the color change, while start-of-rise time and rate of rise were fit to the rising portion of the change detection activity. We selected the rising portion of the change-detection activity using the mean baseline firing rate (μb), the peak firing rate (p), and the standard deviation of the baseline firing rate (σb), by finding the portion of activity satisfying: μb + 2σb < activity < p − σb. The rate-of-rise was the slope of a line fit to the selected rising portion of the change-detection activity with linear regression, and the start-of-rise time was the time where the fitted line had a value equal to the baseline mean (Fig. 7B).
To determine the proportion of joystick release time (jRT) accounted for by the timing of change-detection activity in each unit, we followed a method based on Thompson et al. (1996). We split the distribution of jRT for hits into three quantiles (fast, 0–33rd percentile; medium, 33rd–66th percentile; and slow, 66th–100th percentile; Fig. 7A), and fit the hinge model to the average firing rate of each (Fig. 7B). Change detection activity timing was quantified by setting a threshold firing rate to the smallest “end-of-rise” (p − σb) over the three quantiles, and finding the “threshold time” when each quantile’s hinge-fit crossed threshold. For each unit, we calculated the proportion of jRT accounted for by threshold time (Fig. 7F) as the average slope of the relationship between threshold time and median jRT (Fig. 7E); average slope was determined from the slopes of the two lines connecting the three data points in Figure 7E, producing a single value for each neuron in Figure 7F. The proportion of jRT accounted for by start-of-rise, rate-of-rise, and baseline offset values were calculated similarly: by taking the average slope of the relationship between the value and median jRT (Fig. 7G–I).
Results
Two monkeys performed a covert spatial attention task requiring them to release a joystick in response to saturation changes at a cued location and to maintain their hold for changes at an opposing foil location or if no changes occurred (Fig. 1A). Hit rates (79% and 81%) and false alarm rates (6% in both) were comparable in the two animals (Fig. 1B). In 136 sessions, we recorded extracellular activity from 139 neurons throughout the SC. These neurons were found between 0.25 and 3 mm from the surface of the SC, primarily in the intermediate and deep layers, and the majority (n = 113) were categorized as V (n = 42), VM (n = 45), or VMp (n = 26), as defined in Materials and Methods. RF or movement field centers were at 6–14° of eccentricity, and ±60° from the horizontal.
During the task, SC neuron activity had several components: (1) phasic responses to task events, (2) modulation by cue condition, (3) modulation by stimulus color, and (4) information about response choice and response timing. We start by characterizing how SC activity varied with stimulus color.
Color preferences of SC neurons during the task
The activity of SC neurons during the attention task was somewhat different for differently colored stimuli, and the magnitude of these color preferences varied throughout the trial. The example neuron in Figure 2A had higher firing rate for yellow than for blue directly after stimulus onset and persisting until 500 ms after (Fig. 2A, left). Both yellow and blue stimulus change events evoked large phasic increases in firing rate (Fig. 2A, right), with the activity for yellow rising slightly earlier than for blue (Fig. 2A, right, inset). The same example neuron responded earlier for red than for green at stimulus onset, but the peak onset response was larger for green (Fig. 2C, left). This neuron also showed slightly larger peak activity for red saturation changes than for green (Fig. 2C, right). We quantified individual unit color preferences using bootstrapped ROC area (see Materials and Methods), and combined individual preferences to determine population-level preferences.
Population-level color preferences were strongest at stimulus onset. Approximately 60% (84/139) of neurons showed a preference for yellow over blue at stimulus onset (80–100 ms bin; Fig. 2B, left), and 80% (25/31) preferred red over green (60–80 ms bin; Fig. 2C, left). Preference for yellow over blue settled at just under 25% of neurons ∼250 ms after onset (Fig. 2B, left), and later decreased slightly until just after the stimulus change event (Fig. 2B, right). In contrast, a variable and lower percentage of neurons preferred red over green from 250 ms after onset (Fig. 2D, left) up to the change event (Fig. 2D, right). The increased preference for yellow following stimulus change (Fig. 2B, right) stems from a tendency for yellow change-evoked activity to rise earlier than blue (as in Figure 2B, right, inset). A similar difference in timing also explains the small increase in red preference over green just after the change (Fig. 2D, right). To quantify overall population preferences, we compared the proportion of units exhibiting a preference for yellow versus blue and red versus green in each bin with a χ2-proportion test. The preference for yellow over blue was significant in 153/210 bins, with significant preference for blue over yellow in just 13/210 (all significant χ2 > 3.95). Meanwhile, the preference for red over green was significant in 31/210 bins, and for green over red in 7/210 (all significant χ2 > 3.96; see Materials and Methods for a note regarding Type I errors and large numbers of bins).
Overall, SC neurons showed modest color preferences that were strongest at stimulus onset and around the time of the stimulus change. We next describe the much stronger modulatory effects of the spatial cue, and briefly contrast the effects of cue and color.
Cue-related modulation
Cue-related modulation was evident for neurons in each of the three main categories of SC neurons we studied, and this cue-related modulation was largest for VMp neurons (Fig. 3A).
We measured cue-related modulation in two ways: (1) we computed an attention modulation index (AMI = cue – foil/cue + foil), and (2) we computed the percentage of units in each category showing a significant preference for the cue over the foil using bootstrapped ROC area. Notably, AMI increased in the 500 ms before the window of time during which color changes were possible (1–4 s after stimulus onset), then appeared to ramp up slowly, reaching a maximum just before the stimulus change. On average, AMI was smallest for V, larger for VM, and largest for VMp neurons (Fig. 3B). Similarly, the percentage of units with a significant preference for the cue was smallest for V, larger for VM, and largest for VMp neurons (Fig. 3C).
We quantified the differences in AMI across unit types with two ANOVAs, one in a window aligned on stimulus onset (1–1.5 s) and one on change (-0.5 to 0 s). We found a significant effect of category on AMI in both ANOVAs (onset window: F(2110) = 5.95, p < 0.01; change window: F(2110) = 4.49, p < 0.05). Post hoc testing demonstrated that AMI for both VM and VMp was significantly greater than V, but not different between VM and VMp (α = 0.05). We also assessed the significance of any differences in population percentage cue preference across neuron categories with a χ2-proportion test in each of the 105 20 ms time bins covering the 2.1 s shown in Figure 3C. In a small number of bins, a larger percentage of VM neurons had a significant cue preference than V neurons (16/105 onset aligned bins, 3/105 change aligned; all significant χ2 > 3.92), but in contrast to the ANOVA results, VMp neurons had a greater percentage cue preference than VM in a large number of bins (73/105 onset aligned, 48/105 change aligned; all significant χ2 > 3.84). Thus, despite showing a statistically similar magnitude of cue-related modulation, VMp neurons were more likely to be significantly modulated by the cue than VM.
Cue preferences and color preferences changed in distinct ways during the trial. To compare the time course of cue and color preferences, we computed category-specific yellow-blue preferences across neurons (Fig. 3D). Both cue preference and color preference (yellow over blue) showed an onset transient at stimulus onset, but differences emerged later in the trial. In the window from 0.5 to 1 s after onset, cue preference increased while color preference was static. In the window from −0.5 to 0 s before the stimulus change, cue preference was static or increased while color preference decreased. Immediately following the stimulus change, there was a transient increase in color preference while cue preference decreased. We quantified the opposing trends in preference during the 0.5 to 1 s after onset window and −0.5 to 0 s before change window using ANCOVA. Both included factors of preference type (cue or yellow), unit type (V, VM, or VMp), and interaction terms. In both ANCOVAs, the interaction between time and preference type was significant (onset window: F(1140) = 74.111, p ≪ 0.01; change window: F(1140) = 15.092, p < 0.01), supporting the interpretation that cue and color preferences did not change in unison during the task.
Change-detection activity
The foregoing analysis examined color preferences and cue-related modulation of onset-aligned and change-aligned SC activity. We now focus on the phasic increase in firing rate evoked by the saturation change in the color stimulus, which we term “change-detection activity”; this activity was significant in the vast majority of neurons (96%, 133/139; Wilcoxon rank sum test, all p < 0.05)
Despite the fact that the saturation change was adjusted to be near each monkey’s detection threshold, the magnitude of change-detection activity was about the same as stimulus-onset activity. Irrespective of unit-type category (V, VM, or VMp), the peak of the average firing rate evoked by the stimulus change was similar to the peak evoked by stimulus onset both for the cued stimulus (Fig. 4A) and for the foil (Fig. 4B). To determine quantitatively whether there was any systematic variation in onset-evoked or change-evoked peak firing rate that depended on neuron category, stimulus identity, or combinations thereof, we conducted a three-way ANOVA on single neuron peak firing rates (see Materials and Methods). The factors in this ANOVA were (1) trial epoch (onset or change), (2) stimulus identity (cue or foil), and (3) unit-type category (V, VM, or VMp). The effect of trial epoch was not significant (F(1140) = 2.36, p = 0.13), nor were any of the interaction terms (all p > 0.08), demonstrating that across our population of neurons there was no difference between onset-evoked and change-evoked peak firing rate, despite the fact that the stimulus saturation changes were small (0.03–0.05 DKL units) and gradual (80 ms duration), unlike the large luminance contrast and abrupt onset of the color patches.
Change-detection activity was similar for saturation increases and saturation decreases. For a small set of isolated neurons (n = 22), we alternated blocks of saturation-increase trials and saturation-decrease trials, choosing stimulus parameters to maintain comparable hit and false-alarm rates for the two types of change. The magnitude of change-detection activity was comparable between saturation increases and decreases in the population average firing rate (Fig. 5A), and also for individual neurons (Fig. 5A, inset). A paired t test revealed that peak change-detection activity for saturation increases was not different from the peak for saturation decreases (p = 0.89). However, the average response to saturation decreases appeared to have slower temporal dynamics than saturation increases. Accordingly, across the population of neurons there was a preference for saturation increases between 100 and 200 ms after the change, followed by a preference for saturation decreases from ∼300 to 900 ms (Fig. 5B). The pattern of preferences for individual neurons suggests that these effects were largely separable: one subset of units showed a transient early preference for saturation increases (neurons 9–22), and the remainder showed an overall preference for saturation decreases (neurons 2–8; Fig. 5C).
Behavioral information in change-detection activity
Hits were associated with larger change-detection activity than misses. For all categories of units (V, VM, VMp), average change-detection firing rate was higher when it preceded a hit (i.e., correct joystick release) than when it preceded a miss (Fig. 6A). The difference in average firing rate for hits and misses emerged at the onset of change-detection activity and was maximal at the peak evoked firing rate. We quantified these differences between hits and misses in two ways: (1) we calculated the percentage of neurons in each category with significantly greater activity for hits than misses (from bootstrapped ROC area), and (2) we calculated the population average ROC area, referred to as “detect probability” (Cook and Maunsell, 2002; Nienborg and Cumming, 2009). VM and VMp neurons were both more likely than V to show significantly increased activity for hits over misses (Fig. 6B), and showed a larger detect probability (Fig. 6C). To quantitatively compare activity for hits and misses, we performed a two-way ANOVA on spike counts in a time window comprising the top 90% of the peak of change-detection activity (190–290 ms). The factors in the ANOVA were: (1) behavioral response (hit or miss), and (2) unit-type category (V, VM, or VMp). Both factors proved significant (behavioral response: F(2220) = 7.38, p ≪ 0.01; unit-type: F(1220) = 3.93, p < 0.05), but their interaction was not (F(2220) = 0.26, p = 0.77). The lack of significance of the interaction term suggests that the peak difference between hits and misses did not differ significantly among the categories of units. We also compared the proportion of neurons with a significant preference for hits versus misses across categories with a χ2-proportion test; this test was run in each of several 20 ms bins (Fig. 6B), and also in the 190–290 ms ANOVA window. Although VM and VMp neurons showed larger percentages of neurons with a significant preference (Fig. 6B), in agreement with ANOVA results, % hits preference did not differ significantly across categories (bins: all χ2 < 5.26, all p > 0.07; ANOVA window: χ2 = 2.78, p = 0.25).
The timing of change-evoked activity accounted for a substantial proportion of jRT. To illustrate this, we divided the data from hit trials in each session into three latency quantiles (fast, middle, and slow), and plotted average firing rates for each (Fig. 7A). The clear difference in the timing of firing rate activity along with joystick release time (jRT) led us to ask two questions. (1) What proportion of jRT could be accounted for by the timing of firing rate activity? And (2) what factor or factors gave rise to differences in the timing of the firing rate activity? To address these questions quantitatively, we fit a simple hinge model to each unit’s average firing rate data for each quantile (Fig. 7B). This model consisted of a constant baseline portion and a linear-rise portion (see Materials and Methods). The constant baseline was set to the average firing rate in the 100 ms preceding the stimulus change. The linear-rise portion was the least-squares fit to the segment of the average firing rate trace between 2 SD above baseline and 1 SD below the peak (SD calculated from the 100 ms before change). We termed the intersection point of the baseline and linear-rise portions “start-of-rise”, and the −1 SD from peak values “end-of-rise”. The smallest end-of-rise value for each unit was used as a criterion firing rate value, and the “threshold time” at which each quantile’s hinge-fit crossed this value was calculated.
Using this model, we found a clear trend in the relationship between the threshold time and the median in-quantile jRT for each unit (Fig. 7E). On average, threshold time accounted for 67% of the jRT (Fig. 7F): if the median jRT of the fast and slow groups differed by 100 ms, their threshold times would differ by 67 ms. We next considered the three factors that contribute to threshold time differences in the hinge model; these factors are: (1) start-of-rise (Fig. 7B), (2) rate-of-rise (Fig. 7C), and (3) baseline offset (Fig. 7D). We computed the relative contributions of these components (Fig. 7G–I) and found that rate-of-rise accounted for a substantially greater proportion of the total than the other two factors. A one-way ANOVA on these proportions revealed a significant main effect (F(2312) = 21.38, p ≪ 0.01), and that differences in rate-of-rise among quantiles contributed a significantly greater proportion than did baseline differences or start-of-rise differences (Tukey-Kramer post hoc tests, α = 0.05). Thus, the timing of change-detection activity, in particular, the rate-of-rise after the saturation change, accounted for more than half of the jRT, even though the joystick was released more than 200 ms after this phasic activity.
Discussion
We have shown that neurons in the primate SC exhibit large phasic increases in firing rate shortly after near-threshold changes in color saturation. This change-detection activity was as large as the stimulus-onset response, and was evoked by both saturation increases and decreases. Change-detection activity was significantly larger when monkeys released the joystick (hits) than when they maintained their hold (misses), and its timing accounted for reaction time despite occurring several hundred milliseconds before joystick release. The same neuron types whose change-detection activity best predicted behavioral choice (VM and VMp) also showed the largest cue-related modulation throughout the trial. We also found modest color preferences along the cardinal axes of DKL color space (“blue-yellow” and “red-green”), with the population preferring yellow over blue and red over green.
Change-detection activity integrates event detection and behavioral relevance
Change-detection in our color task is a novel example of the way SC neurons indicate the time and location of behaviorally relevant visual events. The phrase “behavioral relevance” unifies diverse modulatory effects on SC activity, for example: spatial cueing (Ignashchenkova et al., 2004), target probability (Basso and Wurtz, 1998), and target identity (McPeek and Keller, 2002). These effects have primarily been seen as supporting orienting or targeting behaviors. We instead speculate that SC activity guides action selection more generally by signaling which sensory data should be used to choose an appropriate response to a visual event.
Supporting this interpretation, we found SC activity was not only modulated by behavioral relevance but was also predictive of the joystick release, arguably neither an orienting nor a targeting response. Change-detection activity was larger for the cue than for the foil stimulus, but cued change-detection activity was larger still for hits than for misses (Fig. 6). We also found that the timing of change-detection activity accounted for a substantial portion (67%) of the joystick release latency. These findings demonstrate that SC activity indicates if and when the monkey will release the joystick in response to a stimulus change.
Previous work in SC has characterized its role as reporting novel visual onsets primarily at the service of orienting. Lettvin et al. (1961) roughly partitioned SC neurons in the frog as responding either to “newness” or “sameness” based on habituation to stimulus persistence or repetition. Boehnke et al. (2011) described a subpopulation of macaque SC neurons as detecting novel events, finding that both brighter and darker oddballs evoked responses with stronger sustained components (∼75–90 to 200 ms) than nonoddballs. Boehnke and Munoz (2008) proposed that the visual onset response in macaque SC integrates stimulus information as well as relevance and expectation, and influences future actions and immediate orienting. Our results broaden this line of thought, showing that not only onsets, but also near-threshold changes can evoke SC activity, and that this activity can be used not only for orienting, but also for other nonorienting motor responses.
How might the SC accomplish this more general role? One possibility is that the SC reports the detection of salient events to the substantia nigra pars compacta (SNc). Dopaminergic neurons in macaque SNc respond to novel stimulus events or those associated with reward at short latency (<100 ms; Schultz, 1998). Rat SNc dopaminergic neurons have similar responses, mediated by the tectonigral projection from SC to SNc (Comoli et al., 2003; Dommett et al., 2005). Redgrave and Gurney (2006) hypothesize that salient event-detection signals from SC allow SNc to reinforce novel actions that caused the event: if an animal unintentionally triggers a rewarding sensory event, the behaviorally relevant event is reported to SNc by SC, and the preceding actions are reinforced by dopamine release in the striatum. To successfully perform our task, animals instead needed a reliable link between a behaviorally relevant event (cued saturation change) and a subsequent action (joystick release); the recruitment of SNc dopaminergic neurons by SC change-detection activity may have been important for learning that link between event and action.
Change-detection activity is unlikely to be a manifestation of previously observed reach-related activity in the SC and underlying mesencephalic reticular formation (Werner, 1993; Werner et al. 1997b; Stuphorn et al., 1999; 2000). The vast majority of our neurons had visual responses, which is uncommon in reach-sensitive neurons (Werner et al., 1997a). Also, reach-sensitive neurons are mostly found deeper than 3 mm from SC’s surface, but our neurons were all isolated between 0.25 and 3 mm. Lastly, reach-related activity generally persists during the movement and often after (Werner et al., 1997a; 1997b; Stuphorn et al., 1999; 2000), whereas change-detection activity terminated 100–200 ms before the joystick-release. Thus, our results are consistent with a role for the SC in processing visual events regardless of the effector used to respond (Nummela and Krauzlis, 2010).
The relationship between change-detection and cue-related modulation
Among SC neurons, those with VM activity (i.e., VM and VMp) are most strongly implicated in the integration of cue-related and change-detection signals. These neurons showed both the largest cue-related modulation, consistent with previous studies (Kustov and Robinson 1996; Ignashchenkova et al., 2004) and best predicted joystick release. It is probable that cue-related enhancement of activity increases the likelihood of larger change-detection activity and thus of a joystick release, whereas cue-related reduction of foil activity leads to smaller change-detection activity and to joystick hold. Situated in the intermediate and deep layers of SC, these neurons could be the origin of event-detection signals sent to SNc (May et al., 2009), and are anatomically poised to integrate a variety of signals from visual and fronto-parietal areas of cortex (Lock et al., 2003). The excitatory inputs from cortex are also presumably the source of the color-change signals that were instrumental in our task.
The SC also receives inhibitory inputs from the substantia nigra pars reticulata (SNr), and we speculate that these may play an as-yet unappreciated role in modulating SC activity during attention tasks. The SNr projects selectively to intermediate and deep SC (Hikosaka and Wurtz, 1983; Jiang et al., 2003) where it modulates neuronal firing via GABAergic synapses (Isa et al., 1998; Kaneda et al., 2008). Inhibition from the SNr likely mediates reward-based modulation of saccade latency and peak velocity (Ikeda and Hikosaka, 2003), and phasic reductions in SNr neuronal activity have been observed for target-identifying luminance changes (Basso and Wurtz, 2002), but the SNr’s influence need not be limited to saccades. In our task, similar SNr inhibition could be responsible for reducing the excitability of SC neurons responding to the foil stimulus, so that they emit lower activity for behaviorally irrelevant events.
The SC and color
We used color in our task because it offers a good test of whether the SC’s role in the control of spatial attention extends across visual features. Earlier work in anesthetized monkeys concluded the retinotectal projection was not color opponent (Schiller and Malpeli, 1977; de Monasterio, 1978), the indirect geniculostriate input to SC was not color opponent (Schiller et al., 1979), and that SC neurons are chromatically nonopponent (Marrocco and Li, 1977). More recently, work in awake behaving monkeys, perhaps a key difference, found evidence consistent with cortically derived color-opponent inputs to SC (White et al., 2009) and S-cone inputs (Hall and Colby, 2014). White et al. (2009) found that SC neurons respond to isoluminant color stimuli across a range of hues, and speculated that SC’s color information might come from cortical area V4, or from other geniculostriate input. Challenging the widespread belief that S-cone stimuli are “invisible” to the SC (Sumner et al., 2002), Hall and Colby (2014) found that “nearly all visual SC neurons” respond to S-cone isolating stimuli, and suggested that SC’s S-cone sensitivity might arise from previously underappreciated components of the retinotectal projection.
Saturation-change information in the SC may derive from neurons in the “globs” of macaque posterior inferior temporal cortex and area V4. Glob neurons have several properties that argue for their importance in color perception (Conway et al., 2007; Conway and Tsao, 2009; Namima et al., 2014), and modeling work has concluded that their responses are consistent with a sensitivity to saturation (Bohon et al., 2016). V4 neurons are known to show vigorous phasic activity shortly after isoluminant changes in color (Womelsdorf et al., 2006; Fries et al., 2008, Zhou et al., 2016), and to respond strongly to orientation changes (Cohen and Maunsell, 2010). The activity evoked by orientation changes is suggestive of a generic change-detection signal, because it occurs even when the changed orientation is not at the peak of the neuron’s tuning for orientation. These V4 signals could influence SC activity by a direction projection to the intermediate and deep layers (Lock et al., 2003), but whether glob neurons project to SC is unknown.
In summary, visual cortical areas are likely required to generate the saturation-change activity we found in SC neurons, but the mechanisms that give rise to this phasic activity in the SC have not yet been identified. Saturation-change activity in SC is unlikely to result from intrinsic processing for several reasons. Saturation-change activity was evident in neurons throughout the layers of SC, which receive very different inputs: retinotectal projections exclusively target superficial layers (Schiller and Malpeli, 1977), and distinct cortical areas target superficial compared with intermediate and deep layers (Lock et al., 2003). Saturation-change activity was present in neurons with different functional properties: purely V, VM, and VMp neurons all showed this response. Individual neurons showed saturation-change activity for saturation increases and decreases and along multiple directions in DKL color space. Instead, the color-change activity we found with our stimuli appears to involve a mechanism that reads out visual cortical activity specifically for the purpose of detecting changes, and provides signals to the SC consistent with its long evolutionary history of detecting behaviorally relevant events.
Conclusion
We have shown that subtle changes in color saturation evoke vigorous phasic increases in firing rate in primate SC neurons in the absence of orienting behaviors. Change-detection activity is both modulated by spatial cueing, and predictive of manual choice behaviors. We conclude that change-detection activity in SC is suitable to guide action selection, regardless of the visual feature that is monitored or the specific motor response that follows.
Acknowledgments
Acknowledgments: We thank Fabrice Arcizet, Anil Bollimunta, Lupeng Wang, and Amar Bogadhi for helpful input and Denise Richards, Irina Bunea, and Betre Legesse for invaluable assistance with animal care.
Footnotes
The authors declare no competing financial interests.
This work was supported by the National Eye Institute Intramural Research Program at the National Institutes of Health.
This is an open-access article distributed under the terms of the Creative Commons Attribution 4.0 International license, which permits unrestricted use, distribution and reproduction in any medium provided that the original work is properly attributed.