Abstract
Chromatic mechanisms have been studied extensively with psychophysical techniques in humans, but the number and nature of the mechanisms are still controversial. Appeals to monkey neurophysiology are often used to sort out the competing claims and to test hypotheses arising from the experiments in humans, but psychophysical chromatic mechanisms have never been assessed in monkeys. Here we address this issue by measuring color-detection thresholds in monkeys before and after chromatic adaptation, employing a standard approach used to determine chromatic mechanisms in humans. We conducted separate experiments using adaptation configured as either flickering full-field colors or heterochromatic gratings. Full-field colors would favor activity within the visual system at or before the arrival of retinal signals to V1, before the spatial transformation of color signals by the cortex. Conversely, gratings would favor activity within the cortex where neurons are often sensitive to spatial chromatic structure. Detection thresholds were selectively elevated for the colors of full-field adaptation when it modulated along either of the two cardinal chromatic axes that define cone-opponent color space [L vs M or S vs (L + M)], providing evidence for two privileged cardinal chromatic mechanisms implemented early in the visual-processing hierarchy. Adaptation with gratings produced elevated thresholds for colors of the adaptation regardless of its chromatic makeup, suggesting a cortical representation comprised of multiple higher-order mechanisms each selective for a different direction in color space. The results suggest that color is represented by two cardinal channels early in the processing hierarchy and many chromatic channels in brain regions closer to perceptual readout.
Introduction
In an influential paper, Krauskopf et al. (1982) provided the psychophysical foundation for the widely held notion that the neural representation for color, subsequent to the cones (L, M, S), is mediated by two cardinal chromatic mechanisms, one comparing L and M signals and one comparing S signals to a combined L + M signal. Krauskopf et al. demonstrated these mechanisms by determining the cone contrast of targets at detection threshold, before and after adaptation to temporally modulated full-field (FF) colors. They observed that thresholds were specifically elevated for colors of the adapting stimulus if the adaptation selectively modulated only the L and M cones or only the S cones. For example, after viewing flickering reddish/greenish fields (Fig. 1A, horizontal axis), thresholds for reddish and greenish spots were elevated, whereas thresholds for other colors were little affected. Krauskopf et al. observed that thresholds were raised for all color directions around the color circle after adaptation intermediate to the cardinal axes. Subsequent experiments have provided additional support for the special status of the cardinal mechanisms (Stromeyer and Lee, 1988; Krauskopf and Farell, 1990; Nagy and Sanchez, 1990; Krauskopf and Gegenfurtner, 1992; Sankeralli and Mullen, 1997; Giulianini and Eskew, 1998; Eskew et al., 2001; Gunther and Dobkins, 2003). Other experiments have implicated more than two “higher-order” mechanisms each tuned to a different direction in color space. The first such evidence, by Krauskopf et al. (1986), depended on a Fourier analysis of data originally used to endorse the cardinal-mechanism model. Additional evidence, using a range of paradigms, followed (Boynton et al., 1986; Krauskopf et al., 1986; Flanagan et al., 1990; D'Zmura, 1991; Gegenfurtner and Kiper, 1992; Krauskopf and Gegenfurtner, 1992; Sachtler and Zaidi, 1992; Zaidi and Halevy, 1993; Webster and Mollon, 1994; D'Zmura and Knoblauch, 1998; Hansen and Gegenfurtner, 2006).
The various psychophysical results, all obtained in humans, make predictions about the underlying neural representation for color. The cardinal mechanisms have been accounted for by the chromatic tuning of neurons in the parvocellular and koniocellular layers of monkey lateral geniculate nucleus (Derrington et al., 1984; Sun et al., 2006). By the same token, evidence for multiple higher-order mechanisms has been attributed to color-tuned neurons in monkey striate and/or extrastriate cortex (Flanagan et al., 1990; Webster and Mollon, 1994; Hansen and Gegenfurtner, 2006). But psychophysical chromatic mechanisms have never been evaluated in monkeys, undermining these claims. To provide an explicit link between psychophysical chromatic mechanisms and color physiology and to address the controversy regarding the number and nature of chromatic mechanisms (Eskew, 2009), we conducted two sets of psychophysical experiments in monkeys. In one set of experiments, we measured color-detection thresholds after FF chromatic adaptation similar to Krauskopf et al. (1982, 1986). Such stimuli would theoretically affect color-coding mechanisms that are not tuned for spatial structure, such as subcortical cone-opponent signals. In a second set of experiments, we used spatially structured adaptation [heterochromatic gratings (Gr)], which would theoretically affect chromatic mechanisms implemented after the spatial transformation of color signals achieved by the cortex.
Materials and Methods
All experiments were approved by the institutional animal care and use committees at Harvard Medical School and Wellesley College.
Behavioral training and experimental setup.
Two male macaque monkeys were used in the experiment. Head posts were secured to the animals' skulls with standard surgical techniques (Conway et al., 2007). Head fixation enabled accurate eye-movement monitoring. Using positive reinforcement, animals were trained to enter a primate chair, tolerate head fixation, and perform the detection tasks under dark ambient light conditions on a color-calibrated CRT monitor (60 Hz refresh rate; Barco Display Systems) 37 inches away from the animal. Stimuli were presented using software written in MATLAB using Psychophysics Toolbox extensions (Brainard, 1997; Pelli, 1997; Kleiner et al., 2007). A 14-bit digital-to-analog converter (Bits++; Cambridge Research Systems) drove the stimuli, which were determined using custom-written MATLAB code (provided by Thorsten Hansen, Justus Liebig University Giessen, Giessen, Germany) and calibrated with a PR 655 spectroradiometer (Photo Research). An infrared-based eye tracker (ISCAN) monitored eye position.
Stimuli.
Colors were defined by the Derrington–Krauskopf–Lennie (DKL) color space used in many psychophysical experiments in humans (Krauskopf et al., 1982, 1986; Webster and Mollon, 1991; Gegenfurtner and Kiper, 1992; Hansen and Gegenfurtner, 2006) and physiological experiments in monkeys (Derrington et al., 1984; Lennie et al., 1990; Gegenfurtner et al., 1997; Tailby et al., 2008; Lafer-Sousa et al., 2012). The Judd-corrected Commission Internationale de l'Éclairage xyY coordinates of all stimuli can be found at http://www.wellesley.edu/neuroscience/conway/stoughtonetal2012. Within the equiluminant plane of the DKL space, a given color direction includes the colors of increasing cone contrast falling along a vector beginning at the neutral-gray origin (Fig. 1A)—colors progressively farther away from the origin along a vector appear more saturated, although they vary slightly in hue (constant-hue lines are curved, Abney effect). Colors that selectively modulate just the L and M cones are depicted along the x-axis, whereas colors that selectively modulate just the S cones are depicted along the y-axis. Color directions intermediate to these cardinal axes are depicted in the rest of the space and modulate the activity of all three cone types. The maximum attainable cone contrast of the stimuli was constrained by the gamut of the monitor; maxima were defined as 1 within these device-dependent units (DDUs). The highest cone contrast of stimuli used was 0.9 DDUs. Cone activations are linearly related to DDUs (see Fig. 3A). Relative L-cone contrast was calculated as follows: where a was the target, and b was the background. Cone activation was determined as the dot product of the spectral emission of the display and the L-cone absorption spectra. Mcontrast and Scontrast were determined similarly. All stimuli were obtained with Smith and Pokorny (1972, 1975) cone fundamentals. Table 1 shows the cone contrasts along the cardinal axes (L–M and S) for the most saturated target stimuli used. For comparison, Table 1 also provides cone contrasts computed using Stockman and Sharpe fundamentals (2000). We elected to measure detection thresholds to single colors (as opposed to heterochromatic gratings) to reveal possible asymmetries in the detection of the two colors comprising a given adaptation axis.
The adapting colors could be one of four combinations. One combination selectively modulated along the L–M cardinal direction (Card1) (Fig. 1A, horizontal axis, red–green axis). Varying in chromaticity over time (FF) or over time and space (Gr), this stimulus appeared reddish, then gray, then greenish, and the cycle repeated. Another combination modulated along the S cardinal direction (Card2) (Fig. 1A, vertical axis, blue–yellow axis). The two remaining adaptation conditions involved color combinations defined by the 45° intermediate diagonals through DKL space, between cyan and orange (Int1) or between lime and magenta (Int2). Relative cone contrasts for each adapting stimulus (Table 2) were calculated as follows: where a and b were the two colors comprising the adaptation stimulus. Note that the two intermediate conditions (Int1 and Int2) elicited the same average relative activation of the two cardinal mechanisms.
There is no consensus on the appropriate units for habituation stimuli. As described above, the stimuli in this study were generated in DDUs, which is one common practice. Other studies report the magnitude of the adaptation stimulus in multiples of the detection threshold. We measured detection thresholds to individual colors from which the adaptation stimuli were constructed but not for the adaptation stimuli themselves, which precludes us from reporting directly the magnitude of the habituation stimuli in detection-threshold units. To provide an estimate of the magnitude of the L–M adaptation stimuli in units of detection threshold, we first used Equation 2 to determine the L–M cone contrast of the L–M adaptation stimulus (0.177). We then determined the absolute value of the L–M cone contrast of the two components of the adaptation stimulus at detection threshold assuming a linear relationship between L–M cone contrast (obtained from Eq. 1) and DDUs (red = 0.00181; green = 0.000786). The extent of L–M adaptation of the L–M adaptation stimulus in threshold units = 0.177/((0.00181 + 0.000786)/2) = 136. For the S adaptation stimulus, the absolute value of the S cone contrast of the S adaptation stimulus using Equation 2 was 0.804, and the absolute value of the S cone contrast of the two components were 0.00619 (lavender) and 0.00225 (lime). The extent of S adaptation of the S adaptation stimulus in threshold units = 0.804/((0.00619 + 0.00225)/2) = 191.
Within DDUs, the intermediate adaptation stimuli fell along the 45° intermediate axes. For comparison, we calculated the angle of the intermediate adaptation stimuli in threshold units, accommodating the relatively stronger activation (in threshold units) of the S axis. These angles (and the comparable values in DDUs) are 55–235° for Int1 (45–225°) and 125–305° for Int2 (135°-315°). Although the intermediate stimuli are 10° off from the predicted DDU values, the stimuli were sufficiently intermediate to produce selective adaptation, as described in Results. The finding of selective adaptation to Int1 and Int2 despite the fact that Int1 and Int2 were not perfectly intermediate (in threshold units) provides a more stringent test of intermediate mechanisms than would be provided if Int1 and Int2 were perfectly intermediate between the cardinal axes. We do not plot the results in units of detection threshold for two reasons. First, this would complicate the Fourier analysis (see Figs. 8⇓–10). Second, we did not measure detection thresholds directly for the adaptation stimuli; the angles provided are derived values. Leaving the results in DDUs is not only standard practice but also is closer to the raw data. In addition, the adaptation stimuli were considerably above detection threshold. It is likely that the effect of adaptation produced by the stimuli is not linear.
Baseline detection task.
The animals were trained to fixate a small spot at the center of a monitor displaying full-field neutral gray that established a baseline adaptation state of the visual system. After 500 ms, on some trials, a 2° diameter target spot with 0.1° blurred edges (to mitigate chromatic aberration) appeared with a Gaussian time course over 1500 ms (250 ms SD) at a location centered 2° from the center of gaze (Fig. 1B). The monkey was rewarded with a juice drop for directing its gaze to the target location. On one-sixth of the trials, no spot would appear (catch trials), and the monkey was rewarded for maintaining fixation. To make the task more challenging and to increase the precision of our measurements by expanding the possible choices, the location of the target was varied pseudorandomly between four locations, maintaining on average the same number of presentations at each location.
The target could vary in color and contrast (azimuth and radial length in the DKL space; Fig. 1A) but maintained photometric equiluminance with the gray background. Trials with targets of different color and saturation were pseudorandomly interleaved, maintaining approximately the same total number of trials of each target within a given session and ensuring that no performance biases could accumulate for a given color. Targets of 16 colors were used in M1 and eight colors in M2 (Fig. 1A, the odd-numbered colors), each target at six cone-contrast levels. A trial was included in the analysis as a correct choice if the animal did not break fixation during the fixation period (and the adaptation period in subsequent experiments) and either transferred its gaze to the correct location on trials involving a target or maintained its fixation during catch trials. A trial was included in the analysis as an incorrect choice if the animal maintained its gaze during the adaptation and fixation periods, but after the target was displayed selected an incorrect target location or maintained fixation. Trials in which the monkey broke fixation at any point during the fixation period or very early after the target onset (<2 SDs of the mean reaction time to fully saturated stimuli) were discarded. Figure 2 shows histograms of the reaction times averaged over colors as a function of different cone-contrast levels.
Adaptation task.
Detection thresholds were assessed before and after chromatic adaptation to heterochromatic Gr or flickering FF colors. The colored Gr were of spatial frequency ½ cycle/° and drifted back and forth at a rate of 1 Hz, switching directions every 1.8 s. The FF stimulus consisted of uniform FF colors that modulated sinusoidally in time at a rate of 1 Hz. Each adaptation session was begun with a period of 30 s during which the animal was required to fixate for a juice reward. Chromatic detection was then assessed in trials that each began with a “top-up” period of adaptation (Krauskopf et al., 1982), during which the animals were required to maintain fixation without reward. Reward was given after target detection, as for the baseline detection task described above. Results were only analyzed for trials in which the animal achieved top-up times of 5000 ms (M1) and at least 3000 ms (M2). (The task performance of M1 was better than of M2.) In each adaptation session, we used only one adapting type (Gr or FF) and one of the four chromatic adaptation stimuli (Card1, Int1, Card2, or Int2). Interleaved periodically with adaptation sessions, we performed additional detection sessions without chromatic adaptation to ensure stable baseline detection thresholds (see Fig. 3B,C, values after the arrows).
Luminance control.
An important component of the findings of Krauskopf et al. (1982) was that chromatic thresholds were not elevated after adaptation to high-contrast luminance modulations, ruling out adaptation mechanisms before cone opponency. In M1, we tested changes in detection threshold to the four poles of the cardinal axes after adaptation to a 10% luminance Gr (see Fig. 5B, top right panel, blue curve). The chromaticity (xy) coordinates of the two phases of the luminance grating matched the coordinates of the adapting gray.
Data analyses.
The animal's performance (percentage correct) as a function of stimulus saturation was fit with a Weibull function using the Palamedes toolbox (http://www.palamedestoolbox.org): using the maximum-likelihood criterion, where α is the threshold value, and β is the slope of the fit. The psychometric performance was corrected for guess rate (γ) and lapse rate (λ), so that values ranged from 0 to 1. Guess rate corresponded to the fraction of time the animal broke fixation during a catch trial divided by four, for the four possible target choices. The guess rate was stable over time and empirically determined to be 6% (M1) and 7% (M2) (see Fig. 4). In Figures 3A, 5A, and 6A, 95% confidence intervals were defined as twice the SD of α (threshold) values obtained from Weibull fits to 100 parametric bootstrap simulations of the performance data. Vertical error bars on these figures are the SDs of performance values simulated 100 times using a parametric observer. The preadaptation threshold was subtracted from the postadaptation threshold to determine the change in threshold attributable to adaptation (see Figs. 5B, 6B). Changes were considered significant if the preadaptation threshold lay outside the 95% confidence interval of the postadaptation threshold.
Fourier analyses.
MATLAB function fft was used to find the power of the first n/2 harmonics (n is the length of the dataset) and the amplitude and phase of the second harmonic. In Figure 8, power was defined as the square of the Fourier spectrum amplitude divided by n. For each of 100 bootstrap simulations, a Fourier analysis was run on the change in threshold curve obtained using the threshold values from that simulation. Across simulations, the average power values for each harmonic were calculated, along with the SDs, which are reported in Figure 8. The average phases were calculated and are shown with error bars indicating the SDs (see Fig. 9).
Vector sum analysis.
Vector sum values (see Figs. 7C, 10B) were calculated for each lobe of the change in threshold curves as a measure of the sharpness or narrowness of the curves. To do this analysis, for each change in threshold curve, we added to each data point comprising the curve an amount to compensate for any negative values (which would come about because of decreases in detection threshold brought about by adaptation; see Fig. 6B, open symbols). Then, we isolated each lobe of the curve. For instance, if the adapting axis was along the 45–225° line, lobe 1 would include the colors at angles 315° through 135° and lobe 2 would include the colors at angles 135° through 315°. The vector sum, r, was defined as follows: where w is the change in threshold values of the lobe, and θ is the color angles (in radians) of those values in DKL space. The vector sum of a semicircle (indicative of no tuning) is shown by the icon along each axis (see Figs. 7C, 10B). Error bars in Figure 7C are the SDs of vector sum values from 100 bootstrap simulations of change in threshold curves. Error bars in Figure 10B are the square roots of the average variances of these values for each adapting type (cardinal or intermediate).
Results
We tested color-detection thresholds before and after chromatic adaptation in two macaque monkeys (M1, M2). The goal was to determine whether changes in detection threshold differed in a systematic way for different colors after chromatic adaptation defined by different directions in color space. Because of the unique challenges of conducting behavioral experiments in animals and the large number of test conditions needed to assess chromatic mechanisms, the experiments involved thousands of detection trials spanning many testing sessions. Each trial ranged between 2 and 7 s. M1 completed 126,717 trials in 114 sessions over 22 months; M2 completed 104,672 trials in 125 sessions over 14 months.
Detection thresholds
To assess chromatic mechanisms in monkeys, we adapted for head-fixed animals a chromatic detection task used in human studies. Because we aimed to compare changes in chromatic detection for various colors before and after chromatic adaptation, the experiments required obtaining precise measurements of detection threshold with a consistent method applied for all colors, and stable baseline measurements before chromatic adaptation. Figure 3A shows two example psychometric curves for detection of color 1 (red) from M1; one curve was determined in the first testing session (open symbols) and the other in session 18 (filled symbols). The curves represent the average performance over 62 trials (session 1) and 184 trials (session 18). As expected, for both sessions, the monkey's detection performance increased with increasing cone contrast of the target (session 1, goodness of fit, R2 = 0.9; session 18, R2 = 0.99). The monkey's satisfactory performance in the first session is accounted for by the fact that the animal was not naive to the scheme of the behavioral task—both animals had been trained previously to fixate and to perform a different behavioral task not related to the present experiments. But the threshold (α, see Materials and Methods) in session 18 was lower than in session 1, even though the guess rate across testing sessions was constant (Fig. 4). The better performance of the monkey on the same task in the later session is suggestive of perceptual learning and could potentially confound our ability to accurately assess changes in detection threshold after adaptation.
Figure 3B shows the detection thresholds on the task over time for each color separately (gray lines) and the average for all colors combined (black line); the horizontal black bars indicate sessions 1 and 18, corresponding to the psychometric curves in Figure 3A (open triangle, session 1; filled triangle, session 18). The length of the bars indicates the number of trials completed during the session. Figure 3C shows the results for M2. After many trials, the detection thresholds plateaued (Fig. 3B,C, arrows, defined by the crossing of the SD of the curves fit to individual colors with the asymptotic value). At plateau performance, the monkeys had presumably exhausted perceptual learning, ensuring an accurate baseline threshold for comparison with thresholds remeasured during chromatic adaptation sessions. To achieve plateau performance, the monkeys required 26,468 trials (M1) and 5733 trials (M2)—more trials were required for M1 in part because we used double the number of target colors in this animal.
Adaptation experiments: overview
After plateau performance in the detection task, we conducted two sets of tests using chromatic adaptation of different spatiotemporal configurations. In one set of tests, the adaptation stimulus consisted of colored gratings (Fig. 1C, Gr). In another set of tests, the adaptation stimulus consisted of uniform full-field colors (Fig. 1D, FF). The FF condition might be expected to activate subcortical visual circuits better than the Gr condition because subcortical cells respond poorly to spatial chromatic contrast (Wiesel and Hubel, 1966), although the spatial scale of the Gr stimulus was relatively coarse (0.5 cycle/°) compared with the scale of most LGN receptive fields, and the drift rate (1 Hz) was approximately the same as the temporal modulation of the FF stimulus. However, cortical color circuits would respond much more strongly to the Gr condition than to the FF condition because cortical cells respond well to spatial chromatic structure (Conway et al., 2010). For example V1 double-opponent cells respond to adjacent cone-isolating bars of similar width to the Gr frequency (Conway, 2001; Conway et al., 2002). In both FF and Gr adaptation trials, the test probe was the same: a 2° spot, presented near the fovea. This moderately sized spot would elicit activity from LGN cells as well as cortical double-opponent and complex-equiluminant cells (Conway and Livingstone, 2006) that possess receptive fields straddling the spot boundary.
Each adaptation session was begun with a period of 30 s of adaptation, followed by several seconds of top-up adaptation between trials. The adapting colors could be one of four combinations. One combination selectively modulated along the L–M cardinal direction (Card1), traditionally called the “red–green” axis. Another combination modulated along the S cardinal direction (Card2), traditionally called the “blue–yellow” axis. The two remaining adaptation conditions were intermediate to the cardinal axes, involving cyan-to-orange (Int1) or lime-to-magenta (Int2) color combinations. Note that the color names only approximate appearance.
Adaptation experiments: gratings
The top panels in Figure 5A show the animals' psychometric functions for detecting color 1 (on the L–M cardinal axis) as a function of the cone contrast of the color, before (black line; circles) and after (gray lines) adaptation to gratings modulated either along the L–M axis (triangles) or along the orthogonal (S-cone) cardinal axis (squares). Detection thresholds after adaptation to the L–M stimulus were higher than those obtained at baseline or after adaptation to S stimuli. This result is consistent with selective adaptation of a putative L–M cardinal mechanism.
To test for the existence of higher-order chromatic mechanisms tuned to intermediate color directions, we measured the changes in detection thresholds after adaptation to intermediate directions in DKL color space. If there are only two post-receptoral chromatic mechanisms (tuned to the cardinal axes), adaptation along a given intermediate axis should result in uniformly high elevations in detection threshold for any of the four intermediate test colors, because the adaptation would result in equal levels of adaptation of the two cardinal mechanisms (Krauskopf et al., 1982). Our intermediate stimuli were not exactly intermediate between the cardinal directions (in threshold units) but close enough that this prediction still holds. Figure 5A (bottom row) shows the animals' psychometric functions for detecting the saturation of color 3, the orange color of Int1, before (black line; circles) and after (gray lines) adaptation to gratings modulated either along intermediate axis 1 (triangles) or along the orthogonal intermediate axis (squares). The curve fit to the triangles is shifted to the right compared with the curve fit to the squares and circles, showing that the threshold was selectively increased after adaptation to Int1. This result is inconsistent with the hypothesis that color is represented by only two post-receptoral cardinal chromatic mechanisms and provides evidence for an independent higher-order mechanism tuned to Int1.
Figure 5B shows changes in threshold plotted as a polar plot for all 16 colors tested for monkey M1. The radius represents the change in detection threshold for the target color defined by the vector angle in DKL space after chromatic adaptation indicated by the double-headed arrow. The red circle indicates zero change in detection threshold. In the change in threshold plots, solid black symbols show significant increases in detection threshold and solid gray symbols show insignificant increases; open black symbols show significant decreases in threshold, whereas open gray symbols show insignificant decreases (see Fig. 6B). The left column shows the results after adaptation to colors modulated along the cardinal directions (L–M, top left; S, bottom left). The arrowheads show the data points extracted from the psychometric curves shown in Figure 5A, top left: (TL–M − Tbaseline) = 0.024–0.012 = 0.012, where TL–M was the threshold after adaptation to L–M, and Tbaseline was the baseline threshold (solid arrowhead). (TS − Tbaseline) = 0.013–0.012 = 0.001, where TS was the threshold after adaptation to S (open arrowhead).
The observed increase in threshold was selective for the color direction of the adaptation stimulus when the adaptation stimulus was along one of the cardinal directions. This is shown in Figure 5B, left column: the curve connecting the changes in detection threshold is elongated along the adaptation axis. As described above, if detection of the intermediate color directions was mediated by only two mechanisms tuned to the cardinal directions in DKL color space, then adaptation to colors modulated along the intermediate axes would result in elevations in detection thresholds for all colors, not only those of the adaptation stimulus. Contrary to this prediction, the elevations in detection thresholds were selective for the colors of the adaptation stimulus, even for adaptation along intermediate color directions (Fig. 5B, right column). These results provide evidence for higher-order chromatic mechanisms tuned to non-cardinal color directions. The top left of Figure 5B shows the change in threshold curve resulting from adaptation with 10% luminance gratings (blue trace). Thresholds were little affected, confirming results in humans suggesting that luminance mechanisms and chromatic mechanisms are subserved by independent channels (Krauskopf et al., 1982).
Adaptation experiments: FF stimuli
The results shown in Figure 5 provide evidence for more than just the two higher-order chromatic mechanisms originally described for humans by Krauskopf et al. (1982). One explanation for the discrepancy between the monkey results and the interpretation given by Krauskopf et al. (1982) is a difference in the adaptation stimulus: Krauskopf et al. used FF not Gr adapting stimuli. Cells at different stages of the visual-processing hierarchy are tuned differently for spatial chromatic structure. Parvocellular LGN neurons do not respond to color stimuli with spatial structure (Wiesel and Hubel, 1966), whereas cortical color neurons in V1 and extrastriate regions often respond to stimuli with spatial chromatic structure. The Gr condition rather than the FF condition would be more effective at eliciting cortical activity; results arising from the use of gratings might therefore be attributed to the cortex. The results in Figure 5 may reflect adaptation of cortical cells superimposed on adaptation effects unique to the cardinal mechanisms hypothesized to originate in the LGN or as these signals arrive at V1.
To test this hypothesis, in a second set of experiments, we measured changes in detection thresholds using FF adaptation (Fig. 1D). The changes in threshold were strikingly lower than those measured using Gr adaptation for all chromatic-adaptation conditions (Figs. 6A, 7A,B) but were nonetheless selective for the color direction of the adaptation stimulus when using adaptation along the cardinal directions (Fig. 6B, left column). Note that data points indicating adaptation-induced reductions in threshold are plotted inside the red ring. The selectivity of the changes in detection threshold after adaptation along the intermediate directions was considerably less pronounced, evident only as subtle increases in detection threshold (filled symbols) for colors along the adaptation axis and decreases in adaptation threshold (open symbols) for colors along the orthogonal axis (Fig. 6B, right column).
We can quantify the sharpness of the chromatic mechanisms by determining the vector sum of each lobe—values closer to 1 indicate more selectivity or narrower elongation. If the chromatic mechanisms are more apparent when assessed by the Gr condition compared with the FF condition, as predicted by the hypothesis that the higher-order mechanisms arise in the cortex, then the vector sums in experiments involving Gr adaptation will be higher. This is what we found (Fig. 7C; one-tailed paired t test, p = 0.02). These results show that the chromatic mechanisms are more apparent when assessed after spatially structured adaptation and suggest that the spatially tuned mechanisms adapted by the gratings comprise a range of tuning preferences, including both intermediate and cardinal color directions.
To test whether the spatial structure of adaptation has a differential impact on the intermediate versus cardinal mechanisms, one can compare the correlation of the change in detection thresholds produced by Gr versus FF for each target color. For M1, the extent of adaptation was linearly correlated between the FF and Gr conditions after adaptation along the cardinal (Fig. 7A) and intermediate (Fig. 7B) directions, but the correlation coefficient was significantly lower for adaptation along the intermediate direction (adaptation along the cardinal directions, Gr vs FF, n = 32, r = 0.82, p = 10−8; adaptation along the intermediate directions, Gr vs FF, n = 32, r = 0.5, p = 0.003; r values are significantly different, Fisher's r-to-z transformation, z = 2.14; one-tailed, p = 0.01). We observed a similar trend in M2, although we tested only half the number of target colors compared with M1, and the difference in correlations was not significant. Together, the pattern of results in Figure 7, A and B, is consistent with a role of neurons lacking spatial tuning in establishing the cardinal mechanisms and with a role of neurons possessing selectivity for spatial chromatic structure, such as cortical cells, in establishing higher-order mechanisms tuned to cardinal and non-cardinal directions.
Fourier analysis
Following their work identifying the cardinal mechanisms, Krauskopf et al. (1986) pointed out that their adaptation experiments also showed evidence for multiple higher-order mechanisms. They demonstrated this by Fourier analysis to show that the angle (or phase) of the second harmonic of the change in detection threshold curves aligned with the angle of adaptation for both cardinal and non-cardinal conditions. We performed a similar analysis to provide a direct comparison between the monkey and human results. Figure 8 shows the Fourier spectrum of the monkey data shown in Figures 5B and 6B. There is a distinct peak in spectrum at the second harmonic after adaptation along all four color directions tested when using Gr stimuli (Fig. 8, left panels) and after adaptation along cardinal directions when using FF stimuli (Fig. 8, right panels). There was no major peak in the second harmonic for experiments involving FF adaptation along the intermediate directions. Figure 9A shows the data from Figure 5B normalized to the maximum change in detection threshold; the best-fitting second harmonic is superimposed; the double-headed arrow shows the adaptation axis, whereas the gray line down the axis of the bi-lobed curve indicates the phase of the second harmonic. Figure 9B shows the phase of the second harmonic plotted as a function of the angle of the adapting axis for both animals and both experimental conditions. Consistent with the existence of multiple neural color channels not restricted to the cardinal color directions, the phase of the second harmonic was predicted by the color of the adapting axis—this was true for both Gr and FF conditions (combining data from M1 and M2, Gr, r = 0.99, p = 3 × 10−6; FF, r = 0.93, p = 8 × 10−4). Thus, although the amplitude of the second harmonic was negligible after FF adaptation along intermediate mechanisms, the Fourier phase analysis reveals some selective adaptation of higher-order mechanisms even under FF conditions. The SD of the phase angle was typically higher after adaptation along the intermediate directions compared with adaptation along the cardinal directions, especially for the FF condition, confirming the special status of the cardinal mechanisms. Figure 9C shows the difference between the phase of the second harmonic and the phase of the adaptation stimulus, as a function of the phase of the adaptation stimulus.
Cardinal mechanisms are more susceptible to adaptation and have narrower chromatic tuning than non-cardinal mechanisms
Finally, we quantify the degree to which adaptation along the cardinal axes differs from adaptation along the intermediate axes. The Fourier spectra of Figure 8 show that the amplitude of the second harmonic was higher for experiments involving adaptation along cardinal rather than intermediate directions. Figure 10 shows normalized amplitudes of the adaptation effect, averaging across both animals and adaptation conditions. The adaptation effect when using adaptation along cardinal color directions was stronger than when using adaptation along intermediate color directions (Fig. 10A; one-tailed t test, p = 0.001). An additional test of the specialness of the cardinal mechanisms is provided by a comparison of the narrowness of the polar plots after adaptation to cardinal versus intermediate directions using the vector sum analysis. Adaptation along cardinal axes produced significantly sharper plots than adaptation along the intermediate axes (Fig. 10B; one-tailed t test, p = 0.01).
Discussion
A central question in vision research concerns the mechanisms and circuits that transform the outputs of the three classes of cone photoreceptors into the perception of color. Because the activity in the cone classes is highly correlated, the visual system can discount cone identity to relay high-resolution (achromatic) form information. But to relay color, the system must compute the differential signal between two cone types. Psychophysical measurements of color-discrimination thresholds in humans have been critical in building models of such post-receptoral chromatic mechanisms (Stockman and Brainard, 2010), generating numerous hypotheses concerning the underlying physiology that could potentially be tested in monkeys. Although monkeys and humans show similar spectral sensitivity (De Valois et al., 1974) and have almost identical cone types (Schnapf et al., 1990), the two species have obvious differences in brain organization and cognitive ability; it is not clear that monkeys possess the same psychophysical chromatic mechanisms as humans. The results presented here provide the first direct link between psychophysical chromatic mechanisms observed in humans and physiological studies in monkeys and justify the use of monkeys as a model system to investigate color behavior characterized in people. Moreover, the results help resolve a controversy concerning the number and nature of the psychophysical chromatic mechanisms. Changes in detection thresholds after FF adaptation support the special status of the cardinal mechanisms (Fig. 6), whereas changes in detection thresholds after Gr adaptation reveal multiple higher-order mechanisms tuned to color directions in addition to the cardinal directions (Fig. 5). As discussed below, the results with FF adaptation suggest that the cardinal mechanisms are implemented by circuits found early in the visual-processing hierarchy, whereas the results with Gr adaptation suggest that the higher-order mechanisms are implemented subsequently within the cortex.
Strikingly, detection thresholds were much higher after Gr adaptation than FF adaptation (Fig. 7). This result shows that the fundamental neural representation of color is sensitive to spatial structure, which in turn suggests a cortical locus because cortical neurons and not subcortical neurons have spatially chromatically opponent receptive fields: LGN cells are low-pass tuned for color (Wiesel and Hubel, 1966), whereas cortical color cells are typically bandpass tuned for color (Thorell et al., 1984). Cells with spatially cone-opponent receptive fields have been found as early in the visual-processing hierarchy as V1 (Conway, 2001) and are probably found in many regions of extrastriate cortex.
The experiments involving Gr adaptation revealed chromatic mechanisms tuned to color directions intermediate to the cardinal directions (Figs. 5, 8), providing clear evidence for higher-order chromatic mechanisms. These results are consistent with the noise-masking study by Gegenfurtner and Kiper (Gegenfurtner and Kiper, 1992) who found evidence for multiple intermediate mechanisms using Gabor patches as target probes but is at odds with a later study by Giulianini and Eskew (1998); the discrepancy between these results in humans is unresolved (Hansen and Gegenfurtner, 2006). Subcortical structures, including the retina and LGN, are unlikely to be the neural basis for the higher-order chromatic mechanisms we observed because the neurons within these structures not only lack the requisite sensitivity to spatial chromatic structure but also are primarily restricted to chromatic tuning along the cardinal directions (Derrington et al., 1984; Sun et al., 2006). Conversely, cortical brain regions contain many neurons tuned to non-cardinal directions.
One popular candidate for the implementation of the higher-order chromatic mechanisms is V1 (Lennie et al., 1990), but there are other possibilities. Color-tuned neurons with non-cardinal color preferences are found not only in V1 (Hanazawa et al., 2000; Wachtler et al., 2003; Johnson et al., 2004; Conway and Livingstone, 2006; Horwitz and Hass, 2012; Lafer-Sousa et al., 2012) but also in V2 (Hubel and Livingstone, 1985; Xiao et al., 2003), possibly V3 (Gegenfurtner et al., 1997), V4 (Zeki, 1980), and in several locations of inferior temporal (IT) cortex (Komatsu et al., 1992; Conway et al., 2007; Yasuda et al., 2010). Color neurons within posterior IT are the first along the processing hierarchy that have been shown to be selective for color over achromatic luminance and also to possess color tuning that is tolerant to luminance changes (Conway et al., 2007); moreover, these neurons represent a range of color-tuning preferences spanning psychophysical color space (Stoughton and Conway, 2008). These neurons, which reside in dedicated color modules (Conway et al., 2007), might be better candidates than V1 cells for the neural basis of the chromatic mechanisms because their activity more closely correlates with perception (Koida and Komatsu, 2007; Stoughton and Conway, 2008). Consistent with this hypothesis, chromatic adaptation effects assessed with fMRI tend to be somewhat stronger in visual areas downstream from V1 in the processing hierarchy (Engel, 2005). Although we interpret the psychophysical evidence as support for the hypothesis that the brain represents color with multiple chromatic channels, possibly implemented in IT, we acknowledge that it is possible to construct models that can account for the evidence with the same number of mechanisms as the standard cardinal model, so long as the mechanisms possess nonstandard features, such as asymmetries, nonlinearities, or adaptable tuning (Zaidi and Shapiro, 1993; Eskew, 2009). Thus, although consensus is emerging that the standard cardinal model cannot account for psychophysical behavior, there is little agreement about what model should take its place.
The notion that the cardinal mechanisms and not the higher-order mechanisms are fundamental to the representation of color was initially buttressed by the discovery in monkeys of two populations of neurons in the LGN with tuning closely aligned with the cardinal directions (Derrington et al., 1984). These two populations continue to play an important role in models of chromatic mechanisms (for review, see Webster, 1996; Gegenfurtner and Kiper, 2003; Eskew, 2009), even though LGN cells are not susceptible to adaptation (Derrington et al., 1984; Solomon et al., 2004). We found that tuning along the cardinal directions in monkey was narrower than along the intermediate mechanisms under both FF and Gr conditions (Fig. 10). This observation is consistent with the special status of the cardinal mechanisms documented by human studies, although we note that there is no consensus in the human literature on the widths of the psychophysical chromatic mechanisms (Eskew, 2009). The neural basis for the cardinal mechanisms remains a source of investigation. The relatively narrower tuning of the cardinal mechanisms may provide a clue to their neural substrate: it predicts that the bandwidths of neurons tuned for cardinal directions are narrower than the bandwidths of neurons tuned for intermediate mechanisms, a prediction that, to our knowledge, is yet to be tested.
In the results presented here and those of Krauskopf et al. (1982), the cardinal mechanisms were revealed by FF stimuli, suggesting that these mechanisms are realized by neurons lacking spatial tuning. If the underlying neural basis is not the retina or LGN since neurons at these locations do not show chromatic adaptation, the only alternative locus would seem to be cortical color cells lacking spatial tuning. Such cells reside in V1 at or near the arrival of geniculate signals into the cortex (Livingstone and Hubel, 1984). Physiological experiments in monkeys indicate this stage as a potential substrate for the cardinal mechanisms (Tailby et al., 2008). This is an attractive idea because it preserves a defining role of the two major populations of color-coding LGN cells in generating the cardinal mechanisms yet is consistent with a cortical implementation suggested by the multiple lines of evidence discussed here. But given the rich inter-connectedness of cortical areas with each other, and V1 with LGN, it may be impossible to identify any psychophysical phenomenon within a single anatomical stage (Eskew, 2009).
The classic model of color characterized by two post-receptoral cone-opponent channels implies that color is encoded by a population code in which a given hue is represented not by neurons tuned to the hue but rather by the combined response of the population of L–M cells and S–(L + M) cells. Such a model might be considered mutually distinct from a model predicted by the results concerning higher-order mechanisms described here in monkeys and previously in humans (Krauskopf et al., 1986; Zaidi and Halevy, 1993; Webster and Mollon, 1994; Hansen and Gegenfurtner, 2006), in which the perception of a given hue would be encoded by a distinct neural channel with the corresponding hue tuning. But it need not be: the cardinal mechanisms, if they are instantiated by the LGN or within the geniculate-input layers of V1, may simply represent an early stage in the visual processing of color, whereas the higher-order mechanisms, realized by the hue-tuned neurons within the cortex, may represent a later stage in the processing of color, closer to the perceptual readout of the brain. Although the general idea of a multistage model of color is not new (Guth, 1991; De Valois and De Valois, 1993), the evidence for such a model is lacking and there is no agreement on its specific features (Webster, 1996). The results described here, together with physiological work in monkeys and psychophysical work in humans, should inform a plausible, physiologically testable model showing how the LGN channels are transformed by the cortex into the chromatic mechanisms evidenced both physiologically and perceptually.
Notes
Supplemental material for this article is available at http://www.wellesley.edu/neuroscience/conway/stoughtonetal2012. Judd-corrected Commission Internationale de l'Eclairage xyY coordinates of the target stimuli. This material has not been peer reviewed.
Footnotes
This work was supported by the National Science Foundation (Grant 0918064), the Whitehall Foundation, and Wellesley College. Additional support was provided by the Radcliffe Institute (B.R.C.) and a Howard Hughes Medical Institute postgraduate internship (C.M.S.). We thank John Maunsell, Richard Born, Ellen Hildreth, Michael Wiest, Jeremy Wilmer, Mark Histed, Marlene Cohen, Douglas Ruff, Alexandra Smolyanskaya, Margaret Livingstone, Kaushik Ghose, and Alex Trott for useful discussions and Sonja Swanbeck and Reitumetse Pulumo for help with calibration. We thank David Hubel and John Maunsell for providing laboratory space and Thorsten Hansen for providing us with MATLAB code and guidance for DKL color calibration. We thank Qasim Zaidi and Rhea Eskew for very constructive feedback on the manuscript.
- Correspondence should be addressed to Bevil R. Conway, Neuroscience Program, Wellesley College, 106 Central Street, Wellesley, MA 02481. bconway{at}wellesley.edu