The appearance of an object or surface depends strongly on the light from other objects and surfaces in view. This review focuses on color in complex scenes, which have regions of different colors in view simultaneously and/or successively, as in natural viewing. Two fundamental properties distinguish the chromatic representation evoked by a complex scene from the representation for an isolated patch of light. First, in complex scenes, the color of an object is not fully determined by the light from that object reaching the eye. Second, the chromatic representation of a complex scene contributes not only to hue, saturation, and brightness, but also to other percepts such as shape, texture, and object segmentation. These two properties are cornerstones of this review, which examines color perception with context that varies over space or time, including color constancy, and chromatic contributions to such percepts as orientation, contour, depth, and motion.