The early visual system is endowed with adaptive mechanisms that rapidly adjust gain and integration time based on the local luminance (mean intensity) and contrast (standard deviation of intensity relative to the mean). Here we show that these mechanisms are matched to the statistics of the environment. First, we measured the joint distribution of luminance and contrast in patches selected from natural images and found that luminance and contrast were statistically independent of each other. This independence did not hold for artificial images with matched spectral characteristics. Second, we characterized the effects of the adaptive mechanisms in lateral geniculate nucleus (LGN), the direct recipient of retinal outputs. We found that luminance gain control had the same effect at all contrasts and that contrast gain control had the same effect at all mean luminances. Thus, the adaptive mechanisms for luminance and contrast operate independently, reflecting the very independence encountered in natural images.