The neural system underlying face perception must represent the unchanging features of a face that specify identity, as well as the changeable aspects of a face that facilitate social communication. However, the way information about faces is represented in the brain remains controversial. In this study, we used fMR adaptation (the reduction in fMRI activity that follows the repeated presentation of identical images) to ask how different face- and object-selective regions of visual cortex contribute to specific aspects of face perception. We report that activity in the face-selective region of the fusiform gyrus (FG) was reduced following repeated presentations of the same face. Adaptation in this area was not sensitive to changes in image size, but was sensitive to changes in viewpoint. In contrast, face-selective regions in the superior temporal lobe failed to adapt to identical presentations of the same face, but showed an increased response when the same face was shown from different viewpoints and with different expressions. These results reveal a largely size-invariant neural representation in the inferior temporal lobe that could be involved in the recognition of facial identity, and a separate face-selective region in the superior temporal lobe that could be used to detect changeable aspects of faces. The absence of fMR-adaptation in object-selective regions of visual cortex challenges the idea that a more distributed network of areas is used to represent information about faces.