A transcranial magnetic stimulation (TMS) adaptation paradigm was used to investigate the neural representation of observed motor behavior in the inferior parietal lobule (IPL), ventral premotor cortex (PMv), and in the cortex around the superior temporal sulcus (STS). Participants were shown adapting movies of a hand or a foot acting on different objects and were asked to compare to the movie, a motor act shown in test pictures. The invariant features between adapting and test stimuli fitted a 2 x 2 design: same or different action made by the same or different effector. Neuronavigated TMS pulses were delivered at the onset of each test picture. TMS over the left and right PMv and over the left IPL induced a selective shortening of reaction times (RTs) to stimuli showing a repeated (adapted) action, regardless of the effector performing it. In a second experiment, TMS applied over the left STS induced shortening of RTs for adapted actions but only if also the effector was repeated. The results indicate that observed motor behavior is encoded with the body part that performs it in the temporal lobe. A hierarchically higher level of representation is carried by neural populations in the parietofrontal regions, where acts are encoded in an abstract way.