Abstract
While “recalibration by pairing” is now generally held to be the main process responsible for adaptation to intermodal discordance, the conditions under which pairing of heteromodal data occur in spite of a discordance have not been studied systematically. The question has been explored in the case of auditory-visual discordance. Subjects pointed at auditory targets before and after exposure to auditory and visual data from sources 20° apart in azimuth, in conditions varying by (a) the degree of realism of the context, and (b) the synchronization be-tween auditory and visual data. In Experiment 1, the exposure conditions combined the sound of a percussion instrument (bongos) with either the image on a video monitor of the hands of the player (semirealistic situation) or diffuse light modulated by the sound (nonrealistic situation). Experiment 2 featured a voice and either the image of the face of the speaker or light modulated by the voice, and in both situations either sound and image were exactly syn-chronous or the sound was made to lag by 0.35 sec. Desynchronization was found to reduce adaptation significantly, while degree of realism failed to produce an effect. Answers to a question asked at the end of the testing regarding the location of the sound source suggested that the apparent fusion of the auditory and visual data—the phenomenon called “ventriloquism”— was not affected by the conditions in the same way as adaptation. In Experiment 3, subjects were exposed to the experimental conditions of Experiment 2 and were asked to report their impressions of fusion by pressing a key. The results contribute to the suggestion that pairing of registered auditory and visual locations, the hypothetical process at the basis of recalibration, may be a different phenomenon from conscious fusion.
Article PDF
Similar content being viewed by others
References
Bertelson, P., &Radeau, M. Ventriloquism, sensory interaction, and response bias: Remarks on the paper by Choe, Welch, Gilford, and Juola.Perception & Psychophysics, 1976,19, 531–535.
Canon, L. K. Intermodality inconsistency of input and directed attention as determinants of the nature of adaptation.Journal of Experimental Psychology, 1970,84, 141–147.
Choe, C. S., Welch, R. B., Gilford, R. M., &Juola, J. F. The “ventrioloquist effect”: Visual dominance or response bias?Perception & Psychophysics, 1975,18, 55–60.
Epstein, W. Recalibration by pairing: A process of perceptual learning.Perception, 1975,4, 59–72.
Held, R., &Freedman, S. J. Plasticity in human sensorimotor control.Science, 1963,142, 455–461.
Held, R., Efstathiou, A., &Greene, M. Adaptation to displaced and delayed visual feedback from the hand.Journal of Experimental Psychology, 1966,72, 887–891.
Howard, I. P. Displacing the optical array. In S. J. Freedman (Ed.),The neuropsychology of spatially oriented behavior. Homewood, Ill: Dorsey Press, 1968.
Jack, C. E., &Thurlow, W. R. Effects of degree of visual association and angle of displacement on the “ventriloquism” effect.Perceptual and Motor Skills, 1973,37, 967–979.
Jackson, C. V. Visual factors in auditory localization.Quarterly Journal of Experimental Psychology, 1953,5, 52–65.
Radeau, M. The locus of adaptation to auditory-visual conflict.Perception, 1973,2, 327–332.
Radeau, M. Adaptation au déplacement prismatique sur la base d’une discordance entre la vision et l’audition.Année Psychologique, 1974,74. 23–34.
Radeau, M., &Bertelson, P. Adaptation à un déplacement prismatique sur la base de stimulations exafférentes en conflit.Psychologica Belgica, 1969,9, 133–140.
Radeau, M..&Bertelson, P. The aftereffects of ventriloquism.Quarterly Journal of Experimental Psychology, 1974,26, 63–71.
Radeau, M., &Bertelson, P. The effect of a textured visual field on modality dominance in a ventriloquism situation.Perception & Psychophysics, 1976,20, 227–235.
Rock, I.The nature of perceptual adaptation. New York: Basic Books. 1966.
Siegel, S.Non-parametric statistics. New York: McGraw-Hill. 1956.
Thurlow, W. R. G., &Jack, C. E. Certain determinants of the “ventriloquism effect.”Perceptual and Motor Skills, 1973,36, 1171–1184.
Wallach, H. Informational discrepancy as a basis of perceptual adaptation. In S. J. Freedman (Ed.),The neuropsychology of spatially oriented behavior. Homewood, Ill: Dorsey Press, 1968.
Wallach, H. &Karsh, E. B. The modification of stereoscopic depth perception and the kinetic depth effect.American Journal of Psychology, 1963,76, 429–435.
Weerts, T. C., &Thurlow, W. R. The effect of eye position and expectation on sound localization.Perception & Psychophysics, 1971,9. 35–39.
Welch, R. B. The effect of experienced limb identity upon adaptation to simulated displacement of the visual field.Perception & Psychophysics, 1972,12. 453–456.
Author information
Authors and Affiliations
Additional information
The present work was carried out while the first author held a grant as “Chargé de Recherches” from the Belgian “Fonds national de la Recherche scientifique.” The help of this university’s “Centre universitaire du Film scientifique” (CUFS) in preparing the video recordings is gratefully acknowledged.
Rights and permissions
About this article
Cite this article
Radeau, M., Bertelson, P. Adaptation to auditory-visual discordance and ventriloquism in semirealistic situations. Perception & Psychophysics 22, 137–146 (1977). https://doi.org/10.3758/BF03198746
Received:
Accepted:
Issue Date:
DOI: https://doi.org/10.3758/BF03198746