Skip to main content
SpringerLink
Log in

A Putative Model of Multisensory Object Representation

  • Review
  • Published:
Brain Topography Aims and scope Submit manuscript

Abstract

This review surveys the recent literature on visuo-haptic convergence in the perception of object form, with particular reference to the lateral occipital complex (LOC) and the intraparietal sulcus (IPS) and discusses how visual imagery or multisensory representations might underlie this convergence. Drawing on a recent distinction between object- and spatially-based visual imagery, we propose a putative model in which LOtv, a subregion of LOC, contains a modality-independent representation of geometric shape that can be accessed either bottom-up from direct sensory inputs or top-down from frontoparietal regions. We suggest that such access is modulated by object familiarity: spatial imagery may be more important for unfamiliar objects and involve IPS foci in facilitating somatosensory inputs to the LOC; by contrast, object imagery may be more critical for familiar objects, being reflected in prefrontal drive to the LOC.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

References

  • Alivisatos B, Petrides M (1997) Functional activation of the human brain during mental rotation. Neuropsychologia 36:11–118

    Google Scholar 

  • Amedi A, Malach R, Hendler T, Peled S, Zohary E (2001) Visuo-haptic object-related activation in the ventral visual pathway. Nat Neurosci 4:324–330

    Article  PubMed  CAS  Google Scholar 

  • Amedi A, Jacobson G, Hendler T, Malach R, Zohary E (2002) Convergence of visual and tactile shape processing in the human lateral occipital complex. Cereb Cortex 12:1202–1212

    Article  PubMed  Google Scholar 

  • Amedi A, Raz N, Pianka P, Malach R, Zohary E (2003) Early ‘visual’ cortex activation correlates with superior verbal memory performance in the blind. Nat Neurosci 6:758–766

    Article  PubMed  CAS  Google Scholar 

  • Amedi A, Stern WM, Camprodon JA, Bermpohl F, Merabet L, Rotman S, Hemond C, Meijer P, Pascual-Leone A (2007) Shape conveyed by visual-to-auditory sensory substitution activates the lateral occipital complex. Nat Neurosci 10:687–689

    Article  PubMed  CAS  Google Scholar 

  • Arno P, De Volder AG, Vanlierde A, Wanet-Defalque M-C, Streel E, Robert A, Sanabria-Bohórquez S, Veraart C (2001) Occipital activation by pattern recognition in the early blind using auditory substitution for vision. Neuroimage 13:632–645

    Article  PubMed  CAS  Google Scholar 

  • Banati RB, Goerres GW, Tjoa C, Aggleton JP, Grasby P (2000) The functional anatomy of visual-tactile integration in man: a study using positron emission tomography. Neuropsychologia 38:115–124

    Article  PubMed  CAS  Google Scholar 

  • Blajenkova O, Kozhevnikov M, Motes MA (2006) Object-spatial imagery: a new self-report imagery questionnaire. Appl Cognit Psychol 20:239–263

    Article  Google Scholar 

  • Burton H, Snyder AZ, Conturo TE, Akbudak E, Ollinger JM, Raichle ME (2002) Adaptive changes in early and late blind: a fMRI study of Braille reading. J Neurophysiol 87:589–607

    PubMed  CAS  Google Scholar 

  • Carpenter PA, Eisenberg P (1978) Mental rotation and the frame of reference in blind and sighted individuals. Percept Psychophys 23:117–124

    PubMed  CAS  Google Scholar 

  • De Volder AG, Toyama H, Kimura Y, Kiyosawa M, Nakano H, Vanlierde A, Wanet-Defalque M-C, Mishima M, Oda K, Ishiwata K, Senda M (2001) Auditory triggered mental imagery of shape involves visual association areas in early blind humans. Neuroimage 14:129–139

    Article  PubMed  Google Scholar 

  • Dellantonio A, Spagnolo F (1990) Mental rotation of tactual stimuli. Acta Psychol 73:245–257

    Article  CAS  Google Scholar 

  • Deshpande G, Hu X, Stilla R, Sathian K (2008) Effective connectivity during haptic perception: a study using Granger causality analysis of functional magnetic resonance imaging data. Neuroimage 40:1807–1814

    Article  PubMed  Google Scholar 

  • Easton RD, Greene AJ, Srinivas K (1997a) Transfer between vision and haptics: memory for 2-D patterns and 3-D objects. Psychon Bull Rev 4:403–410

    Google Scholar 

  • Easton RD, Srinivas K, Greene AJ (1997b) Do vision and haptics share common representations? Implicit and explicit memory within and between modalities. J Exp Psychol Learn 23:153–163

    Article  CAS  Google Scholar 

  • Feinberg TE, Rothi LJ, Heilman KM (1986) Multimodal agnosia after unilateral left hemisphere lesion. Neurology 36:864–867

    PubMed  CAS  Google Scholar 

  • Gibson G, Stilla R, Sathian K (2008) Segregated visuo-haptic processing of texture and location. Abstract, Human Brain Mapping

  • Grefkes C, Fink G (2005) The functional organization of the intraparietal sulcus in humans and monkeys. J Anat 207:3–17

    Article  PubMed  Google Scholar 

  • Grefkes C, Geyer S, Schormann T, Roland P, Zilles K (2001) Human somatosensory area 2: observer-independent cytoarchitectonic mapping, interindividual variability, and population map. Neuroimage 14:617–631

    Article  PubMed  CAS  Google Scholar 

  • Grefkes C, Weiss PH, Zilles K, Fink GR (2002) Crossmodal processing of object features in human anterior intraparietal cortex: an fMRI study implies equivalencies between humans and monkeys. Neuron 35:173–184

    Article  PubMed  CAS  Google Scholar 

  • Grefkes C, Ritzl A, Zilles K, Fink GR (2004) Human medial intraparietal cortex subserves visuomotor coordinate transformation. Neuroimage 23:1494–1506

    Article  PubMed  Google Scholar 

  • Grill-Spector K, Henson R, Martin A (2006) Repetition and the brain: neural models of stimulus-specific effects. Trends Cogn Sci 10:14–23

    Article  PubMed  Google Scholar 

  • Hadjikhani N, Roland PE (1998) Cross-modal transfer of information between the tactile and the visual representations in the human brain: a positron emission tomographic study. J Neurosci 18:1072–1084

    PubMed  CAS  Google Scholar 

  • Hollins M (1986) Haptic mental rotation: more consistent in blind subjects? J Visual Impair Blin 80:950–952

    Google Scholar 

  • Iwamura Y (1998) Hierarchical somatosensory processing. Curr Opin Neurobiol 8:522–528

    Article  PubMed  CAS  Google Scholar 

  • James TW, Humphrey GK, Gati JS, Servos P, Menon RS, Goodale MA (2002) Haptic study of three-dimensional objects activates extrastriate visual areas. Neuropsychologia 40:1706–1714

    Article  PubMed  Google Scholar 

  • James TW, James KH, Humphrey GK, Goodale MA (2006) Do visual and tactile object representations share the same neural substrate? In: Heller MA, Ballesteros S (eds) Touch and blindness: psychology and neuroscience. Lawrence Erlbaum Associates, Mahwah, NJ, pp 139–155

    Google Scholar 

  • Kosslyn SM (1973) Scanning visual images: some structural implications. Percept Psychophys 14:90–94

    Google Scholar 

  • Kosslyn SM, Ball TM, Reiser BJ (1978) Visual images preserve metric spatial information: evidence from studies of image scanning. J Exp Psychol Human 4:47–60

    Article  CAS  Google Scholar 

  • Kourtzi Z, Kanwisher N (2000) Cortical regions involved in perceiving object shape. J Neurosci 20:3310–3318

    PubMed  CAS  Google Scholar 

  • Kourtzi Z, Kanwisher N (2001) Representation of perceived object shape by the human lateral occipital complex. Science 293:1506–1509

    Article  PubMed  CAS  Google Scholar 

  • Kourtzi Z, Tolias AS, Altmann CF, Augath M, Logothetis NK (2003) Integration of local features into global shapes: monkey and human fMRI studies. Neuron 37:333–346

    Article  PubMed  CAS  Google Scholar 

  • Kozhevnikov M, Hegarty M, Mayer RE (2002) Revising the visualiser-verbaliser dimension: evidence for two types of visualisers. Cognit Instruct 20:47–77

    Article  Google Scholar 

  • Kozhevnikov M, Kosslyn SM, Shephard J (2005) Spatial versus object visualisers: a new characterisation of cognitive style. Mem Cognit 33:710–726

    PubMed  Google Scholar 

  • Krekelberg B, Boynton GM, van Wezel RJA (2006) Adaptation: from single cells to BOLD signals. Trends Neurosci 29:250–256

    Article  PubMed  CAS  Google Scholar 

  • Lacey S, Campbell C (2006) Mental representation in visual/haptic crossmodal memory: evidence from interference effects. Q J Exp Psychol 59:361–376

    Article  Google Scholar 

  • Lacey S, Campbell C, Sathian K (2007a) Vision and touch: multiple or multisensory representations of objects? Perception 36:1513–1521

    Article  PubMed  Google Scholar 

  • Lacey S, Peters A, Sathian K (2007b) Cross-modal object representation is viewpoint-independent. PLoS ONE 2:e890. doi:10.1371/journal.pone0000890

    Article  PubMed  Google Scholar 

  • Large M-E, Aldcroft A, Vilis T (2007) Task-related laterality effects in the lateral occipital complex. Brain Res 1128:130–138

    Article  PubMed  CAS  Google Scholar 

  • Lucan JN, Foxe JJ, Gomez-Ramirez M, Sathian K, Molholm S (2008) The spatio-temporal dynamics of somatosensory shape discrimination. Abstract, International Multisensory Research Forum

  • Malach R, Reppas JB, Benson RR, Kwong KK, Jiang H, Kennedy WA, Ledden PJ, Brady TJ, Rosen BR, Tootell RBH (1995) Object-related activity revealed by functional magnetic resonance imaging in human occipital cortex. Proc Natl Acad Sci USA 92:8135–8139

    Article  PubMed  CAS  Google Scholar 

  • Marmor GS, Zaback LA (1976) Mental rotation by the blind: does mental rotation depend on visual imagery? J Exp Psychol Human 2:515–521

    Article  CAS  Google Scholar 

  • Motes MA, Malach R, Kozhevnikov M (2008) Object-processing neural efficiency differentiates object from spatial visualizers. NeuroReport 19:1727–1731

    Article  PubMed  Google Scholar 

  • Newman SD, Klatzky RL, Lederman SJ, Just MA (2005) Imagining material versus geometric properties of objects: an fMRI study. Cognit Brain Res 23:235–246

    Article  Google Scholar 

  • Pascual-Leone A, Hamilton RH (2001) The metamodal organization of the brain. Prog Brain Res 134:427–445

    Article  PubMed  CAS  Google Scholar 

  • Pascual-Leone A, Amedi A, Fregni F, Merabet LB (2005) The plastic human brain. Annu Rev Neurosci 28:377–401

    Article  PubMed  CAS  Google Scholar 

  • Peltier S, Stilla R, Mariola E, LaConte S, Hu X, Sathian K (2007) Activity and effective connectivity of parietal and occipital cortical regions during haptic shape perception. Neuropsychologia 45:476–483

    Article  PubMed  Google Scholar 

  • Pietrini P, Furey ML, Ricciardi E, Gobbini MI, Wu W-HC, Cohen L, Guazzelli M, Haxby JV (2004) Beyond sensory images: object-based representation in the human ventral pathway. Proc Natl Acad Sci USA 101:5658–5663

    Article  PubMed  CAS  Google Scholar 

  • Prather SC, Votaw JR, Sathian K (2004) Task-specific recruitment of dorsal and ventral visual areas during tactile perception. Neuropsychologia 42:1079–1087

    Article  PubMed  CAS  Google Scholar 

  • Reales JM, Ballesteros S (1999) Implicit and explicit memory for visual and haptic objects: cross-modal priming depends on structural descriptions. J Exp Psychol Learn 25:644–663

    Article  Google Scholar 

  • Reed CL, Shoham S, Halgren E (2004) Neural substrates of tactile object recognition: an fMRI study. Hum Brain Mapp 21:236–246

    Article  PubMed  Google Scholar 

  • Renier L, Collignon O, Tranduy D, Poirier C, Vanlierde A, Veraart C, De Volder A (2004) Visual cortex activation in early blind and sighted subjects using an auditory visual substitution device to perceive depth. Neuroimage 22:S1

    Article  Google Scholar 

  • Renier L, Collignon O, Poirier C, Tranduy D, Vanlierde A, Bol A, Veraart C, De Volder AG (2005) Cross modal activation of visual cortex during depth perception using auditory substitution of vision. Neuroimage 26:573–580

    Article  PubMed  Google Scholar 

  • Röder B, Rösler F (1998) Visual input does not facilitate the scanning of spatial images. J Ment Imag 22:165–181

    Google Scholar 

  • Saito DN, Okada T, Morita Y, Yonekura Y, Sadato N (2003) Tactile-visual cross-modal shape matching: a functional MRI study. Cognit Brain Res 17:14–25

    Article  Google Scholar 

  • Sathian K (2004) Modality, quo vadis?: comment. Behav Brain Sci 27:413–414

    Article  Google Scholar 

  • Sathian K (2005) Visual cortical activity during tactile perception in the sighted and the visually deprived. Dev Psychobiol 46:279–286

    Article  PubMed  CAS  Google Scholar 

  • Sathian K, Lacey S (2007) Journeying beyond classical somatosensory cortex. Can J Exp Psychol 61:254–264

    PubMed  CAS  Google Scholar 

  • Sathian K, Zangaladze A, Hoffman JM, Grafton ST (1997) Feeling with the mind’s eye. Neuroreport 8:3877–3881

    Article  PubMed  CAS  Google Scholar 

  • Shepard RN, Metzler J (1971) Mental rotation of three-dimensional objects. Science 171:701–703

    Article  PubMed  CAS  Google Scholar 

  • Shikata E, McNamara A, Sprenger A, Hamzei F, Glauche V, Büchel C, Binkofski F (2008) Localization of human intraparietal areas AIP, CIP, and LIP using surface orientation and saccadic eye movement tasks. Hum Brain Mapp 29:411–421

    Article  PubMed  Google Scholar 

  • Stilla R, Sathian K (2008) Selective visuo-haptic processing of shape and texture. Hum Brain Mapp 29:1123–1138

    Article  PubMed  Google Scholar 

  • Stilla R, Hanna R, Hu X, Mariola E, Deshpande G, Sathian K (2008a) Neural processing underlying tactile microspatial discrimination in the blind: a functional magnetic resonance imaging study. J Vis 8:1–19. doi:10.1167/8.10.13

    Article  PubMed  Google Scholar 

  • Stilla R, Lacey S, Deshpande G, Hu X, Sathian K (2008b) Effective connectivity during haptic shape perception is modulated by object familiarity. Abstract, Society for Neuroscience

  • Stoesz M, Zhang M, Weisser VD, Prather SC, Mao H, Sathian K (2003) Neural networks active during tactile form perception: common and differential activity during macrospatial and microspatial tasks. Int J Psychophysiol 50:41–49

    Article  PubMed  Google Scholar 

  • Swisher JD, Halko MA, Merabet LB, McMains SA, Somers DC (2007) Visual topography of human intraparietal sulcus. J Neurosci 27:5326–5337

    Article  PubMed  CAS  Google Scholar 

  • Tal N, Amedi A (2008) Using fMR-adaptation to study visual-tactile integration of objects in humans. Abstract, Israel Society for Neuroscience

  • Zhang M, Weisser VD, Stilla R, Prather SC, Sathian K (2004) Multisensory cortical processing of object shape and its relation to mental imagery. Cogn Affect Behav Ne 4:251–259

    Article  Google Scholar 

  • Zhou Y-D, Fuster JM (1997) Neuronal activity of somatosensory cortex in a cross-modal (visuo-haptic) memory task. Exp Brain Res 116:551–555

    Article  PubMed  CAS  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Neurology, Emory University School of Medicine, WMB-6000, 101 Woodruff Circle, Atlanta, 30322, GA, USA

    Simon Lacey & K. Sathian

  2. Physiology Department, Faculty of Medicine, The Hebrew University of Jerusalem, Jerusalem, 91220, Israel

    Noa Tal & Amir Amedi

  3. Cognitive Science Program, The Hebrew University of Jerusalem, Jerusalem, 91220, Israel

    Amir Amedi

  4. Department of Rehabilitation Medicine, Emory University, Atlanta, GA, USA

    K. Sathian

  5. Department of Psychology, Emory University, Atlanta, GA, USA

    K. Sathian

  6. Rehabilitation R&D Center of Excellence, Atlanta VAMC, Decatur, GA, USA

    K. Sathian

Authors
  1. Simon Lacey
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Noa Tal
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Amir Amedi
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. K. Sathian
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to K. Sathian.

Additional information

This article is published as part of the Special Issue on Multisensory Integration.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Lacey, S., Tal, N., Amedi, A. et al. A Putative Model of Multisensory Object Representation. Brain Topogr 21, 269–274 (2009). https://doi.org/10.1007/s10548-009-0087-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10548-009-0087-4

Keywords

Search

Navigation