Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Perspective
  • Published:

Coupling perception to action through incidental sensory consequences of motor behaviour

Nature Reviews Psychology volume 1, pages 112–123 (2022)Cite this article

Subjects

Abstract

Researchers in the field of active perception study how sensory processes coalesce with motor actions to extract information from the world. Such actions intrinsically alter perceptual processing and have intended sensory outcomes, but also lead to incidental sensory consequences, which are side effects of moving the sensory surface to its intended goal. These incidental consequences of actions are generally considered a nuisance to perception that needs to be attenuated or suppressed during movement execution. In this Perspective, we propose instead that incidental sensory consequences of actions shape perceptual processes through action–perception couplings and we review evidence from the domain of active vision. We propose four hallmarks representing the degrees to which actions are an integral part of a perceptual processing architecture. Finally, we outline a research strategy for probing these hallmarks in active perceptual systems and conclude that researchers of perception should embrace the study of action kinematics in pursuit of their questions.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: Three types of sensory consequence of visual actions.
Fig. 2: Evidence that incidental sensory consequences may support visual and motor functions.
Fig. 3: Framework of action–perception coupling and research strategy.

Similar content being viewed by others

References

  1. Churchland, P. S., Ramachandran, V. S. & Sejnowski, T. J. in Large-Scale Neuronal Theories of the Brain (eds Koch, C. & Davis, J. L.) 23–60 (MIT Press, 1994).

  2. O’Regan, J. K. Solving the ‘real’ mysteries of visual perception: the world as an outside memory. Can. J. Psychol. 46, 461–488 (1992).

    PubMed  Google Scholar 

  3. Hayhoe, M. & Ballard, D. Eye movements in natural behavior. Trends Cogn. Sci. 9, 188–194 (2005).

    PubMed  Google Scholar 

  4. Findlay, J. M. & Gilchrist, I. D. Active Vision (Oxford Univ. Press, 2003).

  5. Engel, A. K., Maye, A., Kurthen, M. & König, P. Where’s the action? The pragmatic turn in cognitive science. Trends Cogn. Sci. 17, 202–209 (2013).

    PubMed  Google Scholar 

  6. Goodale, M. A. Transforming vision into action. Vis. Res. 51, 1567–1587 (2011).

    PubMed  Google Scholar 

  7. Schroeder, C. E., Wilson, D. A., Radman, T., Scharfman, H. & Lakatos, P. Dynamics of active sensing and perceptual selection. Curr. Opin. Neurobiol. 20, 172–176 (2010).

    PubMed  PubMed Central  Google Scholar 

  8. Ridderinkhof, K. R. Neurocognitive mechanisms of perception–action coordination: a review and theoretical integration. Neurosci. Biobehav. Rev. 46, 3–29 (2014).

    PubMed  Google Scholar 

  9. Herwig, A. Linking perception and action by structure or process? Toward an integrative perspective. Neurosci. Biobehav. Rev. 52, 105–116 (2015).

    PubMed  Google Scholar 

  10. Hayhoe, M. M. Vision and action. Annu. Rev. Vis. Sci. 3, 389–413 (2017).

    PubMed  Google Scholar 

  11. Awh, E., Armstrong, K. M. & Moore, T. Visual and oculomotor selection: links, causes and implications for spatial attention. Trends Cogn. Sci. 10, 124–130 (2006).

    PubMed  Google Scholar 

  12. Moore, T. & Zirnsak, M. Neural mechanisms of selective visual attention. Annu. Rev. Psychol. 68, 47–72 (2017).

    PubMed  Google Scholar 

  13. Wurtz, R. H. Corollary discharge contributions to perceptual continuity across saccades. Annu. Rev. Vis. Sci. 4, 215–237 (2018).

    PubMed  Google Scholar 

  14. Heuer, A., Ohl, S. & Rolfs, M. Memory for action: a functional view of selection in visual working memory. Vis. Cogn. 28, 388–400 (2020).

    Google Scholar 

  15. Richmond, L. L. & Zacks, J. M. Constructing experience: event models from perception to action. Trends Cogn. Sci. 21, 962–980 (2017).

    PubMed  PubMed Central  Google Scholar 

  16. Hommel, B. & Wiers, R. W. Towards a unitary approach to human action control. Trends Cogn. Sci. 21, 940–949 (2017).

    PubMed  Google Scholar 

  17. Warren, W. H. Information is where you find it: perception as an ecologically well-posed problem. Perception https://doi.org/10.1177/20416695211000366 (2021).

    Article  Google Scholar 

  18. Witt, J. K. & Riley, M. A. Discovering your inner Gibson: reconciling action-specific and ecological approaches to perception–action. Psychon. Bull. Rev. 21, 1353–1370 (2014).

    PubMed  Google Scholar 

  19. Wexler, M. & van Boxtel, J. J. A. Depth perception by the active observer. Trends Cogn. Sci. 9, 431–438 (2005).

    PubMed  Google Scholar 

  20. Rucci, M. & Poletti, M. Control and functions of fixational eye movements. Annu. Rev. Vis. Sci. 1, 499–518 (2015).

    PubMed  PubMed Central  Google Scholar 

  21. Press, C. & Cook, R. Beyond action-specific simulation: domain-general motor contributions to perception. Trends Cogn. Sci. 19, 176–178 (2015).

    PubMed  Google Scholar 

  22. Binda, P. & Morrone, M. C. Vision during saccadic eye movements. Annu. Rev. Vis. Sci. 4, 193–213 (2018).

    PubMed  Google Scholar 

  23. Blakemore, S.-J., Wolpert, D. & Frith, C. Why canʼt you tickle yourself? NeuroReport 11, R11–R16 (2000).

    PubMed  Google Scholar 

  24. Schneider, D. M. & Mooney, R. How movement modulates hearing. Annu. Rev. Neurosci. 41, 553–572 (2018).

    PubMed  PubMed Central  Google Scholar 

  25. Land, M. F. The coordination of rotations of the eyes, head and trunk in saccadic turns produced in natural situations. Exp. Brain Res. 159, 151–160 (2004).

    PubMed  Google Scholar 

  26. Bajcsy, R. Active perception. Proc. IEEE 76, 966–1005 (1988).

    Google Scholar 

  27. Aloimonos, J., Weiss, I. & Bandyopadhyay, A. Active vision. Int. J. Comput. Vis. 1, 333–356 (1988).

    Google Scholar 

  28. Aloimonos, J. Purposive and qualitative active vision. in 10th Int. Conf. Pattern Recognition [1990] Proc. Vol. 1 346–360 (IEEE, 1990).

  29. Ballard, D. H. Animate vision. Artif. Intell. 48, 57–86 (1991).

    Google Scholar 

  30. Ballard, D. H. & Brown, C. M. Principles of animate vision. CVGIP Image Underst. 56, 3–21 (1992).

    Google Scholar 

  31. Tsotsos, J. K. A Computational Perspective on Visual Attention (MIT Press, 2011).

  32. Bajcsy, R., Aloimonos, Y. & Tsotsos, J. K. Revisiting active perception. Auton. Robot. 42, 177–196 (2018).

    Google Scholar 

  33. Schütz, A. C., Braun, D. I. & Gegenfurtner, K. R. Eye movements and perception: a selective review. J. Vis. 11, 9–9 (2011).

    PubMed  Google Scholar 

  34. Rolfs, M. Attention in active vision: a perspective on perceptual continuity across saccades. Perception 44, 900–919 (2015).

    PubMed  Google Scholar 

  35. Rummell, B. P., Klee, J. L. & Sigurdsson, T. Attenuation of responses to self-generated sounds in auditory cortical neurons. J. Neurosci. 36, 12010–12026 (2016).

    PubMed  PubMed Central  Google Scholar 

  36. van Ede, F., van Doren, T. I., Damhuis, J., de Lange, F. P. & Maris, E. Movement preparation improves touch perception without awareness. Cognition 137C, 189–195 (2015).

    Google Scholar 

  37. Schweitzer, R. & Rolfs, M. Intra-saccadic motion streaks as cues to linking object locations across saccades. J. Vis. 20, 17–17 (2020).

    PubMed  PubMed Central  Google Scholar 

  38. Campbell, F. W. & Wurtz, R. H. Saccadic omission: why we do not see a grey-out during a saccadic eye movement. Vis. Res. 18, 1297–1303 (1978).

    PubMed  Google Scholar 

  39. Balsdon, T., Schweitzer, R., Watson, T. L. & Rolfs, M. All is not lost: post-saccadic contributions to the perceptual omission of intra-saccadic streaks. Conscious. Cogn. 64, 19–31 (2018).

    PubMed  Google Scholar 

  40. Castet, E. in Dynamics of Visual Motion Processing (eds Ilg, U. J. & Masson, G. S.) 213–238 (Springer, 2009).

  41. Castet, E. Motion perception of saccade-induced retinal translation. Proc. Natl Acad. Sci. USA 99, 15159–15163 (2002).

    PubMed  PubMed Central  Google Scholar 

  42. Rucci, M. & Victor, J. D. The unsteady eye: an information-processing stage, not a bug. Trends Neurosci. 38, 195–206 (2015).

    PubMed  PubMed Central  Google Scholar 

  43. Rucci, M., Ahissar, E. & Burr, D. Temporal coding of visual space. Trends Cogn. Sci. 22, 883–895 (2018).

    PubMed  PubMed Central  Google Scholar 

  44. James, K. H. Sensori-motor experience leads to changes in visual processing in the developing brain. Dev. Sci. 13, 279–288 (2010).

    PubMed  PubMed Central  Google Scholar 

  45. White, A. L., Rolfs, M. & Carrasco, M. Adaptive deployment of spatial and feature-based attention before saccades. Vis. Res. 85, 26–35 (2013).

    PubMed  Google Scholar 

  46. Masselink, J. & Lappe, M. Visuomotor learning from postdictive motor error. eLife 10, e64278 (2021).

    PubMed  PubMed Central  Google Scholar 

  47. Rolfs, M. & Ohl, S. Moving fast and seeing slow? The visual consequences of vigorous movement. Behav. Brain Sci. 44, e131 (2021).

    PubMed  Google Scholar 

  48. Li, H.-H., Hanning, N. M. & Carrasco, M. To look or not to look: dissociating presaccadic and covert spatial attention. Trends Neurosci. 44, 669–686 (2021).

    PubMed  Google Scholar 

  49. Schneider, W. X., Einhäuser, W. & Horstmann, G. Attentional selection in visual perception, memory and action: a quest for cross-domain integration. Philos. Trans. R. Soc. B 368, 20130053 (2013).

    Google Scholar 

  50. Wurtz, R. H. Neuronal mechanisms of visual stability. Vis. Res. 48, 2070–2089 (2008).

    PubMed  Google Scholar 

  51. Kowler, E., Anderson, E., Dosher, B. & Blaser, E. The role of attention in the programming of saccades. Vis. Res. 35, 1897–1916 (1995).

    PubMed  Google Scholar 

  52. Deubel, H. & Schneider, W. X. Saccade target selection and object recognition: evidence for a common attentional mechanism. Vis. Res. 36, 1827–1837 (1996).

    PubMed  Google Scholar 

  53. Castet, E., Jeanjean, S., Montagnini, A., Laugier, D. & Masson, G. S. Dynamics of attentional deployment during saccadic programming. J. Vis. 6, 2–2 (2006).

    Google Scholar 

  54. Montagnini, A. & Castet, E. Spatiotemporal dynamics of visual attention during saccade preparation: independence and coupling between attention and movement planning. J. Vis. 7, 1–16 (2007).

    PubMed  Google Scholar 

  55. Deubel, H. The time course of presaccadic attention shifts. Psychol. Res. 72, 630–640 (2008).

    PubMed  Google Scholar 

  56. Rolfs, M. & Carrasco, M. Rapid simultaneous enhancement of visual sensitivity and perceived contrast during saccade preparation. J. Neurosci. 32, 13744–13752 (2012).

    PubMed  PubMed Central  Google Scholar 

  57. Shurygina, O., Pooresmaeili, A. & Rolfs, M. Pre-saccadic attention spreads to stimuli forming a perceptual group with the saccade target. Cortex 140, 179–198 (2021).

    PubMed  Google Scholar 

  58. Ohl, S., Kuper, C. & Rolfs, M. Selective enhancement of orientation tuning before saccades. J. Vis. 17, 1–11 (2017).

    Google Scholar 

  59. Li, H.-H., Barbot, A. & Carrasco, M. Saccade preparation reshapes sensory tuning. Curr. Biol. 26, 1564–1570 (2016).

    PubMed  PubMed Central  Google Scholar 

  60. Moore, T., Tolias, A. S. & Schiller, P. H. Visual representations during saccadic eye movements. Proc. Natl Acad. Sci. USA 95, 8981–8984 (1998).

    PubMed  PubMed Central  Google Scholar 

  61. Li, H.-H., Pan, J. & Carrasco, M. Presaccadic attention improves or impairs performance by enhancing sensitivity to higher spatial frequencies. Sci. Rep. 9, 2659 (2019).

    PubMed  PubMed Central  Google Scholar 

  62. Kroell, L. M. & Rolfs, M. The peripheral sensitivity profile at the saccade target reshapes during saccade preparation. Cortex 139, 12–26 (2021).

    PubMed  Google Scholar 

  63. Rolfs, M., Jonikaitis, D., Deubel, H. & Cavanagh, P. Predictive remapping of attention across eye movements. Nat. Neurosci. 14, 252–256 (2011).

    PubMed  Google Scholar 

  64. Cavanagh, P., Hunt, A. R., Afraz, A. & Rolfs, M. Visual stability based on remapping of attention pointers. Trends Cogn. Sci. 14, 147–153 (2010).

    PubMed  PubMed Central  Google Scholar 

  65. Schneider, W. X. Selective visual processing across competition episodes: a theory of task-driven visual attention and working memory. Philos. Trans. R. Soc. B 368, 20130060 (2013).

    Google Scholar 

  66. Irwin, D. E. Memory for position and identity across eye movements. J. Exp. Psychol. Learn. Mem. Cogn. 18, 307–317 (1992).

    Google Scholar 

  67. Bridgeman, B. A theory of visual stability across saccadic eye movements. Behav. Brain Sci. 17, 247–292 (1994).

    Google Scholar 

  68. McConkie, G. W. & Currie, C. B. Visual stability across saccades while viewing complex pictures. J. Exp. Psychol. Hum. Percept. Perform. 22, 563–581 (1996).

    PubMed  Google Scholar 

  69. Hollingworth, A. & Franconeri, S. L. Object correspondence across brief occlusion is established on the basis of both spatiotemporal and surface feature cues. Cognition 113, 150–166 (2009).

    PubMed  PubMed Central  Google Scholar 

  70. Aagten-Murphy, D. & Bays, P. M. in Processes of Visuospatial Attention and Working Memory Vol. 41 (ed. Hodgson, T.) 155–183 (Springer International, 2018).

  71. van der Stigchel, S. & Hollingworth, A. Visuospatial working memory as a fundamental component of the eye movement system. Curr. Dir. Psychol. Sci. 27, 136–143 (2018).

    PubMed  PubMed Central  Google Scholar 

  72. van Ede, F. Visual working memory and action: functional links and bi-directional influences. Vis. Cogn. 28, 401–413 (2020).

    PubMed  PubMed Central  Google Scholar 

  73. Olivers, C. N. L. & Roelfsema, P. R. Attention for action in visual working memory. Cortex 131, 179–194 (2020).

    PubMed  Google Scholar 

  74. Ohl, S. & Rolfs, M. Saccadic eye movements impose a natural bottleneck on visual short-term memory. J. Exp. Psychol. Learn. Mem. Cogn. 43, 736–748 (2017).

    PubMed  Google Scholar 

  75. Ohl, S. & Rolfs, M. Saccadic selection of stabilized items in visuospatial working memory. Conscious. Cogn. 64, 32–44 (2018).

    PubMed  Google Scholar 

  76. Hanning, N. M., Jonikaitis, D., Deubel, H. & Szinte, M. Oculomotor selection underlies feature retention in visual working memory. J. Neurophysiol. 115, 1071–1076 (2016).

    PubMed  Google Scholar 

  77. Hanning, N. M. & Deubel, H. Independent effects of eye and hand movements on visual working memory. Front. Syst. Neurosci. 12, 37 (2018).

    PubMed  PubMed Central  Google Scholar 

  78. Heuer, A. & Schubö, A. Separate and combined effects of action relevance and motivational value on visual working memory. J. Vis. 18, 14–14 (2018).

    PubMed  Google Scholar 

  79. Ohl, S. & Rolfs, M. Bold moves: inevitable saccadic selection in visual short-term memory. J. Vis. 20, 11–11 (2020).

    PubMed  PubMed Central  Google Scholar 

  80. Heuer, A. & Schubö, A. Selective weighting of action-related feature dimensions in visual working memory. Psychon. Bull. Rev. 24, 1129–1134 (2017).

    PubMed  Google Scholar 

  81. Harman, K. L., Humphrey, G. K. & Goodale, M. A. Active manual control of object views facilitates visual recognition. Curr. Biol. 9, 1315–1318 (1999).

    PubMed  Google Scholar 

  82. Michotte, A. The Perception of Causality (Basic Books, 1963).

  83. Scholl, B. J. & Tremoulet, P. D. Perceptual causality and animacy. Trends Cogn. Sci. 4, 299–309 (2000).

    PubMed  Google Scholar 

  84. Rolfs, M., Dambacher, M. & Cavanagh, P. Visual adaptation of the perception of causality. Curr. Biol. 23, 250–254 (2013).

    PubMed  Google Scholar 

  85. Fleischer, F., Christensen, A., Caggiano, V., Thier, P. & Giese, M. A. Neural theory for the perception of causal actions. Psychol. Res. 76, 476–493 (2012).

    PubMed  Google Scholar 

  86. Ullman, S., Harari, D. & Dorfman, N. From simple innate biases to complex visual concepts. Proc. Natl Acad. Sci. USA 109, 18215–18220 (2012).

    PubMed  PubMed Central  Google Scholar 

  87. White, P. A. The experience of force: the role of haptic experience of forces in visual perception of object motion and interactions, mental simulation, and motion-related judgments. Psychol. Bull. 138, 589–615 (2012).

    PubMed  Google Scholar 

  88. Caggiano, V., Fleischer, F., Pomper, J. K., Giese, M. A. & Thier, P. Mirror neurons in monkey premotor area F5 show tuning for critical features of visual causality perception. Curr. Biol. 26, 3077–3082 (2016).

    PubMed  Google Scholar 

  89. Rolfs, M. Visual neuroscience: seeing causality with the motor system? Curr. Biol. 26, R1183–R1185 (2016).

    PubMed  Google Scholar 

  90. Rolfs, M. Microsaccades: small steps on a long way. Vis. Res. 49, 2415–2441 (2009).

    PubMed  Google Scholar 

  91. Otero-Millan, J., Macknik, S. L., Langston, R. E. & Martinez-Conde, S. An oculomotor continuum from exploration to fixation. Proc. Natl Acad. Sci. USA 110, 6175–6180 (2013).

    PubMed  PubMed Central  Google Scholar 

  92. Watson, T. L. & Krekelberg, B. The relationship between saccadic suppression and perceptual stability. Curr. Biol. 19, 1040–1043 (2009).

    PubMed  PubMed Central  Google Scholar 

  93. O’Regan, J. K. & Noë, A. A sensorimotor account of vision and visual consciousness. Behav. Brain Sci. 24, 939–973 (2001). discussion 973-1031.

    PubMed  Google Scholar 

  94. Volkmann, F. C. Vision during voluntary saccadic eye movements. J. Opt. Soc. Am. 52, 571 (1962).

    PubMed  Google Scholar 

  95. Duyck, M., Collins, T. & Wexler, M. Masking the saccadic smear. J. Vis. 16, 1–1 (2016).

    PubMed  Google Scholar 

  96. Cleland, B. G., Dubin, M. W. & Levick, W. R. Sustained and transient neurones in the cat’s retina and lateral geniculate nucleus. J. Physiol. 217, 473–496 (1971).

    PubMed  PubMed Central  Google Scholar 

  97. Croner, L. J. & Kaplan, E. Receptive fields of P and M ganglion cells across the primate retina. Vis. Res. 35, 7–24 (1995).

    PubMed  Google Scholar 

  98. Nagano, T. Temporal sensitivity of the human visual system to sinusoidal gratings. J. Opt. Soc. Am. 70, 711 (1980).

    PubMed  Google Scholar 

  99. Mostofi, N. et al. Spatiotemporal content of saccade transients. Curr. Biol. 30, 3999–4008.e2 (2020).

    PubMed  PubMed Central  Google Scholar 

  100. Boi, M., Poletti, M., Victor, J. D. & Rucci, M. Consequences of the oculomotor cycle for the dynamics of perception. Curr. Biol. 27, 1268–1277 (2017).

    PubMed  PubMed Central  Google Scholar 

  101. Burr, D. C. & Ross, J. Contrast sensitivity at high velocities. Vis. Res. 22, 479–484 (1982).

    PubMed  Google Scholar 

  102. Casile, A., Victor, J. D. & Rucci, M. Contrast sensitivity reveals an oculomotor strategy for temporally encoding space. eLife 8, e40924 (2019).

    PubMed  PubMed Central  Google Scholar 

  103. Zimmermann, E. Saccade suppression depends on context. eLife 9, e49700 (2020).

    PubMed  PubMed Central  Google Scholar 

  104. Scholes, C., McGraw, P. V. & Roach, N. W. Learning to silence saccadic suppression. Proc. Natl Acad. Sci. USA 118, e2012937118 (2021).

    PubMed  PubMed Central  Google Scholar 

  105. Geisler, W. S. Motion streaks provide a spatial code for motion direction. Nature 400, 65–69 (1999).

    PubMed  Google Scholar 

  106. Geisler, W. S., Albrecht, D. G., Crane, A. M. & Stern, L. Motion direction signals in the primary visual cortex of cat and monkey. Vis. Neurosci. 18, 501–516 (2001).

    PubMed  Google Scholar 

  107. Apthorp, D., Cass, J. & Alais, D. Orientation tuning of contrast masking caused by motion streaks. J. Vis. 10, 11–11 (2010).

    PubMed  Google Scholar 

  108. Apthorp, D. et al. Direct evidence for encoding of motion streaks in human visual cortex. Proc. R. Soc. B Biol. Sci. 280, 20122339 (2013).

    Google Scholar 

  109. Krekelberg, B., Vatakis, A. & Kourtzi, Z. Implied motion from form in the human visual cortex. J. Neurophysiol. 94, 4373–4386 (2005).

    PubMed  Google Scholar 

  110. Edwards, M. & Crane, M. F. Motion streaks improve motion detection. Vis. Res. 47, 828–833 (2007).

    PubMed  Google Scholar 

  111. Jancke, D. Orientation formed by a spot’s trajectory: a two-dimensional population approach in primary visual cortex. J. Neurosci. 20, RC86–RC86 (2000).

    PubMed  PubMed Central  Google Scholar 

  112. Schweitzer, R. & Rolfs, M. Intrasaccadic motion streaks jump-start gaze correction. Sci. Adv. 7, eabf2218 (2021).

    PubMed  PubMed Central  Google Scholar 

  113. Flombaum, J. I., Scholl, B. J. & Santos, L. R. in The Origins of Object Knowledge (eds Hood, B. M. & Santos, L. R.) 135–164 (Oxford Univ. Press, 2009).

  114. Kahneman, D., Treisman, A. & Gibbs, B. J. The reviewing of object files: object-specific integration of information. Cognit. Psychol. 24, 175–219 (1992).

    PubMed  Google Scholar 

  115. Mitroff, S. R. & Alvarez, G. A. Space and time, not surface features, guide object persistence. Psychon. Bull. Rev. 14, 1199–1204 (2007).

    PubMed  Google Scholar 

  116. Bahill, A. T., Clark, M. R. & Stark, L. The main sequence, a tool for studying human eye movements. Math. Biosci. 24, 191–204 (1975).

    Google Scholar 

  117. Zuber, B. L., Stark, L. & Cook, G. Microsaccades and the velocity–amplitude relationship for saccadic eye movements. Science 150, 1459–1460 (1965).

    PubMed  Google Scholar 

  118. Engbert, R. in Progress in Brain Research Vol. 154 Part A (eds. Martinez-Conde, S., Macknik, S.L., Martinez, L.M., Alonso, J.-M. & Tse,P.U.) 177–192 (Elsevier, 2006).

  119. Kapoula, Z. A., Robinson, D. A. & Hain, T. C. Motion of the eye immediately after a saccade. Exp. Brain Res. 61, 386–394 (1986).

    PubMed  Google Scholar 

  120. Serences, J. T. Neural mechanisms of information storage in visual short-term memory. Vis. Res. 128, 53–67 (2016).

    PubMed  Google Scholar 

  121. Sreenivasan, K. K., Curtis, C. E. & D’Esposito, M. Revisiting the role of persistent neural activity during working memory. Trends Cogn. Sci. 18, 82–89 (2014).

    PubMed  PubMed Central  Google Scholar 

  122. Christophel, T. B., Klink, P. C., Spitzer, B., Roelfsema, P. R. & Haynes, J.-D. The distributed nature of working memory. Trends Cogn. Sci. 21, 111–124 (2017).

    PubMed  Google Scholar 

  123. Bedell, H. E. & Yang, J. The attenuation of perceived image smear during saccades. Vis. Res. 41, 521–528 (2001).

    PubMed  Google Scholar 

  124. Brooks, B. A., Yates, J. T. & Coleman, R. D. Perception of images moving at saccadic velocities during saccades and during fixation. Exp. Brain Res. 40, 71–78 (1980).

    PubMed  Google Scholar 

  125. Duyck, M., Wexler, M., Castet, E. & Collins, T. Motion masking by stationary objects: a study of simulated saccades. I-Perception https://doi.org/10.1177/2041669518773111 (2018).

    Article  PubMed  PubMed Central  Google Scholar 

  126. Brooks, B. A. & Fuchs, A. F. Influence of stimulus parameters on visual sensitivity during saccadic eye movement. Vis. Res. 15, 1389–1398 (1975).

    PubMed  Google Scholar 

  127. Chekaluk, E. & Llewellyn, K. R. Visual stimulus input, saccadic suppression, and detection of information from the postsaccade scene. Percept. Psychophys. 48, 135–142 (1990).

    PubMed  Google Scholar 

  128. Diamond, M. R., Ross, J. & Morrone, M. C. Extraretinal control of saccadic suppression. J. Neurosci. 20, 3449–3455 (2000).

    PubMed  PubMed Central  Google Scholar 

  129. MacKay, D. M. Elevation of visual threshold by displacement of retinal image. Nature 225, 90–92 (1970).

    PubMed  Google Scholar 

  130. Idrees, S., Baumann, M. P., Franke, F., Münch, T. A. & Hafed, Z. M. Perceptual saccadic suppression starts in the retina. Nat. Commun. 11, 1977 (2020).

    PubMed  PubMed Central  Google Scholar 

  131. Sperling, G. Comparison of perception in the moving and stationary eye. Rev. Oculomot. Res. 4, 307–351 (1990).

    PubMed  Google Scholar 

  132. MacKay, D. M. Mislocation of test flashes during saccadic image displacements. Nature 227, 731–733 (1970).

    PubMed  Google Scholar 

  133. Morrone, M. C., Ross, J. & Burr, D. C. Apparent position of visual targets during real and simulated saccadic eye movements. J. Neurosci. 17, 7941–7953 (1997).

    PubMed  PubMed Central  Google Scholar 

  134. Ostendorf, F., Fischer, C., Gaymard, B. & Ploner, C. J. Perisaccadic mislocalization without saccadic eye movements. Neuroscience 137, 737–745 (2006).

    PubMed  Google Scholar 

  135. Shadmehr, R., Reppert, T. R., Summerside, E. M., Yoon, T. & Ahmed, A. A. Movement vigor as a reflection of subjective economic utility. Trends Neurosci. 42, 323–336 (2019).

    PubMed  PubMed Central  Google Scholar 

  136. Ahissar, E. & Arieli, A. Figuring space by time. Neuron 32, 185–201 (2001).

    PubMed  Google Scholar 

  137. Reppas, J. B., Usrey, W. & Reid, R. Saccadic eye movements modulate visual responses in the lateral geniculate nucleus. Neuron 35, 961–974 (2002).

    PubMed  Google Scholar 

  138. MacEvoy, S. P., Hanks, T. D. & Paradiso, M. A. Macaque V1 activity during natural vision: effects of natural scenes and saccades. J. Neurophysiol. 99, 460–472 (2008).

    PubMed  Google Scholar 

  139. Bosman, C. A., Womelsdorf, T., Desimone, R. & Fries, P. A microsaccadic rhythm modulates gamma-band synchronization and behavior. J. Neurosci. 29, 9471–9480 (2009).

    PubMed  PubMed Central  Google Scholar 

  140. Leszczynski, M. & Schroeder, C. E. The role of neuronal oscillations in visual active sensing. Front. Integr. Neurosci. 13, 32 (2019).

    PubMed  PubMed Central  Google Scholar 

  141. Martinez-Conde, S., Otero-Millan, J. & Macknik, S. L. The impact of microsaccades on vision: towards a unified theory of saccadic function. Nat. Rev. Neurosci. 14, 83–96 (2013).

    PubMed  Google Scholar 

  142. Potter, M. C. Meaning in visual search. Science 187, 965–966 (1975).

    PubMed  Google Scholar 

  143. Potter, M. C. Short-term conceptual memory for pictures. J. Exp. Psychol. [Hum. Learn.] 2, 509–522 (1976).

    Google Scholar 

  144. Raymond, J. E., Shapiro, K. L. & Arnell, K. M. Temporary suppression of visual processing in an RSVP task: an attentional blink? J. Exp. Psychol. Hum. Percept. Perform. 18, 849–860 (1992).

    PubMed  Google Scholar 

  145. Shapiro, K. L., Raymond, J. E. & Arnell, K. M. The attentional blink. Trends Cogn. Sci. 1, 291–296 (1997).

    PubMed  Google Scholar 

  146. Duncan, J., Ward, R. & Shapiro, K. Direct measurement of attentional dwell time in human vision. Nature 369, 313–315 (1994).

    PubMed  Google Scholar 

  147. Ward, R., Duncan, J. & Shapiro, K. The slow time-course of visual attention. Cognit. Psychol. 30, 79–109 (1996).

    PubMed  Google Scholar 

  148. Breitmeyer, B. G. & Ganz, L. Implications of sustained and transient channels for theories of visual pattern masking, saccadic suppression, and information processing. Psychol. Rev. 83, 1–36 (1976).

    PubMed  Google Scholar 

  149. Herzog, M. H., Drissi-Daoudi, L. & Doerig, A. All in good time: long-lasting postdictive effects reveal discrete perception. Trends Cogn. Sci. 24, 826–837 (2020).

    PubMed  Google Scholar 

  150. Choi, H. & Scholl, B. J. Perceiving causality after the fact: postdiction in the temporal dynamics of causal perception. Perception 35, 385–399 (2006).

    PubMed  Google Scholar 

  151. Holcombe, A. O. Seeing slow and seeing fast: two limits on perception. Trends Cogn. Sci. 13, 216–221 (2009).

    PubMed  Google Scholar 

  152. Bouma, H. Interaction effects in parafoveal letter recognition. Nature 226, 177–178 (1970).

    PubMed  Google Scholar 

  153. Levi, D. M. Crowding — an essential bottleneck for object recognition: a mini-review. Vis. Res. 48, 635–654 (2008).

    PubMed  Google Scholar 

  154. Whitney, D. & Levi, D. M. Visual crowding: a fundamental limit on conscious perception and object recognition. Trends Cogn. Sci. 15, 160–168 (2011).

    PubMed  PubMed Central  Google Scholar 

  155. Pelli, D. G. & Tillman, K. A. The uncrowded window of object recognition. Nat. Neurosci. 11, 1129–1135 (2008).

    PubMed  PubMed Central  Google Scholar 

  156. Harrison, W. J., Retell, J. D., Remington, R. W. & Mattingley, J. B. Visual crowding at a distance during predictive remapping. Curr. Biol. 23, 793–798 (2013).

    PubMed  Google Scholar 

  157. Harrison, W. J., Mattingley, J. B. & Remington, R. W. Eye movement targets are released from visual crowding. J. Neurosci. 33, 2927–2933 (2013).

    PubMed  PubMed Central  Google Scholar 

  158. Yildirim, F., Meyer, V. & Cornelissen, F. W. Eyes on crowding: crowding is preserved when responding by eye and similarly affects identity and position accuracy. J. Vis. 15, 21–21 (2015).

    PubMed  Google Scholar 

  159. Ağaoğlu, M. N., Öğmen, H. & Chung, S. T. L. Unmasking saccadic uncrowding. Vis. Res. 127, 152–164 (2016).

    PubMed  Google Scholar 

  160. Ağaoğlu, M. N. & Chung, S. T. L. Interaction between stimulus contrast and pre-saccadic crowding. R. Soc. Open. Sci. 4, 160559 (2017).

    PubMed  PubMed Central  Google Scholar 

  161. Bridgeman, B., Hendry, D. & Stark, L. Failure to detect displacement of the visual world during saccadic eye movements. Vis. Res. 15, 719–722 (1975).

    PubMed  Google Scholar 

  162. Wexler, M. & Collins, T. Orthogonal steps relieve saccadic suppression. J. Vis. 14, 13–13 (2014).

    PubMed  Google Scholar 

  163. Niemeier, M., Crawford, J. D. & Tweed, D. B. Optimal transsaccadic integration explains distorted spatial perception. Nature 422, 76–80 (2003).

    PubMed  Google Scholar 

  164. Greenwood, J. A., Szinte, M., Sayim, B. & Cavanagh, P. Variations in crowding, saccadic precision, and spatial localization reveal the shared topology of spatial vision. Proc. Natl Acad. Sci. USA 23, 201615504 (2017).

    Google Scholar 

  165. Nandy, A. S. & Tjan, B. S. Saccade-confounded image statistics explain visual crowding. Nat. Neurosci. 15, 463–469 (2012).

    PubMed  PubMed Central  Google Scholar 

  166. Zago, M., McIntyre, J., Senot, P. & Lacquaniti, F. Visuo-motor coordination and internal models for object interception. Exp. Brain Res. 192, 571–604 (2009).

    PubMed  Google Scholar 

  167. Hogendoorn, H. Motion extrapolation in visual processing: lessons from 25 years of flash-lag debate. J. Neurosci. 40, 5698–5705 (2020).

    PubMed  PubMed Central  Google Scholar 

  168. Cavanagh, P. & Anstis, S. The flash grab effect. Vis. Res. 91, 8–20 (2013).

    PubMed  Google Scholar 

  169. van Heusden, E., Rolfs, M., Cavanagh, P. & Hogendoorn, H. Motion extrapolation for eye movements predicts perceived motion-induced position shifts. J. Neurosci. 38, 8243–8250 (2018).

    PubMed  PubMed Central  Google Scholar 

  170. Blakemore, S.-J. & Decety, J. From the perception of action to the understanding of intention. Nat. Rev. Neurosci. 2, 561–567 (2001).

    PubMed  Google Scholar 

  171. de’Sperati, C. & Viviani, P. The relationship between curvature and velocity in two-dimensional smooth pursuit eye movements. J. Neurosci. 17, 3932–3945 (1997).

    PubMed  PubMed Central  Google Scholar 

  172. Lacquaniti, F., Terzuolo, C. & Viviani, P. The law relating the kinematic and figural aspects of drawing movements. Acta Psychol. 54, 115–130 (1983).

    Google Scholar 

  173. Viviani, P. & Stucchi, N. The effect of movement velocity on form perception: geometric illusions in dynamic displays. Percept. Psychophys. 46, 266–274 (1989).

    PubMed  Google Scholar 

  174. Viviani, P. & Stucchi, N. Biological movements look uniform: evidence of motor–perceptual interactions. J. Exp. Psychol. Hum. Percept. Perform. 18, 603–623 (1992).

    PubMed  Google Scholar 

  175. Levit-Binnun, N., Schechtman, E. & Flash, T. On the similarities between the perception and production of elliptical trajectories. Exp. Brain Res. 172, 533–555 (2006).

    PubMed  Google Scholar 

  176. Maoz, U. & Flash, T. Spatial constant equi-affine speed and motion perception. J. Neurophysiol. 111, 336–349 (2013).

    PubMed  Google Scholar 

  177. Dayan, E. et al. Neural representations of kinematic laws of motion: evidence for action–perception coupling. Proc. Natl Acad. Sci. USA 104, 20582–20587 (2007).

    PubMed  PubMed Central  Google Scholar 

  178. Kandel, S., Orliaguet, J.-P. & Viviani, P. Perceptual anticipation in handwriting: the role of implicit motor competence. Percept. Psychophys. 62, 706–716 (2000).

    PubMed  Google Scholar 

  179. Flach, R., Knoblich, G. & Prinz, W. The two-thirds power law in motion perception. Vis. Cogn. 11, 461–481 (2004).

    Google Scholar 

  180. White, B. J., Stritzke, M. & Gegenfurtner, K. R. Saccadic facilitation in natural backgrounds. Curr. Biol. 18, 124–128 (2008).

    PubMed  Google Scholar 

  181. Gellman, R. S. & Carl, J. R. Motion processing for saccadic eye movements in humans. Exp. Brain Res. 84, 660–667 (1991).

    PubMed  Google Scholar 

  182. Krakauer, J. W., Ghazanfar, A. A., Gomez-Marin, A., MacIver, M. A. & Poeppel, D. Neuroscience needs behavior: correcting a reductionist bias. Neuron 93, 480–490 (2017).

    PubMed  Google Scholar 

  183. Schweitzer, R. & Rolfs, M. An adaptive algorithm for fast and reliable online saccade detection. Behav. Res. Methods 52, 1122–1139 (2020).

    PubMed  Google Scholar 

  184. Braun, D. I., Schütz, A. C. & Gegenfurtner, K. R. Localization of speed differences of context stimuli during fixation and smooth pursuit eye movements. Vis. Res. 50, 2740–2749 (2010).

    PubMed  Google Scholar 

  185. Maus, G. W., Goh, H. L. & Lisi, M. Perceiving locations of moving objects across eyeblinks. Psychol. Sci. 31, 1117–1128 (2020).

    PubMed  Google Scholar 

  186. Duyck, M., Collins, T. & Wexler, M. Visual continuity during blinks and alterations in time perception. J. Exp. Psychol. Hum. Percept. Perform. 47, 1–12 (2021).

    PubMed  Google Scholar 

  187. Murakami, I. Correlations between fixation stability and visual motion sensitivity. Vis. Res. 44, 751–761 (2004).

    PubMed  Google Scholar 

  188. Poletti, M., Listorti, C. & Rucci, M. Stability of the visual world during eye drift. J. Neurosci. 30, 11143–11150 (2010).

    PubMed  PubMed Central  Google Scholar 

  189. Aytekin, M. & Rucci, M. Motion parallax from microscopic head movements during visual fixation. Vis. Res. 70, 7–17 (2012).

    PubMed  Google Scholar 

  190. Deubel, H. & Bridgeman, B. Perceptual consequences of ocular lens overshoot during saccadic eye movements. Vis. Res. 35, 2897–2902 (1995).

    PubMed  Google Scholar 

  191. Samonds, J. M., Geisler, W. S. & Priebe, N. J. Natural image and receptive field statistics predict saccade sizes. Nat. Neurosci. 21, 1591–1599 (2018).

    PubMed  PubMed Central  Google Scholar 

  192. Crapse, T. B. & Sommer, M. A. Corollary discharge across the animal kingdom. Nat. Rev. Neurosci. 9, 587–600 (2008).

    PubMed  PubMed Central  Google Scholar 

  193. Brimijoin, W. O. & Akeroyd, M. A. The role of head movements and signal spectrum in an auditory front/back illusion. i-Percept 3, 179–182 (2012).

    Google Scholar 

  194. Spering, M. & Carrasco, M. Acting without seeing: eye movements reveal visual processing without awareness. Trends Neurosci. 38, 247–258 (2015).

    PubMed  PubMed Central  Google Scholar 

  195. Richards, W. Saccadic suppression. J. Opt. Soc. Am. 59, 617–623 (1969).

    PubMed  Google Scholar 

  196. Dorr, M. & Bex, P. J. Peri-saccadic natural vision. J. Neurosci. 33, 1211–1217 (2013).

    PubMed  PubMed Central  Google Scholar 

  197. Richards, W. Visual suppression during passive eye movement. J. Opt. Soc. Am. 58, 1159–1160 (1968).

    PubMed  Google Scholar 

  198. Burr, D. C., Holt, J., Johnstone, J. R. & Ross, J. Selective depression of motion sensitivity during saccades. J. Physiol. 333, 1–15 (1982).

    PubMed  PubMed Central  Google Scholar 

  199. Volkmann, F. C., Riggs, L. A., White, K. D. & Moore, R. K. Contrast sensitivity during saccadic eye movements. Vis. Res. 18, 1193–1199 (1978).

    PubMed  Google Scholar 

  200. Burr, D. C., Morrone, M. C. & Ross, J. Selective suppression of the magnocellular visual pathway during saccadic eye movements. Nature 371, 511–513 (1994).

    PubMed  Google Scholar 

  201. Ross, J., Morrone, M. C., Goldberg, M. E. & Burr, D. C. Changes in visual perception at the time of saccades. Trends Neurosci. 24, 113–121 (2001).

    PubMed  Google Scholar 

  202. Sylvester, R., Haynes, J.-D. & Rees, G. Saccades differentially modulate human LGN and V1 responses in the presence and absence of visual stimulation. Curr. Biol. 15, 37–41 (2005).

    PubMed  Google Scholar 

  203. Thilo, K. V., Santoro, L., Walsh, V. & Blakemore, C. The site of saccadic suppression. Nat. Neurosci. 7, 13–14 (2004).

    PubMed  Google Scholar 

  204. Bremmer, F., Kubischik, M., Hoffmann, K.-P. & Krekelberg, B. Neural dynamics of saccadic suppression. J. Neurosci. 29, 12374–12383 (2009).

    PubMed  PubMed Central  Google Scholar 

  205. Thiele, A., Henning, P., Kubischik, M. & Hoffmann, K.-P. Neural mechanisms of saccadic suppression. Science 295, 2460–2462 (2002).

    PubMed  Google Scholar 

  206. Castet, E. & Masson, G. S. Motion perception during saccadic eye movements. Nat. Neurosci. 3, 177–183 (2000).

    PubMed  Google Scholar 

  207. Deubel, H., Elsner, T. & Hauske, G. Saccadic eye movements and the detection of fast-moving gratings. Biol. Cybern. 57, 37–45 (1987).

    PubMed  Google Scholar 

  208. Garcı́a-Pérez, M. A. & Peli, E. Intrasaccadic perception. J. Neurosci. 21, 7313–7322 (2001).

    PubMed  PubMed Central  Google Scholar 

  209. Matin, E. Saccadic suppression: a review and an analysis. Psychol. Bull. 81, 899–917 (1974).

    PubMed  Google Scholar 

  210. Mitrani, L., Mateeff, S. & Yakimoff, N. Smearing of the retinal image during voluntary saccadic eye movements. Vis. Res. 10, 405–409 (1970).

    PubMed  Google Scholar 

  211. Brooks, B. A., Impelman, D. M. K. & Lum, J. T. Backward and forward masking associated with saccadic eye movement. Percept. Psychophys. 30, 62–70 (1981).

    PubMed  Google Scholar 

  212. Matin, E., Clymer, A. B. & Matin, L. Metacontrast and saccadic suppression. Science 178, 179–182 (1972).

    PubMed  Google Scholar 

  213. Crevecoeur, F. & Körding, K. P. Saccadic suppression as a perceptual consequence of efficient sensorimotor estimation. eLife 6, e25073 (2017).

    PubMed  PubMed Central  Google Scholar 

  214. Burton, G. J. & Moorhead, I. R. Color and spatial structure in natural scenes. Appl. Opt. 26, 157 (1987).

    PubMed  Google Scholar 

  215. Tolhurst, D. J., Tadmor, Y. & Chao, T. Amplitude spectra of natural images. Ophthalmic Physiol. Opt. 12, 229–232 (2007).

    Google Scholar 

  216. Coppola, D. M., Purves, H. R., McCoy, A. N. & Purves, D. The distribution of oriented contours in the real world. Proc. Natl Acad. Sci. USA 95, 4002–4006 (1998).

    PubMed  PubMed Central  Google Scholar 

  217. Torralba, A. & Oliva, A. Statistics of natural image categories. Netw. Comput. Neural Syst. 14, 391–412 (2003).

    Google Scholar 

  218. Najemnik, J. & Geisler, W. S. Eye movement statistics in humans are consistent with an optimal search strategy. J. Vis. 8, 4 (2008).

    PubMed  Google Scholar 

  219. Appelle, S. Perception and discrimination as a function of stimulus orientation: the ‘oblique effect’ in man and animals. Psychol. Bull. 78, 266–278 (1972).

    PubMed  Google Scholar 

  220. Field, D. J. Relations between the statistics of natural images and the response properties of cortical cells. J. Opt. Soc. Am. A 4, 2379 (1987).

    PubMed  Google Scholar 

  221. Dragoi, V., Turcu, C. M. & Sur, M. Stability of cortical responses and the statistics of natural scenes. Neuron 32, 1181–1192 (2001).

    PubMed  Google Scholar 

  222. Girshick, A. R., Landy, M. S. & Simoncelli, E. P. Cardinal rules: visual orientation perception reflects knowledge of environmental statistics. Nat. Neurosci. 14, 926–932 (2011).

    PubMed  PubMed Central  Google Scholar 

  223. Cajar, A., Engbert, R. & Laubrock, J. How spatial frequencies and color drive object search in real-world scenes: a new eye-movement corpus. J. Vis. 20, 8 (2020).

    PubMed Central  Google Scholar 

  224. Mohr, J. et al. BOiS — Berlin Object in Scene Database: controlled photographic images for visual search experiments with quantified contextual priors. Front. Psychol. 7, 749 (2016).

    PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgements

The authors thank L. M. Kroell, J.-N. Klanke, S. Ohl, Ł. Grzeczkowski, C. Hübner and W. Nörenberg for feedback on an earlier version of this manuscript, and all members of the Active Perception and Cognition group for helpful discussions. This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No 865715) as well as from the Deutsche Forschungsgemeinschaft (DFG; German Research Foundation) under Germany’s Excellence Strategy — EXC 2002/1 ‘Science of Intelligence’ — project no. 390523135. M.R. was supported by the Heisenberg Programme of the DFG (grants RO 3579/8-1 and RO 3579/12-1).

Author information

Authors and Affiliations

  1. Department of Psychology, Humboldt-Universität zu Berlin, Berlin, Germany

    Martin Rolfs & Richard Schweitzer

  2. Bernstein Center for Computational Neuroscience Berlin, Berlin, Germany

    Martin Rolfs

  3. Exzellenzcluster Science of Intelligence, Technische Universität Berlin, Berlin, Germany

    Martin Rolfs & Richard Schweitzer

Authors
  1. Martin Rolfs
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Richard Schweitzer
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Both authors contributed to all aspects of the article.

Corresponding author

Correspondence to Martin Rolfs.

Ethics declarations

Competing interests

The authors declare no competing interests.

Additional information

Peer review information

Nature Reviews Psychology thanks Grace Edwards, Freek van Ede and the other, anonymous, reviewer for their contribution to the peer review of this work.

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Rolfs, M., Schweitzer, R. Coupling perception to action through incidental sensory consequences of motor behaviour. Nat Rev Psychol 1, 112–123 (2022). https://doi.org/10.1038/s44159-021-00015-x

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s44159-021-00015-x

This article is cited by

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing