Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Article
  • Published:

Transitions in neural oscillations reflect prediction errors generated in audiovisual speech

This article has been updated

Abstract

According to the predictive coding theory, top-down predictions are conveyed by backward connections and prediction errors are propagated forward across the cortical hierarchy. Using MEG in humans, we show that violating multisensory predictions causes a fundamental and qualitative change in both the frequency and spatial distribution of cortical activity. When visual speech input correctly predicted auditory speech signals, a slow delta regime (3–4 Hz) developed in higher-order speech areas. In contrast, when auditory signals invalidated predictions inferred from vision, a low-beta (14–15 Hz) / high-gamma (60–80 Hz) coupling regime appeared locally in a multisensory area (area STS). This frequency shift in oscillatory responses scaled with the degree of audio-visual congruence and was accompanied by increased gamma activity in lower sensory regions. These findings are consistent with the notion that bottom-up prediction errors are communicated in predominantly high (gamma) frequency ranges, whereas top-down predictions are mediated by slower (beta) frequencies.

This is a preview of subscription content, access via your institution

Access options

Rent or buy this article

Prices vary by article type

from$1.95

to$39.95

Prices may be subject to local taxes which are calculated during checkout

Figure 1: Interaction between predictiveness and validity in ERFs.
Figure 2: Late low-frequency oscillatory patterns depend on the validity of inferences.
Figure 3: Late high-frequency activity reflects prediction error computation.

Similar content being viewed by others

Change history

  • 15 May 2011

    In the HTML version of this article initially published online, the International Phonetic Alphabet character “ʒ” was missing in the first sentence of the Results. The error has been corrected in the HTML.

References

  1. Friston, K. A theory of cortical responses. Phil. Trans. R. Soc. Lond. B 360, 815–836 (2005).

    Article  Google Scholar 

  2. Mumford, D. On the computational architecture of the neocortex. II. The role of cortico-cortical loops. Biol. Cybern. 66, 241–251 (1992).

    Article  CAS  Google Scholar 

  3. Rao, R.P. & Ballard, D.H. Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects. Nat. Neurosci. 2, 79–87 (1999).

    Article  CAS  Google Scholar 

  4. Buzsáki, G. & Draguhn, A. Neuronal oscillations in cortical networks. Science 304, 1926–1929 (2004).

    Article  Google Scholar 

  5. Canolty, R.T. et al. High gamma power is phase-locked to theta oscillations in human neocortex. Science 313, 1626–1628 (2006).

    Article  CAS  Google Scholar 

  6. Lakatos, P. et al. An oscillatory hierarchy controlling neuronal excitability and stimulus processing in the auditory cortex. J. Neurophysiol. 94, 1904–1911 (2005).

    Article  Google Scholar 

  7. Wang, X.J. Neurophysiological and computational principles of cortical rhythms in cognition. Physiol. Rev. 90, 1195–1268 (2010).

    Article  Google Scholar 

  8. van Wassenhove, V., Grant, K.W. & Poeppel, D. Visual speech speeds up the neural processing of auditory speech. Proc. Natl. Acad. Sci. USA 102, 1181–1186 (2005).

    Article  CAS  Google Scholar 

  9. Chandrasekaran, C., Trubanova, A., Stillittano, S., Caplier, A. & Ghazanfar, A.A. The natural statistics of audiovisual speech. PLOS Comput. Biol. 5, e1000436 (2009).

    Article  Google Scholar 

  10. Arnal, L.H., Morillon, B., Kell, C.A. & Giraud, A.-L. Dual neural routing of visual facilitation in speech processing. J. Neurosci. 29, 13445–13453 (2009).

    Article  CAS  Google Scholar 

  11. McGurk, H. & MacDonald, J. Hearing lips and seeing voices. Nature 264, 746–748 (1976).

    Article  CAS  Google Scholar 

  12. Shah, A.S. et al. Neural dynamics and the fundamental mechanisms of event-related brain potentials. Cereb. Cortex 14, 476–483 (2004).

    Article  Google Scholar 

  13. Garrido, M.I., Kilner, J.M., Kiebel, S.J., Stephan, K.E. & Friston, K.J. Dynamic causal modelling of evoked potentials: a reproducibility study. Neuroimage 36, 571–580 (2007).

    Article  Google Scholar 

  14. Hickok, G. & Poeppel, D. The cortical organization of speech processing. Nat. Rev. Neurosci. 8, 393–402 (2007).

    Article  CAS  Google Scholar 

  15. Beauchamp, M.S., Lee, K.E., Argall, B.D. & Martin, A. Integration of auditory and visual information about objects in superior temporal sulcus. Neuron 41, 809–823 (2004).

    Article  CAS  Google Scholar 

  16. Ghazanfar, A.A., Chandrasekaran, C. & Logothetis, N.K. Interactions between the superior temporal sulcus and auditory cortex mediate dynamic face/voice integration in rhesus monkeys. J. Neurosci. 28, 4457–4469 (2008).

    Article  CAS  Google Scholar 

  17. Kayser, C. & Logothetis, N.K. Directed interactions between auditory and superior temporal cortices and their role in sensory integration. Front. Integr. Neurosci. 3, 7 (2009).

    Article  Google Scholar 

  18. Koch, S.P., Werner, P., Steinbrink, J., Fries, P. & Obrig, H. Stimulus-induced and state-dependent sustained gamma activity is tightly coupled to the hemodynamic response in humans. J. Neurosci. 29, 13962–13970 (2009).

    Article  CAS  Google Scholar 

  19. Mukamel, R. et al. Coupling between neuronal firing, field potentials, and FMRI in human auditory cortex. Science 309, 951–954 (2005).

    Article  CAS  Google Scholar 

  20. Niessing, J. et al. Hemodynamic signals correlate tightly with synchronized gamma oscillations. Science 309, 948–951 (2005).

    Article  CAS  Google Scholar 

  21. Engel, A.K. & Fries, P. Beta-band oscillations—signalling the status quo? Curr. Opin. Neurobiol. 20, 156–165 (2010).

    Article  CAS  Google Scholar 

  22. von Stein, A., Chiang, C. & Konig, P. Top-down processing mediated by interareal synchronization. Proc. Natl. Acad. Sci. USA 97, 14748–14753 (2000).

    Article  CAS  Google Scholar 

  23. Womelsdorf, T. et al. Modulation of neuronal interactions through neuronal synchronization. Science 316, 1609–1612 (2007).

    Article  CAS  Google Scholar 

  24. Sejnowski, T.J. & Paulsen, O. Network oscillations: emerging computational principles. J. Neurosci. 26, 1673–1676 (2006).

    Article  CAS  Google Scholar 

  25. Wróbel, A. Beta activity: a carrier for visual attention. Acta Neurobiol. Exp. (Warsz.) 60, 247–260 (2000).

    Google Scholar 

  26. Neisser, U. Cognitive Psychology (Appleton-Century-Crofts, New York, 1967).

  27. Poeppel, D., Idsardi, W.J. & van Wassenhove, V. Speech perception at the interface of neurobiology and linguistics. Phil. Trans. R. Soc. Lond. B 363, 1071–1086 (2008).

    Article  Google Scholar 

  28. Stevens, K.N. & Halle, M. Remarks on analysis by synthesis and distinctive features. in Models for the Perception of Speech and Visual Form: Proceedings of a Symposium (ed. Wathen-Dunn, W.) 88–102 (MIT Press, Cambridge, Massachusetts, USA, 1967).

  29. Wolpert, D.M., Ghahramani, Z. & Jordan, M.I. An internal model for sensorimotor integration. Science 269, 1880–1882 (1995).

    Article  CAS  Google Scholar 

  30. Roopun, A.K. et al. Period concatenation underlies interactions between gamma and beta rhythms in neocortex. Front. Cell. Neurosci. 2, 1 (2008).

    Article  Google Scholar 

  31. Spratling, M.W. Reconciling predictive coding and biased competition models of cortical function. Front. Comput. Neurosci. 2, 4 (2008).

    Article  Google Scholar 

  32. Buschman, T.J. & Miller, E.K. Top-down versus bottom-up control of attention in the prefrontal and posterior parietal cortices. Science 315, 1860–1862 (2007).

    Article  CAS  Google Scholar 

  33. George, D. & Hawkins, J. Towards a mathematical theory of cortical micro-circuits. PLOS Comput. Biol. 5, e1000532 (2009).

    Article  Google Scholar 

  34. Kopell, N., Kramer, M.A., Malerba, P. & Whittington, M.A. Are different rhythms good for different functions? Front. Hum. Neurosci. 4, 187 (2010).

    Article  Google Scholar 

  35. Kramer, M.A. et al. Rhythm generation through period concatenation in rat somatosensory cortex. PLOS Comput. Biol. 4, e1000169 (2008).

    Article  Google Scholar 

  36. Hagoort, P., Hald, L., Bastiaansen, M. & Petersson, K.M. Integration of word meaning and world knowledge in language comprehension. Science 304, 438–441 (2004).

    Article  CAS  Google Scholar 

  37. Hald, L.A., Bastiaansen, M.C. & Hagoort, P. EEG theta and gamma responses to semantic violations in online sentence processing. Brain Lang. 96, 90–105 (2006).

    Article  Google Scholar 

  38. Kayser, C., Montemurro, M.A., Logothetis, N.K. & Panzeri, S. Spike-phase coding boosts and stabilizes information carried by spatial and temporal spike patterns. Neuron 61, 597–608 (2009).

    Article  CAS  Google Scholar 

  39. Schroeder, C.E. & Lakatos, P. The gamma oscillation: master or slave? Brain Topogr. 22, 24–26 (2009).

    Article  Google Scholar 

  40. Engel, A.K., Fries, P. & Singer, W. Dynamic predictions: oscillations and synchrony in top-down processing. Nat. Rev. Neurosci. 2, 704–716 (2001).

    Article  CAS  Google Scholar 

  41. Fries, P., Roelfsema, P.R., Engel, A.K., Konig, P. & Singer, W. Synchronization of oscillatory responses in visual cortex correlates with perception in interocular rivalry. Proc. Natl. Acad. Sci. USA 94, 12699–12704 (1997).

    Article  CAS  Google Scholar 

  42. Wyart, V. & Tallon-Baudry, C. Neural dissociation between visual awareness and spatial attention. J. Neurosci. 28, 2667–2679 (2008).

    Article  CAS  Google Scholar 

  43. Schroeder, C.E. & Lakatos, P. Low-frequency neuronal oscillations as instruments of sensory selection. Trends Neurosci. 32, 9–18 (2009).

    Article  CAS  Google Scholar 

  44. Rockland, K.S. & Pandya, D.N. Laminar origins and terminations of cortical connections of the occipital lobe in the rhesus monkey. Brain Res. 179, 3–20 (1979).

    Article  CAS  Google Scholar 

  45. Akam, T. & Kullmann, D.M. Oscillations and filtering networks support flexible routing of information. Neuron 67, 308–320 (2010).

    Article  CAS  Google Scholar 

  46. Colgin, L.L. et al. Frequency of gamma oscillations routes flow of information in the hippocampus. Nature 462, 353–357 (2009).

    Article  CAS  Google Scholar 

  47. Panzeri, S., Brunel, N., Logothetis, N.K. & Kayser, C. Sensory neural codes using multiplexed temporal scales. Trends Neurosci. 33, 111–120 (2010).

    Article  CAS  Google Scholar 

  48. Grill-Spector, K., Henson, R. & Martin, A. Repetition and the brain: neural models of stimulus-specific effects. Trends Cogn. Sci. 10, 14–23 (2006).

    Article  Google Scholar 

  49. Tallon-Baudry, C., Bertrand, O., Delpuech, C. & Pernier, J. Stimulus specificity of phase-locked and non-phase-locked 40 Hz visual responses in human. J. Neurosci. 16, 4240–4249 (1996).

    Article  CAS  Google Scholar 

  50. Maris, E. & Oostenveld, R. Nonparametric statistical testing of EEG- and MEG-data. J. Neurosci. Methods 164, 177–190 (2007).

    Article  Google Scholar 

Download references

Acknowledgements

We thank the staff of the Centre de Neuroimagerie de Recherche and the Magnetoencephalography Center (Hôpital de la Pitié-Salpêtrière), in particular J.-D. Lemarechal, A. Ducorps and D. Schwarz, and the colleagues who commented on this work: V. van Wassenhove, B. Morillon, A. Hyafil, S. Denève, L. Melloni, F. Griffon, A. Kleinschmidt, E. Koechlin, B. Fischer, K. Friston and C. Tallon-Baudry. This work was supported by the Centre National de la Recherche Scientifique (A.-L.G.) and the Fondation Fyssen (V.W.).

Author information

Authors and Affiliations

Authors

Contributions

Conceived and designed the experiments: L.H.A., A.-L.G.; performed the experiments: L.H.A.; analyzed the data: L.H.A., V.W., A.-L.G.; contributed to analysis tools: L.H.A., V.W.; wrote the paper: L.H.A., V.W., A.-L.G.

Corresponding author

Correspondence to Anne-Lise Giraud.

Ethics declarations

Competing interests

The authors declare no competing financial interests.

Supplementary information

Supplementary Text and Figures

Supplementary Data and Supplementary Figures 1–5 (PDF 1038 kb)

Rights and permissions

Reprints and permissions

About this article

Cite this article

Arnal, L., Wyart, V. & Giraud, AL. Transitions in neural oscillations reflect prediction errors generated in audiovisual speech. Nat Neurosci 14, 797–801 (2011). https://doi.org/10.1038/nn.2810

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/nn.2810

This article is cited by

Search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing