The frontal eye field as a prediction map

doi:10.1016/S0079-6123(08)00656-0

Progress in Brain Research

Volume 171, 2008, Pages 383-390

https://doi.org/10.1016/S0079-6123(08)00656-0 Get rights and content

Abstract:

Predictive processes are widespread in the motor and sensory areas of the primate brain. They enable rapid computations despite processing delays and assist in resolving noisy, ambiguous input. Here we propose that the frontal eye field, a cortical area devoted to sensorimotor aspects of eye movement control, implements a prediction map of the postsaccadic visual scene for the purpose of constructing a stable percept despite saccadic eye movements.

Introduction

The brain is an inferential machine. Both its motor areas and sensory networks engage in predictive computations (Miall and Wolpert, 1996; Bullier, 2001; Friston, 2005). Hierarchical models suggest that visual cortical neurons fire predominately to signal deviations from predicted inputs (Rao and Ballard, 1999; Lee and Mumford, 2003). Studies in the motor realm, too, suggest that predictions are used for adaptive control (Miall and Wolpert, 1996; Hwang and Shadmehr, 2005). Here we propose that predictive operations for both the sensory and motor domains find unification in the primate visuosaccadic system for the purpose of constructing a stable transaccadic percept.

The primate visuosaccadic system faces a serious problem (Fig. 1). It must generate a stable percept despite the frequent disruptions in gaze induced by saccadic eye movements. To appreciate this, consider the saccade as an event comprised of three constituent parts: presaccadically, a target is selected; intrasaccadically, the world as sensed actually shifts; and postsaccadically, the target is foveated. The intrasaccadic component is the problem. Were the animal to perceive the world exactly as sensed during the intrasaccadic epoch, the visual scene would seem to leap from place-to-place dozens of times per minute. Yet it is not perceived as such, and the brain can even distinguish what aspects of the jumpy visual inflow are artefactual (due to saccades) as opposed to real (due to changes in the world). How does it do it?

An important cortical gaze control component is the frontal eye field (FEF) (Fig. 2; Sommer and Wurtz, 2008). Located within the anterior bank of the arcuate sulcus, the FEF contains topographically arranged neurons with response properties that span the continuum from purely visual to purely movement. The FEF receives input from many cortical and subcortical areas including the superior colliculus (SC) via thalamic relay neurons. We recently demonstrated that convergent inputs from both SCs provide each FEF with a full-field representation of all saccades and all of visual space (Crapse and Sommer, 2007).

Like several other primate areas, the FEF contains neurons that shift their response fields (RFs) before eye movements (a “shifting RF”) (Umeno and Goldberg, 1997). While a typical RF is firmly retinotopic and samples a new part of the visual field (the new RF) only after the eye moves, a shifting RF is dynamic and starts sampling the new RF location even before a saccade. Such neurons depend on corollary discharge (CD) from the midbrain to trigger the shift and are thought to contribute to a percept of visual stability (Sommer and Wurtz, 2006, Sommer and Wurtz, 2008). How might these neurons influence the rest of the brain?

Section snippets

A prediction map in primate frontal eye field

We propose that shifting neurons of the FEF are components of a larger FEF inferential architecture that engages in predictive coding. That is, the FEF is a prediction map. This scheme assigns a causal role to the FEF in the construction of a stable visual percept despite saccadic interruptions. According to this conception, predictions of the future scene (postsaccadic) are generated based on extrapolations from the current scene (presaccadic). Neurons with shifting RFs, informed

Computational basis of the prediction map

The prediction map may be grounded in an inferential process based on empirical Bayesian principles. Neurons throughout the brain seem to engage in probabilistic and inferential computations (Gold and Shadlen, 2007). The probabilistic aspect seems necessary because of the noise and ambiguity intrinsic to neural computation (Knill and Pouget, 2004). Bayesian inference would allow prior and conditional probability distributions to be utilized to generate a posterior distribution, i.e., the

Physiological mechanism of the prediction map

Mechanistically, frontal modulatory control of the posterior lobes could be implemented through imposed patterns of synchronization mediated by cortico-cortical connections (Womelsdorf et al., 2007). Visually evoked activity of single neurons is surprisingly quite deterministic (Arieli et al., 1996). The oft-encountered variability in single neuron responses seems to emerge from the dynamics of ongoing network activity. This fact could be exploited by the prediction map for purposes of ensuring

The sequence of events

A detailed account of what we propose occurs during each voluntary saccade is as follows (Fig. 4). The imminent saccade would initiate an iteration of recurrent processing between the FEF and an assortment of visual cortical areas, beginning with receipt in the FEF of CD (leftmost line) from the SC. This information represents the when and where (a vector quantity) of the imminent change in gaze and induces a transient alteration in local functional topology of the FEF network. At the centre of

Site of prediction error calculation

The prediction errors could be calculated at any number of visual cortical depots. Virtually every portion of the cortical mantle exhibits some degree of saccadic modulation (Baker et al., 2006). Dorsal stream components seem most likely for two reasons. First, the temporal structure of information flow through the primate visual system points to a dorsal stream speed advantage over the ventral stream (Bullier, 2001; Bar, 2007). Dorsal stream components exhibit activation latencies that often

Relation to previous FEF studies

Previous studies have uncovered a number of FEF response properties consistent with the notion of a prediction map. One implication of prediction error in general is that single neurons should not respond if the stimulus falling in its RF is predicted. Burman and Segraves (1994) found that when monkeys rescanned a previously scanned image, visual activity was virtually unaffected by the contents of the image that fell within the RF. In contrast, these same neurons fired vigorously during the

Other frontal lobe functions

The prediction map is consistent with a number of other phenomena involving the frontal lobes and visual function. Among other things, the frontal lobes are thought to play a role in resolving visual ambiguity. Visual scenes are often ambiguous; the sensory data are consistent with multiple interpretations. This often results in illusions and multistable percepts, i.e., depth reversals, binocular rivalry, ambiguous figures, etc. A host of studies point to a role of the frontal lobes, FEF

References (30)

M. Bar
The proactive brain: using analogies and associations to generate predictions
Trends Cogn. Sci.
(2007)
J. Bullier
Integrated model of visual processing
Brain Res.
(2001)
J. Bullier et al.
The role of feedback connections in shaping the responses of visual cortical neurons
Prog. Brain Res.
(2001)
S.M. Kosslyn et al.
Topographical representations of mental images in primary visual cortex
Nature
(1995)
R.C. Miall et al.
Forward models for physiological motor control
Neural Netw.
(1996)
E.K. Miller et al.
An integrative theory of prefrontal cortex function
Annu. Rev. Neurosci.
(2001)
A. Arieli et al.
Dynamics of ongoing activity: explanation of the large variability in evoked cortical responses
Science
(1996)
J.T. Baker et al.
Distribution of activity across the monkey cerebral cortical surface, thalamus and midbrain during rapid, visually guided saccades
Cereb. Cortex
(2006)
C.J. Bruce et al.
Primate frontal eye fields. I. Single neurons discharging before saccades
J. Neurophysiol.
(1985)
D.D. Burman et al.
Primate frontal eye field activity during natural scanning eye movements
J. Neurophysiol.
(1994)

Crapse, T.B. and Sommer, M.A. (2007) Frontal eye field neurons receiving input from both superior colliculi: are their...

K. Friston

A theory of cortical responses

Philos. Trans. R. Soc. Lond.

(2005)

R. Goebel et al.

The constructive nature of vision: direct evidence from functional magnetic resonance imaging studies of apparent motion and motion imagery

Eur. J. Neurosci.

(1998)

J.I. Gold et al.

The neural basis of decision making

Annu. Rev. Neurosci.

(2007)

F.H. Hamker

The reentry hypothesis: the putative interaction of the frontal eye field, ventrolateral prefrontal cortex, and areas V4, IT for attention and eye movement

Cereb. Cortex

(2005)

Cited by (28)

Corollary Discharge for Action and Cognition
2019, Biological Psychiatry: Cognitive Neuroscience and Neuroimaging
Citation Excerpt :
Neurons that remap sample, before each saccade, the part of visual space that they will “see” after the saccade (12). Presaccadic remapping in the FEF therefore provides a signal in visual coordinates that is appropriate for predicting the consequence of each saccade (11). Analysis of microcircuitry in the FEF suggests that it could generate the remapping signal locally (41), and projections from the FEF could relay the signals back to extrastriate cortex (42,43) for prediction of incoming visual input.
In motor systems, a copy of the movement command known as corollary discharge is broadcast to other regions of the brain to warn them of the impending movement. The premise of this review is that the concept of corollary discharge may generalize in revealing ways to the brain’s cognitive systems. An oculomotor pathway from the brain stem to frontal cortex provides a well-established example of how corollary discharge is instantiated for sensorimotor processing. Building on causal evidence from inactivation of the pathway, we motivate forward models as a tool for understanding the contributions of corollary discharge to perception and movement. Finally, we extend the definition of corollary discharge to account for signals that may be used for cognitive forward models of decision making. This framework may provide new insights into signals and circuits that contribute to sequential decision processes, the breakdown of which may account for some symptoms of psychiatric disorders.
Spatiotopic updating across saccades revealed by spatially-specific fMRI adaptation
2017, NeuroImage
Citation Excerpt :
We can divide the regions involved into two functional clusters. The first set of areas, the frontal eye fields (FEF) and posterior parietal cortex (PPC), have previously been implicated in the representation of spatial saliency maps that keep track of important items and update these representations across saccades (Duhamel et al., 1992; Gottlieb, 2007; Crapse and Sommer, 2008; Melcher and Colby, 2008). Specifically, it has been shown that these areas are involved in attention to salient items and that neurons in these regions change their receptive fields around the time of saccades.
Brain representations of visual space are predominantly eye-centred (retinotopic) yet our experience of the world is largely world-centred (spatiotopic). A long-standing question is how the brain creates continuity between these reference frames across successive eye movements (saccades). Here we use functional magnetic resonance imaging (fMRI) to address whether spatially specific repetition suppression (RS) is evident during trans-saccadic perception. We presented two successive Gabor patches (S1 and S2) in either the upper or lower visual field, left or right of fixation. Spatial congruency was manipulated by having S1 and S2 occur in the same or different upper/lower visual field. On half the trials, a saccade was cued between S1 and S2, placing spatiotopic and retinotopic reference frames in opposition. Equivalent RS was observed in the posterior parietal cortex and frontal eye fields when S1-S2 were spatiotopically congruent, irrespective of whether retinotopic and spatiotopic coordinates were in accord or were placed in opposition by a saccade. Additionally the post-saccadic response to S2 demonstrated spatially-specific RS in retinotopic visual regions, with stronger RS in extrastriate than striate cortex. Collectively, these results are consistent with a robust trans-saccadic spatial updating mechanism for object position that directly influences even the earliest levels of visual processing.
The coding and updating of visuospatial memory for goal-directed reaching and pointing
2011, Vision Research
In this review we discuss evidence from psychophysical, electrophysiological, and neuroimaging studies that demonstrates the updating of remembered visual space in a reference frame that is centred on the eye. We then extend these findings by discussing recent work from our lab. Specifically, we address eye-centred updating of visuospatial memory for arm movements following different types of eye movements, the role of retinal versus extraretinal information in such spatial updating, and the use of allocentric versus egocentric information in coding multiple targets. We provide a conceptual model to explain the relationships among these findings.
Electrophysiological correlates of inter- and intrahemispheric saccade-related updating of visual space
2011, Behavioural Brain Research
Citation Excerpt :
Efference copies of motor commands are believed to underlie the cancellation of self-induced stimulations in perception [1–3]. According to forward models, information about the metrics of a saccade is used to predict its sensory consequences [4,5]. A mismatch between prediction and actual visual reafference is attributed to external motion.
The process of spatial updating is crucial for maintaining perceptual stability despite gross and frequent displacements of space following saccadic eye movements. Efference copies of motor commands are used to update retinal coordinates across saccades. The present study investigated neural correlates of saccadic updating in a perceptual context with regard to temporal dynamics and modulation by intra- versus interhemispheric transfer of updating-related information. Twenty-two subjects engaged in a perceptual localization task which required trans-saccadic spatial updating while event-related potentials (ERPs) were recorded. In accordance with previous studies, post-saccadic perceptual localization of stimuli presented before a saccade was less accurate when relying on efference copy signals (i.e. updating was required) as compared to a control condition not involving updating. Updating-related ERP components emerged before and after saccade onset. There was no clear transfer-dependent modulation of the presaccadic component. A negative deflection between 30 and 70 ms after saccade onset was most pronounced for rightward saccades, and when intrahemispheric transfer was required. A slower positive deflection starting about 170–230 ms after saccade onset had a shorter latency for leftward than for rightward saccades and was not modulated by transfer. In accordance with previous work, this relative positivity is thought to reflect sensory memory, whereas the earlier negative deflection can be more directly linked to the updating process itself.
The anatomy and physiology of the ocular motor system
2011, Handbook of Clinical Neurology
Citation Excerpt :
In humans, functional imaging indicates that the portion of FEF concerned with smooth pursuit lies in the lower anterior wall and adjacent fundus of the precentral sulcus (Petit and Haxby, 1999; Rosano et al., 2002). Lesions of the FEF in both monkeys and humans cause a predominantly ipsidirectional defect of smooth pursuit that mainly involves predictive aspects of the tracking response (MacAvoy et al., 1991; Morrow and Sharpe, 1995; Heide et al., 1996; Crapse and Sommer, 2008). During head rotation the smooth-pursuit system must interact with the vestibular system.
Accurate diagnosis of abnormal eye movements depends upon knowledge of the purpose, properties, and neural substrate of distinct functional classes of eye movement. Here, we summarize current concepts of the anatomy of eye movement control. Our approach is bottom-up, starting with the extraocular muscles and their innervation by the cranial nerves. Second, we summarize the neural circuits in the pons underlying horizontal gaze control, and the midbrain connections that coordinate vertical and torsional movements. Third, the role of the cerebellum in governing and optimizing eye movements is presented. Fourth, each area of cerebral cortex contributing to eye movements is discussed. Last, descending projections from cerebral cortex, including basal ganglionic circuits that govern different components of gaze, and the superior colliculus, are summarized. At each stage of this review, the anatomical scheme is used to predict the effects of lesions on the control of eye movements, providing clinical–anatomical correlation.
Corollary discharge circuits in the primate brain
2008, Current Opinion in Neurobiology
Citation Excerpt :
Psychophysical and physiological evidence provides support that such computations are indeed performed by the brain under a variety of contexts [47–49]. We have proposed elsewhere that predictive operations, grounded in such probabilistic inference, are implemented in the primate visuosaccadic system for the purpose of constructing a stable transaccadic percept [50]. At the center of the model are CD and shifting RFs of the FEF, and we propose they together constitute an inferential architecture that engages in predictive coding.
Movements are necessary to engage the world, but every movement results in sensorimotor ambiguity. Self-movements cause changes to sensory inflow as well as changes in the positions of objects relative to motor effectors (eyes and limbs). Hence the brain needs to monitor self-movements, and one way this is accomplished is by routing copies of movement commands to appropriate structures. These signals, known as corollary discharge (CD), enable compensation for sensory consequences of movement and preemptive updating of spatial representations. Such operations occur with a speed and accuracy that implies a reliance on prediction. Here we review recent CD studies and find that they arrive at a shared conclusion: CD contributes to prediction for the sake of sensorimotor harmony.

View all citing articles on Scopus

View full text

Chapter 5.2 - The frontal eye field as a prediction map

Abstract:

Introduction

Section snippets

A prediction map in primate frontal eye field

Computational basis of the prediction map

Physiological mechanism of the prediction map

The sequence of events

Site of prediction error calculation

Relation to previous FEF studies

Other frontal lobe functions

Trends Cogn. Sci.

Brain Res.

Prog. Brain Res.

Nature

Neural Netw.

Annu. Rev. Neurosci.

Dynamics of ongoing activity: explanation of the large variability in evoked cortical responses

Science

Distribution of activity across the monkey cerebral cortical surface, thalamus and midbrain during rapid, visually guided saccades

Cereb. Cortex

Primate frontal eye fields. I. Single neurons discharging before saccades

J. Neurophysiol.

Primate frontal eye field activity during natural scanning eye movements

J. Neurophysiol.

A theory of cortical responses

Philos. Trans. R. Soc. Lond.

The constructive nature of vision: direct evidence from functional magnetic resonance imaging studies of apparent motion and motion imagery

Eur. J. Neurosci.

The neural basis of decision making

Annu. Rev. Neurosci.

The reentry hypothesis: the putative interaction of the frontal eye field, ventrolateral prefrontal cortex, and areas V4, IT for attention and eye movement

Cereb. Cortex