On sense and reference: Examining the functional neuroanatomy of referential processing
Introduction
At least since Frege (1892), a distinction has been made between a word's general, dictionary-like meaning (sense) and the particular entity it denotes (reference). This distinction becomes clear when one compares, for example, the meaning carried by the word ‘doctor’ in (1) Hannibal Lecter is a doctor, and in (2) Hannibal Lecter is eating a doctor. The lexical item ‘doctor’ has an identical sense in 1 and 2 (i.e., someone who has completed medical school), but it provides information about Hannibal Lecter in 1, while it refers to another, rather unfortunate doctor in 2. Thus, to appreciate the meaning of a word in a given context, one must establish both the general, lexical meaning and the specific, referential meaning it conveys.
In psycholinguistics, sense and reference are often associated with semantic and referential analysis, respectively. Through semantic analysis, we retrieve the meaning of individual words from long-term memory and combine them into larger units of conceptual structure. Through referential analysis, we determine who's who and what's what by establishing the relationships between words in the discourse and entities in our mental representation of the world (e.g., persons, events, concepts, places), real or fictional. Alongside the phonological and syntactic aspects of language comprehension, semantic and referential analysis has been subjected to experimental investigation throughout the past decades (e.g., see Garnham, 2001, Carreiras and Clifton, 2004). Convergent evidence from behavioral measures and event-related potentials (ERPs) has suggested that listeners and readers immediately relate the semantic and referential meaning of incoming words to the global context (e.g., Kutas and Hillyard, 1980, Marslen-Wilson and Tyler, 1980, Nieuwland and Van Berkum, 2006b, Tanenhaus and Trueswell, 1995, Van Berkum et al., 1999).
Importantly, results from ERP experiments show that the brain responds in different ways to problems with reference than to problems with either syntax or semantics. While semantic and syntactic processing are predominantly associated with the N400 and P600/LAN respectively (e.g., Friederici, 2002, Hagoort et al., 1999), referentially ambiguous nouns (e.g., “the girl” in a two-girl context) elicit a frontally dominant and sustained negative shift (Van Berkum et al., 1999, Van Berkum et al., 2003). This referentially induced ERP effect stems from having two suitable referential candidates for a single anaphor (Nieuwland et al., 2007; see Van Berkum et al., 2007, for review), and enables us to selectively track referential processing at the level that is most relevant for comprehension, the mental representation of the situation as described in the discourse (the situation model, Zwaan and Radvansky, 1998).
To examine the causal antecedents of this referentially induced frontal negativity, we have recently investigated how people establish reference for pronouns like “he” or “she”. Pronouns have a rather shallow sense of their own and constitute our linguistic ‘pointers’ for maintaining reference to topics that are in the focus of our conversation (e.g., Gordon et al., 1993, Sanford and Garrod, 1989). Because pronouns are among the most frequently used words in many languages, pronoun resolution is often regarded as the textbook example of referential processing. Consistent with our findings for nouns, referentially ambiguous pronouns (e.g., “Ronald told Frank that he…”) also elicit a frontal negative shift compared to referentially coherent pronouns (Nieuwland and Van Berkum, 2006a, Van Berkum et al., 2004). Furthermore, referentially failing pronouns (when no suitable referent is directly available, e.g., “Rose told Emily that he…”) elicit a P600 effect (Nieuwland and Van Berkum, 2006a, Van Berkum et al., 2004), suggesting that readers initially try to find an appropriate antecedent within the given set of discourse entities (e.g., Garnham, 2001) and therefore initially ascribe the referential failure to a problem with the syntactic gender of the pronoun (e.g., Osterhout and Mobley, 1995; see Van Berkum et al., 2007, for discussion).
Taken together, the ERP findings suggest that the brain responds in qualitatively different ways to distinct problems with referential analysis (failure/ambiguity), and that both responses, in turn, differ from the brain's response to semantically unexpected words (semantic anomaly). However, these findings do not tell the whole story. In other words, although the different ERP effects are indicative of the engagement of at least partially non-overlapping neural systems, they do not tell us which exact cortical networks are involved in the establishing of reference, or what their relationship is to the networks known to be involved in the processing of meaning. In the last decade, language comprehension researchers have begun to use functional neuroimaging techniques (e.g., fMRI) to unravel the neuronal infrastructure of semantic, syntactic and phonological processing (e.g., Bookheimer, 2002, Friederici, 2002, Kaan and Swaab, 2002, Stowe et al., 2005, for reviews). However, with the exception of a few studies focusing on cohesion/coherence processing in discourse comprehension (Ferstl and von Cramon, 2001, Robertson et al., 2000), there has been no functional magnetic resonance imaging (fMRI) work on the neural substrate for referential processing. As a result, we know virtually nothing about how the brain subserves a core aspect of language comprehension, how people work out who's who and what's what in text or conversation.
In the present study, we used fMRI to examine the functional neuroanatomical correlates of referential processing, and to relate the cortical systems involved in establishing reference to the cortical systems recruited by semantic processing. Following up on earlier ERP work, we focused on the processing of referentially ambiguous and referentially failing pronouns. Note that although referential ambiguity and referential failure both seem to involve disruptions of ‘normal’ language comprehension, such disruptions are in fact not uncommon in everyday language use (e.g., Auer, 1984), and understanding how readers deal with them is thus of a priori relevance to the study of reference resolution (e.g., Gordon et al., 1993, Sanford and Garrod, 1989). Furthermore, we know from other language comprehension research that the systems recruited by semantic and syntactic problems are also implicated in the processing of non-problematic language input. For these two reasons, we take our study to provide insights into the cortical networks involved in natural referential processing.
In our experiment, participants read sentences that were presented on a screen one word at a time, and that contained two narrative characters and one singular pronoun (cf., Table 1; see also Appendix A). This pronoun was referentially ambiguous, failing, or coherent depending on whether its gender matched both, none, or one of the characters mentioned in the sentence, respectively. In addition, referentially coherent sentences were either semantically anomalous if it contained a lexical-semantically anomalous word (downstream from the pronoun), or semantically coherent if it contained a non-anomalous control word.
The available literature suggests at least two possible scenarios that link ambiguous pronoun resolution to activity in the left inferior frontal gyrus (LIFG). First, referential ambiguity may require the language system to, from the ambiguous pronoun and onwards, actively maintain two candidate fillers for an unresolved single referential slot in working memory (e.g., Gibson, 1998). In such a scenario, the brain signature of referential ambiguity may resemble that of increased working memory load, eliciting increased activity in the LIFG (BA 44/45/47; Fletcher and Henson, 2001, Kaan and Swaab, 2002, Smith and Jonides, 1999). Second, in a recent review, Novick et al. (2005) posit a similar prediction for LIFG involvement in resolving referential ambiguity, albeit for different reasons. According to Novick et al. (2005), LIFG activations may signal resolution of the representational conflict that arises when semantic representations point towards competing interpretations, an account that makes intuitive sense with regard to referential ambiguity.
Thus, the abovementioned accounts all predict that referential ambiguity (compared to referential coherence) would lead to enhanced activation in the LIFG. This said, both accounts also predict substantial overlap for cortical networks involved in dealing with referential ambiguity and semantic anomaly, as semantic anomalies are known to elicit increased responses in the left anterior inferior frontal gyrus (LIFG, BA 45/47; e.g., Baumgaertner et al., 2002, Hagoort et al., 2004, Kiehl et al., 2002, Kuperberg et al., 2003).
In a third account, the processing of referential ambiguity is not necessarily linked to the LIFG at all. This account entails that ambiguous pronoun resolution requires readers to engage in problem-solving and inference procedures (e.g., Graesser et al., 1994, Greene et al., 1992). For example, readers may launch an additional search through their discourse memory (e.g., Myers and O'Brien, 1998) and infer that one of the referents is the correct one (e.g., through reasoning or decision-making). In this respect, ambiguous pronoun resolution can be considered to be a multi-faceted enterprise that relies on interrelated, domain-independent executive cognitive processes, recruiting a network of prefrontal and parietal regions (e.g., Cabeza and Nyberg, 2000, Fletcher and Henson, 2001, Wagner et al., 2005). Following such an account, increased responses may be observed in brain regions that subserve knowledge-based, non-automatic inferences that require the use of general world knowledge and/or discourse information (e.g., Ferstl, in press, Ferstl and Siebörger, 2007, for reviews). These regions include medial frontal (BA 8/9/10) and parietal (BA 7/23/31) cortices (e.g., Ferstl and von Cramon, 2001, Ferstl and von Cramon, 2002, Ferstl et al., 2005, Kuperberg et al., 2006, Maguire et al., 1999, Volz et al., 2006a, Volz et al., 2006b, Zysset et al., 2002, Zysset et al., 2003), but also right lateral prefrontal (BA 8/9/10/46) and bilateral parietal regions (BA 39/40) involved in monitoring and evaluating of memory retrieval outcomes (e.g., Buckner and Wheeler, 2001, Wagner et al., 2005).1
For referential failure (compared to referential coherence), we predicted that the brain treats referentially failing pronouns as morpho-syntactic violations in the first instance (e.g., Osterhout and Mobley, 1995, Van Berkum et al., 2007), eliciting activity increases in left middle/superior and inferior frontal (BA 6/8/44, Hammer et al., 2007, Newman et al., 2001, Ni et al., 2000), and medial and bilateral parietal regions (BA 7/23/31/40; Kuperberg et al., 2003, Ni et al., 2000). However, because the pronouns as used here are ‘free’ in a Chomskian sense (Chomsky, 1981), there are actually alternative resolutions of the referential failure at hand. For example, readers may stick with ‘blaming’ the syntactic gender of the pronoun, but they may also invoke a third, unmentioned person (e.g., ‘he’ in “Rose told Emily that he had a positive attitude towards life” can very well be taken to refer to Rose‘s new boyfriend). Because these two different interpretations differ in whether or not the referents were only looked for ‘within the sentence’, we will refer to them as sentence-internal and sentence-external referential interpretations. In an ERP study examining referential failure, Osterhout and Mobley (1995) reported that these different interpretations may indeed elicit qualitatively different brain responses. In one of the analyses in the present study, we used the results from a post-experiment debriefing to examine whether the reliance on different resolution strategies was also reflected in differential BOLD response patterns.
In summary, the aim of the present study was to investigate the cortical networks involved in different aspects of referential processing, a crucial facet of language comprehension that has hardly been touched upon by cognitive neuroscientists. We contrasted different functional accounts of referential ambiguity that are associated with different functional neuroanatomical predictions (particularly with respect to involvement of the LIFG, a brain area typically associated with semantic processing). For referential failure, we predicted, based on earlier ERP findings, that failing pronouns would elicit increased activity in brain regions involved in morpho-syntactic processing. Furthermore, we used results from a post-experiment briefing to examine potential individual differences in reference resolution strategies. Finally, the inclusion of a semantic anomaly manipulation in our design allowed us to address whether referential and semantic aspects of language comprehension draw upon common or different brain regions.
Section snippets
Participants
Twenty-two right-handed college students (4 males, mean age 21, S.D. = 2.3) participated in this study for course credits or a small monetary reimbursement. All participants were native speakers of Dutch. None of them used medication or had a history of drug abuse, head trauma, neurological or psychiatric illness. The experiment was approved by the medical ethical committee of the Amsterdam Medical Center, and informed consent was obtained from all subjects.
Stimulus materials
180 sentences (a subset of 270
Results
We first investigated the contrasts involving referential ambiguity, referential failure and semantic anomaly compared to their coherent counterparts. The corresponding statistical results are presented in Table 2. As visible from Fig. 1A, referential ambiguity (ambiguity > coherence; Fig. 1A, Table 2A) was associated with greater activation in a set of regions that included medial frontal gyrus (BA 10), right superior frontal cortex (BA 8/9), medial parietal cortices (precuneus, BA 7/31), and
Discussion
The objective of the present study was to investigate the functional neuroanatomical correlates of referential processing. We compared event-related BOLD-fMRI responses to sentences containing referentially ambiguous (e.g., “Ronald told Frank that he…”), referentially failing pronouns (e.g., “Rose told Emily that he…”) or referentially coherent pronouns (e.g., “Frank told Emily that he…”). To provide a benchmark for these results, we also compared responses to sentences containing semantically
Acknowledgments
We gratefully thank Steven Scholte for his technical advice, Tessa Dekker and Sietske Kleibeuker for help in data collection, and Gina Kuperberg, and Evelyn Ferstl for valuable comments on an earlier draft of this manuscript. This research was supported by an NWO Innovation Impulse Vidi grant to JVB. MSN is currently funded by a Rubicon grant from the Netherlands Organisation for Scientific Research (NWO).
References (79)
Referential problems in conversation
J. Pragmat.
(1984)- et al.
Event-related fMRI reveals cortical sites involved in contextual sentence integration
NeuroImage
(2002) - et al.
Neural systems for word meaning modulated by semantic ambiguity
NeuroImage
(2004) - et al.
Common regions of the human frontal lobe recruited by diverse cognitive demands
Trends Neurosci.
(2000) - et al.
The role of coherence and cohesion in text comprehension: an event-related fMRI study
Cogn. Brain Res.
(2001) - et al.
What does the fronto-medial cortex contribute to language processing: coherence or theory of mind?
NeuroImage
(2002) Towards a neural basis of auditory sentence processing
Trends Cogn. Sci.
(2002)- et al.
Event-related fMRI: characterizing differential responses
NeuroImage
(1998) - et al.
Thresholding of statistical maps in functional neuroimaging using the false discovery rate
NeuroImage
(2002) Linguistic complexity: locality of syntactic dependencies
Cognition
(1998)
Explaining modulation of reasoning by belief
Cognition
Pronouns, names, and the centering of attention in discourse
Cogn. Sci.
On broca, brain, and binding: a new framework
Trends Cogn. Sci.
When sex meets syntactic gender on a neural basis during pronoun processing
Brain Res.
The brain circuitry of syntactic comprehension
Trends Cogn. Sci.
Reading anomalous sentences: an event-related fMRI study of semantic processing
NeuroImage
Making sense of discourse: an fMRI study of causal inferenceing across sentences
NeuroImage
The temporal structure of spoken language understanding
Cognition
Individual differences and contextual bias in pronoun resolution: evidence from ERPs
Brain Res.
Event-related brain potentials elicited by failure to agree
J. Mem. Lang.
Instruction-specific brain activations during episodic encoding. A generalized level of processing effect
Neuroimage
Interaction between a verbal working memory network and the medial temporal lobe
NeuroImage
Rethinking the neurological basis of language
Lingua
Oscillatory gamma activity in humans and its role in object representation
Trends Cogn. Sci.
Sentence comprehension
Early referential context effects in sentence processing: evidence from event-related brain potentials
J. Mem. Lang.
Establishing reference in language comprehension: an electrophysiological perspective
Brain Res.
Parietal lobe contributions to episodic memory retrieval
Trends Cogn. Sci.
The anterior frontomedian cortex and evaluative judgment: an fMRI study
NeuroImage
Functional specialization within the anterior medial prefrontal cortex: a functional magnetic resonance imaging study with human subjects
Neurosci. Lett.
Semantic retrieval, mnemonic control, and prefrontal cortex
Behav. Cogn. Neurosci. Rev.
Functional MRI of language: new approaches to understanding the cortical organization of semantic processing
Annu. Rev. Neurosci.
The problem of functional localization in the human brain
Nat. Rev., Neurosci.
The cognitive neuroscience of remembering
Nat. Rev., Neurosci.
The gateway hypothesis of rostral prefrontal cortex (area 10) function
Imaging cognition II: an empirical review of 275 PET and fMRI studies
J. Cogn. Neurosci.
The On-Line Study of Sentence Comprehension: Eyetracking, ERPs and Beyond
Cited by (76)
Neural correlates of pronoun processing: An activation likelihood estimation meta-analysis
2023, Brain and LanguageCortical networks for recognition of speech with simultaneous talkers
2023, Hearing ResearchDecoding the silence: Neural bases of zero pronoun resolution in Chinese
2022, Brain and LanguageCitation Excerpt :In both fMRI and MEG, Li et al. found that the left middle temporal gyrus (LMTG) is consistently activated for third-person pronoun processing in both English and Chinese. Yet they also found additional medial parietal activity from the MEG data, consistent with Brodbeck and Pylkkänen (2017), Brodbeck et al. (2016), and Nieuwland et al. (2007). To sum up, referential processing has been implicated in a number of regions, including the medial parietal lobe.
Who is “oneself” in Chinese? ERP responses to the Chinese simple reflexive ziji in discourse comprehension
2021, Journal of Neurolinguistics