Neural Pattern Classification Tracks Transfer-Appropriate Processing in Episodic Memory

Abstract

The transfer-appropriate processing (TAP) account holds that episodic memory depends on the overlap between encoding and retrieval processing. In the current study, we employed multivariate pattern analysis (MVPA) of electroencephalography to examine the relevance of spontaneously engaged visual processing during encoding for later retrieval. Human participants encoded word- picture associations, where the picture could be a famous face, a landmark, or an object. At test, we manipulated the retrieval demands by asking participants to retrieve either visual or verbal information about the pictures. MVPA revealed classification between picture categories during early perceptual stages of encoding (∼170 ms). Importantly, these visual category-specific neural patterns were predictive of later episodic remembering, but the direction of the relationship was contingent on the particular retrieval demand of the memory task: a benefit for the visual and a cost for the verbal. A reinstatement of the category-specific neural patterns established during encoding was observed during retrieval, and again the relationship with behavior varied with retrieval demands. Reactivation of visual representations during retrieval was associated with better memory in the visual task, but with lower performance in the verbal task. Our findings support and extend the TAP account by demonstrating that processing of particular aspects during memory formation can also have detrimental effects on later episodic remembering when other aspects of the event are called-for and shed new light on encoding and retrieval interactions in episodic memory.

Significance Statement Episodic memory allows us to mentally travel in time to relive our personal past. The likelihood that a previous episode will be remembered is considered to depend on the extent to which processes engaged during encoding are also engaged at the time of retrieval. In the present study, we leveraged multivariate pattern analysis of high time-resolution oscillatory brain activity to assess participants’ category-specific visual processing during memory formation, and later cortical reinstatement at retrieval. The results demonstrate that category-specific visual processing may predict both benefits and costs in episodic memory, depending on the retrieval requirements. These novel findings elucidate the neurocognitive dynamics of encoding and retrieval, which mediate the access to our personal past.