Abstract
Recent fMRI studies reported transformed representations between perception and visual working memory (VWM) in human early visual cortex (EVC). This is inconsistent with the still widely cited original proposal of the sensory account of VWM, which argues for a shared perception-VWM representation based on successful cross-decoding of the two representations. Although cross-decoding was usually lower than within-VWM decoding and consistent with transformed VWM representations, this has been attributed to experimental differences between perceptual and VWM tasks: once they are equated, the same representation is expected to exist in both. Including human participants of both sexes, this study compared target and distractor representations during the same VWM delay period for the same objects, thereby equating experimental differences. Even with strong VWM representations present throughout occipitotemporal cortex (OTC, including EVC) and posterior parietal cortex (PPC), fMRI cross-decoding revealed significant representational differences between distractors (perception) and targets (VWM) in both regions. Similar differences existed between target encoding (perception) and delay (VWM), being greater in OTC than PPC, indicating more invariant target representations in PPC than OTC. As only part of the sensory input is usually task-relevant, sustaining sensory input in VWM without selection/refinement/consolidation is both taxing and unnecessary. Transformed representations, mediated by task goals and associative areas coding task-relevant information (e.g., PPC), can easily account for these and other recent findings. A task-driven transformed account of VWM thus better captures the nature of VWM representation in the human brain (including EVC) than the sensory representations originally proposed by the sensory account of VWM.
Significance Statement The original proposal of the sensory account of visual working memory (VWM) argues for a shared representation between perception and VWM in sensory areas. This assumption, however, was not thoroughly tested due to differences in experimental settings in prior studies. Using fMRI cross-decoding and closely matched experimental conditions, this study compared object representations when they were VWM targets and distractors and during the encoding and delay period of VWM. Both comparisons revealed significant representational differences between perception and VWM in human sensory areas. These results are inconsistent with the sensory nature of VWM representations as it is originally proposed. Instead, they support a task-driven transformed account of VWM in which sensory input is selected/refined/consolidated before VWM storage in these areas.
Footnotes
Research reported in this publication was supported by the National Eye Institute of the National Institutes of Health under Award Number R01EY030854 to YX. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health. I thank SuKeun Jeong for creating the visual stimuli, Judy Young Hye Kwon and Hillary Nguyen for their assistance in fMRI data collection, and Marvin Chun for general support.
The author declares no competing interests.
This is an open-access article distributed under the terms of the Creative Commons Attribution 4.0 International license, which permits unrestricted use, distribution and reproduction in any medium provided that the original work is properly attributed.
