Differential involvement of EEG oscillatory components in sameness vs. spatial-relation visual reasoning tasks

Abstract

The development of deep convolutional neural networks (CNNs) has recently led to great successes in computer vision and CNNs have become de facto computational models of vision. However, a growing body of work suggests that they exhibit critical limitations beyond image categorization. Here, we study one such fundamental limitation, for judging whether two simultaneously presented items are the same or different (SD) compared to a baseline assessment of their spatial relationship (SR). In both human subjects and artificial neural networks, we test the prediction that SD tasks recruit additional cortical mechanisms which underlie critical aspects of visual cognition that are not explained by current computational models. We thus recorded EEG signals from human participants engaged in the same tasks as the computational models. Importantly, in humans the two tasks were matched in terms of difficulty by an adaptive psychometric procedure: yet, on top of a modulation of evoked potentials, our results revealed higher activity in the low beta (16-24Hz) band in the SD compared to the SR conditions. We surmise that these oscillations reflect the crucial involvement of additional mechanisms, such as working memory and attention, which are missing in current feed-forward CNNs.

Significance statement Convolutional neural networks (CNNs) are currently the best computational models of primate vision. Here, we independently confirm prior results suggesting that CNNs can learn to solve visual reasoning problems involving spatial relations much more easily than problems involving sameness judgments. We hypothesize that these results reflect different computational demands between the two tasks and conducted a human EEG experiment to test this hypothesis. Our results suggest a significant difference – both in evoked potentials and in the oscillatory dynamics– of the EEG signals measured from human participants performing these two tasks. We interpret this difference as the signature for the fundamental involvement of recurrent mechanisms implementing cognitive functions such as working memory and attention.