The Neural Dynamics of Facial Identity Processing: Insights from EEG-Based Pattern Analysis and Image Reconstruction

The time course of EEG-based classification accuracy for across- and within-expression discrimination of facial identity. A, Classification was conducted across consecutive 10-ms window patterns over 12 occipitotemporal electrodes for group-based ERP data. Both types of analysis exhibited above-chance discrimination across extensive temporal intervals (permutation test; FDR correction across time, q < 0.01); shaded areas mark intervals of better-than-chance discrimination for across-expression classification. B, The time course of EEG-based classification accuracy for across- and within-expression discrimination of facial identity for a single representative participant. Classification was conducted across consecutive 10-ms window patterns over 12 occipitotemporal electrodes. Both types of analysis exhibited above-chance discrimination across extensive temporal intervals (permutation test; FDR correction across time, q < 0.01); shaded areas mark intervals of better-than-chance discrimination for across-expression classification.

EEG-based classification accuracy for across- and within-expression discrimination of facial identity with temporally cumulative data (50–650 ms after stimulus onset). Accuracy corresponding to neutral and happy faces are separately shown for (A) group-based ERP data and (B) single-participant data (i.e., pattern classification was conducted individually for each participant and, then, its results averaged across participants). The plots display (A) the results of permutation tests (red solid and dash lines indicate average accuracy and 99% confidence intervals estimated with 10³ permutations) and (B) the distribution of single-participant data (green and purple solid lines indicate medians, boxes represent 1st and 3rd quartiles, whiskers represent minimum and maximum accuracy values, points represent individual participants’ values and red solid lines indicate chance-level discrimination).

Correlation of EEG- and behavioral-based estimates of pairwise face similarity. EEG-based estimates are derived from across-expression discrimination of facial identity for consecutive 10-ms windows of group-based data. Multiple intervals, marked by shaded areas, exhibit significant levels of correlation (permutation test; FDR correction across time, q < 0.01).

Neutral and happy face space estimates along with their fit (after Procrustes alignment). Estimates were derived through MDS analysis of similarity matrices based on within-expression face discrimination of group-based temporally cumulative data. The two face space estimates exhibit a similar topography as found with (A) their visualization across multiple dimensions (red and green circles indicate neutral and happy faces, respectively; solid lines connect face images with the same identity with the thickness of the line proportionally reflecting shorter distances; the first four dimensions shown here account for 40% and 41% variance for neutral and happy face space); (B) badness of fit (SSEs) for the two spaces compared to their permutation-based counterpart (average fits and 95% confidence intervals estimated with 10³ permutations).

Examples of CIMs extracted from EEG-based face space constructs for (A) neutral and (B) happy faces. Pairs of images show raw CIMs (odd columns) and their analysis (even columns) with a pixelwise permutation-based test (FDR-corrected across pixels; q < 0.05). Bright/dark, red/green, and yellow/blue regions in analyzed CIs mark areas of the face brighter (L*), redder (a*), or more yellow (b*) than chance in CIEL*a*b*. Results are shown separately for the first and fourth dimensions of face spaces derived from group-based temporally cumulative data.

Reconstruction results for neutral and happy face images across consecutive 10-ms windows of group-based data. A, Examples of face stimuli along with their corresponding reconstructions at two different times (numbers in the upper left corner indicate image-based estimates of reconstruction accuracy). B, Time course of reconstruction accuracy. Both neutral and happy face images exhibit above-chance discrimination across multiple temporal intervals (permutation test; FDR correction across time, q < 0.05; shaded areas mark intervals of better-than-chance discrimination for neutral faces). Reconstruction accuracy is maximized in the vicinity of the N170 component (Fig. 1) and of the discrimination peak found with pattern classification (Fig. 2).