Improved manual annotation of EEG signals through convolutional neural network guidance

Abstract

The development of validated algorithms for automated handling of artifacts is essential for reliable and fast processing of EEG signals. Recently, there have been methodological advances in designing machine-learning algorithms to improve artifact detection of trained professionals who usually meticulously inspect and manually annotate EEG signals. However, validation of these methods is hindered by the lack of a gold standard as data are mostly private and data annotation is time-consuming and error-prone. In the effort to circumvent these issues, we propose an iterative learning model to speed up and reduce errors of manual annotation of EEG. We use a convolutional neural network (CNN) to train on expert-annotated eyes-open and eyes-closed resting-state EEG data from typically developing children (n = 30) and children with neurodevelopmental disorders (n = 141). To overcome the circular reasoning of aiming to develop a new algorithm and benchmarking to a manually-annotated gold standard, we instead aim to improve the gold standard by revising the portion of the data that was incorrectly learned by the network. When blindly presented with the selected signals for re-assessment (23% of the data), the two independent expert-annotators changed the annotation in 25% of the cases. Subsequently, the network was trained on the expert-revised gold standard, which resulted in improved separation between artifacts and non-artifacts as well as an increase in balanced accuracy from 74% to 80% and precision from 59% to 76%. These results show that CNNs are promising to enhance manual annotation of EEG artifacts and can be improved further with better gold-standard data.

Significance statement

Manual annotation of artifacts in EEGs remains the gold standard in research and clinic but is time-consuming and prone to human oversight. Here, we introduce a convolutional neural network to increase the speed and accuracy of manual annotation of EEG artifacts. We highlight the possibility of using active learning to iteratively improve both the model and the gold standard. With our method, it is possible to vary the decision probability threshold and control the portion of the data that can be labelled automatically by the model or that would require expert judgement. We expect that our new approach will speed up EEG processing and facilitate reliable data analysis in neurodevelopmental disorders.