Optimizing and Benchmarking Machine Learning and Traditional Synaptic Event Detection Pipelines in Neurophysiology Experiments

Abstract

Synaptic physiology experiments are fundamental to neuroscience research. Consequently, accurate detection of synaptic currents is crucial for conducting high quality experiments. Traditionally, detecting inhibitory and excitatory postsynaptic currents (sIPSCs / sEPSCs) relied on hand-counting individual events, and while sEPSCs and sIPSCs are clear to the trained eye, hand analysis is time and labor intensive. Recent advances in applied machine learning promise faster, superior event detectors that may improve data quality and reduce or even completely negate the need for hand curation. While many strategies for sIPSC and sEPSC detection exist, rarely have they been quantitatively compared for accuracy within an experiment. Our study aims to establish practical ground truth event detection in a large experimental dataset through meticulous hand counting, and to assess variance in detection results across different laboratories, analysis techniques, and cell-types. Using thoroughly hand-counted data as our ground-truth comparison we will benchmark current popular detection methods, including a modern supervised deep learning approach. Our results suggest that current analysis strategies vary widely in their results, and that a supervised machine learning approach rivals manual event counting performed by expert electrophysiologists better than other automated approaches.

Significance Statement Our study aims to establish a practical ground truth to measure inter-lab variability and to benchmark specific inhibitory and excitatory synaptic event detection techniques, including hand counting and the main automated approaches used in the field of slice electrophysiology.