The algorithms and neural circuits that process spatio-temporal changes in luminance to extract visual motion cues have been the focus of intense research. An influential model, the Hassenstein-Reichardt correlator, relies on differential temporal filtering of two spatially separated input channels, delaying one input signal with respect to the other. Motion in a particular direction causes these delayed and non-delayed luminance signals to arrive simultaneously at a subsequent processing step in the brain; these signals are then nonlinearly amplified to produce a direction-selective response. Recent work in Drosophila has identified two parallel pathways that selectively respond to either moving light or dark edges. Each of these pathways requires two critical processing steps to be applied to incoming signals: differential delay between the spatial input channels, and distinct processing of brightness increment and decrement signals. Here we demonstrate, using in vivo patch-clamp recordings, that four medulla neurons implement these two processing steps. The neurons Mi1 and Tm3 respond selectively to brightness increments, with the response of Mi1 delayed relative to Tm3. Conversely, Tm1 and Tm2 respond selectively to brightness decrements, with the response of Tm1 delayed compared with Tm2. Remarkably, constraining Hassenstein-Reichardt correlator models using these measurements produces outputs consistent with previously measured properties of motion detectors, including temporal frequency tuning and specificity for light versus dark edges. We propose that Mi1 and Tm3 perform critical processing of the delayed and non-delayed input channels of the correlator responsible for the detection of light edges, while Tm1 and Tm2 play analogous roles in the detection of moving dark edges. Our data show that specific medulla neurons possess response properties that allow them to implement the algorithmic steps that precede the correlative operation in the Hassenstein-Reichardt correlator, revealing elements of the long-sought neural substrates of motion detection in the fly.