Frequency-following responses (FFRs) are sustained potentials based on phase-locked neural activities elicited by low- to medium-frequency periodical sound waveforms. Human brainstem FFRs, which are able to encode some critical acoustic features of speech, can be unmasked by binaural processing. However, the underlying unmasking mechanisms have not previously been reported. In rats, most neurons in the inferior colliculus (IC) exhibit binaural responses which are affected by axonal projections from both the contralateral dorsal nucleus of the lateral lemniscus (DNLL) and the contralateral IC. The present study investigated whether the contralateral DNLL and the contralateral IC modulate binaural unmasking of FFRs recorded in the rat IC. The results show that IC FFRs to the rat pain call (chatter) were enhanced by local injection of the excitatory glutamate receptor antagonist kynurenic acid (KYNA) into the contralateral DNLL but were reduced by KYNA injection into the contralateral IC. Introducing a disparity between the interaural time difference (ITD) of the FFR-eliciting chatter and the ITD of the masking noise enhanced IC FFRs. Moreover, the ITD-disparity-induced FFR enhancement was weakened by injection of KYNA into either the contralateral DNLL or the contralateral IC when the ipsilateral chatter preceded the contralateral chatter. Thus, binaural hearing can improve IC FFRs against noise masking. More importantly, both inhibitory projections from the contralateral DNLL and excitatory projections from the contralateral IC modulate IC FFRs and play a role in forming binaural unmasking of IC FFRs.