Purpose In this study, we investigate temporal adaptation capacities of children with normal hearing and children with cochlear implants and/or hearing aids during verbal exchange. We also address the question of the efficiency of a rhythmic training on temporal adaptation during speech interaction in children with hearing loss. Method We recorded electroencephalogram data in children while they named pictures delivered on a screen, in alternation with a virtual partner. We manipulated the virtual partner's speech rate (fast vs. slow) and the regularity of alternation (regular vs. irregular). The group of children with normal hearing was tested once, and the group of children with hearing loss was tested twice: once after 30 min of auditory training and once after 30 min of rhythmic training. Results Both groups of children adjusted their speech rate to that of the virtual partner and were sensitive to the regularity of alternation with a less accurate performance following irregular turns. Moreover, irregular turns elicited a negative event-related potential in both groups, showing a detection of temporal deviancy. Notably, the amplitude of this negative component positively correlated with accuracy in the alternation task. In children with hearing loss, the effect was more pronounced and long-lasting following rhythmic training compared with auditory training. Conclusion These results are discussed in terms of temporal adaptation abilities in speech interaction and suggest the use of rhythmic training to improve these skills of children with hearing loss.