Prenatal opioid exposure has reached epidemic proportions. In the last 10 years, there has been a 242% increase in the number of babies born with the drug withdrawal syndrome known as Neonatal Opioid Withdrawal Syndrome (NOWS). Developmental outcome studies of infants with prenatal opioid exposure are limited by methodological issues including small sample sizes and lack of control for confounding variables such as exposure to poverty and maternal psychopathology. Thus, there is a critical gap in the literature that limits our ability to predict short-term effects of opioid exposure. Here we review direct neurotoxic, indirect, and stress-related pathophysiologies of prenatal opioid exposure. We describe the literature on short and long-term neurodevelopmental outcomes of children with prenatal opioid exposure, highlighting sex differences and the role of early life stress. We conclude by prioritizing avenues for future research for this group of underserved women and their children at risk for neurodevelopmental delays.