Brain plasticity refers to the remarkable property of cerebral neurons to change their structure and function in response to experience, a fundamental theoretical theme in the field of basic research and a major focus for neural rehabilitation following brain disease. While much of the early work on this topic was based on deprivation approaches relying on sensory experience reduction procedures, major advances have been recently obtained using the conceptually opposite paradigm of environmental enrichment, whereby an enhanced stimulation is provided at multiple cognitive, sensory, social, and motor levels. In this survey, we aim to review past and recent work concerning the influence exerted by the environment on brain plasticity processes, with special emphasis on the underlying cellular and molecular mechanisms and starting from experimental work on animal models to move to highly relevant work performed in humans. We will initiate introducing the concept of brain plasticity and describing classic paradigmatic examples to illustrate how changes at the level of neuronal properties can ultimately affect and direct key perceptual and behavioral outputs. Then, we describe the remarkable effects elicited by early stressful conditions, maternal care, and preweaning enrichment on central nervous system development, with a separate section focusing on neurodevelopmental disorders. A specific section is dedicated to the striking ability of environmental enrichment and physical exercise to empower adult brain plasticity. Finally, we analyze in the last section the ever-increasing available knowledge on the effects elicited by enriched living conditions on physiological and pathological aging brain processes.