Cortical topography can be remapped as a consequence of sensory deprivation, suggesting that cortical circuits are continually modified by experience. To see the effect of altered sensory experience on specific components of cortical circuits, we imaged neurons, labeled with a genetically modified adeno-associated virus, in the intact mouse somatosensory cortex before and after whisker plucking. Following whisker plucking we observed massive and rapid reorganization of the axons of both excitatory and inhibitory neurons, accompanied by a transient increase in bouton density. For horizontally projecting axons of excitatory neurons there was a net increase in axonal projections from the non-deprived whisker barrel columns into the deprived barrel columns. The axon collaterals of inhibitory neurons located in the deprived whisker barrel columns retracted in the vicinity of their somata and sprouted long-range projections beyond their normal reach towards the non-deprived whisker barrel columns. These results suggest that alterations in the balance of excitation and inhibition in deprived and non-deprived barrel columns underlie the topographic remapping associated with sensory deprivation.