The flux of neurotransmitter receptors in and out of synapses depends on receptor interaction with scaffolding molecules. However, the crowd of transmembrane proteins and the rich cytoskeletal environment may constitute obstacles to the diffusion of receptors within the synapse. To address this question, we studied the membrane diffusion of the γ-aminobutyric acid type A receptor (GABA(A)R) subunits clustered (γ2) or not (α5) at inhibitory synapses in rat hippocampal dissociated neurons. Relative to the extrasynaptic region, γ2 and α5 showed reduced diffusion and increased confinement at both inhibitory and excitatory synapses but they dwelled for a short time at excitatory synapses. In contrast, γ2 was ~3-fold more confined and dwelled ~3-fold longer in inhibitory synapses than α5, indicating faster synaptic escape of α5. Furthermore, using a gephyrin dominant-negative approach, we showed that the increased residency time of γ2 at inhibitory synapses was due to receptor-scaffold interactions. As shown for GABA(A)R, the excitatory glutamate receptor 2 subunit (GluA2) of the α-amino-3-hydroxy-5-methyl-4-isoxazolepropionic acid receptor (AMPAR) had lower mobility in both excitatory and inhibitory synapses but a higher residency time at excitatory synapses. Therefore barriers impose significant diffusion constraints onto receptors at synapses where they accumulate or not. Our data further reveal that the confinement and the dwell time but not the diffusion coefficient report on the synapse specific sorting, trapping and accumulation of receptors.