Both the passage of time and external distraction make it difficult to keep attention on the task at hand. We tested the hypothesis that time-on-task and external distraction pose independent challenges to attention and that the brain's cholinergic system selectively modulates our ability to resist distraction. Participants with a polymorphism limiting cholinergic capacity (Ile89Val variant [rs1013940] of the choline transporter gene SLC5A7) and matched controls completed self-report measures of attention and a laboratory task that measured decrements in sustained attention with and without distraction. We found evidence that distraction and time-on-task effects are independent and that the cholinergic system is strongly linked to greater vulnerability to distraction. Ile89Val participants reported more distraction during everyday life than controls, and their task performance was more severely impacted by the presence of an ecologically valid video distractor (similar to a television playing in the background). These results are the first to demonstrate a specific impairment in cognitive control associated with the Ile89Val polymorphism and add to behavioral and cognitive neuroscience studies indicating the cholinergic system's critical role in overcoming distraction.