Smooth pursuit impairment is recognized clinically by the presence of saccadic tracking of a small object and quantified by reduction in pursuit gain, the ratio of smooth eye movement velocity to the velocity of a foveal target. Correlation of the site of brain lesions, identified by imaging or neuropathological examination, with defective smooth pursuit determines brain structures that are necessary for smooth pursuit. Paretic, low gain, pursuit occurs toward the side of lesions at the junction of the parietal, occipital and temporal lobes (area V5), the frontal eye field and their subcortical projections, including the posterior limb of the internal capsule, the midbrain and the basal pontine nuclei. Paresis of ipsiversive pursuit also results from damage to the ventral paraflocculus and caudal vermis of the cerebellum. Paresis of contraversive pursuit is a feature of damage to the lateral medulla. Retinotopic pursuit paresis consists of low gain pursuit in the visual hemifield contralateral to damage to the optic radiation, striate cortex or area V5. Craniotopic paresis of smooth pursuit consists of impaired smooth eye movement generation contralateral to the orbital midposition after acute unilateral frontal or parietal lobe damage. Omnidirectional saccadic pursuit is a most sensitive sign of bilateral or diffuse cerebral, cerebellar or brainstem disease. The anatomical and physiological bases of defective smooth pursuit are discussed here in the context of the effects of lesion in the human brain. (Contains 5 figures and 2 tables.)