This article reviews current knowledge about how the visual system recognizes letters and words, and the impact on reading when parts of the visual system malfunction. The physiology of eye and brain places important constraints on how we process text, and the efficient organization of the neurocognitive systems involved is not inherent but depends on the experience of learning to read. Although studies have reported that many children with severe reading difficulties have problems with visual fixation of words, in this context the concept of "visual dyslexia" remains controversial. Evidence that a significant proportion of children with dyslexia have impairment of the magnocellular component of the visual system (which responds to contrast and movement) has led to alternative theories challenging the phonological deficit model of dyslexia. Deficits in the magno system have also been proposed to explain symptoms of visual stress that many people experience when reading, and to account for the alleviation of these symptoms by the use of coloured overlays or tinted lenses. However, a competing theory that posits cortical hypersensitivity to pattern glare as the cause of visual stress is generally more favoured at the present time. Our scientific understanding of reading would be much improved if visual factors were better integrated into theories that currently focus almost exclusively on phonological factors, but some progress towards this is being made.