We tested proposals that medial temporal lobe (MTL) structures support not just memory but certain kinds of visual perception as well. Patients with hippocampal lesions or larger MTL lesions attempted to identify the unique object among twin pairs of objects that had a high degree of feature overlap. Patients were markedly impaired under the more difficult task conditions. However, the deficit was fully rescued when patients used a pencil to draw lines between the twin pairs, thereby eliminating the need to hold material in memory as they worked at each display. The perceptual demands of the task were presumably the same with or without this memory aid. Accordingly, the results suggest that the deficit on this and similar tasks, which involve comparisons across stimuli, are better understood in terms of impaired memory rather than impaired perception.