We show that retrieval from semantic memory is vulnerable even to the mere presence of speech. Irrelevant speech impairs semantic fluency--namely, lexical retrieval cued by a semantic category name--but only if it is meaningful (forward speech compared to reversed speech or words compared to nonwords). Moreover, speech related semantically to the retrieval category is more disruptive than unrelated speech. That phonemic fluency--in which participants are cued with the first letter of words they are to report--was not disrupted by the mere presence of meaningful speech, only by speech in a related phonemic category, suggests that distraction is not mediated by executive processing load. The pattern of sensitivity to different properties of sound as a function of the type of retrieval cue is in line with an interference-by-process approach to auditory distraction. (Contains 4 tables, 1 figure and 1 footnote.)