Background: Previous research has suggested that speech perception in elderly adults is influenced not only by age-related hearing loss or presbycusis but also by declines in cognitive abilities, by background noise and by the syntactic complexity of the message. Aims: To gain further insight into the influence of these cognitive as well as acoustic and linguistic factors on speech perception in elderly adults by investigating inhibitory control as a listener characteristic and background noise type and syntactic complexity as input characteristics. Methods & Procedures: Phoneme identification was measured in different noise conditions and in different linguistic contexts (single words, sentences with varying syntactic complexity). Additionally, inhibitory control was measured using a visual stimulus--response matching task. Fifty-one adults participated in this study, including elderly adults with age-related hearing loss (n = 9) and with normal hearing (n = 17), and a control group of normal hearing younger adults (n = 25). Outcomes & Results: The analysis revealed that elderly adults with normal hearing and with hearing loss were less likely to identify successfully phonemes in single words than younger normal hearing controls. In the context of sentences, only elderly adults with hearing loss had a lower odds of correct phoneme perception than the control group. Additionally, in elderly adults with hearing loss, phoneme-in-sentence perception was linked to age-related declines in inhibitory control. In all participants, phoneme identification in sentences was influenced by both noise type and syntactic complexity. Conclusions & Implications: Inhibitory control and syntactic complexity might play a significant role in speech perception, especially in elderly listeners. These factors might also influence the results of clinical assessments of speech perception. Testing procedures thus need to be selected and their results interpreted carefully with these influences in mind.