While the importance of phonological sensitivity for understanding reading acquisition and impairment across orthographies is well documented, what underlies deficits in phonological sensitivity is not well understood. Some researchers have argued that speech perception underlies variability in phonological representations. Others have investigated the role of more general auditory sensitivity for reading development and reading difficulties, arguing that poor phonological representations may actually be due to broad underlying auditory deficits, which are not restricted to speech stimuli. We argue that these hypotheses are not necessarily mutually exclusive. In this review, we demonstrate that auditory sensitivity and speech perception can be integrated into a single developmental model, in which auditory sensitivity may have an indirect impact on reading; this impact is mediated by speech perception. In the model, we distinguish general auditory sensitivity as falling into at least two general categories: rhythmic and temporal. Correspondingly, speech perception itself can be distinguished as suprasegmental and segmental. Theoretically, the proposed model integrates a broad range of studies on general auditory and speech perception to suggest a developmental trajectory for reading acquisition that can be explored from before birth. Practically, the proposed model points to different ways of understanding and diagnosing reading difficulties and distinguishing reading difficulties across languages and orthographies.