Auditory perception of English minimal pairs was tested with or without noise background. Each subject was interviewed after the test to collect information regarding their early experience on learning English as a foreign language. This study was designed to examine the differential effects of learning English at three age-starting points and two learning durations. This study hopes to determine how childhood experience of English learning (which is not mandatory in public elementary schools) has affected the auditory competence of university students in distinguishing English minimal pairs. Results showed that age effects were salient only under condition of noise background. Without the interference of background noise, most subjects performed well enough to obliterate any potential differences.