This article reports on the results of a research study that investigated Cantonese English as a second language (ESL) learners' perception of English speech sounds, their perceived relations between "similar" English and Cantonese sounds, as well as the applicability of the claims of the Speech Learning Model (SLM) to second language (L2) learning by Cantonese ESL learners in Hong Kong. Forty university English majors participated in a perception study that required them to identify English minimal pairs, to classify a target English sound as a Cantonese sound and to rate the degree of similarity of a certain English sound to a "similar" Cantonese sound. It is found that Cantonese ESL learners' perception of L2 speech sounds is intimately related to their perceived similarity between certain first language (L1) and L2 contrasts. The research provides empirical support for the SLM, affirming the adverse effects of similarity on L2 learning. It also calls for the inclusion of perception training in an L2 phonology program. Further research is needed to include learners from different linguistic backgrounds and at different English proficiency levels.