Research on native Arabic speakers' perception of non-native English phonemic contrasts has manifested various conclusions. Some studies have revealed that native Arabic speakers experience difficulty in the perception of non-native English phonemic contrasts, where others have concluded that they largely do not experience such difficulty. Due to this discrepancy, the current study aimed to examine the overall perceptual accuracy of only the English labial categories /p, b, f, and v/ in onset positions performed by native Arabic speakers. The study also examined the potential effects of the phonetic contexts and the familiarity of the stimuli being tested on the participants' perceptual performance. Twenty-four Saudi male participants completed consonantal phoneme identification and discrimination tasks. The findings revealed that the participants experienced difficulty in the perception of the categories /p/ and /v/, confusing them with their counterparts /b/ and /f/. The findings also revealed that neither phonetic context nor familiarity had any effect on the participants' perceptual accuracy of all of the categories in question, except in the context in which the category /v/ was followed by back vowels. In this specific context, the participants' performance tended to be lower, which could be attributed to the influence of lip rounding, which results in coarticulation with the fricative noise associated with this category (/v/). Lastly, the results obtained in this study provide additional crucial insights into auditory phonetics and present additional explanation on how the human auditory system recognizes speech sounds.