By the end of their studies, non-native speakers of English studying at English-medium universities have had several years of exposure to English in that setting. Do non-native students, particularly those enrolled in non-language related programs, show different levels of second language (L2) speaking ability in their final semester of studies than non-native students in their first semester, as judged by other students in the university community? In this exploratory cross-sectional study, two matched groups of L2 English university students in their first or final semester of study in non-language-related programs (N = 20) were recorded in mock job interviews. The students were rated by two groups of raters for accentedness, comprehensibility, fluency, and communicative effectiveness. Both rater groups were university students; one group was from diverse academic programs, while the other group was studying human resource management (HRM). Although the first- and final-semester L2 English students differed in how long they had studied in English, no significant difference in ratings between first- and final-semester students was found. However, the two rater groups differed in how they rated accentedness and comprehensibility, suggesting that the nature of listeners' previous academic experience (e.g., with HRM) influences their judgments. The use of holistic rating scales to evaluate L2 speech is discussed, as well as the relationship between the nature of language exposure and the performance of the student and rater groups.