The purpose of the present study was to determine if pre-service music educators could discriminate between novice and expert choral directors irrespective of the proficiency of the choral ensemble. A secondary purpose was to compare study results elicited in the USA with those from participants in Europe, South America and Asia. Previous research has indicated that teaching proficiency can be evaluated by people who are not speakers of the language of the lesson, but these evaluations have always occurred with participants in a natural environment, i.e., good teachers teaching students who perform well and deficient teachers in front of students who are struggling. This study was conducted to see if good teaching could still be identified, irrespective of student proficiency. Participants viewed videotapes of four different choral conductors from the USA. Two conductors were proficient and experienced, and two were novices. Participants viewed these conductors in the following counterbalanced settings: (1) an accomplished conductor conducting a fine choral ensemble; (2) an accomplished conductor conducting a poor choral ensemble; (3) a novice conductor conducting a fine choral ensemble; and (4) a novice conductor conducting a poor choral ensemble. Participants wrote observational comments and gave each teacher an evaluative numeric rating. Results of the quantitative data analysis indicated that participants evaluated the skills of the teacher and not the proficiency of the ensembles. Furthermore, results of the qualitative data analysis indicated that there were not only differences between participants from different continents, but also vast differences regarding the topics on which the participants commented, dependent upon the level of the conductor being observed. (Contains 1 figure and 3 tables.)