The definition and measurement of counsellor trainee competency is an issue that has received increased attention yet lacks quantitative study. This research evaluates item responses, scale reliability and intercorrelations, interrater agreement, and criterion-related validity of the Professional Performance Fitness Evaluation/Professional Counseling Performance Evaluation, a measure used by counsellor educators to evaluate professional fitness. The results identified several scale items that were insensitive to substandard skill levels and indicated that university and site supervisors' trainee ratings did not agree. Both site and university supervisors did identify a majority of students demonstrating substandard skills, and when the site and university supervisors' ratings were combined, the supervisors identified 79% of the noncompetent trainees.