The purpose of this validity study was to improve measurement of athletes' evaluations of their head coach's coaching competency, an important multidimensional construct in models of coaching effectiveness. A revised version of the Coaching Competency Scale (CCS) was developed for athletes of high school teams (APCCS II-HST). Data were collected from athletes (N = 748) of seven relevant sports. Athlete observations were clustered within teams (G = 74). Multigroup confirmatory factor analyses of the asymptotic within-teams covariance matrix provided evidence for factorial invariance, except for one residual variance, by athlete gender (n [subscript male] = 427, n [subscript female] = 321). An exploratory multilevel confirmatory factor analysis provided evidence for close fit of an oblique five-factor within-teams structure and a one-factor between-teams structure. (Contains 2 tables.)