This study evaluated a process for training raters to reliably rate clinicians delivering the Adolescent Community Reinforcement Approach (A-CRA) in a national dissemination project. The unique A-CRA coding system uses specific behavioral anchors throughout its 73 procedure components. Five randomly selected raters each rated "passing" and "not passing" examples of the 19 A-CRA procedures. Ninety-four percent of the final intraclass correlation coefficients were at least "good" (=0.60) and 66.7% were "excellent" (=0.75), and 95% of the ratings exceeded the 60% or better agreement threshold between raters and the gold standard. Raters can be trained to provide reliable A-CRA feedback for large-scale dissemination projects.