Traditionally, treatment adherence has been measured in outcome studies by highly trained, independent raters who code audio- or video-taped treatment sessions to understand therapists' adherence to preestablished treatment session components. Unfortunately, this method of assessing treatment adherence is time-, labor-, and cost-intensive, and does not translate well to community settings. This study compared expert-coded behavior observations, therapist and caregiver report of therapists' adherence to nine teaching technique items assessed in treatment sessions using Alternatives for Families: A Cognitive Behavioral Therapy (AF-CBT) to determine whether other raters (outside of traditional expert coders) could effectively and accurately measure therapist adherence. A total of 533 therapy sessions were coded across experts (n = 2), therapists (n = 20), and caregivers (n = 42). Outcomes indicated strikingly different ratings across all reporters suggesting that therapist and caregiver reports may be supplement to, but not substitute for, observer ratings.