Objective: Few studies have examined concordance between raters of ADHD symptoms in adults; there is less information on how well rating scales function in distinguishing adult ADHD from other disorders. This study examined these variables using the Conners Adult ADHD Rating Scales (CAARS). Method: The sample included 349 adults evaluated for attention problems. Correlations and kappa values were calculated using self- and observer-ratings of item-level symptoms; sensitivity, specificity, and discriminant validity of cluster scores in predicting clinician diagnoses were computed for 269 participants. Results: Item-level concordance rates ranged from slight to fair. Cluster scores demonstrated a poor balance of sensitivity and specificity in predicting ADHD diagnosis; a high percentage of participants with internalizing disorders had scores in the clinical range. Conclusion: Self- and observer-ratings on the CAARS provide clinically relevant data about attention problems in adults, but the instrument does not effectively distinguish between ADHD and other adult psychiatric disorders. (Contains 5 tables.)