A multi-faceted Rasch measurement (MFRM) approach was used to analyze clinical performance ratings of 24 first-year residents in one surgery residency program in Thailand to investigate three types of rater effects: leniency, rater inconsistency, and restriction of range. Faculty from 14 surgical services rated the clinical performance of residents using an 11-item rating instrument. We analyzed the ratings using a three-faceted Rasch model that defines the probability of a particular resident receiving a particular rating as a function of the level of clinical performance of a resident, the difficulty of an item, and the severity of a clinical service. Faculty in 14 clinical services showed significant differences in severity. Faculty in the intensive care unit gave inconsistent ratings, while faculty in the trauma and minor operating room services showed restriction of score range. We recommended using a MFRM approach in analyzing clinical performance ratings. A MFRM analysis not only provides measures of residents' clinical performance that have been adjusted for systematic difference in clinical services severity, but also identifies specific clinical services that exhibit aberrant rating behaviors. The residency program can use this diagnostic information to help determine what changes are needed in the approach to evaluate resident performance. (Contains 1 table, 2 figures, and 1 footnote.)