An Objective Structured Clinical Examination (OSCE) is an assessment approach employed in medical education, in which residents rotate through multiple stations of standardized clinical tasks to evaluate their clinical competence. Because items used to evaluate residents' performance in each OSCE station are linked to the same task and are rated by the same rater, their ratings may be dependent on one another, violating the assumption of conditional item independence that underlies the multi-faceted Rasch measurement (MFRM) model. We employed a MFRM model to analyze a communication skills assessment of 79 residents, using 6 OSCE stations, each scored on 18 five-point rating scale items. When we treated the rating on each item as a separate scoring unit, MFRM analyses showed item dependency in 65% of item pairs within an OSCE station according to Fisher's Z statistic, a modification of Yen's Q3 index of item dependency. This resulted in overestimation of resident separation reliability and inaccurate parameter estimation. Combining item scores in each OSCE station into a station score and using station as a scoring unit reduced the amount of item dependency to 27%. This approach produced more realistic reliability estimates and helped improve the fit of the data to the model. (Contains 2 tables.)