ERIC Number: EJ797099
Record Type: Journal
Publication Date: 2005-Mar
Reference Count: N/A
Investigating Gender-Related Construct-Irrelevant Components of Scores on the Written Assessment Exercise of a High-Stakes Certification Assessment
Boulet, John R.; McKinley, Danette W.
Advances in Health Sciences Education, v10 n1 p53-63 Mar 2005
The ECFMG[R] Clinical Skills Assessment (CSA[R]) was developed to evaluate whether graduates of international medical schools (IMGs) are ready to enter graduate training programs in the United States. The patient note (PN) exercise is specifically used to assess a candidate's ability to summarize and synthesize the data collected in a simulated patient interview. In a 1-year period, over 7700 first time takers completed the CSA, resulting in over 77,000 physician-based performance ratings. An initial pilot study indicated that, based solely on handwriting, the raters were able to correctly classify the gender of the candidate approximately 70% of the time. This result, combined with the fact that the notes are holistically scored, suggests that rating bias is possible. The purpose of this study was to investigate whether the gender of the candidate, the gender of the performing standardized patient, and the gender of the rater had any impact on scores. An analysis of covariance (ANCOVA) indicated that there was no significant interaction between candidate and rater gender. Female candidates significantly outperformed males, regardless of rater gender (p less than 0.01, effect size=0.23). The results of this study suggest that, based on rater, SP, and candidate characteristics, the validity of the PN ratings is not compromised.
Descriptors: Graduate Study, Medical Schools, Program Effectiveness, Statistical Analysis, Clinical Experience, Gender Differences, Scores, Pilot Projects, Interviews, High Stakes Tests, Data Collection, Simulation
Springer. 233 Spring Street, New York, NY 10013. Tel: 800-777-4643; Tel: 212-460-1500; Fax: 212-348-4505; e-mail: email@example.com; Web site: http://www.springerlink.com
Publication Type: Journal Articles; Reports - Evaluative
Education Level: N/A
Authoring Institution: N/A
Identifiers - Location: United States