ERIC Number: EJ861377
Record Type: Journal
Publication Date: 2009-Dec
Abstractor: As Provided
Reference Count: 0
Does Scale Length Matter? A Comparison of Nine- versus Five-Point Rating Scales for the Mini-CEX
Cook, David A.; Beckman, Thomas J.
Advances in Health Sciences Education, v14 n5 p655-664 Dec 2009
Educators must often decide how many points to use in a rating scale. No studies have compared interrater reliability for different-length scales, and few have evaluated accuracy. This study sought to evaluate the interrater reliability and accuracy of mini-clinical evaluation exercise (mini-CEX) scores, comparing the traditional mini-CEX nine-point scale to a five-point scale. Methods: The authors conducted a validity study in an academic internal medicine residency program. Fifty-two program faculty participated. Participants rated videotaped resident-patient encounters using the mini-CEX with both a nine-point scale and a five-point scale. Some cases were scripted to reflect a specific level of competence (unsatisfactory, satisfactory, superior). Outcome measures included mini-CEX scores, accuracy (scores compared to scripted competence level), interrater reliability, and domain intercorrelation. Results: Interviewing, exam, counseling, and overall ratings varied significantly across levels of competence (P less than 0.0001). Nine-point scale scores accurately classified competence more often (391/720 [54%] for overall ratings) than five-point scores (316/723 [44%], P less than 0.0001). Interrater reliability was similar for scores from the nine- and five-point scales (0.43 and 0.40, respectively, for overall ratings). With the exception of correlation between exam and counseling scores using the five-point scale (r = 0.38, P = 0.13), score correlations among all domain combinations were high (r = 0.46-0.89) and statistically significant (P [less than or equal to] 0.015) for both scales. Conclusions: Mini-CEX scores demonstrated modest interrater reliability and accuracy. Although interrater reliability is similar for nine- and five-point scales, nine-point scales appear to provide more accurate scores. This has implications for many educational assessments.
Descriptors: Interrater Reliability, Rating Scales, Internal Medicine, Test Validity, Video Technology, Medical Students, Medical Education, Educational Assessment, Measures (Individuals), Measurement Techniques, Evaluation Methods, Clinical Teaching (Health Professions), Medical School Faculty, Medical Evaluation, Student Evaluation, Graduate Students
Springer. 233 Spring Street, New York, NY 10013. Tel: 800-777-4643; Tel: 212-460-1500; Fax: 212-348-4505; e-mail: firstname.lastname@example.org; Web site: http://www.springerlink.com
Publication Type: Journal Articles; Reports - Evaluative
Education Level: Higher Education; Postsecondary Education
Authoring Institution: N/A