ERIC Number: EJ1067825
Record Type: Journal
Publication Date: 2015-Aug
Abstractor: As Provided
Examiners and Content and Site: Oh My! a National Organization's Investigation of Score Variation in Large-Scale Performance Assessments
Sebok, Stefanie S.; Roy, Marguerite; Klinger, Don A.; De Champlain, André F.
Advances in Health Sciences Education, v20 n3 p581-594 Aug 2015
Examiner effects and content specificity are two well known sources of construct irrelevant variance that present great challenges in performance-based assessments. National medical organizations that are responsible for large-scale performance based assessments experience an additional challenge as they are responsible for administering qualification examinations to physician candidates at several locations and institutions. This study explores the impact of site location as a source of score variation in a large-scale national assessment used to measure the readiness of internationally educated physician candidates for residency programs. Data from the Medical Council of Canada's National Assessment Collaboration were analyzed using Hierarchical Linear Modeling and Rasch Analyses. Consistent with previous research, problematic variance due to examiner effects and content specificity was found. Additionally, site location was also identified as a potential source of construct irrelevant variance in examination scores.
Descriptors: Performance Based Assessment, Site Selection, Medical Students, Physicians, Readiness, Graduate Medical Education, Examiners, Scores, Hierarchical Linear Modeling, Foreign Countries
Springer. 233 Spring Street, New York, NY 10013. Tel: 800-777-4643; Tel: 212-460-1500; Fax: 212-348-4505; e-mail: email@example.com; Web site: http://www.springerlink.com
Publication Type: Journal Articles; Reports - Evaluative
Education Level: Higher Education; Postsecondary Education
Authoring Institution: N/A
Identifiers - Location: Canada
Grant or Contract Numbers: N/A