ERIC Number: EJ797109
Record Type: Journal
Publication Date: 2005-Jun
Reference Count: 0
The Role of a Computerised Case-Based Testing Procedure in Practice Performance Assessment
Schuwirth, L.; Gorter, S.; Van der Heijde, D.; Rethans, J. J.; Brauer, J.; Houben, H.; Van der Linden, S.; Van der Vleuten, C.; Scherpbier, A.
Advances in Health Sciences Education, v10 n2 p145-155 Jun 2005
Introduction: For postgraduate training of doctors there is a need for valid and reliable instruments to assess their daily performance. Various instruments have been suggested, some of which use incognito simulated patients (SPs). These methods are resource intensive. Computerised Case-based testing (CCT) is logistically simpler and may still predict performance well. The research question was to evaluate the predictive validity of CCT for performance. Methods: Seventeen rheumatologists were each visited by eight incognito SPs presenting various rheumatological complaints, and scoring the performance of the rheumatologists using a predefined checklist. From this checklist a panel of experts identified essential items. In addition the rheumatologists sat a CCT test containing 55 cases with a total of 121 items. Results: Negative correlations were found between the SP scores and the CCT scores. This was unexpected. Therefore, background variables on experience were used to compare both methods. The correlation between these and CCT were high and positive and with the SP scores high and negative. This pattern did not differ when using the essential items of the checklist. Reliabilities of the SP scores were markedly high. Discussion: Although CCT was not predictive of SP scores, it was related to working experience. There are good reasons to assume that although SP-scores were more authentic, they were less valid than CCT scores, mainly because they focussed more on thoroughness than on efficiency in data gathering. The results underpin the assumption that for valid performance assessment the most important issue is what information about the candidate is collected and now how authentic the method is.
Descriptors: Check Lists, Performance Based Assessment, Testing, Predictive Validity, Computer Assisted Testing, Measures (Individuals), Evaluation Research, Evaluation Methods, Physicians, Professional Continuing Education, Medical Education, Correlation, Experience, Test Reliability, Simulation, Educational Technology
Springer. 233 Spring Street, New York, NY 10013. Tel: 800-777-4643; Tel: 212-460-1500; Fax: 212-348-4505; e-mail: firstname.lastname@example.org; Web site: http://www.springerlink.com
Publication Type: Journal Articles; Reports - Research
Education Level: Adult Education; Postsecondary Education
Authoring Institution: N/A