ERIC Number: EJ812994
Record Type: Journal
Publication Date: 2004-Mar
Abstractor: As Provided
Reference Count: 0
Using a Longitudinal Database to Assess the Validity of Preceptors' Ratings of Clerkship Performance
Ferguson, Kristi J.; Kreiter, Clarence D.
Advances in Health Sciences Education, v9 n1 p39-46 Mar 2004
Purpose: To examine the validity of using scores from a clinical evaluation form as an assessment of clinical competence. Method: Investigators collected a longitudinal clinical skills assessment database that included scores reflecting performance on standardized patient interactions, case-based learning performance, scores on multiple-choice clinical examinations, and preceptors' ratings of students during their clerkships. Pearson correlation coefficients and coefficients corrected for attenuation were calculated between the mean preceptor rating CEF score and the other measures collected during years one and two. Results: Estimates from an earlier across-clerkship generalizability study of the CEF demonstrated that a mean rating computed across 28 forms and four clerkships (the average measurement frequencies observed in this study), yielded an estimated G-coefficient of 0.62 (Kreiter and Ferguson, 2001). Reliabilities for the other measures ranged from 0.23 to 0.56. Nine of the twelve clinical skill measures correlated with the CEF. For those significantly correlated measures where reliabilities could be calculated, correlations corrected for attenuation ranged from 0.46 to 0.58. Conclusion: This study indicates that the skills measured by the CEF are related to other clinical performance measures and, conversely, that pre-clinical measures of skills that are believed to be important during clinical years are in fact predictors of preceptors' ratings of clinical performance later in medical school. In addition, the magnitude of the disattenuated coefficients suggests that ratings on the CEF are dependent on important aspects such as clinical knowledge. This study suggests that when averaged over a large number of observations, mean CEF scores demonstrate validity coefficients large enough to support their use as part of an evaluation of students' clinical performance.
Descriptors: Generalizability Theory, Medical Students, Validity, Program Effectiveness, Correlation, Databases, Clinical Experience, Longitudinal Studies, Graduate Medical Education, Scores, Multiple Choice Tests, Reliability
Springer. 233 Spring Street, New York, NY 10013. Tel: 800-777-4643; Tel: 212-460-1500; Fax: 212-348-4505; e-mail: firstname.lastname@example.org; Web site: http://www.springerlink.com
Publication Type: Journal Articles; Reports - Research
Education Level: Higher Education
Authoring Institution: N/A