ERIC Number: ED260104
Record Type: RIE
Publication Date: 1985-Apr
Reference Count: 0
Metacognition of Performance Raters.
Littlefield, John H.; And Others
Sixteen Family Practice faculty members completed ratings on 59 senior medical students after a 6-week primary care clerkship. Each student was rated by seven to ten faculty members and the chief residents who worked with them, resulting in a total of 353 ratings. The rating scale covered: (1) attainment of learning objectives; (2) progress during the clerkship; (3) overall performance, (4) frequency of contact between student and rater (number of patients discussed); and (5) confidence in the rating, to indicate raters' metacognition. A two-factor analysis of variance was performed on the results to explore the relationships among rater accuracy, level of contact, and rater confidence in the score assigned. It was concluded that confidence in the validity of a rating was not related to the accuracy of that rating. Level of rater-student contact was, however, related to accuracy, with the most accurate ratings based upon discussion of seven to eleven patients. Low levels of contact were associated with overly stringent ratings, and high levels of contact were associated with lenient ratings. Individual raters differed in the leniency of scores, the tendency to make extreme judgments, and confidence in each rating. (GDC)
Descriptors: Analysis of Variance, Clinical Experience, Confidence Testing, Evaluators, Graduate Medical Education, Higher Education, Interrater Reliability, Medical School Faculty, Medical Students, Metacognition, Rating Scales, Student Evaluation, Teacher Student Relationship, Test Reliability, Test Validity
Publication Type: Speeches/Meeting Papers; Reports - Research
Education Level: N/A
Authoring Institution: N/A
Note: Paper presented at the Annual Meeting of the American Educational Research Association (69th, Chicago, IL, March 31-April 4, 1985).