ERIC Number: ED228314
Record Type: RIE
Publication Date: 1983-Apr
Reference Count: 0
Controlling Rater Stringency Error in Clinical Performance Rating: Further Validation of a Performance Rating Theory.
Cason, Gerald J.; And Others
Prior research in a single clinical training setting has shown Cason and Cason's (1981) simplified model of their performance rating theory can improve rating reliability and validity through statistical control of rater stringency error. Here, the model was applied to clinical performance ratings of 14 cohorts (about 250 students and 200 raters) in a Junior year clerkship at a different medical school. The model fit every cohort's data. In the 11 cohorts where a significant rater effect was observed, reliability and validity of ratings were improved by using the model to obtain adjusted ratings from which rater stringency error was removed. Statistical control of stringency error in these cohorts produced adjusted ratings with a mean standard error of about 2.9 percent compared with 8.4 percent for (unadjusted) observed ratings. These improvements can be had at a fraction of the cost for methods such as rater training. The results provided further support for the validity, extended the generalizability, and illustrated the practical value of the Casons' model. Contained in the appendices are: "A Theory of Performance Rating"; and "MERLIN's Magic: The Estimation of RRP's and SAP's." (Author/PN)
Publication Type: Speeches/Meeting Papers; Reports - Research
Education Level: N/A
Authoring Institution: N/A
Identifiers: Generalizability (Tests); Performance Rating Theory (Cason and Cason); Rater Reliability; Rater Stringency Error
Note: Paper presented at the Annual Meeting of the American Educational Research Association (67th, Montreal, Quebec, April 11-15, 1983).