ERIC Number: ED254554
Record Type: RIE
Publication Date: 1985-Mar
Reference Count: 0
Interrater Reliability and Internal Consistency of Student and Staff Ratings of Medical Instruction.
Dielman, T. E.; Horvatich, Paula K.
The purposes of this study were to establish the interrater reliability, dimensionality, and internal consistency of an instruction evaluation instrument used at The University of Michigan Medical School. Using the nine-item rating scale, 1,758 student ratings and 88 staff ratings were gathered on 61 faculty. Interrater agreement ranged from .28 to .53 among students, from .11 to .70 among staff, and from .28 to .70 between students and staff. Separate factor analyses of student and staff data showed all items except one exhibited high loadings on a single factor. The eight items forming this factor were summed with unit weighting to form a total "Teaching Quality" index score for each instructor. Internal consistency (Cronbach's alpha) coefficients for this index were .92 for student raters and .93 for staff raters. Although this evidence for reliability is compelling, the question of validity remains. Validation methods depend on how the instructional ratings are used. The instructor evaluation form is appended. (Author/BS)
Publication Type: Speeches/Meeting Papers; Tests/Questionnaires; Reports - Research
Education Level: N/A
Authoring Institution: N/A
Identifiers: University of Michigan Medical School
Note: Paper presented at the Annual Meeting of the American Educational Research Association (69th, Chicago, IL, March 31-April 4, 1985).