NotesFAQContact Us
Search Tips
Peer reviewed Peer reviewed
Direct linkDirect link
ERIC Number: EJ764179
Record Type: Journal
Publication Date: 2007-Feb
Pages: 16
Abstractor: Author
Reference Count: 27
ISSN: ISSN-1380-3611
Multiple Teacher Ratings: An Evaluation of Measurement Strategies
Muschkin, Clara G.; Malone, Patrick S.
Educational Research and Evaluation, v13 n1 p71-86 Feb 2007
This study addresses the questions that arise when collecting, describing, and analyzing information from multiple informants regarding attributes of individual students. Using data from the Fast Track study, we evaluate alternative measurement strategies for using multiple teacher ratings of student adjustment to middle school among a sample of 326 Grade-6 pupils. One goal of the study was to compare the advantages of three measurement strategies using multiple and single informants in terms of their correlation with contemporaneous measures of behavior and academic achievement. Comparisons of residual variance using an aggregated rating, the rating from an "optimal informant," and a score selected at random from the response set, indicate that aggregation provides the highest criterion-related validity. As part of these analyses, we explore the significance of inter-rater concordance, measured in terms of the intraclass correlation coefficient (ICC). Results indicate that for some aggregated scores, reliability can significantly limit their interpretability. The second main goal of the study was to evaluate the effects of variation in the number of teacher ratings on residual variance estimates for aggregate measures in selected behavioral domains. We conclude that the advantages of using multiple ratings are significant with a larger number of informants. (Contains 2 figures, 2 tables and 1 note.) [This study was authored in part by the Conduct Problems Prevention Research Group.]
Routledge. Available from: Taylor & Francis, Ltd. 325 Chestnut Street Suite 800, Philadelphia, PA 19106. Tel: 800-354-1420; Fax: 215-625-2940; Web site:
Publication Type: Journal Articles; Reports - Research
Education Level: Grade 6; Middle Schools
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A