NotesFAQContact Us
Search Tips
Peer reviewed Peer reviewed
Direct linkDirect link
ERIC Number: EJ927921
Record Type: Journal
Publication Date: 2011
Pages: 10
Abstractor: As Provided
Reference Count: 37
ISSN: ISSN-0731-1745
The Effect of Ignoring Classroom-Level Variance in Estimating the Generalizability of School Mean Scores
Wei, Xin; Haertel, Edward
Educational Measurement: Issues and Practice, v30 n1 p13-22 Spr 2011
Contemporary educational accountability systems, including state-level systems prescribed under No Child Left Behind as well as those envisioned under the "Race to the Top" comprehensive assessment competition, rely on school-level summaries of student test scores. The precision of these score summaries is almost always evaluated using models that ignore the classroom-level clustering of students within schools. This paper reports balanced and unbalanced generalizability analyses investigating the consequences of ignoring variation at the level of classrooms within schools when analyzing the reliability of such school-level accountability measures. Results show that the reliability of school means cannot be determined accurately when classroom-level effects are ignored. Failure to take between-classroom variance into account biases generalizability (G) coefficient estimates downward and standard errors (SEs) upward if classroom-level effects are regarded as fixed, and biases G-coefficient estimates upward and SEs downward if they are regarded as random. These biases become more severe as the difference between the school-level intraclass correlation (ICC) and the class-level ICC increases. School-accountability systems should be designed so that classroom (or teacher) level variation can be taken into consideration when quantifying the precision of school rankings, and statistical models for school mean score reliability should incorporate this information.
Wiley-Blackwell. 350 Main Street, Malden, MA 02148. Tel: 800-835-6770; Tel: 781-388-8598; Fax: 781-388-8232; e-mail:; Web site:
Publication Type: Journal Articles; Reports - Research
Education Level: Elementary Secondary Education
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A