NotesFAQContact Us
Search Tips
ERIC Number: ED523698
Record Type: Non-Journal
Publication Date: 2011-Apr
Pages: 38
Abstractor: As Provided
Reference Count: 20
The Evaluation of Bias of the Weighted Random Effects Model Estimators. Research Report. ETS RR-11-13
Jia, Yue; Stokes, Lynne; Harris, Ian; Wang, Yan
Educational Testing Service
Estimation of parameters of random effects models from samples collected via complex multistage designs is considered. One way to reduce estimation bias due to unequal probabilities of selection is to incorporate sampling weights. Many researchers have been proposed various weighting methods (Korn, & Graubard, 2003; Pfeffermann, Skinner, Holmes, Goldstein, & Rasbash, 1998) in estimating the parameters of hierarchical models, including random effects models as a special case. In this paper, the bias of the weighted analysis of variance (ANOVA) estimators of the variance components for a two-level, one-way random effects model is evaluated. For these estimators, analytic bias expressions are first developed, the expressions are then used to examine the impact of sample size, intraclass correlation coefficient (ICC), and the sampling design on the bias of the estimators. In addition, two-stage sampling designs are considered, with a general probability design at the first stage (Level 2) and simple random sampling without replacement (SRS) at the second stage (Level 1). The study shows that first-order weighted variance component estimators perform well when for moderate cluster sizes and ICC values. However, noticeable estimation bias can be found with this weighting method for small cluster sizes (less than 20), particularly when ICC is small (less than 0.2). In such scenarios, scaled first-order weighted estimators can be an alternative. This paper is discussed in the context of National Assessment of Educational Progress (NAEP) 2003 4th Grade Reading National and State Assessment data, with Level 1 being the student level and Level 2 being the school level. Bias Expression of First-Order Weighted Estimators is appended. (Contains 4 tables and 4 figures.)
Educational Testing Service. Rosedale Road Mailstop 19R, Princeton, NJ 08541-0001. Tel: 609-921-9000; Fax: 609-734-5410; Web site:
Publication Type: Reports - Research
Education Level: Elementary Education; Elementary Secondary Education; Grade 4; Intermediate Grades
Audience: N/A
Language: English
Sponsor: American Educational Research Association (AERA)
Authoring Institution: Educational Testing Service
Identifiers - Assessments and Surveys: National Assessment of Educational Progress