Download full text
Download full text
ERIC Number: ED562265
Record Type: Non-Journal
Publication Date: 2015
Reference Count: 8
Small-Sample Adjustments for Tests of Moderators and Model Fit in Robust Variance Estimation in Meta-Regression
Tipton, Elizabeth; Pustejovsky, James E.
Society for Research on Educational Effectiveness
Randomized experiments are commonly used to evaluate the effectiveness of educational interventions. The goal of the present investigation is to develop small-sample corrections for multiple contrast hypothesis tests (i.e., F-tests) such as the omnibus test of meta-regression fit or a test for equality of three or more levels of a categorical moderator. Drawing on work that addresses related, simpler problems and special cases of cluster-robust variance estimation, the authors develop three small-sample tests based on different approximations to the distribution of a robust Wald test statistic. In the remainder of the study, the authors describe their modeling assumptions, proposed tests, and some initial simulation result. These approximations are drawn from a wide array of areas within statistics, ranging from econometrics to survey sampling. The paper presents both new analytic work describing these small-sample corrected test statistics and the results of a large simulation study that compares these potential solutions, as well as a discussion of the implications of our findings for practice. Tables and figures are appended.
Descriptors: Randomized Controlled Trials, Sample Size, Effect Size, Hypothesis Testing, Regression (Statistics), Robustness (Statistics), Mathematical Models, Simulation, Statistical Bias, Sampling, Economics, Hierarchical Linear Modeling, Meta Analysis
Society for Research on Educational Effectiveness. 2040 Sheridan Road, Evanston, IL 60208. Tel: 202-495-0920; Fax: 202-640-4401; e-mail: firstname.lastname@example.org; Web site: http://www.sree.org
Publication Type: Reports - Research
Education Level: N/A
Authoring Institution: Society for Research on Educational Effectiveness (SREE)