NotesFAQContact Us
Search Tips
Peer reviewed Peer reviewed
PDF on ERIC Download full text
ERIC Number: ED562823
Record Type: Non-Journal
Publication Date: 2014
Pages: 10
Abstractor: ERIC
Reference Count: 16
Synthesizing Results from Replication Studies Using Robust Variance Estimation: Corrections When the Number of Studies Is Small
Tipton, Elizabeth
Society for Research on Educational Effectiveness
Replication studies allow for making comparisons and generalizations regarding the effectiveness of an intervention across different populations, versions of a treatment, settings and contexts, and outcomes. One method for making these comparisons across many replication studies is through the use of meta-analysis. A recent innovation in meta-analysis is the introduction of a robust variance estimator (RVE) that allows for the inclusion of multiple, correlated effect sizes in a meta-analysis. While the RVE estimator is unbiased in large-samples, its small sample properties are often not ideal. This paper investigates possible approaches to adjusting the RVE estimator when the number of studies is small (less than 40), which is common in the both meta-analyses and replication studies in education. Results indicate that, even with m = 40 studies, the probability of a Type I error for the standard RVE estimator can be much larger than a = 0.05. For this reason the author argues that it is best if the corrections provided within the study are implemented in all RVE analyses, even those of moderate to large sizes. Tables and figures are appended.
Society for Research on Educational Effectiveness. 2040 Sheridan Road, Evanston, IL 60208. Tel: 202-495-0920; Fax: 202-640-4401; e-mail:; Web site:
Publication Type: Reports - Research
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: Society for Research on Educational Effectiveness (SREE)