Download full text
Download full text
ERIC Number: ED562264
Record Type: Non-Journal
Publication Date: 2015
Reference Count: 14
Meaningful Effect Sizes, Intra-Class Correlations, and Proportions of Variance Explained by Covariates for Planning 3 Level Cluster Randomized Experiments in Prevention Science
Dong, Nianbo; Reinke, Wendy M.; Herman, Keith C.; Bradshaw, Catherine P.; Murray, Desiree W.
Society for Research on Educational Effectiveness
Cluster randomized experiments are now widely used to examine intervention effects in prevention science. It is meaningful to use empirical benchmarks for interpreting effect size in prevention science. The effect size (i.e., the standardized mean difference, calculated by the difference of the means between the treatment and control groups, divided by the pooled standard deviation of the two groups) provides one way to interpret the substantive or practical significance as compared to the statistical significance of interventions. The purpose of this study is to provide reference values of design parameters by analyzing data from four large scaled intervention studies. In particular, to estimate: (1) the empirical benchmark of meaningful effect sizes regarding the developmental annual growth, and policy-relevant demographic performance gaps (race/ethnicity, gender, and SES); (2) ICC for schools or/and teacher/class; and (3) R-squared at different levels (school, teacher/class, and student). The authors will focus on 3-level cluster randomized experiments. The data structure of a 3-level cluster randomized experiments can be students (level 1) nested within classrooms (level 2), and classrooms nested within schools (level 3). Data for this study came from four separate cluster randomized trials (RCTs) evaluating the effectiveness of school-based prevention interventions. Two of the cluster RCTs evaluated the efficacy of the Incredible Years Teacher Classroom Management (IY TCM) program. Data from two additional cluster RCTs investigated the impact of a school-level prevention intervention called Positive Behavior Intervention and Support (PBIS). Findings included: (1) Normative expectations for change; (2) Demographic performance gaps among subgroups; and (3) Intraclass correlations (ICCs) and proportions of variance explained by the covariates (R-squared). Tables are appended.
Descriptors: Effect Size, Correlation, Multivariate Analysis, Statistical Analysis, Prevention, Randomized Controlled Trials, Benchmarking, Program Effectiveness, Elementary Schools, Positive Behavior Supports
Society for Research on Educational Effectiveness. 2040 Sheridan Road, Evanston, IL 60208. Tel: 202-495-0920; Fax: 202-640-4401; e-mail: email@example.com; Web site: http://www.sree.org
Publication Type: Reports - Research
Education Level: Elementary Education
Authoring Institution: Society for Research on Educational Effectiveness (SREE)
Identifiers - Location: Maryland; Missouri; North Carolina