ERIC Number: EJ1106857
Record Type: Journal
Publication Date: 2016-Aug
Pages: 28
Abstractor: As Provided
ISBN: N/A
ISSN: ISSN-1076-9986
EISSN: N/A
On the Hedges Correction for a "t"-Test
VanHoudnos, Nathan M.; Greenhouse, Joel B.
Journal of Educational and Behavioral Statistics, v41 n4 p392-419 Aug 2016
When cluster randomized experiments are analyzed as if units were independent, test statistics for treatment effects can be anticonservative. Hedges proposed a correction for such tests by scaling them to control their Type I error rate. This article generalizes the Hedges correction from a posttest-only experimental design to more common designs used in practice. We show that for many experimental designs, the generalized correction controls its Type I error while the Hedges correction does not. The generalized correction, however, necessarily has low power due to its control of the Type I error. Our results imply that using the Hedges correction as prescribed, for example, by the What Works Clearinghouse can lead to incorrect inferences and has important implications for evidence-based education.
Descriptors: Statistical Analysis, Randomized Controlled Trials, Error of Measurement, Scaling, Research Design, Statistical Inference, Meta Analysis, Equations (Mathematics), Effect Size, Evidence Based Practice, Hierarchical Linear Modeling
SAGE Publications. 2455 Teller Road, Thousand Oaks, CA 91320. Tel: 800-818-7243; Tel: 805-499-9774; Fax: 800-583-2665; e-mail: journals@sagepub.com; Web site: http://sagepub.com
Publication Type: Journal Articles; Reports - Evaluative
Education Level: N/A
Audience: N/A
Language: English
Sponsor: Institute of Education Sciences (ED)
Authoring Institution: N/A
IES Funded: Yes
Grant or Contract Numbers: R305B090023; R305B100027