ERIC Number: EJ782126
Record Type: Journal
Publication Date: 2008
Reference Count: 18
The File Drawer Problem in Reliability Generalization: A Strategy to Compute a Fail-Safe N with Reliability Coefficients
Howell, Ryan T.; Shields, Alan L.
Educational and Psychological Measurement, v68 n1 p120-128 2008
Meta-analytic reliability generalizations (RGs) are limited by the scarcity of reliability reporting in primary articles, and currently, RG investigators lack a method to quantify the impact of such nonreporting. This article introduces a stepwise procedure to address this challenge. First, the authors introduce a formula that allows researchers to estimate the lower bound population average reliability for a desired instrument. Second, they present an equation to determine the Fail-Safe N for RG. This equation estimates the number of "file drawer" studies required to drop the aggregate score reliability of an instrument below a specified criterion value. Finally, the authors demonstrate the utility of these equations using published RG studies. Comments on the conclusions drawn from each RG application are provided.
Descriptors: Reliability, Meta Analysis, Generalization, Evaluation Methods, Data Analysis, Research Methodology, Equations (Mathematics)
SAGE Publications. 2455 Teller Road, Thousand Oaks, CA 91320. Tel: 800-818-7243; Tel: 805-499-9774; Fax: 800-583-2665; e-mail: firstname.lastname@example.org; Web site: http://sagepub.com
Publication Type: Journal Articles; Reports - Research
Education Level: N/A
Authoring Institution: N/A