ERIC Number: ED319784
Record Type: RIE
Publication Date: 1987-Apr
Reference Count: N/A
Putting the "But" Back in Meta-Analysis: Issues Affecting the Validity of Quantitative Reviews.
L'Hommedieu, Randi; And Others
Some of the frustrations inherent in trying to incorporate qualifications of statistical results into meta-analysis are reviewed, and some solutions are proposed to prevent the loss of information in meta-analytic reports. The validity of a meta-analysis depends on several factors, including the: thoroughness of the literature search; selection of studies for inclusion; appropriate coding and analysis of studies; and report format selected. The solution proposed to the problem of methodological quality is to include all selected studies and report an average effect size for the aggregate. The report on the meta-analysis then should be a qualitative, discursive argument rather than a simple statistic. Proposals for putting the "but" back in meta-analysis are: (1) assure that it is not a substitute for qualitative review; (2) offer the reader information necessary to evaluate the validity of decisions made at the individual level; and (3) assure that qualifications of studies are not excluded from the analysis. A thorough quantitative review should include: a discursive review of each study; a report on how each effect size was calculated; the location of the summary statistics upon which each effect size was based; and a discussion of study limitations and the factors that affect validity of effect size. Suggestions are also given for appropriate reporting and avoiding publication bias. (SLD)
Publication Type: Reports - Evaluative; Speeches/Meeting Papers
Education Level: N/A
Authoring Institution: N/A
Note: Paper presented at the Annual Meeting of the American Educational Research Association (Washington, DC, April 20-24, 1987).