ERIC Number: ED238999
Record Type: RIE
Publication Date: 1983
Reference Count: 0
Dilemmas in Meta-Analysis: A Reply to Reanalyses of the Desegregation-Achievement Synthesis.
Crain, Robert L.
The decision by the National Institute of Education panel on the effects of school desegregation to select (for meta-analysis) a small group of preferred studies based upon criteria chosen in advance of examining the studies was, in principle, a mistake. One usually cannot know until the data have been examined which of several competing methodological criteria are most important. In the case of the effects of desegregation on minority achievement, Crain and Mahard in their 1982 review of 93 desegregation studies found a methodological error so specific to desegregation research that it was not even recognized as an error until the review was done. The error was that studies of the effects of desegregation on minority achievement will underestimate any effects when using subjects who have not been in desegregated settings since kindergarten or Grade 1. Whereas Crain and Mahard found 20 studies of blacks in desegregated settings since kindergarten or Grade 1, the panel discarded all but one of them because they did not fit their chosen-in-advance criteria. Of the 20 studies identified by Crain and Mahard, 16 showed consistent positive outcomes and only 2 were negative. If the principal function of selecting a superior subgroup of studies is to find the consistency of results which is masked by error in an unselected sample, Crain and Mahard succeeded, and the panel did not. (CMG)
Descriptors: Academic Achievement, Achievement Gains, Black Students, Desegregation Effects, Elementary Secondary Education, Evaluation Criteria, Grade 1, Kindergarten, Meta Analysis, Outcomes of Education, Program Effectiveness, Program Evaluation, Research Methodology, Research Reports, School Desegregation, Validity
Publication Type: Information Analyses; Opinion Papers
Education Level: N/A
Sponsor: National Inst. of Education (ED), Washington, DC.
Authoring Institution: N/A
Note: For related documents, see UD 023 303-308. Paper submitted as one of a collection from the National Institute of Education Panel on the Effects of School Desegregation. Document may not reproduce well.