NotesFAQContact Us
Search Tips
ERIC Number: ED514496
Record Type: Non-Journal
Publication Date: 2010
Pages: 8
Abstractor: ERIC
Reference Count: 5
Evaluating Math Recovery: Measuring Fidelity of Implementation
Munter, Charles; Garrison, Anne; Cobb, Paul; Cordray, David
Society for Research on Educational Effectiveness
In this paper, the authors describe a case of measuring implementation fidelity within an evaluation study of Math Recovery (MR), a pullout tutoring program aimed at increasing the mathematics achievement of low-performing first graders, thereby closing the school-entry achievement gap by enabling them to achieve at the level of their higher-performing peers in the regular mathematics classroom. Two research questions guided the conduct and analysis of the larger study: 1) Does participation in MR raise the mathematics achievement of low performing first-grade students? 2) If so, do participating students maintain the gains made in first grade through the end of second grade? The analysis reported in this paper follows from a third question: 3) To what extent does fidelity of implementation influence the effectiveness of MR? The two-year evaluation of Math Recovery was conducted in 20 elementary schools (five urban, ten suburban and five rural), representing five districts in two states. The findings suggest it is possible to create a reliable instrument to measure implementation fidelity for differentiated interventions--an endeavor that has, heretofore, been largely avoided in evaluations of educational interventions. Many potentially high-quality interventions are unscripted, instead relying on teacher knowledge and professional development, requiring considerable differentiation by implementers. As individuals work to rigorously evaluate such programs, they need to develop reliable fidelity measures that are both feasible and true to program components, so that evaluators can adequately link measures of treatment integrity to outcomes, to more accurately determine the relative strength of interventions (Cordray & Pion, 2006). This paper outlines the development and use of one such measure as a case of how such fidelity instruments might be developed and used in the future. Critical aspects of the process included: (1) the identification of the core implementation components of the intervention (Fixsen et al., 2005); (2) close work with program developers to operationalize those components; (3) training of coders in both the program itself and the coding schemes/process; and (4) collaborating with the coding team to further refine operationalizations and coding decisions, to strike a balance of feasibility and adherence to program components.
Society for Research on Educational Effectiveness. 2040 Sheridan Road, Evanston, IL 60208. Tel: 202-495-0920; Fax: 202-640-4401; e-mail:; Web site:
Publication Type: Reports - Research
Education Level: Elementary Education; Grade 1; Primary Education
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: Society for Research on Educational Effectiveness (SREE)