ERIC Number: ED514644
Record Type: Non-Journal
Publication Date: 2010
Reference Count: 2
A Procedure for Assessing Fidelity of Implementation in Experiments Testing Educational Interventions
Nelson, Michael C.; Cordray, David S.; Hulleman, Chris S.; Darrow, Catherine L.; Sommer, Evan C.
Society for Research on Educational Effectiveness
An educational intervention's effectiveness is judged by whether it produces positive outcomes for students, with the randomized controlled trial (CRT) as a valuable tool for determining intervention effects. However, the intervention-as-implemented in an experiment frequently differs from the intervention-as-designed, making it unclear whether unfavorable results are due to an ineffective intervention model or the failure to implement the model fully. It is therefore vital to assess fidelity of implementation and, where possible, to incorporate fidelity data in the analysis of outcomes. The objectives of this study are to: (1) elaborate a five-step procedure for fidelity assessment; (2) describe the advantages of assessing fidelity with this approach when conducting experiments testing educational interventions; and (3) use examples based on published studies to illustrate how this procedure can be applied. The authors find that, while most analyses of fidelity in educational studies include one or more steps in the five-step fidelity assessment process, the typical study does not include all steps and/or executes steps incompletely. As a result, the usefulness of fidelity data for interpreting experimental results is diminished. The authors demonstrate that a more complete assessment of fidelity in such cases would lead to a more meaningful analysis that interprets experimental results in light of actual implementation.
Descriptors: Intervention, Program Effectiveness, Demonstration Programs, Experimental Programs, Feasibility Studies, Pilot Projects, Evaluation Methods, Cognitive Mapping, Research Methodology, Sequential Approach, Educational Research, Literature Reviews, Meta Analysis
Society for Research on Educational Effectiveness. 2040 Sheridan Road, Evanston, IL 60208. Tel: 202-495-0920; Fax: 202-640-4401; e-mail: email@example.com; Web site: http://www.sree.org
Publication Type: Reports - Research
Education Level: Elementary Secondary Education; Higher Education
Authoring Institution: Society for Research on Educational Effectiveness (SREE)