NotesFAQContact Us
Collection
Advanced
Search Tips
Peer reviewed Peer reviewed
Direct linkDirect link
ERIC Number: EJ1189181
Record Type: Journal
Publication Date: 2018
Pages: 19
Abstractor: As Provided
ISBN: N/A
ISSN: ISSN-0013-1881
EISSN: N/A
Methodological Challenges in Education RCTs: Reflections from England's Education Endowment Foundation
Dawson, Anneka; Yeomans, Emily; Brown, Elena Rosa
Educational Research, v60 n3 p292-310 2018
Background: The Education Endowment Foundation (EEF) is an independent charity that was established in 2011 with the explicit aim of breaking the link between family income and educational achievement in England. Over the seven years since its inception, EEF has contributed to the existing evidence base by funding over one hundred randomised controlled trials (RCTs) in education. Purpose: This article discusses five of the key challenges EEF experienced in running RCTs in English schools; namely: (1) ensuring interventions are ready for trial, (2) recruiting schools and retaining them in trials, (3) calculating the appropriate sample size and ensuring cost effectiveness, (4) carrying out robust implementation and process evaluation, and (5) choosing and delivering appropriate testing. It is hoped that this discussion may provide insights for others who may be engaged in funding and/or conducting trials in education. Sources of evidence: The challenges in conducting RCTs in education are explored by drawing on EEF's experience in designing and reporting on trials. Examples from some of the EEF trials and wider methodological literature are used to illustrate the points of discussion. Main argument: The five challenges summarised here span the full process of running a trial in schools, from intervention design to the collection of outcome measures. These have had the biggest impact on the quality of trials and the confidence that EEF has in the results. The first lesson focuses on ensuring that the intervention is sufficiently developed and there is a level of assurance in both its scalability and feasibility prior to trialling. Key questions concerning an intervention's readiness are described and ways to improve interventions through piloting are discussed. The second lesson focuses on the recruitment of schools to a trial. Issues include the importance of communicating roles to schools and the need for time to do this effectively. Thirdly, the sample sizes of early EEF trials were informed by the size of impact seen in previous research which led to an overestimation of the effects that EEF was likely to see. Current considerations that influence the size of the trials are described and there is a discussion of why, sometimes, even small effects can be important. Fourthly, EEF trials have developed a more integrated approach to implementation and process evaluation (IPE). This is discussed in the context of how it has led to a better understanding of programme delivery. Finally, testing is discussed in light of what EEF has learned about ensuring testing success in educational trials. Conclusions: The main lessons that EEF has learned from the major challenges faced so far span a variety of facets of evaluation. Ultimately, though, these challenges can all be addressed at least partially, by two key approaches: namely by keeping schools central to research by considering their needs and by the early planning of trials to improve implementation, communication and delivery.
Routledge. Available from: Taylor & Francis, Ltd. 530 Walnut Street Suite 850, Philadelphia, PA 19106. Tel: 800-354-1420; Tel: 215-625-8900; Fax: 215-207-0050; Web site: http://www.tandf.co.uk/journals
Publication Type: Journal Articles; Reports - Descriptive
Education Level: Elementary Secondary Education
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Identifiers - Location: United Kingdom (England)
Grant or Contract Numbers: N/A