NotesFAQContact Us
Search Tips
Peer reviewed Peer reviewed
PDF on ERIC Download full text
ERIC Number: ED567633
Record Type: Non-Journal
Publication Date: 2016
Pages: 7
Abstractor: ERIC
Comparing a Program Implemented under the Constraints of an RCT and in the Wild
Newman, Denis; Lazarev, Valeriy; Zacamy, Jenna
Society for Research on Educational Effectiveness
This paper examines the "ecological validity" of a randomized controlled trial (RCT) conducted in 42 high schools to measure the impact of a content literacy program. This study of comparable schools implementing the same program in different contexts highlights characteristics that are often not attended to in rigorous effectiveness research but are pertinent to understanding the effectiveness and scalability of the program. In the current study, a direct comparison of school achievement was not possible since the outcome measure used in the RCT was researcher-administered. However, the authors are able to look at teacher characteristics associated with higher student achievement in the RCT and they can look at teacher (and principal) characteristics associated with school-level gains in program participation in Scale-up. The authors found that RCT schools had lower levels of program-related characteristics associated with scale-up. Scale-up schools had similar levels of program-related teacher characteristics associated with greater achievement. Schools that were successful in growing participating internally were not more likely to have the characteristics associated with achievement gains. This suggests that program effectiveness in a controlled study may not be indicative of the program's prospects of wider adoption and, in fact, such studies may not create the best conditions for future program sustainability. Tables and figures are appended. [SREE documents are structured abstracts of SREE conference symposium, panel, and paper or poster submissions.]
Society for Research on Educational Effectiveness. 2040 Sheridan Road, Evanston, IL 60208. Tel: 202-495-0920; Fax: 202-640-4401; e-mail:; Web site:
Publication Type: Reports - Research
Education Level: High Schools; Secondary Education
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: Society for Research on Educational Effectiveness (SREE)
Identifiers - Location: California; Pennsylvania