NotesFAQContact Us
Search Tips
Peer reviewed Peer reviewed
PDF on ERIC Download full text
ERIC Number: ED562424
Record Type: Non-Journal
Publication Date: 2015
Pages: 11
Abstractor: ERIC
Reference Count: 9
Site Selection in Experiments: A Follow-Up Evaluation of Site Recruitment in Two Scale-Up Studies
Tipton, Elizabeth; Fellers, Lauren; Caverly, Sarah; Vaden-Kiernan, Michael; Borman, Geoffrey; Sullivan, Kate; Ruiz de Castillo, Veronica
Society for Research on Educational Effectiveness
Randomized experiments are commonly used to evaluate if particular interventions improve student achievement. While these experiments can establish that a treatment actually "causes" changes, typically the participants are not randomly selected from a well-defined population and therefore the results do not readily generalize. Three streams of research methodologies have been developed to improve generalizations from large-scale experiments: (1) "assessing" the degree of similarity between the convenience sample of schools or districts in a completed experiment (e.g., Stuart, Cole, Bradshaw, & Leaf, 2011; Olsen, Orr, Bell, & Stuart, 2013; Tipton, in press); (2) "reweighting" this convenience sample to be more similar to one or more well-defined inference populations (e.g., O'Muircheartaigh & Hedges, 2014; Tipton, 2013); and (3) improvements through design and improved recruitment strategies (e.g. Tipton et al, 2014; Tipton, 2014; Roschelle et al, 2014). Tipton et al (2014) provide a design-based approach that uses propensity score methodology to first compare an inference population to those eligible for recruitment in the experiment, and then creates strata for site-selection. The goal is to help recruiters create a recruitment strategy that is targeted and, that when perfectly implemented, results in a sample of sites that is like a miniature of the inference population of interest. This paper is a follow up study to the examples proposed and carried out in Tipton et al (2014), with the goal of evaluating the success of these methods in practice, as well as addressing additional problems that arose in recruitment. One figure and three tables are appended.
Society for Research on Educational Effectiveness. 2040 Sheridan Road, Evanston, IL 60208. Tel: 202-495-0920; Fax: 202-640-4401; e-mail:; Web site:
Publication Type: Reports - Research
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: Society for Research on Educational Effectiveness (SREE)