NotesFAQContact Us
Collection
Advanced
Search Tips
Peer reviewed Peer reviewed
PDF on ERIC Download full text
ERIC Number: ED554442
Record Type: Non-Journal
Publication Date: 2015-Feb
Pages: 49
Abstractor: As Provided
Reference Count: 21
ISBN: N/A
ISSN: N/A
How Methodology Decisions Affect the Variability of Schools Identified as Beating the Odds. REL 2015-071
Abe, Yasuyo; Weinstock, Phyllis; Chan, Vincent; Meyers, Coby; Gerdeman, R. Dean; Brandt, W. Christopher
Regional Educational Laboratory Midwest
Schools that show better academic performance than would be expected given characteristics of the school and student populations are often described as "beating the odds" (BTO). State and local education agencies often attempt to identify such schools as a means of identifying strategies or practices that might be contributing to the schools' relative success. Key decisions on how to identify BTO schools may affect whether schools make the BTO list and thereby the identification of practices used to beat the odds. The purpose of this study was to examine how a list of BTO schools might change depending on the methodological choices and selection of indicators used in the BTO identification process. This study considered whether choices of methodologies and type of indicators affect the schools that are identified as BTO. The three indicators were (1) type of performance measure used to compare schools, (2) the types of school characteristics used as controls in selecting BTO schools, and (3) the school sample configuration used to pool schools across grade levels. The study applied statistical models involving the different methodologies and indicators and documented how the lists schools identified as BTO changed based on the models. Public school and student data from one midwest state from 2007-08 through 2010-11 academic years were used to generate BTO school lists. By performing pairwise comparisons among BTO school lists and computing agreement rates among models, the project team was able to gauge the variation in BTO identification results. Results indicate that even when similar specifications were applied across statistical methods, different sets of BTO schools were identified. In addition, for each statistical method used, the lists of BTO schools identified varied with the choice of indicators. Fewer than half of the schools were identified as BTO in more than one year. The results demonstrate that different technical decisions can lead to different identification results. Three appendices include: (1) Measures of classification accuracy; (2) Data, measures, and outliers and missing data; and (3) Classification tables.
Regional Educational Laboratory Midwest. Available from: Institute of Education Sciences. 555 New Jersey Avenue NW, Washington, DC 20208. Tel: 800-872-5327; Web site: http://ies.ed.gov/ncee/edlabs/
Publication Type: Reports - Research
Education Level: Elementary Secondary Education
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: National Center for Education Evaluation and Regional Assistance (ED); Regional Educational Laboratory Midwest (ED); American Institutes for Research
Identifiers - Location: Michigan
IES Funded: Yes
Grant or Contract Numbers: ED-IES-12-C-0004