NotesFAQContact Us
Collection
Advanced
Search Tips
ERIC Number: ED506849
Record Type: Non-Journal
Publication Date: 2004-May
Pages: 32
Abstractor: ERIC
Reference Count: N/A
ISBN: N/A
ISSN: N/A
Evaluation of Bias Correction Methods for "Worst-case" Selective Non-participation in NAEP
McLaughlin, Don; Gallagher, Larry; Stancavage, Fran
American Institutes for Research
With the advent of No Child Left Behind (NCLB), the context for NAEP participation is changing. Whereas in the past participation in NAEP has always been voluntary, participation is now mandatory for some grade and subjects among schools receiving Title I funds. While this will certainly raise school-level participation rates in the mandated grades and subjects, it could have the opposite effect on non-mandated grades and subjects, particularly in light of the increased burden of state testing also required by NCLB. At the student level, participation remains voluntary in all subject areas as it has been in the past. However, participation rates could be influenced negatively by the NCLB requirement for more aggressive notification of students and parents regarding their rights to opt out of NAEP testing. Random non-participation introduces random error into NAEP estimates. More worrisome is the possibility of selective non-participation at the top or bottom of the ability distribution, which would introduce a bias into statewide mean scores. Although NAEP has not, as had once been proposed, been given an official confirmatory role under NCLB, one can expect greater scrutiny of the relationship between state scores and state NAEP scores in the coming years. This could lead to subtle pressures that depress participation among schools or students near the bottom of the distribution. Conversely, if non-Title I schools decide to take advantage of their exemption from mandatory participation, this could remove a disproportionate number of high performing schools from the sample, given that Title I funds are targeted at schools serving disadvantaged students. Furthermore, participation rates among high performing students could be differentially affected by a variety of factors. These could include a greater reluctance to lose instructional time for testing, or simply a greater willingness among affluent students and parents to assert their rights under the law. It may take several years for these countervailing forces to play themselves out and for any serious problems in NAEP participation rates to manifest. In the meantime, the purpose of this study is to estimate the potential bias from "worst-case" scenarios of selective non-participation, and to examine the extent to which statistical methods can correct for that bias. Correction for non-participation bias is partially effective, eliminating roughly half of the bias when the linear equating method is used. A useful finding is that state test scores can be replaced with quartiles in these non-participation bias corrections without serious degradation of results. (Contains 25 figures and 6 footnotes.)
American Institutes for Research. 1000 Thomas Jefferson Street NW, Washington, DC 20007. Tel: 202-403-5000; Fax: 202-403-5001; e-mail: inquiry@air.org; Web site: http://www.air.org
Publication Type: Numerical/Quantitative Data; Reports - Evaluative
Education Level: Elementary Secondary Education
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: American Institutes for Research
Identifiers - Laws, Policies, & Programs: No Child Left Behind Act 2001