ERIC Number: EJ1111232
Record Type: Journal
Publication Date: 2008-Apr
Pages: 65
Abstractor: As Provided
ISBN: N/A
ISSN: EISSN-2330-8516
EISSN: N/A
Evaluating the Effectiveness of a Full-Population Estimation Method. Research Report. ETS RR-08-18
Braun, Henry; Zhang, Jinming; Vezzu, Sailesh
ETS Research Report Series, Apr 2008
At present, although the percentages of students with disabilities (SDs) and/or students who are English language learners (ELL) excluded from a NAEP administration are reported, no statistical adjustment is made for these excluded students in the calculation of NAEP results. However, the exclusion rates for both SD and ELL students vary substantially across jurisdictions at a given administration, and, in some cases, have changed substantially over time within a jurisdiction. Consequently, comparisons of performance based on reported NAEP scores may indeed be biased by differential exclusion and identification practices. Using only NAEP data, this report investigates plausible explanations for the observed heterogeneity among jurisdictions in exclusion rates. It also examines the operating characteristics of a particular class of methods that carry out statistical adjustments to NAEP's reported scores to address the possible bias due to differential exclusion rates. The final results of such adjustments are termed full-population estimates (FPEs). The conclusions are that there is both a strong likelihood of bias and that neither the current NAEP procedure nor the FPE methodologies constitutes an ideal solution. The former because it assumes that all excluded students could not meaningfully participate in NAEP, and the latter because they implicitly assume that all students could obtain a proper NAEP score.
Descriptors: Research Methodology, Computation, Disabilities, English Language Learners, National Competency Tests, Scores, Evaluation Methods, Test Bias, Sampling, Grade 4, Reading Achievement, Statistical Analysis, Comorbidity, Geographic Location
Educational Testing Service. Rosedale Road, MS19-R Princeton, NJ 08541. Tel: 609-921-9000; Fax: 609-734-5410; e-mail: RDweb@ets.org; Web site: https://www.ets.org/research/policy_research_reports/ets
Publication Type: Journal Articles; Reports - Research
Education Level: Grade 4; Intermediate Grades; Elementary Education
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Identifiers - Location: United States
Identifiers - Assessments and Surveys: National Assessment of Educational Progress
Grant or Contract Numbers: N/A