NotesFAQContact Us
Search Tips
ERIC Number: ED499213
Record Type: Non-Journal
Publication Date: 2007-Sep
Pages: 282
Abstractor: ERIC
Validity Study of the NAEP Mathematics Assessment: Grades 4 and 8
Daro, Phil; Stancavage, Frances; Ortega, Moreica; DeStefano, Lizanne; Linn, Robert
American Institutes for Research
In Spring 2006,. the NAEP Validity Studies (NVS) Panel was asked by the National Center for Education Statistics (NCES) to undertake a validity study to examine the quality of the NAEP Mathematics Assessments at grades 4 and 8. Specifically, NCES asked the NVS Panel to address five questions: (1) Does the NAEP framework offer reasonable content and skill-based coverage compared to the assessments of states and other nations? (2) Does the NAEP item pool and assessment design accurately reflect the NAEP framework? (3) Is NAEP mathematically accurate and not unduly oriented to a particular curriculum, philosophy or pedagogy? (4) Does NAEP properly consider the spread of abilities in the assessable population? (5) Does NAEP provide information that is representative of all students, including students who are unable to demonstrate their achievements on the standard assessment? The Panel's central finding of the validity study is that the NAEP mathematics assessment is sufficiently robust to support the main conclusions that have been drawn about the U.S. and state progress in mathematics since 1990. NAEP results show achievement in mathematics rising steadily over the years for all subgroups, although gaps among subgroups persist. Other findings are that: The NAEP framework is reasonable; however, the NAEP framework and specifications do not provide as much guidance for test developers as they could. The NAEP item pool broadly aligns with the framework with some important exceptions. Item quality is typical of large-scale assessments but could be better. Measurement precision is good over a broad range of proficiency but could be better for lower-achieving students. Nine recommendations flow from this study. They are: (1) Sharpen the framework; (2) Provide detailed implementation plans; (3) Define a larger role for exemplar items; (4) Improve quality assurance for the overall item pool and for individual items; (5) Attend particularly to the various aspects of item quality; (6) Undertake a program of evidence-based research on item design; (7) Expand the range of item difficulty and curricular reach; (8) Manage changes in the item pool; and (9) Move NAEP in the direction of adaptive testing. These recommendations hold promise for improving the quality of assessment, not only within the NAEP program, but for U.S. education overall. [The NAEP Validity Studies Panel was formed by the American Institutes for Research under contract with the National Center for Education Statistics.]
American Institutes for Research. 1000 Thomas Jefferson Street NW, Washington, DC 20007. Tel: 202-403-5000; Fax: 202-403-5001; e-mail:; Web site:
Publication Type: Reports - Evaluative
Education Level: Grade 4; Grade 8
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: American Institutes for Research, Washington, DC
Grant or Contract Numbers: N/A