ERIC Number: ED415289
Record Type: Non-Journal
Publication Date: 1997-Jun-18
What Do NAEP State Comparisons and Rankings Really Mean?
Robinson, Glen E.
This analysis addresses two important questions regarding appropriate uses of the National Assessment of Educational Progress (NAEP) statewide average test score comparisons. First is whether NAEP state test score averages provide meaningful and valid comparisons of the relative educational quality or proficiency of specific states. The other question is what NAEP state test score comparisons really mean. These questions are addressed through data from the 1996 NAEP and the 1992 NAEP State Trial Assessments in Mathematics for eighth graders. Analyses indicate that 89% of the state differences in the NAEP-92 Trial State Assessment test score averages can be explained by variations in four demographic variables over which schools have no control. This suggests that rather than measuring differences in the quality or proficiency of the state's educational programs, the NAEP-92 Trial State Assessment appear to reflect differences in what might be called the difficulty of the educational tasks or challenges facing the states. Where the differences in NAEP state test score averages are found to correlate highly with certain student demographic variables, such correlations should not be used to expect less learning from children in adverse circumstances. Instead, these findings should be rough indicators of the need for appropriate resources and instructional support to help students. In addition to the demographic influences on NEAP state assessment score averages, certain nondemographic factors, such as nonresponse bias, may influence NAEP state test scores substantially, materially affecting state rankings and state comparisons. This is an additional reason for not considering state test score averages as indicators of the relative quality of proficiency of state educational programs. (Contains 10 figures and 20 references.) (SLD)
Publication Type: Reports - Evaluative; Speeches/Meeting Papers
Education Level: N/A
Authoring Institution: N/A
Identifiers - Assessments and Surveys: National Assessment of Educational Progress