ERIC Number: ED380777
Record Type: Non-Journal
Publication Date: 1995-Feb
Reference Count: N/A
NAEP Reading Revisit: An Evaluation of the 1992 Achievement Levels Descriptions.
American Coll. Testing Program, Iowa City, IA.
A study was designed to provide recommendations regarding the use of the achievement levels set in 1992 for reporting National Assessment of Educational Progress (NAEP) reading results in 1994 and in future NAEP reading assessments. Two procedures were used: the Item Difficulty Categorization (IDC) procedure involved an evaluation of the achievement levels descriptions (ALDs) via a statistical categorization of items; and the Judgmental Item Categorization (JIC) evaluated the ALDs via a judgmental item mapping. A total of 56 panelists were trained in the reading framework and achievement levels set in 1992. The high correspondence between the judgmental classification and the statistical classification provided compelling evidence that the achievement levels descriptions communicate clearly and accurately with respect to student performance. Further, the recommendations developed by the panelists involved in the two evaluation methods were quite similar. The fact that the recommendations made by the two sets of panelists were similar and confirming of the achievement levels seemed a sufficiently positive outcome to support the use of the achievement levels for reporting the 1994 results. To the extent that panelists recommended changes in the achievement levels descriptions, it was to increase the requirements for the Basic level descriptions, based on actual student performance. Contains nine tables of data. Appendixes present a list of observers, panelists, and staff; nomination material; achievement levels descriptions and procedures for evaluating; the agenda; examples of IDC and JIC lists; grade-level organizational and format suggestions for ALDs; and an alternative method of computing "hits." (RS)
Publication Type: Reports - Evaluative
Education Level: N/A
Sponsor: National Assessment Governing Board, Washington, DC.
Authoring Institution: American Coll. Testing Program, Iowa City, IA.
Identifiers - Assessments and Surveys: National Assessment of Educational Progress