ERIC Number: ED422351
Record Type: Non-Journal
Publication Date: 1998-Apr
Reference Count: N/A
Booklet Classification Study.
Hanson, Bradley A.; Bay, Luz; Loomis, Susan Cooper
Research studies using booklet classification were implemented by the American College Testing Program to investigate the linkage between the National Assessment of Educational Progress (NAEP) Achievement Levels Descriptions and the cutpoints set to represent student performance with respect to the achievement levels. This paper describes the process and reports the results of the booklet classification study (BCS) implemented for the science achievement levels. It explores the possibility of using booklet classification as a way to set achievement levels by investigating methodologies for computing achievement level cutpoints using booklet classification data. These methodologies were applied to BCS data for science in this study and had been applied to geography and U.S. history. The BCS for science achievement levels involved grades 4 and 8, with 13 panelists for each grade level. Eighteen booklets were selected from NAEP forms, and 22 from other sources. The BCS for science, geography, and U.S. history have all resulted in panelists' classifying student performance at a lower level than plausible values scores indicate. These results indicate that cutpoints computed from booklet classification data would be higher than cutpoints based on the item-by-item rating methods that were used operationally. Procedures using the proportional odds model and nonparametric discriminant analysis were developed as a way to compute Achievement Level cutpoints using booklet classification data. Further refinements to these procedures, especially the nonparametric discriminant analysis, are needed before they could be used operationally to set cutpoints. (Contains 15 tables, 3 figures, and 13 references.) (SLD)
Publication Type: Numerical/Quantitative Data; Reports - Evaluative; Speeches/Meeting Papers
Education Level: N/A
Sponsor: National Assessment Governing Board, Washington, DC.
Authoring Institution: ACT, Inc., Iowa City, IA.
Identifiers - Assessments and Surveys: ACT Assessment; National Assessment of Educational Progress