ERIC Number: ED442843
Record Type: Non-Journal
Publication Date: 2000-Apr-27
Reference Count: N/A
Research Study of the 1998 Civics NAEP Achievement Levels.
Loomis, Susan Cooper
In 1999 a validation study of the 1998 Civics Assessment of the National Assessment of Educational Progress (NAEP) was conducted, combining a similarity classification study and a booklet classification study. The rationale was that if the teachers who had participated in the 5-day achievement levels-setting process could not use the descriptions to judge student performance, it is unlikely that anyone could. If their classifications were very different from the performances in the test booklets, it would suggest that the cutpoints did not denote performance consistent with the achievement level descriptions (ALDs). Eleven eighth-grade teachers from the pilot study of achievement level setting panel participated. Teacher panelists tended to classify their own students higher than their performance levels on the special form of the NAEP developed for this study. When the same teachers were asked to classify the performance of students represented in the special Civics NAEP test booklets, they tended to classify those at or below the empirical score classification of the students' performance. These findings suggest that standards set with a booklet classification method will be higher than those set with the item-by-item method used for the NAEP achievement level setting process. Even teachers well trained in the NAEP achievement levels tend to overestimate the knowledge and skills of their students with respect to the ALDs. Overall, however, results provide information needed to confirm that the general achievement levels-setting process appeared to "work" in that panelists were able to use the ALDs in a different setting and for different purposes and the "translations" with respect to the score scale seem reasonably on target. An appendix contains classification forms from the similarities classification study, a list of participants in the validation portion of the study, and an agenda for the study session. (Contains 14 references.) (SLD)
Publication Type: Reports - Research; Speeches/Meeting Papers
Education Level: N/A
Sponsor: National Assessment Governing Board, Washington, DC.
Authoring Institution: N/A
Identifiers - Assessments and Surveys: National Assessment of Educational Progress