ERIC Number: ED442844
Record Type: Non-Journal
Publication Date: 2000-Apr-27
Reference Count: N/A
Feedback in the NAEP Achievement Levels Setting Process.
Loomis, Susan Cooper
This paper describes the feedback included in operational achievement level setting (ALS) procedures for the National Assessment of Educational Progress (NAEP). It does not describe the feedback form field trials, pilot studies, or other research studies related to the NAEP. The NAEP ALS process includes three rounds of item-by-item ratings, and feedback is provided after each round. This means that panelists have additional information to consider for subsequent rounds of ratings, but additional training is required for no more than one of two new forms of feedback for each round. Most of the feedback is aimed at increasing interjudge and intrajudge consistency. The types of feedback provided are: (1) cutscores and standard deviations; (2) student performance data; (3) interrater consistency data; (4) whole booklet data; (5) interrater consistency data, including "Reckase" charts of performance data; and (6) consequences data. The use of each type of feedback is discussed, and panelists' evaluations of the feedback provided are noted. Panelists' responses indicate high levels of understanding about and confidence in using the feedback. The Reckase charts appear to be a significant addition to the array of feedback information. (Contains 9 tables, 17 figures, and 22 references.) (SLD)
Publication Type: Numerical/Quantitative Data; Reports - Descriptive; Speeches/Meeting Papers
Education Level: N/A
Sponsor: National Assessment Governing Board, Washington, DC.
Authoring Institution: N/A
Identifiers - Assessments and Surveys: National Assessment of Educational Progress