ERIC Number: ED389736
Record Type: RIE
Publication Date: 1995-Apr
Reference Count: N/A
Setting Standards on NAEP Performance Items.
Reckase, Mark D.; And Others
The research reported in this paper was conducted to gain information to guide the selection of standard setting procedures for use with polytomous items to set achievement levels on the National Assessment of Educational Progress (NAEP) assessments in U.S. History and geography. Standard-setting procedures were evaluated to determine the relative level of the standards, the relationships of the level of the standard set by each method to the standard set using dichotomously scored items, and the practicality of the procedure for operational standard setting. The methods explored were: (1) estimated score point percentage (P method); (2) modified percentage estimate (PGE2); (3) estimated mean score method (M method); and (4) the hybrid method (H method), a combination of the previously used paper-selection method and the M method. Data were from the full NAEP pool for each content area with 20 panelists at each level for grades 4, 8, and 12. The P method tended to set the highest standard overall, and the PGE2 method, the lowest overall. The M method, which set standards between P and PGE2, but above the H method when they could be compared directly, was selected as the operational method for achievement level setting, with raters trained in paper-selection as a first step. (Contains two figures, seven tables, and nine references.) (SLD)
Publication Type: Reports - Evaluative; Speeches/Meeting Papers
Education Level: N/A
Authoring Institution: N/A
Identifiers: Dichotomous Scoring; National Assessment of Educational Progress; Polytomous Variables
Note: Paper presented at the Annual Meeting of the American Educational Research Association and the National Council on Measurement in Education (San Francisco, CA, April 18-22, 1995).