NotesFAQContact Us
Collection
Advanced
Search Tips
Peer reviewed Peer reviewed
Direct linkDirect link
ERIC Number: EJ960250
Record Type: Journal
Publication Date: 2012-Apr
Pages: 21
Abstractor: As Provided
Reference Count: 29
ISBN: N/A
ISSN: ISSN-0013-1644
Examining Rounding Rules in Angoff-Type Standard-Setting Methods
Wyse, Adam E.; Reckase, Mark D.
Educational and Psychological Measurement, v72 n2 p224-244 Apr 2012
This study investigates how different rounding rules and ways of providing Angoff standard-setting judgments affect cut-scores. A simulation design based on data from the National Assessment of Education Progress was used to investigate how rounding judgments to the nearest whole number (e.g., 0, 1, 2, etc.), nearest 0.05, or nearest two decimal places for individual items or clusters of items affected cut-scores for individual panelists and a group of panelists across four different pools of items. For the simulated ratings from a group of panelists, the recovery of the cut-scores was examined using the mean and the median. Results showed that rounding to nearest whole number had the potential to produce fairly large statistical biases in cut-score estimates. Biases were less when judgments were simulated across cluster of items. The largest biases were found at the advanced cut-score, but the greatest potential changes in the percentage of students that would be above the cut-score were found for the basic cut-score. Rounding to the nearest 0.05 or nearest two decimals places did not have a large impact on cut-score estimates and had little effect on the percentage of students above the cut-score. Implications for policy and future standard-setting practices are provided. (Contains 4 tables and 1 figure.)
SAGE Publications. 2455 Teller Road, Thousand Oaks, CA 91320. Tel: 800-818-7243; Tel: 805-499-9774; Fax: 800-583-2665; e-mail: journals@sagepub.com; Web site: http://sagepub.com
Publication Type: Journal Articles; Reports - Research
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Identifiers - Assessments and Surveys: National Assessment of Educational Progress