NotesFAQContact Us
Search Tips
Peer reviewed Peer reviewed
Direct linkDirect link
ERIC Number: EJ798018
Record Type: Journal
Publication Date: 2008
Pages: 22
Abstractor: As Provided
Reference Count: 23
ISSN: ISSN-0146-6216
Conversion of Proportion-Correct Standard-Setting Judgments to Cutoff Scores on the Item Response Theory Theta Scale
Hurtz, Gregory M.; Jones, J. Patrick; Jones, Christian N.
Applied Psychological Measurement, v32 n5 p385-406 2008
This study compares the efficacy of different strategies for translating item-level, proportion-correct standard-setting judgments into a theta-metric test cutoff score for use with item response theory (IRT) scoring, using Monte Carlo methods. Simulated Angoff-type ratings, consisting of 1,000 independent 75 Item x13 Rater matrices, were generated at five points along the theta continuum, at three levels of rater fit to the item characteristics curves, yielding 14,625,000 ratings as the basis of the analyses. These simulated proportion-correct ratings were converted to the IRT theta scale using test-level and item-level methods explicated by Kane (1987). Kane's optimally weighted, item-level conversion method initially produced anomalous results; however, it was discovered that imposing a restriction on the weights avoided these anomalies and rendered the optimally weighted method the most statistically efficient. Six areas for future research are outlined for advancing the integration of these classical standard-setting ratings into IRT methodology. (Contains 2 figures, 6 tables, and 2 notes.)
SAGE Publications. 2455 Teller Road, Thousand Oaks, CA 91320. Tel: 800-818-7243; Tel: 805-499-9774; Fax: 800-583-2665; e-mail:; Web site:
Publication Type: Journal Articles; Reports - Evaluative
Education Level: N/A
Audience: Researchers
Language: English
Sponsor: N/A
Authoring Institution: N/A