ERIC Number: ED456156
Record Type: Non-Journal
Publication Date: 2001-Apr
Reference Count: N/A
Use of the Rasch IRT Model in Standard Setting: An Item Mapping Method.
Wang, Ning; Wiser, Randall F.; Newman, Larry S.
This paper provides both logical and empirical evidence to justify the use of an item mapping method for establishing passing scores for multiple-choice licensure and certification examinations. After describing the item-mapping standard setting process, the paper discusses the theoretical basis and rationale for this newly developed method and presents the practical advantages of the item mapping method. Empirical evidence supporting the use of the item mapping method is provided by comparing the results from four standard setting studies for diverse licensure and certification examinations. The 4 cut scores, involving 79, 73, 43, and 53 items respectively, were conducted using both the item mapping and Angoff methods. Rating data from the four standard setting studies using each of the two methods were analyzed using item-by-rater random effects generalizability and dependability studies to examine which method yields higher inter-judge consistency. Results indicate that the item-mapping method produced higher inter-judge consistency and achieved greater rater agreement than the Angoff method. Another finding is that the item mapping method set consistently lower cut scores than the Angoff method. This result is consistent with other research findings that show unrealistically high ratings from the Angoff method or other test-centered standard setting methods that require judges to rate item by item. (Contains 3 tables, 3 figures, and 30 references.) (Author/SLD)
Publication Type: Reports - Evaluative; Speeches/Meeting Papers
Education Level: N/A
Authoring Institution: N/A