NotesFAQContact Us
Collection
Advanced
Search Tips
Peer reviewed Peer reviewed
Direct linkDirect link
ERIC Number: EJ979205
Record Type: Journal
Publication Date: 2012-Oct
Pages: 23
Abstractor: As Provided
Reference Count: 20
ISBN: N/A
ISSN: ISSN-0146-6216
An Evaluation of Item Response Theory Classification Accuracy and Consistency Indices
Wyse, Adam E.; Hao, Shiqi
Applied Psychological Measurement, v36 n7 p602-624 Oct 2012
This article introduces two new classification consistency indices that can be used when item response theory (IRT) models have been applied. The new indices are shown to be related to Rudner's classification accuracy index and Guo's classification accuracy index. The Rudner- and Guo-based classification accuracy and consistency indices are evaluated and compared with estimates from the more commonly applied IRT-recursive procedure using a simulation study and data from two large-scale assessments. Results from the simulation study and practical examples suggested that the Guo- and Rudner-based indices tended to produce estimates that were closer to the simulated values and exceeded those from the IRT-recursive-based procedure. However, results did suggest that the Rudner- and Guo-based indices can have some undesirable features that are important to keep in mind when applying them in practice. The values of the classification accuracy and consistency indices appeared to be affected by a number of factors including the distribution of examinees, test length, the placement of the cut-scores, and the proficiency estimators applied to estimate examinee ability. Suggestions are made that an important part of investigations evaluating classification accuracy and consistency indices should be the creation of figures that show the value of the classification accuracy and classification consistency for individual examinees across the range of possible scores as these figures can help provide indications into subtle and important differences between indices. (Contains 5 tables and 3 figures.)
SAGE Publications. 2455 Teller Road, Thousand Oaks, CA 91320. Tel: 800-818-7243; Tel: 805-499-9774; Fax: 800-583-2665; e-mail: journals@sagepub.com; Web site: http://sagepub.com
Publication Type: Journal Articles; Reports - Research
Education Level: High Schools; Secondary Education
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Identifiers - Location: Michigan