ERIC Number: ED550195
Record Type: Non-Journal
Publication Date: 2011
Abstractor: As Provided
Reference Count: N/A
Item-Analysis Methods and Their Implications for the ILTA Guidelines for Practice: A Comparison of the Effects of Classical Test Theory and Item Response Theory Models on the Outcome of a High-Stakes Entrance Exam
Ellis, David P.
ProQuest LLC, Ph.D. Dissertation, University of Maryland, College Park.
The current version of the International Language Testing Association (ILTA) Guidelines for Practice requires language testers to pretest items before including them on an exam, or when pretesting is not possible, to conduct post-hoc item analysis to ensure any malfunctioning items are excluded from scoring. However, the guidelines are devoid of guidance with respect to which item-analysis method is appropriate for any given examination. The purpose of this study is to determine what influence choice of item-analysis method has on the outcome of a high-stakes university entrance exam. Two types of classical-test-theory (CTT) item analysis and three item-response-theory (IRT) models were applied to responses generated from a single administration of a 70-item dichotomously scored multiple-choice test of English proficiency, administered to 2,320 examinees applying to a prestigious private university in western Japan. Results illustrate that choice of item-analysis method greatly influences the ordinal ranking of examinees. The implications of these findings are discussed and recommendations are made for revising the ILTA Guidelines for Practice to delineate more explicitly how language testers should apply item analysis in their testing practice. [The dissertation citations contained here are published with the permission of ProQuest LLC. Further reproduction is prohibited without permission. Copies of dissertations may be obtained by Telephone (800) 1-800-521-0600. Web page: http://www.proquest.com/en-US/products/dissertations/individuals.shtml.]
Descriptors: Item Response Theory, High Stakes Tests, College Entrance Examinations, Item Analysis, Pretests Posttests, Test Theory, Multiple Choice Tests, Private Colleges, Guidelines, Language Proficiency, English Language Learners, Language Tests, Foreign Countries
Publication Type: Dissertations/Theses - Doctoral Dissertations
Education Level: Higher Education; Postsecondary Education
Authoring Institution: N/A
Identifiers - Location: Japan