NotesFAQContact Us
Collection
Advanced
Search Tips
Peer reviewed Peer reviewed
Direct linkDirect link
ERIC Number: EJ1006848
Record Type: Journal
Publication Date: 2013-Jan
Pages: 21
Abstractor: As Provided
Reference Count: 29
ISBN: N/A
ISSN: ISSN-0146-6216
Comparing the Performance of Five Multidimensional CAT Selection Procedures with Different Stopping Rules
Yao, Lihua
Applied Psychological Measurement, v37 n1 p3-23 Jan 2013
Through simulated data, five multidimensional computerized adaptive testing (MCAT) selection procedures with varying test lengths are examined and compared using different stopping rules. Fixed item exposure rates are used for all the items, and the Priority Index (PI) method is used for the content constraints. Two stopping rules, standard error (SE) and predicted standard error reduction (PSER), are proposed; each MCAT selection process is stopped if either the required precision has been achieved or the selected number of items has reached the maximum limit. The five procedures are as follows: minimum angle (Ag), volume (Vm), minimize the error variance of the linear combination (V[subscript 1]), minimize the error variance of the composite score with the optimized weight (V[subscript 2]), and Kullback-Leibler (KL) information. The recovery for the domain scores or content scores and their overall score, test length, and test reliability are compared across the five MCAT procedures and between the two stopping rules. It is found that the two stopping rules are implemented successfully and that KL uses the least number of items to reach the same precision level, followed by Vm; Ag uses the largest number of items. On average, to reach a precision of SE = 0.35, 40, 55, 63, 63, and 82 items are needed for KL, Vm, V[subscript 1], V[subscript 2], and Ag, respectively, for the "S"E stopping rule. PSER yields 38, 45, 53, 58, and 68 items for KL, Vm, V[subscript 1], V[subscript 2], and Ag, respectively; PSER yields only slightly worse results than "SE", but with much fewer items. Overall, KL is recommended for varying-length MCAT. (Contains 2 tables and 4 figures.)
SAGE Publications. 2455 Teller Road, Thousand Oaks, CA 91320. Tel: 800-818-7243; Tel: 805-499-9774; Fax: 800-583-2665; e-mail: journals@sagepub.com; Web site: http://sagepub.com
Publication Type: Journal Articles; Reports - Evaluative
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Identifiers - Assessments and Surveys: Armed Forces Qualification Test