ERIC Number: ED478985
Record Type: RIE
Publication Date: 2003-Apr
Assessing the Efficiency of Item Selection in Computerized Adaptive Testing.
This study investigated the efficiency of item selection in a computerized adaptive test (CAT), where efficiency was defined in terms of the accumulated test information at an examinee's true ability level. A simulation methodology compared the efficiency of 2 item selection procedures with 5 ability estimation procedures for CATs of 5, 10, 15, and 25 items in length. The two item selection procedures included maximum Fisher information (FI) and maximum Fisher interval information (FII) item selection. The five ability estimation procedures included maximum likelihood, modal a posteriori (MAP), golden section search (GSS), and two procedures proposed in this study. These were ML/Alt and MAP/Alt, adjusted ML or MAP estimates according to a specific decision rule based on hypothesis testing. For the conventional item selection procedure (FI) and ability estimation procedures (ML and MAP), the best performance was observed for FI with MAP at middle ability levels, with efficiency attaining or exceeding 90% even for the shortest test length. In contrast, large gaps in efficiency were observed for FI with MAP at extreme ability levels, and for FI with ML across al ability levels. Utilizing FII item selection with ML and MAP narrowed the gaps in efficiency at the lowest ability levels for 5- and 10-item tests. The greatest increase in test efficiency was observed when the alternative ability estimation procedures (ML/Alt, MAP/Alt, and GSS) were used. The gains in efficiency were most pronounced for shorter tests, but were noticeable even for longer tests. Overall, it appears that the ability estimation procedure impacts the efficiency of item selection to a large extent than the item selection procedure. (Author/SLD)
Publication Type: Reports - Research; Speeches/Meeting Papers
Education Level: N/A
Authoring Institution: N/A
Note: Paper presented at the Annual Meeting of the American Educational Research Association (Chicago, IL, April 21-25, 2003).