ERIC Number: ED421549
Record Type: Non-Journal
Publication Date: 1998
Reference Count: N/A
Quality Control of On-Line Calibration in Computerized Assessment. Research Report 98-03.
Glas, Cees A. W.
In computerized adaptive testing, updating parameter estimates using adaptive testing data is often called online calibration. In this paper, how to evaluate whether the adaptive testing model used for online calibration fits the item response model used sufficiently is studied. Three approaches are investigated, based on a Lagrange multiplier (LM) statistic (J. Aitchison and S. Silvey, 1958), a Wald statistic, and a cumulative sum (CUMSUM) statistic (W. Veerkamp, 1996). The power of the tests was evaluated with a number of simulation studies. The theoretical advantage of the CUMSUM procedure was that it is based on a directional hypothesis and can be used iteratively. The power of the procedures ranged from rather moderate to good, depending on the change. It was also found that all three tests were equally sensitive to changes in item difficulty and the guessing parameter. All these statistics detected that something has happened to the parameters, but it is very difficult to attribute misfit to specific parameters with these methods. (Contains 3 tables and 18 references.) (Author/SLD)
Descriptors: Adaptive Testing, Computer Assisted Testing, Foreign Countries, Item Response Theory, Models, Online Systems, Power (Statistics), Quality Control, Simulation, Test Items
Faculty of Educational Science and Technology, University of Twente, P.O. Box 217, 7500 AE Enschede, The Netherlands.
Publication Type: Reports - Evaluative
Education Level: N/A
Authoring Institution: Twente Univ., Enschede (Netherlands). Faculty of Educational Science and Technology.