Download full text
Download full text
ERIC Number: EJ1111003
Record Type: Journal
Publication Date: 2005-Feb
Abstractor: As Provided
Reference Count: 31
A Comparison of Two Procedures for Constrained Adaptive Test Construction. Research Report. ETS RR-04-39
Robin, Frédéric; van der Linden, Wim J.; Eignor, Daniel R.; Steffen, Manfred; Stocking, Martha L.
ETS Research Report Series, Feb 2005
The relatively new shadow test approach (STA) to computerized adaptive testing (CAT) proposed by Wim van der Linden is a potentially attractive alternative to the weighted deviation algorithm (WDA) implemented at ETS. However, it has not been evaluated under testing conditions representative of current ETS testing programs. Of interest was whether STA would, under typical high-stakes on-demand testing situations, produce tests of comparable or better psychometric quality as those produced by the current weighted deviation algorithm. Based on simulated data, we found that the STA performed as well or slightly better than the WDA on two of the three commonly accepted testing objectives: measurement and content. The WDA appeared to perform slightly better than the STA when the issue is security or item exposure control. The paper provides a review of the rationale that led to the specific testing objectives employed, an outline of the test construction steps common to the two procedures investigated, and a description of the specific models and algorithms employed by both procedures. Detailed description of the simulation study conducted and the results obtained from both of the procedures for one of the pools are also provided. Results are summarized and further research needs in particular areas discussed.
Descriptors: Test Construction, Computer Assisted Testing, Simulation, Evaluation Methods, Evaluation Research, Measurement Techniques, Item Response Theory, Test Content, Test Items
Educational Testing Service. Rosedale Road, MS19-R Princeton, NJ 08541. Tel: 609-921-9000; Fax: 609-734-5410; e-mail: RDweb@ets.org; Web site: https://www.ets.org/research/policy_research_reports/ets
Publication Type: Journal Articles; Reports - Research
Education Level: N/A
Authoring Institution: N/A