NotesFAQContact Us
Collection
Advanced
Search Tips
ERIC Number: ED507841
Record Type: Non-Journal
Publication Date: 2008-May
Pages: 62
Abstractor: As Provided
Reference Count: 22
ISBN: N/A
ISSN: N/A
The Missing Data Assumptions of the Nonequivalent Groups with Anchor Test (NEAT) Design and Their Implications for Test Equating. Research Report. ETS RR-09-16
Sinharay, Sandip; Holland, Paul W.
Educational Testing Service
The nonequivalent groups with anchor test (NEAT) design involves missing data that are missing by design. Three popular equating methods that can be used with a NEAT design are the poststratification equating method, the chain equipercentile equating method, and the item-response-theory observed-score-equating method. These three methods each make different assumptions about the missing data in the NEAT design. Though studies have compared the equating performance of the three methods under the NEAT design, none has examined the missing data assumptions and their implications for such comparisons. The missing data assumptions can affect equating studies because it is necessary to fill in the missing data or their distribution in some way in order to have a true, or criterion, equating function to compare the accuracy and bias of the different methods. If the missing data or their distribution are filled in using missing data assumptions that correspond to a given method, that may favor that method in any comparison with the others. This paper first describes the missing data assumptions of the three equating methods and then performs a fair comparison of the 3 methods using data from 3 different operational tests. For each data set, we examine how the 3 equating methods perform when the missing data satisfy the assumptions made by only 1 of these equating methods. The chain equating method is somewhat more satisfactory overall than the other methods in our fair comparison of the methods; hence, we recommend that equating practitioners seriously consider the chain equating method when using the NEAT design. In addition, we conclude that the results from the different equating methods will tend to agree with each other when proper equating conditions are in place. Moreover, to uncover problems that might not reveal themselves otherwise, it is important for operational testing programs to apply multiple equating methods and study the differences among their results. Appendices include: (1) Raking; and (2) Proof of Theorem 1. (Contains 15 tables, 7 figures, and 4 notes.)
Educational Testing Service. Rosedale Road Mailstop 19R, Princeton, NJ 08541-0001. Tel: 609-921-9000; Fax: 609-734-5410; Web site: http://www.ets.org
Publication Type: Reports - Evaluative
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: Educational Testing Service