ERIC Number: EJ1081153
Record Type: Journal
Publication Date: 2015-Dec
Abstractor: As Provided
Does Matching Quality Matter in Mode Comparison Studies?
Zeng, Ji; Yin, Ping; Shedden, Kerby A.
Educational and Psychological Measurement, v75 n6 p1045-1062 Dec 2015
This article provides a brief overview and comparison of three matching approaches in forming comparable groups for a study comparing test administration modes (i.e., computer-based tests [CBT] and paper-and-pencil tests [PPT]): (a) a propensity score matching approach proposed in this article, (b) the propensity score matching approach used by Lottridge, Nicewander, and Mitzel, and (c) a modified approach of matched samples comparability analyses (MSCA) mentioned by Way, Davis, and Fitzpatrick. Different matching approaches resulted in different matched data with differing degrees of matching quality, and matched data from each matching approach were then used in the mode comparison investigation. Construct equivalence was examined and the level of invariance was found to be consistent across modes for all three matching approaches. Raw-to-scale score conversion tables were created, and the impact on CBT students' proficiency classification was examined. The comparison of the number of CBT students whose proficiency classification would be affected and the equality of score distributions between modes on raw scores and scale scores across the three matching approaches indicate that the propensity score matching approach delineated in this article led to the most consistent evidence for the conclusion of the mode comparison.
Descriptors: Comparative Analysis, Computer Assisted Testing, Probability, Classification, Scores, Item Response Theory
SAGE Publications. 2455 Teller Road, Thousand Oaks, CA 91320. Tel: 800-818-7243; Tel: 805-499-9774; Fax: 800-583-2665; e-mail: email@example.com; Web site: http://sagepub.com
Publication Type: Journal Articles; Reports - Research
Education Level: N/A
Authoring Institution: N/A