NotesFAQContact Us
Search Tips
ERIC Number: ED533258
Record Type: Non-Journal
Publication Date: 2007
Pages: 370
Abstractor: As Provided
Reference Count: 0
ISBN: ISBN-978-1-1095-9612-0
A Comparison Study of IRT Calibration Methods for Mixed-Format Tests in Vertical Scaling
Meng, Huijuan
ProQuest LLC, Ph.D. Dissertation, The University of Iowa
The purpose of this dissertation was to investigate how different Item Response Theory (IRT)-based calibration methods affect student achievement growth pattern recovery. Ninety-six vertical scales (4 x 2 x 2 x 2 x 3) were constructed using different combinations of IRT calibration methods (separate, pair-wise concurrent, semi-concurrent, and concurrent), lengths of common-item set (10 vs. 20 common items), types of common-item set (dichotomous-only vs. mixed-format), and numbers of polytomous items (6 vs. 12) for three simulated datasets differing in the number of examinees sampled per grade (500, 1000, 5000). Three indexes (absolute bias, standard error of equating and root mean square error) were used to evaluate the performance of the calibration methods on proficiency score distribution recovery over 80 replications. These indexes were derived for seven growth distribution criterion parameters (mean, standard deviation, effect size, and proportions of examines within four proficiency categories). Although exceptions were found in the results for all criterion parameters, important general trends did emerge. Pair-wise concurrent and semi-concurrent calibration methods performed better than concurrent and separate calibration methods for most criterion parameters and combinations of research conditions. Separate calibration, the vertical scaling method used most often in practice, provided the poorest results in most instances. Accuracy of vertical scaling also typically improved with larger samplings of examinees, more common items, mixing item formats in the common-item set, and increases in the number of polytomous items in the common-item set. General trends and exceptional cases from the various analyses are described in detail with tables provided for choosing an appropriate vertical scaling method in different decision contexts. [The dissertation citations contained here are published with the permission of ProQuest LLC. Further reproduction is prohibited without permission. Copies of dissertations may be obtained by Telephone (800) 1-800-521-0600. Web page:]
ProQuest LLC. 789 East Eisenhower Parkway, P.O. Box 1346, Ann Arbor, MI 48106. Tel: 800-521-0600; Web site:
Publication Type: Dissertations/Theses - Doctoral Dissertations
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A